From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id AAE0B26236 for ; Mon, 11 Feb 2019 13:21:59 -0500 (EST) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SgZw0px_Nsls for ; Mon, 11 Feb 2019 13:21:59 -0500 (EST) Received: from smtp35.i.mail.ru (smtp35.i.mail.ru [94.100.177.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 67BB42618F for ; Mon, 11 Feb 2019 13:21:59 -0500 (EST) Received: from [185.6.245.178] (port=60502 helo=atlas.local) by smtp35.i.mail.ru with esmtpa (envelope-from ) id 1gtGDN-0000Ht-Mt for tarantool-patches@freelists.org; Mon, 11 Feb 2019 21:21:57 +0300 Date: Mon, 11 Feb 2019 21:21:57 +0300 From: Konstantin Osipov Subject: [tarantool-patches] Re: [PATCH v2 6/8] vinyl: randomize range compaction to avoid IO load spikes Message-ID: <20190211182157.GB10433@chai> References: <1d11af88e61d4ddb86f3406202ad178c1c7672cf.1548349067.git.vdavydov.dev@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1d11af88e61d4ddb86f3406202ad178c1c7672cf.1548349067.git.vdavydov.dev@gmail.com> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-subscribe: List-owner: List-post: List-archive: To: tarantool-patches@freelists.org * Vladimir Davydov [19/01/24 20:16]: > Since all ranges constituting an LSM tree have the same configuration, > they tend to get compacted at approximately the same time. This entails > IO load spikes, which, in turn, lead to deviation of the LSM tree from > the target shape and hence increased read amplification. To prevent this > from happening, this patch implements compaction randomization: with 10% > probability we defer compaction at each LSM tree level, i.e. if the > number of runs at a level exceeds the configured run_count_per_level, > the level will be compacted with 90%-probability, but with 10% > probability it won't - compaction will be deferred until another run > is added to the level. > > Our simulations show that such a simple algorithm performs fairly well: > it randomizes compaction pace among ranges, spreading IO load evenly in > time, while the write amplification is increased by not more than 5-10%, > which seems to be a reasonable price for elimination of IO load spikes. > > Closes #3944 OK to push. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov