From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Tue, 12 Feb 2019 13:16:15 +0300 From: Vladimir Davydov Subject: Re: [tarantool-patches] Re: [PATCH v2 6/8] vinyl: randomize range compaction to avoid IO load spikes Message-ID: <20190212101615.epw3hyfnuxnzy7go@esperanza> References: <1d11af88e61d4ddb86f3406202ad178c1c7672cf.1548349067.git.vdavydov.dev@gmail.com> <20190211182157.GB10433@chai> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190211182157.GB10433@chai> To: Konstantin Osipov Cc: tarantool-patches@freelists.org List-ID: On Mon, Feb 11, 2019 at 09:21:57PM +0300, Konstantin Osipov wrote: > * Vladimir Davydov [19/01/24 20:16]: > > Since all ranges constituting an LSM tree have the same configuration, > > they tend to get compacted at approximately the same time. This entails > > IO load spikes, which, in turn, lead to deviation of the LSM tree from > > the target shape and hence increased read amplification. To prevent this > > from happening, this patch implements compaction randomization: with 10% > > probability we defer compaction at each LSM tree level, i.e. if the > > number of runs at a level exceeds the configured run_count_per_level, > > the level will be compacted with 90%-probability, but with 10% > > probability it won't - compaction will be deferred until another run > > is added to the level. > > > > Our simulations show that such a simple algorithm performs fairly well: > > it randomizes compaction pace among ranges, spreading IO load evenly in > > time, while the write amplification is increased by not more than 5-10%, > > which seems to be a reasonable price for elimination of IO load spikes. > > > > Closes #3944 > > OK to push. Pushed to 2.1 and 1.10.