Re: [tarantool-patches] Re: [PATCH v2 6/8] vinyl: randomize range compaction to avoid IO load spikes

[Vladimir Davydov <vdavydov.dev@gmail.com>]

On Mon, Feb 11, 2019 at 09:21:57PM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [19/01/24 20:16]:
> > Since all ranges constituting an LSM tree have the same configuration,
> > they tend to get compacted at approximately the same time. This entails
> > IO load spikes, which, in turn, lead to deviation of the LSM tree from
> > the target shape and hence increased read amplification. To prevent this
> > from happening, this patch implements compaction randomization: with 10%
> > probability we defer compaction at each LSM tree level, i.e. if the
> > number of runs at a level exceeds the configured run_count_per_level,
> > the level will be compacted with 90%-probability, but with 10%
> > probability it won't - compaction will be deferred until another run
> > is added to the level.
> > 
> > Our simulations show that such a simple algorithm performs fairly well:
> > it randomizes compaction pace among ranges, spreading IO load evenly in
> > time, while the write amplification is increased by not more than 5-10%,
> > which seems to be a reasonable price for elimination of IO load spikes.
> > 
> > Closes #3944
> 
> OK to push.

Pushed to 2.1 and 1.10.