[tarantool-patches] Re: [PATCH v2 4/8] vinyl: keep track of dumps per compaction for each LSM tree

[Konstantin Osipov <kostja@tarantool.org>]

* Vladimir Davydov <vdavydov.dev@gmail.com> [19/02/08 20:42]:
> On Thu, Jan 24, 2019 at 08:12:40PM +0300, Vladimir Davydov wrote:
> > This patch adds dumps_per_compaction metric to per index statistics. It
> > shows the number of dumps it takes to trigger a major compaction of a
> > range in a given LSM tree. We need it to automatically choose the
> > optimal number of ranges that would smooth out the load generated by
> > range compaction.
> > 
> > To calculate this metric, we assign dump_count to each run. It shows how
> > many dumps it took to create the run. If a run was created by a memory
> > dump, it is set to 1. If a run was created by a minor compaction, it is
> > set to the sum of dump counts of compacted ranges. If a run was created
> > by a major compaction, it is set to the sum of dump counts of compacted
> > ranges minus dump count of the last level run. The dump_count is stored
> > in vylog.
> > 
> > This allows us to estimate the number of dumps that triggers compaction
> > in a range as dump_count of the last level run stored in the range.
> > Finally, we report dumps_per_compaction of an LSM tree as the minimal
> > dumps_per_compaction among all ranges constituting the tree. To achieve
> > that, we maintain a heap of ranges per each LSM tree ordered by
> > dumps_per_compaction.
> > 
> > Needed for #3944
> 
> Rewritten this patch without using a heap - see below.

OK to push.


-- 
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov