From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 1134725DD0 for ; Mon, 11 Feb 2019 13:17:52 -0500 (EST) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kjOjcJr2BUua for ; Mon, 11 Feb 2019 13:17:51 -0500 (EST) Received: from smtp62.i.mail.ru (smtp62.i.mail.ru [217.69.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id C370622F08 for ; Mon, 11 Feb 2019 13:17:51 -0500 (EST) Received: from [185.6.245.178] (port=33242 helo=atlas.local) by smtp62.i.mail.ru with esmtpa (envelope-from ) id 1gtG9O-0007co-9b for tarantool-patches@freelists.org; Mon, 11 Feb 2019 21:17:50 +0300 Date: Mon, 11 Feb 2019 21:17:50 +0300 From: Konstantin Osipov Subject: [tarantool-patches] Re: [PATCH v2 4/8] vinyl: keep track of dumps per compaction for each LSM tree Message-ID: <20190211181750.GD9960@chai> References: <8fb3fe3256e5423d50cb22abda3cc5a3219d6a40.1548349067.git.vdavydov.dev@gmail.com> <20190208174205.qccmsfucrfrztk6b@esperanza> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190208174205.qccmsfucrfrztk6b@esperanza> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-subscribe: List-owner: List-post: List-archive: To: tarantool-patches@freelists.org * Vladimir Davydov [19/02/08 20:42]: > On Thu, Jan 24, 2019 at 08:12:40PM +0300, Vladimir Davydov wrote: > > This patch adds dumps_per_compaction metric to per index statistics. It > > shows the number of dumps it takes to trigger a major compaction of a > > range in a given LSM tree. We need it to automatically choose the > > optimal number of ranges that would smooth out the load generated by > > range compaction. > > > > To calculate this metric, we assign dump_count to each run. It shows how > > many dumps it took to create the run. If a run was created by a memory > > dump, it is set to 1. If a run was created by a minor compaction, it is > > set to the sum of dump counts of compacted ranges. If a run was created > > by a major compaction, it is set to the sum of dump counts of compacted > > ranges minus dump count of the last level run. The dump_count is stored > > in vylog. > > > > This allows us to estimate the number of dumps that triggers compaction > > in a range as dump_count of the last level run stored in the range. > > Finally, we report dumps_per_compaction of an LSM tree as the minimal > > dumps_per_compaction among all ranges constituting the tree. To achieve > > that, we maintain a heap of ranges per each LSM tree ordered by > > dumps_per_compaction. > > > > Needed for #3944 > > Rewritten this patch without using a heap - see below. OK to push. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov