From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Fri, 24 Aug 2018 11:18:40 +0300 From: Vladimir Davydov Subject: Re: [PATCH 17/18] vinyl: set quota timer period to 100 ms Message-ID: <20180824081840.t3bd7yoikmw4w4vb@esperanza> References: <20180823204936.GE24711@chai> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180823204936.GE24711@chai> To: Konstantin Osipov Cc: tarantool-patches@freelists.org List-ID: On Thu, Aug 23, 2018 at 11:49:36PM +0300, Konstantin Osipov wrote: > * Vladimir Davydov [18/08/16 23:03]: > > Currently, it's 1 second, which is OK for calculating watermark, but > > too long for throttling (think of latency of 1 seconds that would be > > introduced by throttling if such timeout were used). > > 1) I think the timer period should self-adjust based on the amount > of available quota. Do we really need to bother? An execution of a timer callback is pretty cheap - it's merely a few arithmetic calculations. I set it to 100 ms to make sure the latency won't be too high when throttling is enabled. > > 2) We narrowed down the problem to throttling only. > We never do anticipatory dump/compaction today. Let's design a > system which swings the pendulum both ways - not only throttles > the client when the load is high but performs an anticipatory > dump/compaction when the load is low. See for example > https://github.com/tarantool/tarantool/issues/3225 It totally makes sense, of course, but it's a completely different issue, which can be done separately on top of this patch set. > > In other words, I think we should not only enable "stalls" in > the timer callback when the write stream is strong, but > trigger compaction when the write stream is thin. Unfortunately > we do not collect read statistics per lsm as a secondary > indicator of compaction priority, so we may only trigger > compaction based simply on the number of levels, but never the > less.