From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 6 Sep 2018 10:37:51 +0300 From: Konstantin Osipov Subject: Re: [PATCH 5/8] vinyl: use separate thread pools for dump and compaction tasks Message-ID: <20180906073751.GE8205@chai> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: To: Vladimir Davydov Cc: tarantool-patches@freelists.org List-ID: * Vladimir Davydov [18/09/04 20:59]: > Using the same thread pool for both dump and compaction tasks makes > estimation of dump bandwidth unstable. For instance, if we have four > worker threads, then the observed dump bandwidth may vary from X if > there's high compaction demand and all worker threads tend to be busy > with compaction tasks to 4 * X if there's no compaction demand. As a > result, we can overestimate the dump bandwidth and trigger dump when > it's too late, which will result in hitting the limit before dump is > complete and hence stalling write transactions, which is unacceptable. > > To avoid that, let's separate thread pools used for dump and compaction > tasks. Since LSM tree based design typically implies high levels of > write amplification, let's allocate 1/4th of all threads for dump tasks > and use the rest exclusively for compaction. Please follow this up by changing the default number of write threads to 4 and making 4 the lower bound for this configuration setting. Thanks. > -- -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov