From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id D93C36EC60; Wed, 31 Mar 2021 11:28:43 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D93C36EC60 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617179323; bh=+y58jii40d13t/xwy6YjTGu6m2kp0dVLnBsrASmOiuc=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=iTcW7UGIm2XrILFr+raI91pT3rEs9SJd1cglXLpHzdsVHigi7rlLWmTduyQTKJlTz HVIt95zJhajblsJ7ys6Qg5i3LPD6nA9GGiXVmcYuMysfgGNAoNw6MTWQIQzqxU7DJN dnMDEALhIsspxUN7uNp/RFPEANdEIQti2sdKIvCU= Received: from smtp40.i.mail.ru (smtp40.i.mail.ru [94.100.177.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C3C7C6EC60 for ; Wed, 31 Mar 2021 11:28:41 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C3C7C6EC60 Received: by smtp40.i.mail.ru with esmtpa (envelope-from ) id 1lRWDP-0000c5-HS; Wed, 31 Mar 2021 11:28:39 +0300 Date: Wed, 31 Mar 2021 08:28:39 +0000 To: Cyrill Gorcunov Message-ID: <20210331082839.ozes6v4nyv4nnfpr@tarantool.org> References: <20210327111310.37504-1-gorcunov@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20210327111310.37504-1-gorcunov@gmail.com> User-Agent: NeoMutt/20170113 (1.7.2) X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9ED7173E37F4E32947A0146560F8BA709498CFB6209D8582A182A05F538085040D0A2611A86A8462FEC8826F0D184EC0B2133CC81E2226719FDEFB2F92BFAABBF X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE79961E86438F5BDAEEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063795DC1BB912715AE68638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95CE99938B3FD79E1DFAB4780029B0EFD183CC94BD419642F4DA471835C12D1D9774AD6D5ED66289B5278DA827A17800CE767883B903EA3BAEA9FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C390BCC82C2C62A6D1117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CF7C60770734F29A93BA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B60099AFFAF51EA17376E601842F6C81A1F004C906525384307823802FF610243DF43C7A68FF6260569E8FC8737B5C2249B372FE9A2E580EFC725E5C173C3A84C30584FF81F342DA0735872C767BF85DA2F004C90652538430E4A6367B16DE6309 X-C1DE0DAB: 0D63561A33F958A5341313261A00426E53AEA76D982FE10DB39A3568B89E07CED59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34CAFBC0A7A4BEEE014683A3FFBCE5D418538E08CB20B220A8611315D03D16B57CD55245C1DFBB5BCD1D7E09C32AA3244CC22AE4703A5E4BBA4F37A8194C18E26EFE8DA44ABE2443F7927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojfQIxm2xDGoaaYJc6n/tOSQ== X-Mailru-Sender: 05EB39F83D09414F9B5D7F52D7CDFDD28911498D8755C4D1482023C5BBAC66013120E6DF4C9998C9CA16B95394F0DD5CE99530A0C0F27B5268329DCED823713783C0E760C018FF54112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v6 0/3] gc/xlog: delay xlog cleanup until relays are subscribed X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Kirill Yukhin via Tarantool-patches Reply-To: Kirill Yukhin Cc: tml , Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hello, On 27 мар 14:13, Cyrill Gorcunov wrote: > Take a look please. > > v2: > - rebase code to the fresh master branch > - keep wal_cleanup_delay option name > - pass wal_cleanup_delay as an option to gc_init, so it > won't be dependent on cfg engine > - add comment about gc_delay_unref in plain bootstrap mode > - allow to setup wal_cleanup_delay dynamically > - update comment in gc_wait_cleanup and call it conditionally > - declare wal_cleanup_delay as a double > - rename gc.cleanup_is_paused to gc.is_paused and update output > - do not show ref counter in box.info.gc() output > - update documentation > - move gc_delay_unref inside relay_subscribe call which runs > in tx context (instead of relay's context) > - update tests: > - add a comment why we need a temp space on replica node > - use explicit insert/snapshot operations > - shrkink the number of insert/snapshot to speedup testing > - use "restart" instead of stop/start pair > - use wait_log helper instead of own function > - add is_paused test > v3: > - fix changelog > - rework box_check_wal_cleanup_delay, the replication_anon > setting is considered only in box_set_wal_cleanup_delay, > ie when config is checked and parsed, moreover the order > of setup is set to be behind "replication_anon" option > processing > - delay cycle now considers deadline instead of per cycle > calculation > - use `double` type for timestamp > - test update > - verify `.is_paused` value > - minimize number of inserts > - no need to use temporary space, regular space works as well > - add comments on why we should restart the master node > v4: > - drop argument from gc_init(), since we're configuring delay > value from load_cfg.lua script there is no need to read the > delay early, simply start gc paused and unpause it on demand > - move unpause message to main wait cycle > - test update: > - verify tests and fix replication/replica_rejoin since it waits > for xlogs to be cleaned up too early > - use 10 seconds for XlogGapError instead of 0.1 second, this is > a common deadline value > v5: > - define limits for `wal_cleanup_delay`: it should be either 0, > or in range [0.001; TIMEOUT_INFINITY]. This is done to not consider > fp epsilon as a meaningul value > - fix comment about why anon replica is not using delay > - rework cleanup delay'ed cycle > - test update: > - update vinyl/replica_rejoin -- we need to disable cleanup > delay explicitly > - update replication/replica_rejoin for same reason > - drop unneded test_run:switch() calls > - add a testcase where timeout is decreased and cleanup > fiber is kicked to run even with stuck replica > > v6: > - test update: > - replica_rejoin.lua simplified to drop not needed data > - update main test to check if _cluster sleanup triggers > the fiber to run > > issue https://github.com/tarantool/tarantool/issues/5806 > branch gorcunov/gh-5806-xlog-gc-6 I've checked your patch into 2.6, 2.7 and master. -- Regards, Kirill Yukhin