From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 10EBB6EC56; Fri, 19 Mar 2021 02:05:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 10EBB6EC56 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1616108722; bh=ZLlk6DRlWjBwfDt0h2ybFcJshDEjgqU/6fvIDA/A1P0=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=IBEsgckEMhDXHoWJRRuyt2mXhR3JLOEkL2CvKOWG3uf3FuyL3K6fD7txOSYu4cPY4 4OaF1pI43XKEBgr2GYuEb+pqlOsLjySTIRhJ8fln4aODkcI/HSse03W4vS/cqq6YXq ZNF97S/8w/wiJCETimiDaIxxinkro0OArD8v+s/4= Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id BE6716EC57 for ; Fri, 19 Mar 2021 02:04:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org BE6716EC57 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1lN1ga-0007Dv-0k; Fri, 19 Mar 2021 02:04:12 +0300 To: Cyrill Gorcunov , tml Cc: Mons Anderson References: <20210318184138.1077807-1-gorcunov@gmail.com> <20210318184138.1077807-3-gorcunov@gmail.com> Message-ID: <117fc87c-0317-db58-ea7a-df1325a6829d@tarantool.org> Date: Fri, 19 Mar 2021 00:04:11 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <20210318184138.1077807-3-gorcunov@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD96485A7A9FC13189348F8A4374722721B2CC8EA71D516B2F1182A05F5380850409517F01CA0E8D9EE65D153ABE244D16B8BC5E3E51CD7D8A7625BC56C9F4E668B X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7DD4CC05688AD5D0CEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006375121C8F3070B83748638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C81A23F326053F8274F381D7358BC452138CF41DBEE918BF7A471835C12D1D9774AD6D5ED66289B5278DA827A17800CE7328B01A8D746D8839FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C317B107DEF921CE79117882F4460429728AD0CFFFB425014E868A13BD56FB6657A7F4EDE966BC389F9E8FC8737B5C22499DC5D3EDC80DC730089D37D7C0E48F6CCF19DD082D7633A0E7DDDDC251EA7DABAAAE862A0553A39223F8577A6DFFEA7C6492012CB0DFA22D43847C11F186F3C5E7DDDDC251EA7DABCC89B49CDF41148F6B16E133A9586FB82623479134186CDE6BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2BBE337FB72E923155C0AF1600DCBC20BD52826C51A2876AB6EFD3514611DFFDB X-C1DE0DAB: 0D63561A33F958A5FA1BE3383E8F4F9983D59A07399AADCFEEEC1940CFDA71F6D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75F04B387B5D7535DE410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34D8FD11F74BDD6E8D15EA92773B45F334704E6B9DFB292007A1942B2A0012816BC75A1FB8C2D1A19A1D7E09C32AA3244C8B5118F521CA683FFAE3145394C38FC4259227199D06760AFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojyKyJYJ15DtJI3DZijZ5Nww== X-Mailru-Sender: 689FA8AB762F73936BC43F508A06382212B045BF5EDEAE19EBA0E266F7A2F4043841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 2/2] test: add a test for wal_cleanup_delay option X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for the patch! See 8 comments below. On 18.03.2021 19:41, Cyrill Gorcunov via Tarantool-patches wrote: > Part-of #5806 > > Signed-off-by: Cyrill Gorcunov > --- > test/replication/gh-5806-master.lua | 13 + > test/replication/gh-5806-slave.lua | 13 + > test/replication/gh-5806-xlog-cleanup.result | 336 ++++++++++++++++++ > .../replication/gh-5806-xlog-cleanup.test.lua | 148 ++++++++ > 4 files changed, 510 insertions(+) > create mode 100644 test/replication/gh-5806-master.lua > create mode 100644 test/replication/gh-5806-slave.lua > create mode 100644 test/replication/gh-5806-xlog-cleanup.result > create mode 100644 test/replication/gh-5806-xlog-cleanup.test.lua > > diff --git a/test/replication/gh-5806-master.lua b/test/replication/gh-5806-master.lua > new file mode 100644 > index 000000000..0404965d3 > --- /dev/null > +++ b/test/replication/gh-5806-master.lua > @@ -0,0 +1,13 @@ > +#!/usr/bin/env tarantool > + > +require('console').listen(os.getenv('ADMIN')) > + > +function func_xlog_snap(space, value) 1. This fails luacheck (which I don't like having in the tests, but still we must keep it green): Checking test/replication/gh-5806-master.lua 1 warning test/replication/gh-5806-master.lua:5:10: (W111) setting non-standard global variable func_xlog_snap Checking test/replication/gh-5806-slave.lua 1 warning test/replication/gh-5806-slave.lua:5:10: (W111) setting non-standard global variable func_xlog_snap Also why does it have 'func' prefix? It is a function obviously, we don't add 'func' prefix to all functions. > + space:insert(value) > + box.snapshot() > +end > diff --git a/test/replication/gh-5806-xlog-cleanup.result b/test/replication/gh-5806-xlog-cleanup.result > new file mode 100644 > index 000000000..97355a8bf > --- /dev/null > +++ b/test/replication/gh-5806-xlog-cleanup.result > @@ -0,0 +1,336 @@ > +-- test-run result file version 2 > +-- > +-- gh-5806: defer xlog cleanup to keep xlogs until > +-- replicas present in "_cluster" are connected. > +-- Otherwise we are getting XlogGapError since > +-- master might go far forwad from replica and 2. forwad -> forward. > +-- replica won't be able to connect without full > +-- rebootstrap. > +-- > + > +fiber = require('fiber') > + | --- > + | ... > +test_run = require('test_run').new() > + | --- > + | ... > +engine = test_run:get_cfg('engine') > + | --- > + | ... > + > +-- > +-- Case 1. > +-- > +-- First lets make sure we're getting XlogGapError in > +-- case if wal_cleanup_delay is not used. > +-- > + > +test_run:cmd('create server master with script="replication/gh-5806-master.lua"') > + | --- > + | - true > + | ... > +test_run:cmd('start server master with wait=True, wait_load=True') > + | --- > + | - true > + | ... > + > +test_run:switch('master') > + | --- > + | - true > + | ... > +box.schema.user.grant('guest', 'replication') > + | --- > + | ... > + > +-- > +-- Keep small number of snaps to force cleanup > +-- procedure be more intensive. > +box.cfg{checkpoint_count = 1} > + | --- > + | ... > + > +engine = test_run:get_cfg('engine') > + | --- > + | ... > +s = box.schema.space.create('test', {engine = engine}) > + | --- > + | ... > +_ = s:create_index('pk') > + | --- > + | ... > + > +test_run:switch('default') > + | --- > + | - true > + | ... > +test_run:cmd('create server replica with rpl_master=master,\ > + script="replication/gh-5806-slave.lua"') > + | --- > + | - true > + | ... > +test_run:cmd('start server replica with wait=True, wait_load=True') > + | --- > + | - true > + | ... > + > +test_run:switch('replica') > + | --- > + | - true > + | ... > +box.cfg{checkpoint_count = 1} > + | --- > + | ... > +s = box.schema.space.create('testtemp', {temporary = true}) > + | --- > + | ... > +_ = s:create_index('pk') > + | --- > + | ... > +for i=1,2 do func_xlog_snap(box.space.testtemp, {i}) end 3. Honestly, it would look much simpler if it would be just 4 lines with 2 inserts and 2 snapshots. 4. Why do you do rw requests both on the replica and master? And why do you need 2 spaces? > + | --- > + | ... > + > +-- > +-- Stop the replica node and generate > +-- first range of xlogs on the master. > +test_run:switch('default') > + | --- > + | - true > + | ... > +test_run:cmd('stop server replica') > + | --- > + | - true > + | ... > + > +test_run:switch('master') > + | --- > + | - true > + | ... > +for i=1,2 do func_xlog_snap(box.space.test, {i}) end > + | --- > + | ... > + > +-- > +-- Restart the masted and generate the 5. masted -> master. > +-- next range of xlogs. > +test_run:switch('default') > + | --- > + | - true > + | ... > +test_run:cmd('stop server master') > + | --- > + | - true > + | ... > +test_run:cmd('start server master with wait_load=True') 6. Does 'restart server master' command work? > + | --- > + | - true > + | ... > +test_run:switch('master') > + | --- > + | - true > + | ... > +for i=3,4 do func_xlog_snap(box.space.test, {i}) end > + | --- > + | ... > + > +-- > +-- Restart master node and the replica then. 7. Why do you need to restart the master 2 times? > +test_run:switch('default') > + | --- > + | - true > + | ... > +test_run:cmd('stop server master') > + | --- > + | - true > + | ... > +test_run:cmd('start server master with wait_load=True') > + | --- > + | - true > + | ... > +test_run:cmd('start server replica with wait=False, wait_load=False') > + | --- > + | - true > + | ... > + > +-- > +-- Wait error to appear. > +while test_run:grep_log("master", "XlogGapError") == nil do fiber.sleep(0.01) end 8. We have test_run:wait_log().