From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9D7336EC55; Tue, 27 Jul 2021 18:10:59 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9D7336EC55 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627398659; bh=ilzjMeWjImJp1dGAgiBRn7jT75+4MRVbOa9jsnthzbk=; h=In-Reply-To:Date:References:To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=R9SZD6FN50F1a7MmCwKpAq4NYJtt1Kf7WGXjGvNawoW6t3r2n43TaxWfKcUxkj5hy SkzbPxzilnZfRpKghOcaEIrI26Rk3RQxlg5lnRy0bi7cYPeWruqwrUMpCmzR/1Ke2F 5OORrMcXHpKnR+DQWlrGDAA9Trx/aaLys4OGaXOk= Received: from smtp32.i.mail.ru (smtp32.i.mail.ru [94.100.177.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 818866EC55 for ; Tue, 27 Jul 2021 18:10:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 818866EC55 Received: by smtp32.i.mail.ru with esmtpa (envelope-from ) id 1m8OjQ-00050F-7o; Tue, 27 Jul 2021 18:10:56 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) In-Reply-To: Date: Tue, 27 Jul 2021 18:10:54 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Vladislav Shpilevoy X-Mailer: Apple Mail (2.3654.100.0.2.22) X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD941C43E597735A9C3104FC76DFAAAAF7DA068FE323FAC4379182A05F53808504021141F201F3BD413967748DEEC7A91D1021C197977964EE9BD85B9CC454A1DE0 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE73F64378C139886A6EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006374ECA27954A00B6C3EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F263187F0419173E7DF4EA009833BCC0D3CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F6B57BC7E64490618DEB871D839B7333395957E7521B51C2DFABB839C843B9C08941B15DA834481F8AA50765F7900637D0FEED2715E18529389733CBF5DBD5E9B5C8C57E37DE458B9E9CE733340B9D5F3BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E73542F54486E6D6388DC4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5F76C2F91C84561D7CF134B65FECDB6780969D7C76D0039E2D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75FA7FF33AA1A4D21C410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3452F7993FB7281BFD3D29AAE2CF549D6D1F56683BECB4954B023F1BCB97C3BBF08517797A2758E66A1D7E09C32AA3244C91DFC82C1B4171FC5A96946C5A696C46C3B3ADDA61883BB5927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojMEANdStWW58Umd8wSno7QA== X-Mailru-Sender: 3B9A0136629DC912F4AABCEFC589C81E1C6701593D56EA59F0E505D44C05DC18ADF077640591CC52AD07DD1419AC565FA614486B47F28B67C5E079CCF3B0523AED31B7EB2E253A9E112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 1/1] replication: set replica ID before _cluster commit X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Ostanevich via Tarantool-patches Reply-To: Sergey Ostanevich Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thanks for the patch! Please, see my 3 comments below. > On 25 Jul 2021, at 19:53, Vladislav Shpilevoy via Tarantool-patches = wrote: >=20 > Replica registration works via looking for the smallest not > occupied ID in _cluster and inserting it into the space. >=20 > It works not so good when mvcc is enabled. In particular, if more > than 1 replica try to register at the same time, they might get > the same replica_id because don't see changes of each other until > the registration in _cluster is complete. >=20 > This in the end leads to all replicas failing the registration > except one with the 'duplicate key' error (primary index in > _cluster is replica ID). >=20 > The patch makes the replicas occupy their ID before they commit it > into _cluster. And new replica ID search now uses the replica ID > map instead of _cluster iterator. >=20 > This way the registration works like before - like MVCC does not > exist which is fine. Did you discuss the MVCC capabilities - if we can address it there, by=20= setting a dedicated flag for this particular - and perhaps for some=20 other internal - space(s) to suppress MVCC =E2=80=98coverage=E2=80=99 = for them. The=20 solution will be way more common with supposedly less hassle with=20 local structures, triggers, and so on? >=20 > Part of #5430 > --- > Branch: = http://github.com/tarantool/tarantool/tree/gerold103/gh-5430-cluster-dupli= cate > Issue: https://github.com/tarantool/tarantool/issues/5430 >=20 > .../gh-5430-cluster-mvcc-duplicate.md | 7 + > src/box/alter.cc | 96 ++++++------ > src/box/box.cc | 19 +-- > src/box/replication.cc | 13 ++ > src/box/replication.h | 4 + > test/replication/gh-5430-cluster-mvcc.result | 146 ++++++++++++++++++ > .../replication/gh-5430-cluster-mvcc.test.lua | 62 ++++++++ > test/replication/gh-5430-mvcc-master.lua | 11 ++ > test/replication/gh-5430-mvcc-replica1.lua | 10 ++ > test/replication/gh-5430-mvcc-replica2.lua | 1 + > test/replication/suite.cfg | 1 + > test/replication/suite.ini | 2 +- > 12 files changed, 306 insertions(+), 66 deletions(-) > create mode 100644 = changelogs/unreleased/gh-5430-cluster-mvcc-duplicate.md > create mode 100644 test/replication/gh-5430-cluster-mvcc.result > create mode 100644 test/replication/gh-5430-cluster-mvcc.test.lua > create mode 100644 test/replication/gh-5430-mvcc-master.lua > create mode 100644 test/replication/gh-5430-mvcc-replica1.lua > create mode 120000 test/replication/gh-5430-mvcc-replica2.lua >=20 > diff --git a/changelogs/unreleased/gh-5430-cluster-mvcc-duplicate.md = b/changelogs/unreleased/gh-5430-cluster-mvcc-duplicate.md > new file mode 100644 > index 000000000..59b90f026 > --- /dev/null > +++ b/changelogs/unreleased/gh-5430-cluster-mvcc-duplicate.md > @@ -0,0 +1,7 @@ > +## bugfix/replication > + > +* Fixed a rare error appearing when MVCC = (`box.cfg.memtx_use_mvcc_engine`) was > + enabled and more than one replica was joined to a cluster. The join = could fail > + with the error `"ER_TUPLE_FOUND: Duplicate key exists in unique = index > + 'primary' in space '_cluster'"`. The same could happen at bootstrap = of a > + cluster having >=3D 3 nodes (gh-5430). > diff --git a/src/box/alter.cc b/src/box/alter.cc > index 89bb5946c..64ba09021 100644 > --- a/src/box/alter.cc > +++ b/src/box/alter.cc > @@ -4178,47 +4178,11 @@ on_replace_dd_schema(struct trigger * /* = trigger */, void *event) > return 0; > } >=20 > -/** > - * A record with id of the new instance has been synced to the > - * write ahead log. Update the cluster configuration cache > - * with it. > - */ > -static int > -register_replica(struct trigger *trigger, void * /* event */) > -{ > - struct tuple *new_tuple =3D (struct tuple *)trigger->data; > - uint32_t id; > - if (tuple_field_u32(new_tuple, BOX_CLUSTER_FIELD_ID, &id) !=3D = 0) > - return -1; > - tt_uuid uuid; > - if (tuple_field_uuid(new_tuple, BOX_CLUSTER_FIELD_UUID, &uuid) = !=3D 0) > - return -1; > - struct replica *replica =3D replica_by_uuid(&uuid); > - if (replica !=3D NULL) { > - replica_set_id(replica, id); > - } else { > - try { > - replica =3D replicaset_add(id, &uuid); > - /* Can't throw exceptions from on_commit trigger = */ > - } catch(Exception *e) { > - panic("Can't register replica: %s", e->errmsg); > - } > - } > - return 0; > -} > - > +/** Unregister the replica affected by the change. */ > static int > -unregister_replica(struct trigger *trigger, void * /* event */) > +on_replace_cluster_clear_id(struct trigger *trigger, void * /* event = */) > { > - struct tuple *old_tuple =3D (struct tuple *)trigger->data; > - > - struct tt_uuid old_uuid; > - if (tuple_field_uuid(old_tuple, BOX_CLUSTER_FIELD_UUID, = &old_uuid) !=3D 0) > - return -1; > - > - struct replica *replica =3D replica_by_uuid(&old_uuid); > - assert(replica !=3D NULL); > - replica_clear_id(replica); > + replica_clear_id((struct replica *)trigger->data); > return 0; > } >=20 > @@ -4280,14 +4244,34 @@ on_replace_dd_cluster(struct trigger *trigger, = void *event) > "updates of instance uuid"); > return -1; > } > - } else { > - struct trigger *on_commit; > - on_commit =3D = txn_alter_trigger_new(register_replica, > - new_tuple); > - if (on_commit =3D=3D NULL) > - return -1; > - txn_stmt_on_commit(stmt, on_commit); > + return 0; > + } > + /* > + * With read-views enabled there might be already a = replica > + * whose registration is in progress in another = transaction. > + * With the same replica ID. > + */ > + if (replica_by_id(replica_id) !=3D NULL) { > + diag_set(ClientError, ER_UNSUPPORTED, = "Tarantool", > + "more than 1 replica with the same = ID"); There should be more details on replica ID/UUID/name here, same as in = unregister panic below. > + return -1; > } > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > I couldn't test this check because of the bug in mvcc: > https://github.com/tarantool/tarantool/issues/6246 >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + struct trigger *on_rollback =3D txn_alter_trigger_new( > + on_replace_cluster_clear_id, NULL); > + if (on_rollback =3D=3D NULL) > + return -1; > + /* > + * Register the replica before commit so as to occupy = the > + * replica ID now. While WAL write is in progress, new = replicas > + * might come, they should see the ID is already in use. > + */ > + struct replica *replica =3D = replica_by_uuid(&replica_uuid); > + if (replica !=3D NULL) > + replica_set_id(replica, replica_id); > + else > + replica =3D replicaset_add(replica_id, = &replica_uuid); > + on_rollback->data =3D replica; > + txn_stmt_on_rollback(stmt, on_rollback); > } else { > /* > * Don't allow deletion of the record for this instance > @@ -4300,9 +4284,23 @@ on_replace_dd_cluster(struct trigger *trigger, = void *event) > if (replica_check_id(replica_id) !=3D 0) > return -1; >=20 > - struct trigger *on_commit; > - on_commit =3D txn_alter_trigger_new(unregister_replica, > - old_tuple); > + struct replica *replica =3D replica_by_id(replica_id); > + if (replica =3D=3D NULL) { > + /* > + * Impossible, but it is important not to leave > + * undefined behaviour if there is a bug. Too = sensitive > + * subsystem is affected. > + */ > + panic("Tried to unregister a replica not stored = in " > + "replica_by_id map, id is %u", = replica_id); > + } > + /* > + * Unregister only after commit. Otherwise if the = transaction > + * would be rolled back, there might be already another = replica > + * taken the freed ID. > + */ > + struct trigger *on_commit =3D txn_alter_trigger_new( > + on_replace_cluster_clear_id, replica); > if (on_commit =3D=3D NULL) > return -1; > txn_stmt_on_commit(stmt, on_commit); > diff --git a/src/box/box.cc b/src/box/box.cc > index 8c10a99dd..5c10aceff 100644 > --- a/src/box/box.cc > +++ b/src/box/box.cc > @@ -2407,22 +2407,9 @@ box_on_join(const tt_uuid *instance_uuid) > return; /* nothing to do - already registered */ >=20 > box_check_writable_xc(); > - > - /** Find the largest existing replica id. */ > - struct space *space =3D space_cache_find_xc(BOX_CLUSTER_ID); > - struct index *index =3D index_find_system_xc(space, 0); > - struct iterator *it =3D index_create_iterator_xc(index, = ITER_ALL, > - NULL, 0); > - IteratorGuard iter_guard(it); > - struct tuple *tuple; > - /** Assign a new replica id. */ > - uint32_t replica_id =3D 1; > - while ((tuple =3D iterator_next_xc(it)) !=3D NULL) { > - if (tuple_field_u32_xc(tuple, > - BOX_CLUSTER_FIELD_ID) !=3D = replica_id) > - break; > - replica_id++; > - } > + uint32_t replica_id; > + if (replica_find_new_id(&replica_id) !=3D 0) > + diag_raise(); Any info on why register fails? > box_register_replica(replica_id, instance_uuid); > } >=20 > diff --git a/src/box/replication.cc b/src/box/replication.cc > index 45ad03dfd..1288bc9b1 100644 > --- a/src/box/replication.cc > +++ b/src/box/replication.cc > @@ -1032,3 +1032,16 @@ replica_by_id(uint32_t replica_id) > { > return replicaset.replica_by_id[replica_id]; > } > + > +int > +replica_find_new_id(uint32_t *replica_id) > +{ > + for (uint32_t i =3D 1; i < VCLOCK_MAX; ++i) { > + if (replicaset.replica_by_id[i] =3D=3D NULL) { > + *replica_id =3D i; > + return 0; > + } > + } > + diag_set(ClientError, ER_REPLICA_MAX, VCLOCK_MAX); > + return -1; > +} > diff --git a/src/box/replication.h b/src/box/replication.h > index 57e0f10ae..5d1fa1255 100644 > --- a/src/box/replication.h > +++ b/src/box/replication.h > @@ -360,6 +360,10 @@ replica_by_uuid(const struct tt_uuid *uuid); > struct replica * > replica_by_id(uint32_t replica_id); >=20 > +/** Find the smallest free replica ID in the available range. */ > +int > +replica_find_new_id(uint32_t *replica_id); > + > /** > * Find a node in the replicaset on which the instance can try to = register to > * join the replicaset. > diff --git a/test/replication/gh-5430-cluster-mvcc.result = b/test/replication/gh-5430-cluster-mvcc.result > new file mode 100644 > index 000000000..831689fc2 > --- /dev/null > +++ b/test/replication/gh-5430-cluster-mvcc.result > @@ -0,0 +1,146 @@ > +-- test-run result file version 2 > +-- > +-- gh-5430: when MVCC was enabled for memtx, new replica registration = attempt > +-- could fail with 'duplicate error' in _cluster space. This was = happening, > +-- because _cluster is memtx. Changes to it were not visible for = newer requests > +-- until commit. > +-- New replica ID was looked up in the space by its full scan. The = scan used a > +-- plain iterator and didn't see replicas, whose registration was in = progress of > +-- being written to WAL. > +-- As a result, if 2 replicas came to register at the same time, they = got the > +-- same replica ID because didn't see each other in _cluster. One of = them would > +-- fail to register in the end due to the conflict. > +-- > +-- The test reproduces it by doing anon replica registration. Because = during > +-- normal join there are more than one access to WAL, the ID = assignment is the > +-- last one. It makes hard to block the ID assignment only. With anon = replica > +-- ID assignment the join is already done, the only WAL write is the = ID > +-- assignment, easy to block and yet not block new replicas = registration > +-- attempts. > +-- > +test_run =3D require('test_run').new() > + | --- > + | ... > + > +test_run:cmd('create server master with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-master.lua"') > + | --- > + | - true > + | ... > +test_run:cmd('start server master') > + | --- > + | - true > + | ... > + > +test_run:cmd('create server replica1 with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-replica1.lua"') > + | --- > + | - true > + | ... > +test_run:cmd('start server replica1') > + | --- > + | - true > + | ... > + > +test_run:cmd('create server replica2 with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-replica2.lua"') > + | --- > + | - true > + | ... > +test_run:cmd('start server replica2') > + | --- > + | - true > + | ... > + > +test_run:switch('master') > + | --- > + | - true > + | ... > +box.error.injection.set('ERRINJ_WAL_DELAY', true) > + | --- > + | - ok > + | ... > + > +test_run:switch('replica1') > + | --- > + | - true > + | ... > +_ =3D require('fiber').create(function() box.cfg{replication_anon =3D = false} end) > + | --- > + | ... > +str =3D string.format('registering replica %s', = box.info.uuid):gsub('-', '%%-') > + | --- > + | ... > +_ =3D test_run:wait_log('master', str) > + | --- > + | ... > + > +test_run:switch('replica2') > + | --- > + | - true > + | ... > +_ =3D require('fiber').create(function() box.cfg{replication_anon =3D = false} end) > + | --- > + | ... > +str =3D string.format('registering replica %s', = box.info.uuid):gsub('-', '%%-') > + | --- > + | ... > +_ =3D test_run:wait_log('master', str) > + | --- > + | ... > + > +test_run:switch('master') > + | --- > + | - true > + | ... > +box.error.injection.set('ERRINJ_WAL_DELAY', false) > + | --- > + | - ok > + | ... > + > +test_run:switch('replica1') > + | --- > + | - true > + | ... > +test_run:wait_cond(function() return box.info.id > 1 end) > + | --- > + | - true > + | ... > + > +test_run:switch('replica2') > + | --- > + | - true > + | ... > +test_run:wait_cond(function() return box.info.id > 1 end) > + | --- > + | - true > + | ... > + > +test_run:switch('default') > + | --- > + | - true > + | ... > +test_run:cmd('stop server replica2') > + | --- > + | - true > + | ... > +test_run:cmd('delete server replica2') > + | --- > + | - true > + | ... > +test_run:cmd('stop server replica1') > + | --- > + | - true > + | ... > +test_run:cmd('delete server replica1') > + | --- > + | - true > + | ... > +test_run:cmd('stop server master') > + | --- > + | - true > + | ... > +test_run:cmd('delete server master') > + | --- > + | - true > + | ... > diff --git a/test/replication/gh-5430-cluster-mvcc.test.lua = b/test/replication/gh-5430-cluster-mvcc.test.lua > new file mode 100644 > index 000000000..bb34ba540 > --- /dev/null > +++ b/test/replication/gh-5430-cluster-mvcc.test.lua > @@ -0,0 +1,62 @@ > +-- > +-- gh-5430: when MVCC was enabled for memtx, new replica registration = attempt > +-- could fail with 'duplicate error' in _cluster space. This was = happening, > +-- because _cluster is memtx. Changes to it were not visible for = newer requests > +-- until commit. > +-- New replica ID was looked up in the space by its full scan. The = scan used a > +-- plain iterator and didn't see replicas, whose registration was in = progress of > +-- being written to WAL. > +-- As a result, if 2 replicas came to register at the same time, they = got the > +-- same replica ID because didn't see each other in _cluster. One of = them would > +-- fail to register in the end due to the conflict. > +-- > +-- The test reproduces it by doing anon replica registration. Because = during > +-- normal join there are more than one access to WAL, the ID = assignment is the > +-- last one. It makes hard to block the ID assignment only. With anon = replica > +-- ID assignment the join is already done, the only WAL write is the = ID > +-- assignment, easy to block and yet not block new replicas = registration > +-- attempts. > +-- > +test_run =3D require('test_run').new() > + > +test_run:cmd('create server master with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-master.lua"') > +test_run:cmd('start server master') > + > +test_run:cmd('create server replica1 with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-replica1.lua"') > +test_run:cmd('start server replica1') > + > +test_run:cmd('create server replica2 with '.. = \ > + 'script=3D"replication/gh-5430-mvcc-replica2.lua"') > +test_run:cmd('start server replica2') > + > +test_run:switch('master') > +box.error.injection.set('ERRINJ_WAL_DELAY', true) > + > +test_run:switch('replica1') > +_ =3D require('fiber').create(function() box.cfg{replication_anon =3D = false} end) > +str =3D string.format('registering replica %s', = box.info.uuid):gsub('-', '%%-') > +_ =3D test_run:wait_log('master', str) > + > +test_run:switch('replica2') > +_ =3D require('fiber').create(function() box.cfg{replication_anon =3D = false} end) > +str =3D string.format('registering replica %s', = box.info.uuid):gsub('-', '%%-') > +_ =3D test_run:wait_log('master', str) > + > +test_run:switch('master') > +box.error.injection.set('ERRINJ_WAL_DELAY', false) > + > +test_run:switch('replica1') > +test_run:wait_cond(function() return box.info.id > 1 end) > + > +test_run:switch('replica2') > +test_run:wait_cond(function() return box.info.id > 1 end) > + > +test_run:switch('default') > +test_run:cmd('stop server replica2') > +test_run:cmd('delete server replica2') > +test_run:cmd('stop server replica1') > +test_run:cmd('delete server replica1') > +test_run:cmd('stop server master') > +test_run:cmd('delete server master') > diff --git a/test/replication/gh-5430-mvcc-master.lua = b/test/replication/gh-5430-mvcc-master.lua > new file mode 100644 > index 000000000..6c7d0de3d > --- /dev/null > +++ b/test/replication/gh-5430-mvcc-master.lua > @@ -0,0 +1,11 @@ > +#!/usr/bin/env tarantool > + > +require('console').listen(os.getenv('ADMIN')) > + > +box.cfg({ > + listen =3D 'unix/:./master.sock', > + replication_timeout =3D 0.1, > + memtx_use_mvcc_engine =3D true, > +}) > + > +box.schema.user.grant('guest', 'super') > diff --git a/test/replication/gh-5430-mvcc-replica1.lua = b/test/replication/gh-5430-mvcc-replica1.lua > new file mode 100644 > index 000000000..f1ad7b8a4 > --- /dev/null > +++ b/test/replication/gh-5430-mvcc-replica1.lua > @@ -0,0 +1,10 @@ > +#!/usr/bin/env tarantool > +require('console').listen(os.getenv('ADMIN')) > + > +box.cfg({ > + listen =3D os.getenv("LISTEN"), > + replication =3D 'unix/:./master.sock', > + replication_timeout =3D 0.1, > + read_only =3D true, > + replication_anon =3D true, > +}) > diff --git a/test/replication/gh-5430-mvcc-replica2.lua = b/test/replication/gh-5430-mvcc-replica2.lua > new file mode 120000 > index 000000000..28c968a8a > --- /dev/null > +++ b/test/replication/gh-5430-mvcc-replica2.lua > @@ -0,0 +1 @@ > +gh-5430-mvcc-replica1.lua > \ No newline at end of file > diff --git a/test/replication/suite.cfg b/test/replication/suite.cfg > index e0bbe2676..0acc66816 100644 > --- a/test/replication/suite.cfg > +++ b/test/replication/suite.cfg > @@ -18,6 +18,7 @@ > "gh-5213-qsync-applier-order.test.lua": {}, > "gh-5213-qsync-applier-order-3.test.lua": {}, > "gh-5426-election-on-off.test.lua": {}, > + "gh-5430-cluster-mvcc.test.lua": {}, > "gh-5433-election-restart-recovery.test.lua": {}, > "gh-5445-leader-inconsistency.test.lua": {}, > "gh-5506-election-on-off.test.lua": {}, > diff --git a/test/replication/suite.ini b/test/replication/suite.ini > index 18981996d..907fd0be9 100644 > --- a/test/replication/suite.ini > +++ b/test/replication/suite.ini > @@ -3,7 +3,7 @@ core =3D tarantool > script =3D master.lua > description =3D tarantool/box, replication > disabled =3D consistent.test.lua > -release_disabled =3D catch.test.lua errinj.test.lua gc.test.lua = gc_no_space.test.lua before_replace.test.lua qsync_advanced.test.lua = qsync_errinj.test.lua quorum.test.lua recover_missing_xlog.test.lua = sync.test.lua long_row_timeout.test.lua gh-4739-vclock-assert.test.lua = gh-4730-applier-rollback.test.lua gh-5140-qsync-casc-rollback.test.lua = gh-5144-qsync-dup-confirm.test.lua gh-5167-qsync-rollback-snap.test.lua = gh-5506-election-on-off.test.lua gh-5536-wal-limit.test.lua = hang_on_synchro_fail.test.lua anon_register_gap.test.lua = gh-5213-qsync-applier-order.test.lua = gh-5213-qsync-applier-order-3.test.lua = gh-6027-applier-error-show.test.lua gh-6032-promote-wal-write.test.lua = gh-6057-qsync-confirm-async-no-wal.test.lua = gh-5447-downstream-lag.test.lua gh-4040-invalid-msgpack.test.lua > +release_disabled =3D catch.test.lua errinj.test.lua gc.test.lua = gc_no_space.test.lua before_replace.test.lua qsync_advanced.test.lua = qsync_errinj.test.lua quorum.test.lua recover_missing_xlog.test.lua = sync.test.lua long_row_timeout.test.lua gh-4739-vclock-assert.test.lua = gh-4730-applier-rollback.test.lua gh-5140-qsync-casc-rollback.test.lua = gh-5144-qsync-dup-confirm.test.lua gh-5167-qsync-rollback-snap.test.lua = gh-5430-cluster-mvcc.test.lua gh-5506-election-on-off.test.lua = gh-5536-wal-limit.test.lua hang_on_synchro_fail.test.lua = anon_register_gap.test.lua gh-5213-qsync-applier-order.test.lua = gh-5213-qsync-applier-order-3.test.lua = gh-6027-applier-error-show.test.lua gh-6032-promote-wal-write.test.lua = gh-6057-qsync-confirm-async-no-wal.test.lua = gh-5447-downstream-lag.test.lua gh-4040-invalid-msgpack.test.lua > config =3D suite.cfg > lua_libs =3D lua/fast_replica.lua lua/rlimit.lua > use_unix_sockets =3D True > --=20 > 2.24.3 (Apple Git-128) >=20