From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp36.i.mail.ru (smtp36.i.mail.ru [94.100.177.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 6033D46970E for ; Wed, 25 Dec 2019 15:47:25 +0300 (MSK) From: sergepetrenko Date: Wed, 25 Dec 2019 15:46:57 +0300 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH v2 0/5] introduce anonymous replicas List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: v.shpilevoy@tarantool.org, georgy@tarantool.org Cc: tarantool-patches@dev.tarantool.org https://github.com/tarantool/tarantool/tree/sp/gh-3186-anon-replica https://github.com/tarantool/tarantool/issues/3186 The first patch alters the comment regarding join protocol to be in sync with latest changes: join isn't done off a snapshot, but from a read-view rather, created at the moment the request is received. The second patch removes unnecessary decoding of replicaset uuid on master side, since master just ignores it now. The third patch is a preparation for anonymous replica. We need to split join protocol into two stages: first, fetch a snapshot. This is done by both anonymous and regular replicas. The second stage is registration, i. e. the addition of replica to _cluster table. It is done right away when joining a regular replica, or it can be done by a request from an anonymous replica to transition it to normal one. The patch itself just splits applier join into appropriate stages. The fourth patch modifies vclock logic to ignore 0 component in comparisons. This is also used by anonymous replicas, which will be the first users of the 0-th vclock component. This component will be used to keep track of anonymous replica local changes, which are not replicated. These local changes will vary from instance to instance, and the anonymous instances will all use 0th component to track them, so no need to compare this component. The fifth patch does the main job of adding anonymous replicas. New requests, FETCH_SNAPSHOT and REGISTER are introduced. Anonymous replicas are allowed to have a 0 id. They aren't registered as gc consumers and aren't present in _cluster table. Changes in v2: - review fixes as per reviews from Georgy Kirichenko and Vlad Shpilevoy. Serge Petrenko (5): box: update comment describing join protocol replication: do not decode replicaset uuid when processing a subscribe applier: split join processing into two stages vclock: ignore 0th component in comparisons. replication: introduce anonymous replica. src/box/applier.cc | 124 ++++++++++-- src/box/applier.h | 4 + src/box/box.cc | 276 +++++++++++++++++++++++++-- src/box/box.h | 11 +- src/box/errcode.h | 1 + src/box/iproto.cc | 16 +- src/box/iproto_constants.h | 6 + src/box/lua/cfg.cc | 14 +- src/box/lua/info.c | 4 +- src/box/lua/load_cfg.lua | 4 + src/box/recovery.cc | 7 +- src/box/relay.cc | 28 ++- src/box/replication.cc | 41 +++- src/box/replication.h | 24 +++ src/box/vclock.h | 12 +- src/box/wal.c | 4 + src/box/xrow.c | 49 ++++- src/box/xrow.h | 68 ++++++- test/app-tap/init_script.result | 49 ++--- test/box/admin.result | 2 + test/box/cfg.result | 4 + test/box/misc.result | 1 + test/replication/anon.lua | 13 ++ test/replication/anon.result | 326 ++++++++++++++++++++++++++++++++ test/replication/anon.test.lua | 118 ++++++++++++ test/replication/anon1.lua | 1 + test/replication/anon2.lua | 1 + test/replication/suite.cfg | 1 + test/unit/vclock.cc | 8 +- 29 files changed, 1123 insertions(+), 94 deletions(-) create mode 100644 test/replication/anon.lua create mode 100644 test/replication/anon.result create mode 100644 test/replication/anon.test.lua create mode 120000 test/replication/anon1.lua create mode 120000 test/replication/anon2.lua -- 2.20.1 (Apple Git-117)