From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6383D6EC5E; Mon, 12 Apr 2021 22:44:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6383D6EC5E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618256698; bh=Dn/3+67plSQZ8KA4aEm5EdqTCoy/3noPekBOG4FuspU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=v6yBA+E9oy9k9lMmNoD4HkauoZD23dXIgwcd/l9QLO+2p1MrSPSF5CX1H0shbjqc6 NZgncR5ZsXSoYgNpAgAD6bzzutGcm2QmKJf/R/ipeeUvjXaxueHbg9lBYCnZ7QHb6Y y15AjGze/pbqODQ4D4TwdrMvDadsfuBxYn4ezKBc= Received: from smtp53.i.mail.ru (smtp53.i.mail.ru [94.100.177.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2183D6BD24 for ; Mon, 12 Apr 2021 22:40:34 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2183D6BD24 Received: by smtp53.i.mail.ru with esmtpa (envelope-from ) id 1lW2QD-0008Az-5A; Mon, 12 Apr 2021 22:40:33 +0300 To: v.shpilevoy@tarantool.org, gorcunov@gmail.com Date: Mon, 12 Apr 2021 22:40:22 +0300 Message-Id: <36cb842c6a98cbe7a0cd1f13ad0d2a36152a34a4.1618256019.git.sergepetrenko@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480257C85EA0BB7A95D5E28B957962BB550182A05F5380850403F8E2BEFF1D69536B0E92E2D6B68A991C4FB7D09FFBE354FD5979E80576F7CF5 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B65E04026579D649EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006372CAA106849E7D531EA1F7E6F0F101C67CDEEF6D7F21E0D1D9295C2E9FA3191EE1B59CA4C82EFA6589DEEF070CF1B13B860E37DD25B352920F6B57BC7E64490618DEB871D839B73339E8FC8737B5C22498424CA1AAF98A6958941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B68424CA1AAF98A6958941B15DA834481F9449624AB7ADAF37BA3038C0950A5D3613377AFFFEAFD2690E30A4C9C8E338DA8A17506121500BB37B076A6E789B0E97A8DF7F3B2552694A1E7802607F20496D49FD398EE364050FCCD848CCB6FE560C3DBBCB839D0549ACB3661434B16C20AC78D18283394535A9E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B66F1C5D350F9AE87C089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A24209795067102C07E8F7B195E1C97831E4A7F6021002B3293657BD1E38EB9DB5 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C4C7A0BC55FA0FE5FCE4F16467A2949FE96CAF9740096D857F5F3AF040193CEDE3B1881A6453793CE9C32612AADDFBE061C61BE10805914D3804EBA3D8E7E5B87ABF8C51168CD8EBDB63AF70AF8205D7DCDC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34AC6E62257D6CD1C95613DC4C659265FC106EC87178839A47E475A4B55077139D671AD97E2AEEB8451D7E09C32AA3244C4F6AF6A8D2CE0698047E0E355BEB6C7BA90944CA99CF22E3927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojq8JA+pXcDuldnJdstxwcIA== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F8AFCB052CDF231E7BD80636EC06F88700D7AFE68DAC5C114C424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v2 9/9] box.ctl: rename clear_synchro_queue to promote X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" New function name will be `box.ctl.promote()`. It's much shorter and closer to the function's now enriched functionality. Old name `box.ctl.clear_synchro_queue()` remains in Lua for the sake of backward compatibility. Follow-up #5445 Closes #3055 @TarantoolBot document Title: deprecate `box.ctl.clear_synchro_queue()` in favor of `box.ctl.promote()` Replace all the mentions of `box.ctl.clear_synchro_queue()` with `box.ctl.promote()` and add a note that `box.ctl.clear_synchro_queue()` is a deprecated alias to `box.ctl.promote()` --- changelogs/unreleased/box-ctl-promote.md | 8 ++ src/box/box.cc | 20 ++-- src/box/box.h | 2 +- src/box/lua/ctl.c | 8 +- src/box/raft.c | 4 +- test/replication/election_basic.result | 25 +++++ test/replication/election_basic.test.lua | 10 ++ .../gh-3055-election-promote.result | 105 ++++++++++++++++++ .../gh-3055-election-promote.test.lua | 43 +++++++ test/replication/suite.cfg | 1 + 10 files changed, 210 insertions(+), 16 deletions(-) create mode 100644 changelogs/unreleased/box-ctl-promote.md create mode 100644 test/replication/gh-3055-election-promote.result create mode 100644 test/replication/gh-3055-election-promote.test.lua diff --git a/changelogs/unreleased/box-ctl-promote.md b/changelogs/unreleased/box-ctl-promote.md new file mode 100644 index 000000000..15f6fb206 --- /dev/null +++ b/changelogs/unreleased/box-ctl-promote.md @@ -0,0 +1,8 @@ +## feature/replication + +* Introduce `box.ctl.promote()` and the concept of manual elections (enabled + with `election_mode='manual'`). Once the instance is in `manual` election + mode, it acts like a `voter` most of the time, but may trigger elections and + become a leader, once `box.ctl.promote()` is called. + When `election_mode ~= 'manual'`, `box.ctl.promote()` replaces + `box.ctl.clear_synchro_queue()`, which is now deprecated (gh-3055). diff --git a/src/box/box.cc b/src/box/box.cc index dc7f434e4..ca5d2dd50 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -1509,12 +1509,12 @@ box_wait_quorum(uint32_t lead_id, int64_t target_lsn, int quorum, } int -box_clear_synchro_queue(bool try_wait) +box_promote(bool try_wait) { /* A guard to block multiple simultaneous function invocations. */ - static bool in_clear_synchro_queue = false; - if (in_clear_synchro_queue) { - diag_set(ClientError, ER_UNSUPPORTED, "clear_synchro_queue", + static bool in_promote = false; + if (in_promote) { + diag_set(ClientError, ER_UNSUPPORTED, "promote", "simultaneous invocations"); return -1; } @@ -1569,7 +1569,7 @@ box_clear_synchro_queue(bool try_wait) int64_t wait_lsn = txn_limbo.confirmed_lsn; int rc = 0; int quorum = replication_synchro_quorum; - in_clear_synchro_queue = true; + in_promote = true; if (run_elections) { /* @@ -1586,13 +1586,13 @@ box_clear_synchro_queue(bool try_wait) raft_cfg_is_candidate(box_raft(), false, false); if (!box_raft()->is_enabled) { diag_set(ClientError, ER_RAFT_DISABLED); - in_clear_synchro_queue = false; + in_promote = false; return -1; } if (box_raft()->state != RAFT_STATE_LEADER) { diag_set(ClientError, ER_INTERFERING_PROMOTE, box_raft()->leader); - in_clear_synchro_queue = false; + in_promote = false; return -1; } } @@ -1616,13 +1616,13 @@ box_clear_synchro_queue(bool try_wait) if (former_leader_id != txn_limbo.owner_id) { diag_set(ClientError, ER_INTERFERING_PROMOTE, txn_limbo.owner_id); - in_clear_synchro_queue = false; + in_promote = false; return -1; } } /* - * clear_synchro_queue() is a no-op on the limbo owner, so all the rows + * promote() is a no-op on the limbo owner, so all the rows * in the limbo must've come through the applier meaning they already * have an lsn assigned, even if their WAL write hasn't finished yet. */ @@ -1659,7 +1659,7 @@ promote: req.term); } } - in_clear_synchro_queue = false; + in_promote = false; return rc; } diff --git a/src/box/box.h b/src/box/box.h index e2321b9b0..89c6fe1a1 100644 --- a/src/box/box.h +++ b/src/box/box.h @@ -274,7 +274,7 @@ extern "C" { typedef struct tuple box_tuple_t; int -box_clear_synchro_queue(bool try_wait); +box_promote(bool try_wait); /* box_select is private and used only by FFI */ API_EXPORT int diff --git a/src/box/lua/ctl.c b/src/box/lua/ctl.c index d039a059f..f06af8588 100644 --- a/src/box/lua/ctl.c +++ b/src/box/lua/ctl.c @@ -82,9 +82,9 @@ lbox_ctl_on_schema_init(struct lua_State *L) } static int -lbox_ctl_clear_synchro_queue(struct lua_State *L) +lbox_ctl_promote(struct lua_State *L) { - if (box_clear_synchro_queue(true) != 0) + if (box_promote(true) != 0) return luaT_error(L); return 0; } @@ -124,7 +124,9 @@ static const struct luaL_Reg lbox_ctl_lib[] = { {"wait_rw", lbox_ctl_wait_rw}, {"on_shutdown", lbox_ctl_on_shutdown}, {"on_schema_init", lbox_ctl_on_schema_init}, - {"clear_synchro_queue", lbox_ctl_clear_synchro_queue}, + {"promote", lbox_ctl_promote}, + /* An old alias. */ + {"clear_synchro_queue", lbox_ctl_promote}, {"is_recovery_finished", lbox_ctl_is_recovery_finished}, {"set_on_shutdown_timeout", lbox_ctl_set_on_shutdown_timeout}, {NULL, NULL} diff --git a/src/box/raft.c b/src/box/raft.c index 47d4fd56d..45baf5dd8 100644 --- a/src/box/raft.c +++ b/src/box/raft.c @@ -91,7 +91,7 @@ box_raft_update_synchro_queue(struct raft *raft) * If the node became a leader, it means it will ignore all records from * all the other nodes, and won't get late CONFIRM messages anyway. Can * clear the queue without waiting for confirmations. - * It's alright that the user may have called clear_synchro_queue + * It's alright that the user may have called promote * manually. In this case the call below will exit immediately and we'll * simply log a warning. */ @@ -100,7 +100,7 @@ box_raft_update_synchro_queue(struct raft *raft) int rc = 0; uint32_t errcode = 0; do { - rc = box_clear_synchro_queue(false); + rc = box_promote(false); if (rc != 0) { struct error *err = diag_last_error(diag_get()); errcode = box_error_code(err); diff --git a/test/replication/election_basic.result b/test/replication/election_basic.result index d5320b3ff..78c911245 100644 --- a/test/replication/election_basic.result +++ b/test/replication/election_basic.result @@ -108,6 +108,31 @@ assert(box.info.election.leader == box.info.id) | - true | ... +-- Manual election mode. A voter most of the time, a leader once +-- `box.ctl.promote()` is called. +box.cfg{election_mode = 'manual'} + | --- + | ... + +assert(box.info.election.state == 'follower') + | --- + | - true + | ... +term = box.info.election.term + | --- + | ... +box.ctl.promote() + | --- + | ... +assert(box.info.election.state == 'leader') + | --- + | - error: assertion failed! + | ... +assert(box.info.election.term > term) + | --- + | - error: assertion failed! + | ... + box.cfg{ \ election_mode = 'off', \ election_timeout = old_election_timeout \ diff --git a/test/replication/election_basic.test.lua b/test/replication/election_basic.test.lua index 821f73cea..5fc398848 100644 --- a/test/replication/election_basic.test.lua +++ b/test/replication/election_basic.test.lua @@ -39,6 +39,16 @@ assert(box.info.election.term > term) assert(box.info.election.vote == box.info.id) assert(box.info.election.leader == box.info.id) +-- Manual election mode. A voter most of the time, a leader once +-- `box.ctl.promote()` is called. +box.cfg{election_mode = 'manual'} + +assert(box.info.election.state == 'follower') +term = box.info.election.term +box.ctl.promote() +assert(box.info.election.state == 'leader') +assert(box.info.election.term > term) + box.cfg{ \ election_mode = 'off', \ election_timeout = old_election_timeout \ diff --git a/test/replication/gh-3055-election-promote.result b/test/replication/gh-3055-election-promote.result new file mode 100644 index 000000000..6f5af13bc --- /dev/null +++ b/test/replication/gh-3055-election-promote.result @@ -0,0 +1,105 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... + +-- +-- gh-3055 box.ctl.promote(). Call on instance with election_mode='manual' +-- in order to promote it to leader. +SERVERS = {'election_replica1', 'election_replica2', 'election_replica3'} + | --- + | ... +-- Start in candidate state in order for bootstrap to work. +test_run:create_cluster(SERVERS, 'replication', {args='2 0.1 candidate'}) + | --- + | ... +test_run:wait_fullmesh(SERVERS) + | --- + | ... + +cfg_set_manual =\ + "box.cfg{election_mode='manual'} "..\ + "assert(box.info.election.state == 'follower') "..\ + "assert(box.info.ro)" + | --- + | ... + +for _, server in pairs(SERVERS) do\ + ok, res = test_run:eval(server, cfg_set_manual)\ + assert(ok)\ +end + | --- + | ... + +-- Promote without living leader. +test_run:switch('election_replica1') + | --- + | - true + | ... +assert(box.info.election.state == 'follower') + | --- + | - true + | ... +term = box.info.election.term + | --- + | ... +box.ctl.promote() + | --- + | ... +assert(box.info.election.state == 'leader') + | --- + | - true + | ... +assert(not box.info.ro) + | --- + | - true + | ... +assert(box.info.election.term > term) + | --- + | - true + | ... + +-- Test promote when there's a live leader. +test_run:switch('election_replica2') + | --- + | - true + | ... +term = box.info.election.term + | --- + | ... +assert(box.info.election.state == 'follower') + | --- + | - true + | ... +assert(box.info.ro) + | --- + | - true + | ... +assert(box.info.election.leader ~= 0) + | --- + | - true + | ... +box.ctl.promote() + | --- + | ... +assert(box.info.election.state == 'leader') + | --- + | - true + | ... +assert(not box.info.ro) + | --- + | - true + | ... +assert(box.info.election.term > term) + | --- + | - true + | ... + +-- Cleanup. +test_run:switch('default') + | --- + | - true + | ... +test_run:drop_cluster(SERVERS) + | --- + | ... diff --git a/test/replication/gh-3055-election-promote.test.lua b/test/replication/gh-3055-election-promote.test.lua new file mode 100644 index 000000000..cbc3ed206 --- /dev/null +++ b/test/replication/gh-3055-election-promote.test.lua @@ -0,0 +1,43 @@ +test_run = require('test_run').new() + +-- +-- gh-3055 box.ctl.promote(). Call on instance with election_mode='manual' +-- in order to promote it to leader. +SERVERS = {'election_replica1', 'election_replica2', 'election_replica3'} +-- Start in candidate state in order for bootstrap to work. +test_run:create_cluster(SERVERS, 'replication', {args='2 0.1 candidate'}) +test_run:wait_fullmesh(SERVERS) + +cfg_set_manual =\ + "box.cfg{election_mode='manual'} "..\ + "assert(box.info.election.state == 'follower') "..\ + "assert(box.info.ro)" + +for _, server in pairs(SERVERS) do\ + ok, res = test_run:eval(server, cfg_set_manual)\ + assert(ok)\ +end + +-- Promote without living leader. +test_run:switch('election_replica1') +assert(box.info.election.state == 'follower') +term = box.info.election.term +box.ctl.promote() +assert(box.info.election.state == 'leader') +assert(not box.info.ro) +assert(box.info.election.term > term) + +-- Test promote when there's a live leader. +test_run:switch('election_replica2') +term = box.info.election.term +assert(box.info.election.state == 'follower') +assert(box.info.ro) +assert(box.info.election.leader ~= 0) +box.ctl.promote() +assert(box.info.election.state == 'leader') +assert(not box.info.ro) +assert(box.info.election.term > term) + +-- Cleanup. +test_run:switch('default') +test_run:drop_cluster(SERVERS) diff --git a/test/replication/suite.cfg b/test/replication/suite.cfg index 8ae2fc14d..00118e9f6 100644 --- a/test/replication/suite.cfg +++ b/test/replication/suite.cfg @@ -2,6 +2,7 @@ "anon.test.lua": {}, "anon_register_gap.test.lua": {}, "gh-2991-misc-asserts-on-update.test.lua": {}, + "gh-3055-election-promote.test.lua": {}, "gh-3111-misc-rebootstrap-from-ro-master.test.lua": {}, "gh-3160-misc-heartbeats-on-master-changes.test.lua": {}, "gh-3247-misc-iproto-sequence-value-not-replicated.test.lua": {}, -- 2.24.3 (Apple Git-128)