From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 35DF66EC40; Sun, 4 Jul 2021 15:14:36 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 35DF66EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1625400876; bh=nC97gMiyAHpCwfUOt8CbCsWDmpf21a+LUExPk0d5f2A=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=fxGXb/pbp3taYPw8/3fE0qgojpNIif7SL/vKGJj8aS5R4y3aq6YBP9gpdp0BfwBc/ tFZdnFFUzAKJUqc9J+LAc2CS2c2Z7L1j3TzFKHnCV+zN9h5sXu6BnW5mnTMPk6wM9d 6laj2iQ85GWQUZPJnw6z9rrWEkdm3lOSNP4cYG5Q= Received: from smtpng3.i.mail.ru (smtpng3.i.mail.ru [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C1ED96EC40 for ; Sun, 4 Jul 2021 15:14:34 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C1ED96EC40 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1m0118-0006cB-2r; Sun, 04 Jul 2021 15:14:34 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <46c167f8c9bfe0e35370b0cbf10505d9d4a888ba.1624918077.git.sergepetrenko@tarantool.org> Message-ID: Date: Sun, 4 Jul 2021 14:14:33 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <46c167f8c9bfe0e35370b0cbf10505d9d4a888ba.1624918077.git.sergepetrenko@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD954DFF1DC42D673FB703477AD6D36A6E3513B6A65BE508F5F182A05F53808504027DB3AA6C7B646CB18E376B0EFAB76B878C3FE11E6EEDB46592F7A5FDA7AEEBA X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7FFA0C4A2B39D8DD8EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637205505A8D8EF484BEA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F231CC83F455ED73A659E32E0EB0773F82CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F3E38EE449E3E2AE389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8744B801E316CB65FF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B851EDB9C5A93305EEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C97831C9A19D3EF7673A91FA0551508B2C535B X-C1DE0DAB: 0D63561A33F958A53DB04C08EAEF65E542681346F0AA9D25D36D3A32A286BB55D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7569E77FCA7B33833F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3463DBE2ADA183F62FE8C20F030B4DD62192E44238D2F898DA198074C439C97C58B0287C312901E8741D7E09C32AA3244C4062C9D5B5CD4CC0EA113FE48FE408BA3A76366E8A9DE7CAFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj5fH2RN9TpJk75XGnaVhHOA== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D58FCC3E0DE31159C25F92FCECC4CAAA43841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v3 04/12] box: make promote always bump the term X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for the patch! Did you think about making it NOP when the node is already a leader (even in manual/off mode)? The current solution is all good except that it makes the current leader temporary read-only until it wins the election again, which looks strange. I would say "unexpected" for users. See 4 comments below. > diff --git a/src/box/box.cc b/src/box/box.cc > index 6a0950f44..ce37b307d 100644 > --- a/src/box/box.cc > +++ b/src/box/box.cc > @@ -1687,16 +1687,19 @@ box_promote(void) > rc = -1; > } else { > promote: > - /* We cannot possibly get here in a volatile state. */ > - assert(box_raft()->volatile_term == box_raft()->term); > - txn_limbo_write_promote(&txn_limbo, wait_lsn, > - box_raft()->term); > + if (try_wait) { > + raft_new_term(box_raft()); > + if (box_raft_wait_persisted() < 0) 1. What if during term WAL write another node also started promote, won the elections and delivered the promote to us? I suppose after the WAL write we will silently write PROMOTE for the term which was won by somebody else, right? Can it be covered by a test? > + return -1; > + } > + uint64_t term = box_raft()->term; > + txn_limbo_write_promote(&txn_limbo, wait_lsn, term); > struct synchro_request req = { > .type = IPROTO_PROMOTE, > .replica_id = former_leader_id, > .origin_id = instance_id, > .lsn = wait_lsn, > - .term = box_raft()->term, > + .term = term, > }; > txn_limbo_process(&txn_limbo, &req); > assert(txn_limbo_is_empty(&txn_limbo)); > diff --git a/src/box/raft.c b/src/box/raft.c > index 7f787c0c5..17caf6f54 100644 > --- a/src/box/raft.c > +++ b/src/box/raft.c > @@ -354,6 +354,42 @@ box_raft_wait_leader_found(void) > return 0; > } > > +struct raft_wait_persisted_data { > + struct fiber *waiter; > + uint64_t term; > +}; > + > +static int > +box_raft_wait_persisted_f(struct trigger *trig, void *event) > +{ > + struct raft *raft = event; > + struct raft_wait_persisted_data *data = trig->data; > + if (raft->term >= data->term) > + fiber_wakeup(data->waiter); > + return 0; > +} > + > +int > +box_raft_wait_persisted(void) > +{ > + if (box_raft()->term == box_raft()->volatile_term) 2. Since it only waits for term being persisted, I would rather call it 'wait_term_persisted'. Because there is also vote, and you do not look at it. > + return 0; > + struct raft_wait_persisted_data data = { > + .waiter = fiber(), > + .term = box_raft()->volatile_term, > + }; > + struct trigger trig; > + trigger_create(&trig, box_raft_wait_persisted_f, &data, NULL); > + raft_on_update(box_raft(), &trig); > + fiber_yield(); 3. What about spurious wakeups? I could call fiber.wakeup() from Lua on this fiber. > + trigger_clear(&trig); > + if (fiber_is_cancelled()) { > + diag_set(FiberIsCancelled); > + return -1; > + } > + return 0; > +} > diff --git a/test/replication/gh-4114-local-space-replication.result b/test/replication/gh-4114-local-space-replication.result > index 9b63a4b99..e71eb60a8 100644 > --- a/test/replication/gh-4114-local-space-replication.result > +++ b/test/replication/gh-4114-local-space-replication.result > @@ -45,9 +45,8 @@ test_run:cmd('switch replica') > | --- > | - true > | ... > -box.info.vclock[0] > +a = box.info.vclock[0] or 0 > | --- > - | - null > | ... > box.cfg{checkpoint_count=1} > | --- > @@ -77,9 +76,9 @@ box.space.test:insert{3} > | - [3] > | ... > > -box.info.vclock[0] > +assert(box.info.vclock[0] == a + 3) > | --- > - | - 3 > + | - true 4. Why do you need these changes? I reverted this test and it passed.