From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id A335F6EC5F; Wed, 21 Apr 2021 01:31:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A335F6EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618957904; bh=tvKZ+KQx41cGfC77cmQEHKWw4h6AeqTOoAsESG4NHNo=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Uti/rmXOSP3gD1xx5PmyUNg3PMObIhL3iRDVZxqikbmQC25kRVYHYff0dZnuek61j OS0uuVslKJTGCgCyTGR9nT8upDyWerZFLRkN6LrMUsZpD9EIwqAiSDnK/A3pgT4E4m bVAXY/iX5gxw8MYc3hVDQv8aJ3RonwnSjtf6i+8w= Received: from smtp58.i.mail.ru (smtp58.i.mail.ru [217.69.128.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C210E6EC5F for ; Wed, 21 Apr 2021 01:31:03 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C210E6EC5F Received: by smtp58.i.mail.ru with esmtpa (envelope-from ) id 1lYytb-0003tW-4J; Wed, 21 Apr 2021 01:31:03 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <5cbaefa9-078a-b00c-2aec-75cf01f732d4@tarantool.org> <83e7df81-078c-def7-1f73-8810676bf241@tarantool.org> <6e626b42-dddd-5ac0-3e0f-f2b92d3ac8fe@tarantool.org> <35351452-fbd8-926f-886b-8210ccb8f74e@tarantool.org> Message-ID: <6e612f9a-b8bf-07fb-5964-f6cc7c8494c0@tarantool.org> Date: Wed, 21 Apr 2021 00:31:02 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E748094FADAEB10E66ADA4C48BE3C291E66DA182A05F538085040DBACBE9C271EFFFC79D74925AF34CE11F050C9FF7942BE1BFAB03FE5489C1149 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE79173C6E970493712EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637D3219148F76DED538638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B22CCF4F4C331F17C0A186466237762BAB40B83DF1158C6122D2E47CDBA5A96583C09775C1D3CA48CF62968DCAA3E4B45B117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE78A80DFD3A0D2C7BC9FA2833FD35BB23DF004C90652538430302FCEF25BFAB3454AD6D5ED66289B5278DA827A17800CE701A9DF589746230F7B076A6E789B0E97A8DF7F3B2552694A1E7802607F20496D49FD398EE364050FB1593CA6EC85F86DBDB03A3F2A65D472B3661434B16C20AC78D18283394535A9E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B654CE8ED7C2D004275ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F6089093C9A16E5BC824A2A04A2ABAA09D25379311020FFC8D4AD4A55790139E197E6330F74400C8E7770 X-C1DE0DAB: 0D63561A33F958A5C36117ADA6CE026E46B142DD52EFCF320EC1EDE140B3C9EED59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D343D50AEDB859DBAD927A97B769C51BA3810B161037FF60D5984DEF3FB138E8C9E794D70CEC63D03661D7E09C32AA3244C4AA39B2DD5470C666FB826EEFFDA02A1B4DF56057A86259FFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojlPRl29Bx4WGmf8DsgtESLw== X-Mailru-Sender: 504CC1E875BF3E7D9BC0E5172ADA311094DE23CA03102C420E1C581A742A419814ABFD4A8FBEAC7907784C02288277CA03E0582D3806FB6A5317862B1921BA260ED6CFD6382C13A6112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v4 13/12] replication: send accumulated Raft messages after relay start X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for the patch! As discussed in private, there is a way to drop this flag, which I did in a separate commit on top of this one, see below and on the branch sp/gh-5445-election-fixes-review: ==================== [tosquash] Remove flag do_restart_recovery diff --git a/src/box/relay.cc b/src/box/relay.cc index 85f335cd7..ff43c2fc7 100644 --- a/src/box/relay.cc +++ b/src/box/relay.cc @@ -95,7 +95,6 @@ struct relay_raft_msg { struct cmsg_hop route[2]; struct raft_request req; struct vclock vclock; - bool do_restart_recovery; struct relay *relay; }; @@ -433,6 +432,12 @@ relay_final_join(int fd, uint64_t sync, struct vclock *start_vclock, relay_delete(relay); }); + /* + * Save the first vclock as 'received'. Because firstly, it was really + * received. Secondly, recv_vclock is used by recovery restart and must + * always be valid. + */ + vclock_copy(&relay->recv_vclock, start_vclock); relay->r = recovery_new(wal_dir(), false, start_vclock); vclock_copy(&relay->stop_vclock, stop_vclock); @@ -660,13 +665,12 @@ struct relay_is_raft_enabled_msg { }; static void -relay_push_raft_msg(struct relay *relay, bool do_restart_recovery) +relay_push_raft_msg(struct relay *relay) { if (!relay->tx.is_raft_enabled || relay->tx.is_raft_push_sent) return; struct relay_raft_msg *msg = &relay->tx.raft_msgs[relay->tx.raft_ready_msg]; - msg->do_restart_recovery = do_restart_recovery; cpipe_push(&relay->relay_pipe, &msg->base); relay->tx.raft_ready_msg = (relay->tx.raft_ready_msg + 1) % 2; relay->tx.is_raft_push_sent = true; @@ -681,16 +685,9 @@ tx_set_is_raft_enabled(struct cmsg *base) (struct relay_is_raft_enabled_msg *)base; struct relay *relay = msg->relay; relay->tx.is_raft_enabled = msg->value; - /* - * Send saved raft message as soon as relay becomes operational. - * Do not restart recovery upon the message arrival. Recovery is - * positioned at replica_clock initially, i.e. already "restarted" and - * restarting it once again would position it at the oldest xlog - * possible, because relay reader hasn't received replica vclock yet. - */ - if (relay->tx.is_raft_push_pending) { - relay_push_raft_msg(msg->relay, false); - } + /* Send saved raft message as soon as relay becomes operational. */ + if (relay->tx.is_raft_push_pending) + relay_push_raft_msg(msg->relay); } /** Relay thread part of the Raft flag setting, second hop. */ @@ -901,6 +898,12 @@ relay_subscribe(struct replica *replica, int fd, uint64_t sync, }); vclock_copy(&relay->local_vclock_at_subscribe, &replicaset.vclock); + /* + * Save the first vclock as 'received'. Because firstly, it was really + * received. Secondly, recv_vclock is used by recovery restart and must + * always be valid. + */ + vclock_copy(&relay->recv_vclock, replica_clock); relay->r = recovery_new(wal_dir(), false, replica_clock); vclock_copy(&relay->tx.vclock, replica_clock); relay->version_id = replica_version_id; @@ -1003,8 +1006,7 @@ relay_raft_msg_push(struct cmsg *base) * would be ignored again. */ relay_send(msg->relay, &row); - if (msg->req.state == RAFT_STATE_LEADER && - msg->do_restart_recovery) + if (msg->req.state == RAFT_STATE_LEADER) relay_restart_recovery(msg->relay); } catch (Exception *e) { relay_set_error(msg->relay, e); @@ -1018,7 +1020,7 @@ tx_raft_msg_return(struct cmsg *base) struct relay_raft_msg *msg = (struct relay_raft_msg *)base; msg->relay->tx.is_raft_push_sent = false; if (msg->relay->tx.is_raft_push_pending) - relay_push_raft_msg(msg->relay, true); + relay_push_raft_msg(msg->relay); } void @@ -1042,7 +1044,7 @@ relay_push_raft(struct relay *relay, const struct raft_request *req) cmsg_init(&msg->base, msg->route); msg->relay = relay; relay->tx.is_raft_push_pending = true; - relay_push_raft_msg(relay, true); + relay_push_raft_msg(relay); } /** Send a single row to the client. */