From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 62A1B6EC5F; Sun, 18 Apr 2021 19:03:17 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 62A1B6EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618761797; bh=EnD1cGsySdExdzxDCGQzEuWLiQmkgLzoGpKLlIV0xqc=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=cR9gv8d0rmuvmbqLKA4DqIRERcm5ZaOMr52czu7jl96lFLJ12uLioMQztU458dsEM f18a+QNHMqbQtt+XuN/4qEs2mgAOjUz8NUK2bESIyZUeACz0XY6abpjXIntZzTAZu0 E/fBppiLubX5ML2AbihuMh0nKif2vIFjVLBXsumA= Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 4B0FC6EC5F for ; Sun, 18 Apr 2021 19:03:16 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4B0FC6EC5F Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1lY9tD-0002HB-LI; Sun, 18 Apr 2021 19:03:16 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <5cbaefa9-078a-b00c-2aec-75cf01f732d4@tarantool.org> Message-ID: <83e7df81-078c-def7-1f73-8810676bf241@tarantool.org> Date: Sun, 18 Apr 2021 18:03:14 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.1 MIME-Version: 1.0 In-Reply-To: <5cbaefa9-078a-b00c-2aec-75cf01f732d4@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480257C85EA0BB7A95D0F00AE41BB9A5343182A05F538085040F5E9F78CAAB38A4FC9D06BA0C32452263104FF20E607C6505770ED3C49CFFF8E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7BA0D57D3459E5640EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006373CF509324F67ED188638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B246B59E324D74A787D8B61180E2287F1AE8578C74B6F99271D2E47CDBA5A96583C09775C1D3CA48CF27ED053E960B195E117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE764603B5C71CE8B8F9FA2833FD35BB23DF004C906525384302BEBFE083D3B9BA73A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E73542F54486E6D6388DC4224003CC83647689D4C264860C145E X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C97831CF8A65526205C84EBC778C3F84A8D0C8 X-C1DE0DAB: 0D63561A33F958A5B01A7A16ED4691869550BED87630D1D043E1E8EE0676B2EFD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34F0A5F58274334C95947811C0B0199C1730148F6776993E52CB08EE038EE61956A374DAFB99E082951D7E09C32AA3244C6B4F00A770B4686512E0E3F00A1680F681560E2432555DBBFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj1t4H7vLuVFXi7IqBH4kJAQ== X-Mailru-Sender: 689FA8AB762F73936BC43F508A0638224F75D1D0FB8348EC21486348FADDB2093841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v4 13/12] replication: send accumulated Raft messages after relay start X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Good job on the patch! See 4 comments below. > diff --git a/src/box/relay.cc b/src/box/relay.cc > index 7be33ee31..9fdd02bc1 100644 > --- a/src/box/relay.cc > +++ b/src/box/relay.cc > @@ -160,6 +160,16 @@ struct relay { >           * anonymous replica, for example. >           */ >          bool is_raft_enabled; > +        /** Is set to true by the first Raft broadcast which comes while 1. Should be a new line after /**. > +         * the relay is not yet ready to dispatch Raft messages. > +         */ > +        bool has_pending_broadcast; > +        /** > +         * A Raft broadcast which should be pushed once relay notifies > +         * tx it needs Raft updates. Otherwise this message would be > +         * lost until some new Raft event happens. > +         */ > +        struct raft_request pending_broadcast; 2. I wouldn't call them 'broadcasts'. Relay sends a single message to the remote node, not to all the nodes. This is a broadcast on the raft level. On relay level it is just a single message to one node. >      } tx; >  }; > @@ -635,14 +649,28 @@ tx_set_is_raft_enabled(struct cmsg *base) >      struct relay_is_raft_enabled_msg *msg = >          (struct relay_is_raft_enabled_msg *)base; >      msg->relay->tx.is_raft_enabled = msg->value; > +    if (msg->relay->tx.has_pending_broadcast) { > +        msg->has_pending_broadcast = true; > +        msg->req = msg->relay->tx.pending_broadcast; 3. Since you will deliver the broadcast now, it is not pending anymore. Hence there must be msg->relay->tx.has_pending_broadcast = false in the end. > +    } >  } > @@ -964,12 +1008,15 @@ void >  relay_push_raft(struct relay *relay, const struct raft_request *req) >  { >      /* > -     * Raft updates don't stack. They are thrown away if can't be pushed > -     * now. This is fine, as long as relay's live much longer that the > -     * timeouts in Raft are set. > +     * Remember the latest Raft update. It might be a notification that > +     * this node is a leader. If sometime later we find out this node needs > +     * Raft updates, we need to send is_leader notification. >       */ > -    if (!relay->tx.is_raft_enabled) > +    if (!relay->tx.is_raft_enabled) { > +        relay->tx.has_pending_broadcast = true; > +        relay->tx.pending_broadcast = *req; 4. Vclock memory does not belong to the request. This is why below we copy it into the message's memory. You might need to do the same here. >          return; > +    } >      /* >       * XXX: the message should be preallocated. It should >       * work like Kharon in IProto. Relay should have 2 raft We could also fix it like described in this XXX, could we?