From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6444C6E200; Sat, 19 Jun 2021 01:52:17 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6444C6E200 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1624056737; bh=eeP0fJxCsQ/yU3YJM1Ef1eeLLVt8OCnQY306cAYEIXg=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=OUp7+iVRGU5TDiQ6otar54WD/8Xsa66LnNolUVBHNwhwYEnTrQZuhPwqLB+HdRQW2 r+VOTc4hqqWh5jlthkQoj+fe9pKYbGJxWmyBPCg/ZAWmYGuJnL/bF7MQ3gHJWp5m/Q XNaj3VAMCOJnMSEiHmg6LrPi7wbtHi8hQh9Gqf3A= Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C657F6E200 for ; Sat, 19 Jun 2021 01:52:15 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C657F6E200 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1luNLS-00011G-VF; Sat, 19 Jun 2021 01:52:15 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <93a4765cb6918512946bf9366fe4ad478345bce1.1623331925.git.sergepetrenko@tarantool.org> <76d19171-2257-35e3-8f07-b1f0323c728f@tarantool.org> <3a61c787-8a92-0191-1565-118915adbfcc@tarantool.org> Message-ID: <12dc56c1-c6b6-154d-1e19-99c621eaea47@tarantool.org> Date: Sat, 19 Jun 2021 00:52:13 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <3a61c787-8a92-0191-1565-118915adbfcc@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD91C2C07775F13263A612E9F961DCF5576EBB1C736C2A4021700894C459B0CD1B9C61D22776BB856182401288527755CF01423CBEC9FF1E7EF70ABB576495EA039 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7ACA11F7F2395C8CCEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063795DC1BB912715AE68638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8B45F21314050F5370E0A9067F418E4D6117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC974A882099E279BDA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735201E561CDFBCA1751F618001F51B5FD3F9D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B65D56369A3576CBA5089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C978311E0918C89E3BCC34EEF3C316914408DE X-C1DE0DAB: 0D63561A33F958A5D7D9E5C2AFE17F2DA6DECF924EE67A89EE3DE6CAB28B4DECD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75448CF9D3A7B2C848410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340A4C04F5DECA7EE92B3BE6F490D16FC7870E663F2DD718CF9F67CF1A3BF3848EAD5146FF738F2E031D7E09C32AA3244CFDE16A961A975036E0DC0AF226AB153035DA7DC5AF9B58C0FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj0Roc5o5ut97IDJdspIdsew== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822E5ED45BC192D15D8D23BCFC28F9925753841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 5/7] replication: send latest effective promote in initial join X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Good job on the fixes! On 17.06.2021 23:00, Serge Petrenko via Tarantool-patches wrote: > > > 16.06.2021 00:00, Vladislav Shpilevoy пишет: >> Thanks for working on this! >> >> Hm. The patch makes me think why don't we send the Raft >> checkpoint on join too? >> >> Otherwise it might happen that a replica joined, didn't >> get the most actual Raft term yet, then the leader died, >> and the replica would start elections with term 1. Even if >> the latest term was 1000. >> >> Nothing critical, Raft will probably work, but it could >> be an "optimization"? Also it would be consistent with the >> libmo state - send all the snapshot data on join. > I tried to implement such a patch, but faced the following problem: > > Unfortunately, we don't have information about replica's version > during join, so I can only send raft state based on term > 1. But you do send the limbo state. Won't it break the old versions like 1.10? I tried to replicate 1.10 from the new version and got UNKNOWN_REQUEST_TYPE error. 2021-06-19 00:50:49.781 [85883] main/105/applier/ applier.cc:345 E> ER_UNKNOWN_REQUEST_TYPE: Unknown request type 31 2021-06-19 00:50:49.781 [85883] main/101/interactive applier.cc:345 E> ER_UNKNOWN_REQUEST_TYPE: Unknown request type 31 2021-06-19 00:50:49.781 [85883] main/101/interactive F> can't initialize storage: Unknown request type 31 And that is fine. Because the replication does not need to be compatible "forward". You don't need to be able to replicate a new instance into an old instance. > Also, while writing a commit message I understood that this patch > doesn't help much. Even if a node joins, but doesn't subscribe to the > leader, it will still subscribe to someone else and receive the latest > Raft state. And what if it does not? What if there are just 2 nodes? > After all, our doc states full-mesh is required for Raft to work, so we'll > have someone else to subscribe to and receive Raft state from for sure. > The patch's small and simple but I think it's not needed. > > I've made a tiny amendment to this commit though, please find the diff below. The term limbo affects the filtering. Assume you sent the limbo state without raft to 2 new replicas. They now has a big limbo term, say 1000, and raft term 1. Now the master dies. The replicas start new election. One of them wins with some small raft term (let it be 10). It starts doing synchro txns. The other node's applier will turn them all to NOPs, because txn_limbo_is_replica_outdated() will return true. Even though it is not outdated. Will it happen?