From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 218996EC58; Mon, 21 Jun 2021 13:13:01 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 218996EC58 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1624270381; bh=DOn8DD+y+AzzVYVQnCK5PU6oz5+cuXeRQv5aX4OPhyw=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=tm63F+bEba6WTxsbevHV2mNhNSIv8/6xuxCRovyyUzjhP9Ll2NI63scAwNlGqN/o7 A0sjyQV0UqkOEqjt21Bnt0IqIvSZh1A6291CVXbEwArCfQ1FZ9LybdAzu4lHzHlolP 3ucy6uC5FSi5E2/Y2YfC0QlpgJRzoXSsrJojfghQ= Received: from smtp53.i.mail.ru (smtp53.i.mail.ru [94.100.177.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 39FBE6EC58 for ; Mon, 21 Jun 2021 13:12:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 39FBE6EC58 Received: by smtp53.i.mail.ru with esmtpa (envelope-from ) id 1lvGvI-0006F0-EQ; Mon, 21 Jun 2021 13:12:56 +0300 To: Vladislav Shpilevoy , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <93a4765cb6918512946bf9366fe4ad478345bce1.1623331925.git.sergepetrenko@tarantool.org> <76d19171-2257-35e3-8f07-b1f0323c728f@tarantool.org> <3a61c787-8a92-0191-1565-118915adbfcc@tarantool.org> <12dc56c1-c6b6-154d-1e19-99c621eaea47@tarantool.org> Message-ID: Date: Mon, 21 Jun 2021 13:12:55 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <12dc56c1-c6b6-154d-1e19-99c621eaea47@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD91C2C07775F13263A612E9F961DCF5576EBB1C736C2A4021700894C459B0CD1B963153DDDE19EEE1E70BD10674D65E6187BF517ABB5E8AC2EE41C5801A3FD0B27 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7495A032B936E882FEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC18FED211962C318638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8C53C93FF25428F86E3C7AFE5C2378C1F117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18BDFBBEFFF4125B51D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B6753C3A5E0A5AB5B7089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: 0D63561A33F958A504909074D89019781343812D106C785A2DD57C368D000125D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75448CF9D3A7B2C848410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34D8C933888226C841BDDB03CA645E2CD808A721A8DB3614998741E3F4BFAB5A8885B1DCAA982E994A1D7E09C32AA3244C879FDEEF39172471EDC25851E139F6973FD9C8CA1B0515E0FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj8x+Gb+jwA+RthH4iAmmRYA== X-Mailru-Sender: 3B9A0136629DC9125D61937A2360A4466F5A51994DB4FB8B9BF21208FEF3A3CEA1BFC3EE533CC8C1424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 5/7] replication: send latest effective promote in initial join X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 19.06.2021 01:52, Vladislav Shpilevoy пишет: > Good job on the fixes! Thanks for the review! > > On 17.06.2021 23:00, Serge Petrenko via Tarantool-patches wrote: >> >> 16.06.2021 00:00, Vladislav Shpilevoy пишет: >>> Thanks for working on this! >>> >>> Hm. The patch makes me think why don't we send the Raft >>> checkpoint on join too? >>> >>> Otherwise it might happen that a replica joined, didn't >>> get the most actual Raft term yet, then the leader died, >>> and the replica would start elections with term 1. Even if >>> the latest term was 1000. >>> >>> Nothing critical, Raft will probably work, but it could >>> be an "optimization"? Also it would be consistent with the >>> libmo state - send all the snapshot data on join. >> I tried to implement such a patch, but faced the following problem: >> >> Unfortunately, we don't have information about replica's version >> during join, so I can only send raft state based on term > 1. > But you do send the limbo state. Won't it break the old versions > like 1.10? I only send it if limbo is used. I.e. when limbo->owner_id != 0. I've just realized, the problem is not only with versions like 1.10. It's the same for versions like 2.8.1, 2.7.2 and so on, which will error with UNKNOWN_REQUEST_TYPE, when receive limbo or raft state in initial join stream. What can we do about that? > > I tried to replicate 1.10 from the new version and got UNKNOWN_REQUEST_TYPE > error. > > 2021-06-19 00:50:49.781 [85883] main/105/applier/ applier.cc:345 E> ER_UNKNOWN_REQUEST_TYPE: Unknown request type 31 > 2021-06-19 00:50:49.781 [85883] main/101/interactive applier.cc:345 E> ER_UNKNOWN_REQUEST_TYPE: Unknown request type 31 > 2021-06-19 00:50:49.781 [85883] main/101/interactive F> can't initialize storage: Unknown request type 31 > > And that is fine. Because the replication does not need to be > compatible "forward". You don't need to be able to replicate a > new instance into an old instance. > >> Also, while writing a commit message I understood that this patch >> doesn't help much. Even if a node joins, but doesn't subscribe to the >> leader, it will still subscribe to someone else and receive the latest >> Raft state. > And what if it does not? What if there are just 2 nodes? > >> After all, our doc states full-mesh is required for Raft to work, so we'll >> have someone else to subscribe to and receive Raft state from for sure. >> The patch's small and simple but I think it's not needed. >> >> I've made a tiny amendment to this commit though, please find the diff below. > The term limbo affects the filtering. Assume you sent the limbo > state without raft to 2 new replicas. They now has a big limbo term, > say 1000, and raft term 1. Now the master dies. > > The replicas start new election. One of them wins with some small > raft term (let it be 10). It starts doing synchro txns. The other node's > applier will turn them all to NOPs, because txn_limbo_is_replica_outdated() > will return true. Even though it is not outdated. > > Will it happen? Looks like this may happen, indeed. -- Serge Petrenko