From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9D7276EC57; Wed, 14 Jul 2021 21:29:25 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9D7276EC57 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1626287365; bh=GCBgPbryBNlz1Ec2le2uKPO1Ym0Yc6eJfkjQ2OjxX5A=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=KAIUpuHuhXtpkmfVE0vQIsF40i9MWGuYCY+9fFB4DJZ4r27WVsCKCj191Ds/r1Ixh MsVUwIgyT2ls0nk3cf0ygBsOYq7/pVwXkEIC24R+9FlQlUbINwAgvbiBlEMOHOaq8i ++/PcEp29IQq2lotAGjJ4N40zS3EBLTtOzWpIMcE= Received: from smtp58.i.mail.ru (smtp58.i.mail.ru [217.69.128.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id D37EE6EC70 for ; Wed, 14 Jul 2021 21:26:03 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D37EE6EC70 Received: by smtp58.i.mail.ru with esmtpa (envelope-from ) id 1m3ja7-0007Q7-5k; Wed, 14 Jul 2021 21:26:03 +0300 To: v.shpilevoy@tarantool.org, gorcunov@gmail.com Date: Wed, 14 Jul 2021 21:25:35 +0300 Message-Id: X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD97BB0EF39AD2B33D5F2EB549EE938DAF5BD6C597CA890F198182A05F53808504039AFB7627262443F62EFB15CF8ECC34C188EE43B2098728C880DEDA0F5F54619 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7CDB54ABCDA26BE78EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637D0C488966F20D4908638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D87A56FB79BB79B95931B1C0F0961D0A24117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC1F8789D36234D406A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520B28585415E75ADA9C26CFBAC0749D213D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B6D635BA3ABDB36C18089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975CE68746B1F2AB10C60986273A8465F5FBD28CAB77290DA1029C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF309DFB797F6729CB699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34829444FF2D8CB89BFBC33D6372C226641762243004D0E688B6ED527B66F14CD9C4B1D64AAF97D73B1D7E09C32AA3244C10D1AC3E4630EDBFBD4EC72B6FB1C6BD259227199D06760A927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojDdSFIg49M1SW4Ao9wmGbkQ== X-Mailru-Sender: 3B9A0136629DC9125D61937A2360A446DCF67A6ACA5D040DA43DEC5E0340C12220768630229FC1CD424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v4 07/16] replication: send current Raft term in join response X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Make Raft nodes send out their latest persisted term to joining replicas. This is needed to avoid the situation when txn_limbo-managed 'promote greatest term' is greater than current Raft term. Otherwise the following may happen: replica joins off some instance and receives its latest limbo state. The state includes "greatest term seen" and makes limbo filter out any data coming from instances with smaller terms. Imagine that master this replica has joined from dies before replica has a chance to subscribe to it. Then it doesn't receive its current Raft term and start elections at smallest term possible, 2 (when there are no suitable Raft nodes besides the replica). Once the elections in a small term number are won, a ton of problems arises: starting with filtering out PROMOTE requests for "old" term and nop-ifying any data coming from terms smaller than "greatest term seen". Prerequisite #6034 --- src/box/applier.cc | 5 +++++ src/box/relay.cc | 6 ++++++ 2 files changed, 11 insertions(+) diff --git a/src/box/applier.cc b/src/box/applier.cc index 4088fcc21..92ec088ea 100644 --- a/src/box/applier.cc +++ b/src/box/applier.cc @@ -459,6 +459,11 @@ applier_wait_snapshot(struct applier *applier) if (xrow_decode_synchro(&row, &req) != 0) diag_raise(); txn_limbo_process(&txn_limbo, &req); + } else if (iproto_type_is_raft_request(row.type)) { + struct raft_request req; + if (xrow_decode_raft(&row, &req, NULL) != 0) + diag_raise(); + box_raft_recover(&req); } else if (row.type != IPROTO_JOIN_SNAPSHOT) { tnt_raise(ClientError, ER_UNKNOWN_REQUEST_TYPE, (uint32_t)row.type); diff --git a/src/box/relay.cc b/src/box/relay.cc index 4b102a777..70f1a045b 100644 --- a/src/box/relay.cc +++ b/src/box/relay.cc @@ -428,7 +428,9 @@ relay_initial_join(int fd, uint64_t sync, struct vclock *vclock, diag_raise(); struct synchro_request req; + struct raft_request raft_req; txn_limbo_checkpoint(&txn_limbo, &req); + box_raft_checkpoint_local(&raft_req); /* Respond to the JOIN request with the current vclock. */ struct xrow_header row; @@ -451,6 +453,10 @@ relay_initial_join(int fd, uint64_t sync, struct vclock *vclock, row.sync = sync; coio_write_xrow(&relay->io, &row); + xrow_encode_raft(&row, &fiber()->gc, &raft_req); + row.sync = sync; + coio_write_xrow(&relay->io, &row); + /* Mark the end of the metadata stream. */ row.type = IPROTO_JOIN_SNAPSHOT; coio_write_xrow(&relay->io, &row); -- 2.30.1 (Apple Git-130)