From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id A73286BD15; Sun, 11 Apr 2021 20:58:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A73286BD15 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618163890; bh=vrfwo4wh7DooqDvR1WUT2kwu3ef4GJ2DGuh6ctKHSUc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=dVIRas4/Hx52avs/t6btAdNIQteV1J37pkRd9xBhbxBsx1b5vkVBLJ7VRm+2TT3lZ btRyxy48m7nwbD876ZUNOj/x8lcAsLYgd9S5jyp6llbKKONen4/CkxwbFm7MpLpHDb 6fAEa/65+pbssex0/S13FZ16n2JOodVPkcyp3ZrE= Received: from smtp42.i.mail.ru (smtp42.i.mail.ru [94.100.177.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 923886BD15 for ; Sun, 11 Apr 2021 20:56:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 923886BD15 Received: by smtp42.i.mail.ru with esmtpa (envelope-from ) id 1lVeJf-0001Ua-RX; Sun, 11 Apr 2021 20:56:12 +0300 To: v.shpilevoy@tarantool.org, gorcunov@gmail.com Date: Sun, 11 Apr 2021 20:55:59 +0300 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480D608FE24BC85426BB1B55F651FED8C70182A05F5380850404B3AB0E1DAE56AC17D737C60D35AE4DC92CCE0093E634C56D1F599B6B2E865BC X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE75A6765C746F51968EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006379023F77A4439C3FDEA1F7E6F0F101C67CDEEF6D7F21E0D1D9295C2E9FA3191EE1B59CA4C82EFA658062816736BBD864BB9D9621D974C294BF6B57BC7E64490618DEB871D839B73339E8FC8737B5C22495FF0BFC5AEE34BE6CC7F00164DA146DAFE8445B8C89999729449624AB7ADAF37F6B57BC7E64490611E7FA7ABCAF51C92176DF2183F8FC7C058C1844A7A85E7B68941B15DA834481F9449624AB7ADAF37BA3038C0950A5D3613377AFFFEAFD269176DF2183F8FC7C0C67AC315686ED4D3D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE30A9EC8C3488E7643C0837EA9F3D19764C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8BDB96E037C9407EA075ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A24209795067102C07E8F7B195E1C9783191471F51605917AD8FF110C78C8E6CED X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8183A4AFAF3EA6BDC44C234C8B12C006B7AFD80A448CD634CA58381BB4D4565BB8694D8F9C1FDB8B313B1881A6453793CE9C32612AADDFBE061C61BE10805914D3804EBA3D8E7E5B87ABF8C51168CD8EBDB63AF70AF8205D7DCDC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D345DE7771146E56B085CA243EE9B57DDAA6F02B90F250690F3B9F4905D730209394A47E33133CF8E0D1D7E09C32AA3244C99F9FC59827C6880B41793EA170BD22B30452B15D76AEC14927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbL9S8ysBdXjz3uqod8pbhWA2+ArV0Z7a X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F8A3D5B6EFD6943869B0F7A28DE9A55D4E64F355F2DF6DC2AA424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH 4/9] box: make clear_synchro_queue() write a PROMOTE entry instead of CONFIRM + ROLLBACK X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" A successful box_clear_synchro_queue() call results in writing CONFIRM(N) ROLLBACK(N+1) pair, where N is the confirmed lsn. Let's write a single PROMOTE(N) entry instead. It'll have the same meaning as CONFIRM + ROLLBACK and it will give followers some additional information regarding leader state change later. Part of #5445 --- src/box/applier.cc | 4 +- src/box/box.cc | 14 ++++++- src/box/iproto_constants.h | 5 +++ src/box/txn_limbo.c | 79 +++++++++++++++++++++----------------- src/box/txn_limbo.h | 10 ++++- 5 files changed, 72 insertions(+), 40 deletions(-) diff --git a/src/box/applier.cc b/src/box/applier.cc index 971b2e64c..e8cbbe27a 100644 --- a/src/box/applier.cc +++ b/src/box/applier.cc @@ -763,7 +763,7 @@ applier_txn_wal_write_cb(struct trigger *trigger, void *event) struct synchro_entry { /** Encoded form of a synchro record. */ - struct synchro_body_bin body_bin; + struct promote_body_bin body_bin; /** xrow to write, used by the journal engine. */ struct xrow_header row; @@ -822,7 +822,7 @@ synchro_entry_new(struct xrow_header *applier_row, } struct journal_entry *journal_entry = &entry->journal_entry; - struct synchro_body_bin *body_bin = &entry->body_bin; + struct synchro_body_bin *body_bin = &entry->body_bin.base; struct xrow_header *row = &entry->row; journal_entry->rows[0] = row; diff --git a/src/box/box.cc b/src/box/box.cc index b846ba8f5..8aba051a2 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -1556,7 +1556,19 @@ box_clear_synchro_queue(bool try_wait) "new synchronous transactions appeared"); rc = -1; } else { - txn_limbo_force_empty(&txn_limbo, wait_lsn); + /* + * Term parameter is unused now, We'll pass + * box_raft()->term there later. + */ + txn_limbo_write_promote(&txn_limbo, wait_lsn, 0); + struct synchro_request req = { + .type = 0, /* unused */ + .replica_id = 0, /* unused */ + .origin_id = instance_id, + .lsn = wait_lsn, + .term = 0, /* unused */ + }; + txn_limbo_read_promote(&txn_limbo, &req); assert(txn_limbo_is_empty(&txn_limbo)); } } diff --git a/src/box/iproto_constants.h b/src/box/iproto_constants.h index 816a308d8..da78ac4d4 100644 --- a/src/box/iproto_constants.h +++ b/src/box/iproto_constants.h @@ -362,6 +362,11 @@ iproto_type_is_synchro_request(uint32_t type) type == IPROTO_PROMOTE; } +static inline bool +iproto_type_is_promote_request(uint32_t type) +{ + return type == IPROTO_PROMOTE; +} static inline bool iproto_type_is_raft_request(uint32_t type) { diff --git a/src/box/txn_limbo.c b/src/box/txn_limbo.c index cf0ad9350..664f9d369 100644 --- a/src/box/txn_limbo.c +++ b/src/box/txn_limbo.c @@ -314,21 +314,25 @@ txn_limbo_write_cb(struct journal_entry *entry) } static void -txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn) +txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn, + uint64_t term) { - assert(lsn > 0); + assert(lsn >= 0); struct synchro_request req = { .type = type, .replica_id = limbo->owner_id, .lsn = lsn, + .term = term, }; /* - * This is a synchronous commit so we can - * allocate everything on a stack. + * This is a synchronous commit so we can allocate everything on a + * stack. Promote body includes synchro body. */ - struct synchro_body_bin body; + struct promote_body_bin body; + struct synchro_body_bin *base = &body.base; + struct xrow_header row; char buf[sizeof(struct journal_entry) + sizeof(struct xrow_header *)]; @@ -336,7 +340,7 @@ txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn) struct journal_entry *entry = (struct journal_entry *)buf; entry->rows[0] = &row; - xrow_encode_synchro(&row, &body, &req); + xrow_encode_synchro(&row, base, &req); journal_entry_create(entry, 1, xrow_approx_len(&row), txn_limbo_write_cb, fiber()); @@ -368,14 +372,14 @@ txn_limbo_write_confirm(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->confirmed_lsn = lsn; - txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn); + txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn, 0); } /** Confirm all the entries <= @a lsn. */ static void txn_limbo_read_confirm(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; rlist_foreach_entry_safe(e, &limbo->queue, in_queue, tmp) { @@ -431,7 +435,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->is_in_rollback = true; - txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn); + txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn, 0); limbo->is_in_rollback = false; } @@ -439,7 +443,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) static void txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; struct txn_limbo_entry *last_rollback = NULL; @@ -487,6 +491,32 @@ txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) box_update_ro_summary(); } +void +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term) +{ + limbo->confirmed_lsn = lsn; + /* + * We make sure that promote is only written once everything this + * instance has may be confirmed. + */ + struct txn_limbo_entry *e = txn_limbo_last_synchro_entry(limbo); + assert(e == NULL || e->lsn <= lsn); + (void) e; + txn_limbo_write_synchro(limbo, IPROTO_PROMOTE, lsn, term); + limbo->is_in_rollback = false; +} + +void +txn_limbo_read_promote(struct txn_limbo *limbo, + const struct synchro_request *req) +{ + txn_limbo_read_confirm(limbo, req->lsn); + txn_limbo_read_rollback(limbo, req->lsn + 1); + assert(txn_limbo_is_empty(&txn_limbo)); + limbo->owner_id = req->origin_id; + limbo->confirmed_lsn = 0; +} + void txn_limbo_ack(struct txn_limbo *limbo, uint32_t replica_id, int64_t lsn) { @@ -649,38 +679,15 @@ txn_limbo_process(struct txn_limbo *limbo, const struct synchro_request *req) case IPROTO_ROLLBACK: txn_limbo_read_rollback(limbo, req->lsn); break; + case IPROTO_PROMOTE: + txn_limbo_read_promote(limbo, req); + break; default: unreachable(); } return; } -void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t confirm_lsn) -{ - struct txn_limbo_entry *e, *last_quorum = NULL; - struct txn_limbo_entry *rollback = NULL; - rlist_foreach_entry(e, &limbo->queue, in_queue) { - if (txn_has_flag(e->txn, TXN_WAIT_ACK)) { - if (e->lsn <= confirm_lsn) { - last_quorum = e; - } else { - rollback = e; - break; - } - } - } - - if (last_quorum != NULL) { - txn_limbo_write_confirm(limbo, last_quorum->lsn); - txn_limbo_read_confirm(limbo, last_quorum->lsn); - } - if (rollback != NULL) { - txn_limbo_write_rollback(limbo, rollback->lsn); - txn_limbo_read_rollback(limbo, rollback->lsn); - } -} - void txn_limbo_on_parameters_change(struct txn_limbo *limbo) { diff --git a/src/box/txn_limbo.h b/src/box/txn_limbo.h index af0addf8d..ec317ed20 100644 --- a/src/box/txn_limbo.h +++ b/src/box/txn_limbo.h @@ -275,7 +275,15 @@ txn_limbo_wait_confirm(struct txn_limbo *limbo); * immediately. */ void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t last_confirm); +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term); + +/** + * Process a PROMOTE request, i.e. confirm all entries <= @lsn and rollback all + * entries > @lsn. + */ +void +txn_limbo_read_promote(struct txn_limbo *limbo, + const struct synchro_request *req); /** * Update qsync parameters dynamically. -- 2.24.3 (Apple Git-128)