From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id BEC416F87A; Fri, 16 Apr 2021 19:28:20 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org BEC416F87A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618590500; bh=3DNpkvwCNuRC1+nb3vUYqyU/sPLKwAtnkiJi/XsNQmg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=GDmHnxrkhL08FwBP7gOsWITWi+/u/1K23roOhvwUii74zBAV8ZZKnx6VSl1EMN+4m rQCkabae0hhkXgwCq/Sz1Vo834hsSm1uCrkdk6K/xZO6yPtAfzkat8Q1Z1Qp8E60J1 ZlDo61DkLuSyLXr36Qf6SASpNFUusHEjasQQEsmg= Received: from smtp17.mail.ru (smtp17.mail.ru [94.100.176.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 406B374B70 for ; Fri, 16 Apr 2021 19:25:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 406B374B70 Received: by smtp17.mail.ru with esmtpa (envelope-from ) id 1lXRHz-00008e-RK; Fri, 16 Apr 2021 19:25:52 +0300 To: v.shpilevoy@tarantool.org, gorcunov@gmail.com Date: Fri, 16 Apr 2021 19:25:36 +0300 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480D608FE24BC85426BB1B55F651FED8C70182A05F538085040AB311BD6C8A1DDA2632DAA0BAB08F3A043563D2A7ACA9CF6EFCFAF669CAE7A7A X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B2CE06D3E4B8AFEBEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637CD877809B688EA868638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2961855EAF964C8A1ED7221F30B4EDD6B088CACE0DB6A842FD2E47CDBA5A96583C09775C1D3CA48CF90BCC82C2C62A6D1117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE75A31C54DA8CF07A79FA2833FD35BB23DF004C90652538430302FCEF25BFAB3454AD6D5ED66289B5278DA827A17800CE7A78FC90518FECA56D32BA5DBAC0009BE395957E7521B51C20BC6067A898B09E4090A508E0FED62991661749BA6B977356CC74F0925D5E26BCD04E86FAF290E2D7E9C4E3C761E06A71DD303D21008E298D5E8D9A59859A8B6B372FE9A2E580EFC725E5C173C3A84C3CE9959E2676FD87735872C767BF85DA2F004C90652538430E4A6367B16DE6309 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A24209795067102C07E8F7B195E1C978319E09578807932652916D7973594ED144 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8183A4AFAF3EA6BDC44E1F4276B80994196BF1196BB3248DD4B2151AE923667D1AF78BAD097B7D7B8099C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF0417BEADF48D1460699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A246DFF68F0EE19E96ED2594E0451F0C0053C48B74D9DE2230AF2FBB36FAD15E3F8E6F7F8CCD220F1D7E09C32AA3244CC4C4CB4B5D3975F967B25E00C69CF01455E75C8D0ED9F6EE927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj3S6P1v0GIqS4ewuUh+kugw== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F88BD3238A7C907672E41E99B2D0019C130F81FB139F4278C2424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v4 05/12] box: make clear_synchro_queue() write a PROMOTE entry instead of CONFIRM + ROLLBACK X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" A successful box_clear_synchro_queue() call results in writing CONFIRM(N) ROLLBACK(N+1) pair, where N is the confirmed lsn. Let's write a single PROMOTE(N) entry instead. It'll have the same meaning as CONFIRM + ROLLBACK and it will give followers some additional information regarding leader state change later. Part of #5445 --- src/box/box.cc | 14 +++++++- src/box/txn_limbo.c | 78 +++++++++++++++++++++++++-------------------- src/box/txn_limbo.h | 9 ++---- 3 files changed, 60 insertions(+), 41 deletions(-) diff --git a/src/box/box.cc b/src/box/box.cc index 2d2ae233c..e0bed74f1 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -1556,7 +1556,19 @@ box_clear_synchro_queue(bool try_wait) "new synchronous transactions appeared"); rc = -1; } else { - txn_limbo_force_empty(&txn_limbo, wait_lsn); + /* + * Term parameter is unused now, We'll pass + * box_raft()->term there later. + */ + txn_limbo_write_promote(&txn_limbo, wait_lsn, 0); + struct synchro_request req = { + .type = IPROTO_PROMOTE, + .replica_id = former_leader_id, + .origin_id = instance_id, + .lsn = wait_lsn, + .term = 0, /* unused */ + }; + txn_limbo_process(&txn_limbo, &req); assert(txn_limbo_is_empty(&txn_limbo)); } } diff --git a/src/box/txn_limbo.c b/src/box/txn_limbo.c index c96e497c6..2346331c7 100644 --- a/src/box/txn_limbo.c +++ b/src/box/txn_limbo.c @@ -317,19 +317,21 @@ txn_limbo_write_cb(struct journal_entry *entry) } static void -txn_limbo_write_synchro(struct txn_limbo *limbo, uint16_t type, int64_t lsn) +txn_limbo_write_synchro(struct txn_limbo *limbo, uint16_t type, int64_t lsn, + uint64_t term) { - assert(lsn > 0); + assert(lsn >= 0); struct synchro_request req = { .type = type, .replica_id = limbo->owner_id, .lsn = lsn, + .term = term, }; /* - * This is a synchronous commit so we can - * allocate everything on a stack. + * This is a synchronous commit so we can allocate everything on a + * stack. Note, that promote body includes synchro body. */ char body[XROW_SYNCHRO_BODY_LEN_MAX]; struct xrow_header row; @@ -371,14 +373,14 @@ txn_limbo_write_confirm(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->confirmed_lsn = lsn; - txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn); + txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn, 0); } /** Confirm all the entries <= @a lsn. */ static void txn_limbo_read_confirm(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; rlist_foreach_entry_safe(e, &limbo->queue, in_queue, tmp) { @@ -425,7 +427,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->is_in_rollback = true; - txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn); + txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn, 0); limbo->is_in_rollback = false; } @@ -433,7 +435,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) static void txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; struct txn_limbo_entry *last_rollback = NULL; @@ -464,6 +466,37 @@ txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) box_update_ro_summary(); } +void +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term) +{ + limbo->confirmed_lsn = lsn; + limbo->is_in_rollback = true; + /* + * We make sure that promote is only written once everything this + * instance has may be confirmed. + */ + struct txn_limbo_entry *e = txn_limbo_last_synchro_entry(limbo); + assert(e == NULL || e->lsn <= lsn); + (void) e; + txn_limbo_write_synchro(limbo, IPROTO_PROMOTE, lsn, term); + limbo->is_in_rollback = false; +} + +/** + * Process a PROMOTE request, i.e. confirm all entries <= @req.lsn and rollback all + * entries > @req.lsn. + */ +static void +txn_limbo_read_promote(struct txn_limbo *limbo, + const struct synchro_request *req) +{ + txn_limbo_read_confirm(limbo, req->lsn); + txn_limbo_read_rollback(limbo, req->lsn + 1); + assert(txn_limbo_is_empty(&txn_limbo)); + limbo->owner_id = req->origin_id; + limbo->confirmed_lsn = 0; +} + void txn_limbo_ack(struct txn_limbo *limbo, uint32_t replica_id, int64_t lsn) { @@ -626,38 +659,15 @@ txn_limbo_process(struct txn_limbo *limbo, const struct synchro_request *req) case IPROTO_ROLLBACK: txn_limbo_read_rollback(limbo, req->lsn); break; + case IPROTO_PROMOTE: + txn_limbo_read_promote(limbo, req); + break; default: unreachable(); } return; } -void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t confirm_lsn) -{ - struct txn_limbo_entry *e, *last_quorum = NULL; - struct txn_limbo_entry *rollback = NULL; - rlist_foreach_entry(e, &limbo->queue, in_queue) { - if (txn_has_flag(e->txn, TXN_WAIT_ACK)) { - if (e->lsn <= confirm_lsn) { - last_quorum = e; - } else { - rollback = e; - break; - } - } - } - - if (last_quorum != NULL) { - txn_limbo_write_confirm(limbo, last_quorum->lsn); - txn_limbo_read_confirm(limbo, last_quorum->lsn); - } - if (rollback != NULL) { - txn_limbo_write_rollback(limbo, rollback->lsn); - txn_limbo_read_rollback(limbo, rollback->lsn); - } -} - void txn_limbo_on_parameters_change(struct txn_limbo *limbo) { diff --git a/src/box/txn_limbo.h b/src/box/txn_limbo.h index f2a98c8bb..f35771dc9 100644 --- a/src/box/txn_limbo.h +++ b/src/box/txn_limbo.h @@ -272,14 +272,11 @@ int txn_limbo_wait_confirm(struct txn_limbo *limbo); /** - * Make txn_limbo confirm all the entries with lsn less than or - * equal to the given one, and rollback all the following entries. - * The function makes txn_limbo write CONFIRM and ROLLBACK - * messages for appropriate lsns, and then process the messages - * immediately. + * Write a PROMOTE request, which has the same effect as CONFIRM(@a lsn) and + * ROLLBACK(@a lsn + 1) combined. */ void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t last_confirm); +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term); /** * Update qsync parameters dynamically. -- 2.24.3 (Apple Git-128)