From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 3E3546EC5B; Mon, 12 Apr 2021 22:42:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3E3546EC5B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618256546; bh=t/kT4KzWFvKPB8dI7Pxeg1aCqTF577LRC7rvYXVqxC8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=XfXetxrHwJXBdFDItqt/fqty55NuHiJNstpQ2ThMTpmGgYita/3qL4t9TZeczjpDb /YUCDfzZ2wwikt1wZYjgBI1q65yn+6riV/ps6nzpUlECEdDT/ydI+Stx7F1IeNfIFz pWrqy+eu7tXgk5VVPQ0+ZwIQhzONpNukdstjUGSc= Received: from smtp53.i.mail.ru (smtp53.i.mail.ru [94.100.177.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 6E4DA6BD0B for ; Mon, 12 Apr 2021 22:40:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6E4DA6BD0B Received: by smtp53.i.mail.ru with esmtpa (envelope-from ) id 1lW2Q6-0008Az-H7; Mon, 12 Apr 2021 22:40:27 +0300 To: v.shpilevoy@tarantool.org, gorcunov@gmail.com Date: Mon, 12 Apr 2021 22:40:17 +0300 Message-Id: <150875bcbaaa0b7c270c1d53dc623892312fe16d.1618256019.git.sergepetrenko@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E748094FADAEB10E66ADA4C48BE3C291E66DA182A05F538085040E4708B321454B636486A04FCD414D9D1D53A41BEDD0B859C2D03B0F8DF793BCF X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE71D6E7AC8529A326CEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637B20ACC28402695E48638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2817B1779992D11220E215C5EB66FE6F428451B159A507268D2E47CDBA5A96583C09775C1D3CA48CF7EF884183F8E4D67117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE7E4DF6D1C10F22F599FA2833FD35BB23DF004C90652538430302FCEF25BFAB3454AD6D5ED66289B5278DA827A17800CE79C49E88482E10C617B076A6E789B0E97A8DF7F3B2552694A1E7802607F20496D49FD398EE364050F26055571C92BF10F3D5BA627BF9F2FCFB3661434B16C20AC78D18283394535A9E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B66F1C5D350F9AE87C089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F6089093C9A16E5BC824A2A04A2ABAA09D25379311020FFC8D4ADF048BB164F7B76AF354880391DE99277 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8183A4AFAF3EA6BDC44C234C8B12C006B7AE4F16467A2949FE901E7B6FF2227D0C31A88C3A95B918B5EB1881A6453793CE9C32612AADDFBE061C61BE10805914D3804EBA3D8E7E5B87ABF8C51168CD8EBDB63AF70AF8205D7DCDC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D346840168BCAD8054E3CFBDDDF1C6B2C8F5A038A3654BBDDED70A24AF832708CA3E9B9664045A5FF3B1D7E09C32AA3244C4F3FF0F2FABB0E9FCE1F3CFFB9F3F30BF94338140B71B8EE927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojq8JA+pXcDunTWB0jdQhgiA== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F8AFCB052CDF231E7BC5032526F242A6188D50A612EA6A48DA424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v2 4/9] box: make clear_synchro_queue() write a PROMOTE entry instead of CONFIRM + ROLLBACK X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" A successful box_clear_synchro_queue() call results in writing CONFIRM(N) ROLLBACK(N+1) pair, where N is the confirmed lsn. Let's write a single PROMOTE(N) entry instead. It'll have the same meaning as CONFIRM + ROLLBACK and it will give followers some additional information regarding leader state change later. Part of #5445 --- src/box/applier.cc | 4 +- src/box/box.cc | 14 ++++++- src/box/iproto_constants.h | 5 +++ src/box/txn_limbo.c | 79 +++++++++++++++++++++----------------- src/box/txn_limbo.h | 10 ++++- 5 files changed, 72 insertions(+), 40 deletions(-) diff --git a/src/box/applier.cc b/src/box/applier.cc index 971b2e64c..e8cbbe27a 100644 --- a/src/box/applier.cc +++ b/src/box/applier.cc @@ -763,7 +763,7 @@ applier_txn_wal_write_cb(struct trigger *trigger, void *event) struct synchro_entry { /** Encoded form of a synchro record. */ - struct synchro_body_bin body_bin; + struct promote_body_bin body_bin; /** xrow to write, used by the journal engine. */ struct xrow_header row; @@ -822,7 +822,7 @@ synchro_entry_new(struct xrow_header *applier_row, } struct journal_entry *journal_entry = &entry->journal_entry; - struct synchro_body_bin *body_bin = &entry->body_bin; + struct synchro_body_bin *body_bin = &entry->body_bin.base; struct xrow_header *row = &entry->row; journal_entry->rows[0] = row; diff --git a/src/box/box.cc b/src/box/box.cc index b846ba8f5..8aba051a2 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -1556,7 +1556,19 @@ box_clear_synchro_queue(bool try_wait) "new synchronous transactions appeared"); rc = -1; } else { - txn_limbo_force_empty(&txn_limbo, wait_lsn); + /* + * Term parameter is unused now, We'll pass + * box_raft()->term there later. + */ + txn_limbo_write_promote(&txn_limbo, wait_lsn, 0); + struct synchro_request req = { + .type = 0, /* unused */ + .replica_id = 0, /* unused */ + .origin_id = instance_id, + .lsn = wait_lsn, + .term = 0, /* unused */ + }; + txn_limbo_read_promote(&txn_limbo, &req); assert(txn_limbo_is_empty(&txn_limbo)); } } diff --git a/src/box/iproto_constants.h b/src/box/iproto_constants.h index 816a308d8..da78ac4d4 100644 --- a/src/box/iproto_constants.h +++ b/src/box/iproto_constants.h @@ -362,6 +362,11 @@ iproto_type_is_synchro_request(uint32_t type) type == IPROTO_PROMOTE; } +static inline bool +iproto_type_is_promote_request(uint32_t type) +{ + return type == IPROTO_PROMOTE; +} static inline bool iproto_type_is_raft_request(uint32_t type) { diff --git a/src/box/txn_limbo.c b/src/box/txn_limbo.c index a22e0861a..f119c35b6 100644 --- a/src/box/txn_limbo.c +++ b/src/box/txn_limbo.c @@ -317,21 +317,25 @@ txn_limbo_write_cb(struct journal_entry *entry) } static void -txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn) +txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn, + uint64_t term) { - assert(lsn > 0); + assert(lsn >= 0); struct synchro_request req = { .type = type, .replica_id = limbo->owner_id, .lsn = lsn, + .term = term, }; /* - * This is a synchronous commit so we can - * allocate everything on a stack. + * This is a synchronous commit so we can allocate everything on a + * stack. Promote body includes synchro body. */ - struct synchro_body_bin body; + struct promote_body_bin body; + struct synchro_body_bin *base = &body.base; + struct xrow_header row; char buf[sizeof(struct journal_entry) + sizeof(struct xrow_header *)]; @@ -339,7 +343,7 @@ txn_limbo_write_synchro(struct txn_limbo *limbo, uint32_t type, int64_t lsn) struct journal_entry *entry = (struct journal_entry *)buf; entry->rows[0] = &row; - xrow_encode_synchro(&row, &body, &req); + xrow_encode_synchro(&row, base, &req); journal_entry_create(entry, 1, xrow_approx_len(&row), txn_limbo_write_cb, fiber()); @@ -371,14 +375,14 @@ txn_limbo_write_confirm(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->confirmed_lsn = lsn; - txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn); + txn_limbo_write_synchro(limbo, IPROTO_CONFIRM, lsn, 0); } /** Confirm all the entries <= @a lsn. */ static void txn_limbo_read_confirm(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; rlist_foreach_entry_safe(e, &limbo->queue, in_queue, tmp) { @@ -434,7 +438,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) assert(lsn > limbo->confirmed_lsn); assert(!limbo->is_in_rollback); limbo->is_in_rollback = true; - txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn); + txn_limbo_write_synchro(limbo, IPROTO_ROLLBACK, lsn, 0); limbo->is_in_rollback = false; } @@ -442,7 +446,7 @@ txn_limbo_write_rollback(struct txn_limbo *limbo, int64_t lsn) static void txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) { - assert(limbo->owner_id != REPLICA_ID_NIL); + assert(limbo->owner_id != REPLICA_ID_NIL || txn_limbo_is_empty(limbo)); assert(limbo == &txn_limbo); struct txn_limbo_entry *e, *tmp; struct txn_limbo_entry *last_rollback = NULL; @@ -490,6 +494,32 @@ txn_limbo_read_rollback(struct txn_limbo *limbo, int64_t lsn) box_update_ro_summary(); } +void +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term) +{ + limbo->confirmed_lsn = lsn; + /* + * We make sure that promote is only written once everything this + * instance has may be confirmed. + */ + struct txn_limbo_entry *e = txn_limbo_last_synchro_entry(limbo); + assert(e == NULL || e->lsn <= lsn); + (void) e; + txn_limbo_write_synchro(limbo, IPROTO_PROMOTE, lsn, term); + limbo->is_in_rollback = false; +} + +void +txn_limbo_read_promote(struct txn_limbo *limbo, + const struct synchro_request *req) +{ + txn_limbo_read_confirm(limbo, req->lsn); + txn_limbo_read_rollback(limbo, req->lsn + 1); + assert(txn_limbo_is_empty(&txn_limbo)); + limbo->owner_id = req->origin_id; + limbo->confirmed_lsn = 0; +} + void txn_limbo_ack(struct txn_limbo *limbo, uint32_t replica_id, int64_t lsn) { @@ -652,38 +682,15 @@ txn_limbo_process(struct txn_limbo *limbo, const struct synchro_request *req) case IPROTO_ROLLBACK: txn_limbo_read_rollback(limbo, req->lsn); break; + case IPROTO_PROMOTE: + txn_limbo_read_promote(limbo, req); + break; default: unreachable(); } return; } -void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t confirm_lsn) -{ - struct txn_limbo_entry *e, *last_quorum = NULL; - struct txn_limbo_entry *rollback = NULL; - rlist_foreach_entry(e, &limbo->queue, in_queue) { - if (txn_has_flag(e->txn, TXN_WAIT_ACK)) { - if (e->lsn <= confirm_lsn) { - last_quorum = e; - } else { - rollback = e; - break; - } - } - } - - if (last_quorum != NULL) { - txn_limbo_write_confirm(limbo, last_quorum->lsn); - txn_limbo_read_confirm(limbo, last_quorum->lsn); - } - if (rollback != NULL) { - txn_limbo_write_rollback(limbo, rollback->lsn); - txn_limbo_read_rollback(limbo, rollback->lsn); - } -} - void txn_limbo_on_parameters_change(struct txn_limbo *limbo) { diff --git a/src/box/txn_limbo.h b/src/box/txn_limbo.h index f2a98c8bb..10db4fc2d 100644 --- a/src/box/txn_limbo.h +++ b/src/box/txn_limbo.h @@ -279,7 +279,15 @@ txn_limbo_wait_confirm(struct txn_limbo *limbo); * immediately. */ void -txn_limbo_force_empty(struct txn_limbo *limbo, int64_t last_confirm); +txn_limbo_write_promote(struct txn_limbo *limbo, int64_t lsn, uint64_t term); + +/** + * Process a PROMOTE request, i.e. confirm all entries <= @lsn and rollback all + * entries > @lsn. + */ +void +txn_limbo_read_promote(struct txn_limbo *limbo, + const struct synchro_request *req); /** * Update qsync parameters dynamically. -- 2.24.3 (Apple Git-128)