From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 16E946F3C7; Fri, 26 Mar 2021 23:49:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 16E946F3C7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1616791751; bh=sAWsLx+n4VMCUiC2p/jSX7+oNLrBehnaudjyfGS+EK8=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=jioUq7rtTJIJpLPCzYg6kQ8U2B9ii0heeUIwuJ7u5NZAMUeXbdqQ332ccqoUxXUd2 kuQXF8GGUITbPqidhfaTYo9yy4j1uefRbUyqhSUhjS2TD5LE9LlbDKHfIw+obZDQwR Qmi2FjVONTQho05QKLrPdjAWVqNm/7cgPYfpnfyg= Received: from smtp46.i.mail.ru (smtp46.i.mail.ru [94.100.177.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 67CEE6F3C7 for ; Fri, 26 Mar 2021 23:49:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 67CEE6F3C7 Received: by smtp46.i.mail.ru with esmtpa (envelope-from ) id 1lPtOH-0000z0-R4; Fri, 26 Mar 2021 23:49:10 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <5cfa8f8e4d733aabd53138dd3ffc6f524ffac743.1616588119.git.sergepetrenko@tarantool.org> Message-ID: Date: Fri, 26 Mar 2021 21:49:08 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <5cfa8f8e4d733aabd53138dd3ffc6f524ffac743.1616588119.git.sergepetrenko@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9ED7173E37F4E32947427BE79D20CABD4ABD7C98AF5DBFD37182A05F5380850401CA6B2C69DF5946A83F83D707CE3F44433B5CF02078B6ABC04F09AD65F0344D4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE704FFE27C31EF363AEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006378D70459434292EC88638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C5DD32608FC869F5DE922107390099F28FFEBA5DD9DA551D5A471835C12D1D9774AD6D5ED66289B5259CC434672EE6371117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE77A825AB47F0FC8649FA2833FD35BB23DF004C90652538430302FCEF25BFAB3454AD6D5ED66289B5278DA827A17800CE742E3D5CE41F5C245D32BA5DBAC0009BE395957E7521B51C20BC6067A898B09E4090A508E0FED6299176DF2183F8FC7C0907D60A27468CB01CD04E86FAF290E2D7E9C4E3C761E06A71DD303D21008E29813377AFFFEAFD269A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE76515C59FC18CEA6D731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F6089093C9A16E5BC824AC8B6CDF511875BC4E8F7B195E1C978314A460336A7179A60D792DD908DDF4B51 X-C1DE0DAB: 0D63561A33F958A530C8B8D865354F43594D4D9400573AFCC39EFA403AE0C0F6D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34E318F287A436F24C804BC906921F4F8BDD6676C0C8662B5672C8655EC3A5B87FA9CC0EAAD682F8F81D7E09C32AA3244CE34E23A5F402CC55535E3F52A09AD8C47101BF96129E4011FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojapPp7P/VpAjXcvcdvPHcvA== X-Mailru-Sender: 504CC1E875BF3E7D9BC0E5172ADA311029B0AD94EF0A590FA14C6BE48DA1F95F05FF9F3A45A34F7C07784C02288277CA03E0582D3806FB6A5317862B1921BA260ED6CFD6382C13A6112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 5/7] applier: make final join transactional X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" I appreciate the work you did here! See 3 comments below. On 24.03.2021 13:24, Serge Petrenko wrote: > Now applier assembles rows into transactions not only on subscribe > stage, but also during final join / register. > > This was necessary for correct handling of rolled back synchronous > transactions in final join stream. > > Part of #5566 > --- > src/box/applier.cc | 126 ++++++++++++++++++++++----------------------- > 1 file changed, 61 insertions(+), 65 deletions(-) > > diff --git a/src/box/applier.cc b/src/box/applier.cc > index d53f13711..9a8b0f0fc 100644 > --- a/src/box/applier.cc > +++ b/src/box/applier.cc > @@ -524,27 +509,19 @@ applier_wait_register(struct applier *applier, uint64_t row_count) > * Receive final data. > */ > while (true) { > - coio_read_xrow(coio, ibuf, &row); > - applier->last_row_time = ev_monotonic_now(loop()); > - if (iproto_type_is_dml(row.type)) { > - vclock_follow_xrow(&replicaset.vclock, &row); > - if (apply_final_join_row(&row) != 0) > - diag_raise(); > - if (++row_count % 100000 == 0) > - say_info("%.1fM rows received", row_count / 1e6); > - } else if (row.type == IPROTO_OK) { > - /* > - * Current vclock. This is not used now, > - * ignore. > - */ 1. The comment was helpful, lets keep it. > - ++row_count; > - break; /* end of stream */ > - } else if (iproto_type_is_error(row.type)) { > - xrow_decode_error_xc(&row); /* rethrow error */ > - } else { > - tnt_raise(ClientError, ER_UNKNOWN_REQUEST_TYPE, > - (uint32_t) row.type); > + struct stailq rows; > + applier_read_tx(applier, &rows, &row_count); > + struct xrow_header *first_row = > + &stailq_first_entry(&rows, struct applier_tx_row, > + next)->row; > + if (first_row->type == IPROTO_OK) { > + assert(first_row == > + &stailq_last_entry(&rows, struct applier_tx_row, > + next)->row); > + break; > } > + if (apply_final_join_tx(&rows) != 0) > + diag_raise(); > } > > return row_count; > @@ -646,8 +613,11 @@ applier_read_tx_row(struct applier *applier) > * messages so we can't assume that if we haven't heard > * from the master for quite a while the connection is > * broken - the master might just be idle. > + * Also there are no timeouts during final join and register. > */ > - if (applier->version_id < version_id(1, 7, 7)) > + if (applier->version_id < version_id(1, 7, 7) || > + applier->state == APPLIER_FINAL_JOIN || > + applier->state == APPLIER_REGISTER) 2. Maybe it would be better to pass the timeout from the upper level and always use coio_read_xrow_timeout_xc(). For the mentioned conditions it would be infinity. Anyway the non-timed version in the end uses TIMEOUT_INFINITY too (coio_read_ahead). That way it would be less tricky conditions checks in the generic code. > coio_read_xrow(coio, ibuf, row); > else > coio_read_xrow_timeout_xc(coio, ibuf, row, timeout); > @@ -731,6 +702,9 @@ applier_read_tx(struct applier *applier, struct stailq *rows) > do { > struct applier_tx_row *tx_row = applier_read_tx_row(applier); > tsn = set_next_tx_row(rows, tx_row, tsn); > + > + if (row_count != NULL && ++*row_count % 100000 == 0) > + say_info("%.1fM rows received", *row_count / 1e6); 3. Hm. That adds branching and heavy '%' operation. Maybe you could make it return number of rows and in the caller code do this check + log. So it would affect only the joins. > @@ -1254,7 +1250,7 @@ applier_subscribe(struct applier *applier) > } > > struct stailq rows; > - applier_read_tx(applier, &rows); > + applier_read_tx(applier, &rows, NULL); > > /* > * In case of an heartbeat message wake a writer up >