From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id AD9776EC5B; Tue, 30 Mar 2021 00:51:42 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org AD9776EC5B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617054702; bh=fhaGuBWtnxEJJVxGlMofADy1BAROjj3+/xrcLnDaU4E=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=OXVBm4BH3D7Nw7C9oAr6B7CAoSjxQb03+I6AInkW/VPfLne0IAj7AEUOJ9IAOoDIH RqegYbU20kw1JmqY33DvpW1+NoMioXVZDmOof/jkdpO4Uq+4BsdOienU8S7+ZqtSsm 1fDsA4OghkWU8mn9rWrk7RWrZK9cfj97bsPkuFk4= Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id ACC596EC60 for ; Tue, 30 Mar 2021 00:51:05 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org ACC596EC60 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1lQzmq-0002C3-Sg; Tue, 30 Mar 2021 00:51:05 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <5cfa8f8e4d733aabd53138dd3ffc6f524ffac743.1616588119.git.sergepetrenko@tarantool.org> <63555ea6-6f0a-964e-65e2-6213eecb2ea1@tarantool.org> Message-ID: <30275fb1-4797-6923-c2a0-17e670720e65@tarantool.org> Date: Mon, 29 Mar 2021 23:51:03 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <63555ea6-6f0a-964e-65e2-6213eecb2ea1@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9ED7173E37F4E32947A0146560F8BA70927CAA5B950F38D9F182A05F5380850405F0139DFB2034C0317B71A5B285F4C08F04A67E3C2944B90D1AEAE6F736475CA X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE785DAC9A352760686C2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE77806F8AEFB2C8BE8EA1F7E6F0F101C67CDEEF6D7F21E0D1D174C73DBBBFC76648736E98A3ECD1341166FDBAC99392F2035E465D1C89C1EA6389733CBF5DBD5E913377AFFFEAFD269A417C69337E82CC2CC7F00164DA146DAFE8445B8C89999729449624AB7ADAF37F6B57BC7E64490611E7FA7ABCAF51C92176DF2183F8FC7C0D9442B0B5983000E8941B15DA834481F9449624AB7ADAF372E808ACE2090B5E14AD6D5ED66289B5259CC434672EE6371089D37D7C0E48F6C8AA50765F7900637AD0424077D726551EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F60894D097F901B7558688B39FA754255D0DFB8CCF60F6B7FD3948E1CD14B953EB46D29663CB11026F891355D89D7DBCDD132 X-C1DE0DAB: 0D63561A33F958A52EE6DA258BF3A414C2B6B7858C34F0CBBD657066E8A13EFBD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34BF5454112BD5BFD76E8819AD55F88C69F59B4E60FC7EE8D161FE6C913DDC42042F7EED5552E43C221D7E09C32AA3244CDC672DBD4D9BA3E03FD89F2EBF2496077C0C08F7987826B9FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojljIiQOC84rQyTMpvq5yLiw== X-Mailru-Sender: 689FA8AB762F73936BC43F508A0638220520B22001EFACB5525928D52B31D6323841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 5/7] applier: make final join transactional X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Good job on the fixes! See 4 comments below. > ================================= > diff --git a/src/box/applier.cc b/src/box/applier.cc > index b96eb360b..0f4492fe3 100644 > --- a/src/box/applier.cc > +++ b/src/box/applier.cc > @@ -497,6 +496,7 @@ struct applier_tx_row { >  static uint64_t >  applier_wait_register(struct applier *applier, uint64_t row_count) >  { > +#define ROWS_PER_LOG 100000 1. Better avoid in-function macro. This can be done as 'const uint64_t' or as a enum in the beginning of the file. >      /* >       * Tarantool < 1.7.0: there is no "final join" stage. >       * Proceed to "subscribe" and do not finish bootstrap > @@ -505,16 +505,23 @@ applier_wait_register(struct applier *applier, uint64_t row_count) >      if (applier->version_id < version_id(1, 7, 0)) >          return row_count; > > +    uint64_t next_log_cnt = > +        row_count + ROWS_PER_LOG - row_count % ROWS_PER_LOG; >      /* >       * Receive final data. >       */ >      while (true) { >          struct stailq rows; > -        applier_read_tx(applier, &rows, &row_count); > +        row_count += applier_read_tx(applier, &rows, TIMEOUT_INFINITY); > +        if (row_count >= next_log_cnt) { > +            say_info("%.1fM rows received", next_log_cnt / 1e6); > +            next_log_cnt += ROWS_PER_LOG; 2. What if row_count > ROWS_PER_LOG? Then it would be printed on the next transaction immediately again (although I don't know if it is possible to have such a big transaction). > +        } >          struct xrow_header *first_row = >              &stailq_first_entry(&rows, struct applier_tx_row, >                          next)->row; >          if (first_row->type == IPROTO_OK) { > +            /* Current vclock. This is not used now, ignore. */ >              assert(first_row == >                     &stailq_last_entry(&rows, struct applier_tx_row, >                            next)->row); > @@ -1234,6 +1229,15 @@ applier_subscribe(struct applier *applier) >          trigger_clear(&on_rollback); >      }); > > +    /* > +     * Tarantool < 1.7.7 does not send periodic heartbeat > +     * messages so we can't assume that if we haven't heard > +     * from the master for quite a while the connection is > +     * broken - the master might just be idle. > +     */ > +    double timeout = applier->version_id < version_id(1, 7, 7) ? > +             TIMEOUT_INFINITY : replication_disconnect_timeout(); 3. What if replication_timeout is changed after first box.cfg{}? It seems it won't affect the running appliers now, will it? > + >      /* >       * Process a stream of rows from the binary log. >       */ <...> > +/** A simpler version of applier_apply_tx() for final join stage. */ > +static int > +apply_final_join_tx(struct stailq *rows) > +{ > + struct xrow_header *first_row = > + &stailq_first_entry(rows, struct applier_tx_row, next)->row; > + struct xrow_header *last_row = > + &stailq_last_entry(rows, struct applier_tx_row, next)->row; > + int rc = 0; > + /* WAL isn't enabled yet, so follow vclock manually. */ > + vclock_follow_xrow(&replicaset.vclock, last_row); > + if (unlikely(iproto_type_is_synchro_request(first_row->type))) { > + assert(first_row == last_row); > + rc = apply_synchro_row(first_row); > + goto end; > + } 4. You don't really need the 'end' label here: ==================== --- a/src/box/applier.cc +++ b/src/box/applier.cc @@ -970,11 +970,9 @@ apply_final_join_tx(struct stailq *rows) if (unlikely(iproto_type_is_synchro_request(first_row->type))) { assert(first_row == last_row); rc = apply_synchro_row(first_row); - goto end; + } else { + rc = apply_plain_tx(rows, false, false); } - - rc = apply_plain_tx(rows, false, false); -end: fiber_gc(); return rc; }