From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 486FC70349; Fri, 22 Oct 2021 01:06:40 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 486FC70349 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1634854000; bh=4X5d3FNTsmZJfedubDhD8V1EkwEGyM+iyuwragZ48RY=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=W1ptuY+QKMf8mls0v0Sd/8GHlIZoAP99Q5b1zCKzHGY2jPpXtAxtJM458f7CuwoRP uikfK47w9K1dHOrZOhiwR5lJLoNui9RfwIKG4ygJr2wO8kueUKEFgJipdwpSG3huy6 KSBBDmns01y2hcw9y2RCVJytr6wgklUP0aLlfoEg= Received: from smtp58.i.mail.ru (smtp58.i.mail.ru [217.69.128.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 62C0F71057 for ; Fri, 22 Oct 2021 01:06:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 62C0F71057 Received: by smtp58.i.mail.ru with esmtpa (envelope-from ) id 1mdgCP-0001t5-E4; Fri, 22 Oct 2021 01:06:09 +0300 Message-ID: Date: Fri, 22 Oct 2021 00:06:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Content-Language: en-US To: Serge Petrenko , Cyrill Gorcunov Cc: tml References: <20211014215622.49732-1-gorcunov@gmail.com> <20211014215622.49732-4-gorcunov@gmail.com> <5ce44467-f9a9-0743-3394-c2a40cff463c@tarantool.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9C7814344C8C501C83E7F929BE7EA671116302D09E268010D182A05F53808504027D03993998CB1ECA9652420AEA5AE0F6DF9AADB6DCA9DB2F857DCDD307606F5 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7233EAFDDCC869EB0EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637A6A20D80F0832BC78638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D854349398434595F5EFC2A614D0A747B7117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC081CF0AE924DC023A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520CCD848CCB6FE560CF04B652EEC242312D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE41BF15D38FB6CB3A28765F5520A300B2D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE360910C30DCD593B1C0837EA9F3D19764C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F79006377AA2284B41911753EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79D6FCB372DF57C6C43292F599E5B1FD2EA X-C1DE0DAB: 0D63561A33F958A571077E105E798F9657F648A4BADA93F52C26B6569935EE19D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75FA7FF33AA1A4D21C410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34197948450CB5442A0ADD3F37AE57DD78827F4494F70D7CA9648088D2AF936A16824A62A31AD64A051D7E09C32AA3244C34928172DCC9E9840819FAC3594ECC2A7101BF96129E4011FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojXelGRflEPnA8fYRMs/Lk2g== X-Mailru-Sender: 1F3202E75A95DDEF0AE1703729B258988462B1BDEC572CD9907567AC1084B199A687CD3158AEA01E07784C02288277CA03E0582D3806FB6A5317862B1921BA260ED6CFD6382C13A6112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v23 3/3] test: add gh-6036-qsync-order test X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" >>> Actually you do need to count writes here. >>> The wait_cond for ERRINJ_WAL_WRITE_COUNT == write_cnt + 3 >>> is needed to make sure you receive (and thus try to process) >>> insert {3} **before** the replica is re-enabled. >>> >>> Otherwise we can't be sure that the test is correct. You may simply >>> perform a select before insert{3} has reached the replica. >> You know, I spent a few hours trying to pass the test waiting for >> ERRINJ_WAL_WRITE_COUNT == write_cnt + 3 and finally realized that >> it seems that is what happens: the replica1 is not longer a leader >> and when this record reach our replica3 node we NOPify it then >> we run >> >> apply_row >>    if (request.type == IPROTO_NOP) >>      return process_nop() >> >> thus this record even not reaching the journal at all and that is >> why waiting for write_cnt + 3 lasts forever. If only I didn't miss >> something obvious. > > Unfortunately, this is not the case. A NOP entry still reaches WAL. > That's why we need NOP entries: they reside in WAL but do nothing. > That's for vclock bump sake. Otherwise we could skip such entries > completely, without nopifying them. > > So, even if the entry is nopified, it would enter WAL sooner or later. > > I just realised what the problem is: the entry is waiting on a limbo latch > inside the NOPify procedure. That's why it never reaches the journal > (until we re-enable replica3, at least). > > I don't know how to wait for this entry's arrival then. > The current test version looks OK to me. > > Vlad, do you have any ideas here? I think it might worth adding an errinj for the number of blocked fibers waiting on the limbo latch. Could even expose that to box.info.qsync, seems like useful info. Would help to measure contention.