From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id DF6DC70349; Fri, 22 Oct 2021 09:36:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org DF6DC70349 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1634884587; bh=OxUTP4M1Jlw6BVik8KKfZHSon6DrpTq3TAE8RtLZpEI=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=KruiCrBHxI54rtk55od4VEp5LdvioOhvcY2obtJIhNRL5SqPrnwQFO3BCMmeoX9TW VgsA0+vqt7bG4B39lqeAr7q1OaEE7CGN9p+fTfrG5xV/p9Gp7bLoFL7vcfGBwp8AqH ZrSO/F+y57ZGEO1+gIbfTCro2xcj2z59URz0ArCw= Received: from smtp30.i.mail.ru (smtp30.i.mail.ru [94.100.177.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id F1B6470349 for ; Fri, 22 Oct 2021 09:36:24 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org F1B6470349 Received: by smtp30.i.mail.ru with esmtpa (envelope-from ) id 1mdoAB-0006pp-TE; Fri, 22 Oct 2021 09:36:24 +0300 To: Vladislav Shpilevoy , Cyrill Gorcunov Cc: tml References: <20211014215622.49732-1-gorcunov@gmail.com> <20211014215622.49732-4-gorcunov@gmail.com> <5ce44467-f9a9-0743-3394-c2a40cff463c@tarantool.org> Message-ID: <79f80140-8c2f-4c34-10ae-186982db5ef1@tarantool.org> Date: Fri, 22 Oct 2021 09:36:23 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9C7814344C8C501C81DF2D982FCC3642ABC592EA95DC3FE9F182A05F538085040E3782BB67E02D04F14F7449BDFBCF727AD6D3401C3F93E041ABEA34E8353F01B X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE75CCA5FD4E6ACD987EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637C3BF94FB392044A18638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8BDEAC2EB727FC1653FB4EBD540CC39B7117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC63AF70AF8205D7DCA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520CCD848CCB6FE560CF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C5EAF9D062D4E28432D242C3BD2E3F4C64AD6D5ED66289B52698AB9A7B718F8C46E0066C2D8992A16725E5C173C3A84C39396EF034D1BBB17BA3038C0950A5D36B5C8C57E37DE458B0BC6067A898B09E46D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE79E9721B410A3B6ED731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B5059E6111B1B64AE5F69FA19E2D67A511AF X-C1DE0DAB: 0D63561A33F958A5698A62E072F8BED1404CB8EA5B3BA5444F83C32EE16D7707D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75FA7FF33AA1A4D21C410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340CB2836B82369449BEC5E970889C46D7C0BC0A252E3FB770773262292CABA43FDD245D396AF620AC1D7E09C32AA3244CAF572E88FDBFE25B5532102A6ADF9DEFF522A1CF68F4BE05FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojXelGRflEPnDg6tHi0tQRug== X-Mailru-Sender: 11C2EC085EDE56FA38FD4C59F7EFE4072F7F296023D40FA1CF0E0D16ECFD396684A8476422EC601E6BB2E709EA627F343C7DDD459B58856F0E45BC603594F5A135B915D4279FF0579437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v23 3/3] test: add gh-6036-qsync-order test X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 22.10.2021 01:06, Vladislav Shpilevoy пишет: >>>> Actually you do need to count writes here. >>>> The wait_cond for ERRINJ_WAL_WRITE_COUNT == write_cnt + 3 >>>> is needed to make sure you receive (and thus try to process) >>>> insert {3} **before** the replica is re-enabled. >>>> >>>> Otherwise we can't be sure that the test is correct. You may simply >>>> perform a select before insert{3} has reached the replica. >>> You know, I spent a few hours trying to pass the test waiting for >>> ERRINJ_WAL_WRITE_COUNT == write_cnt + 3 and finally realized that >>> it seems that is what happens: the replica1 is not longer a leader >>> and when this record reach our replica3 node we NOPify it then >>> we run >>> >>> apply_row >>>    if (request.type == IPROTO_NOP) >>>      return process_nop() >>> >>> thus this record even not reaching the journal at all and that is >>> why waiting for write_cnt + 3 lasts forever. If only I didn't miss >>> something obvious. >> Unfortunately, this is not the case. A NOP entry still reaches WAL. >> That's why we need NOP entries: they reside in WAL but do nothing. >> That's for vclock bump sake. Otherwise we could skip such entries >> completely, without nopifying them. >> >> So, even if the entry is nopified, it would enter WAL sooner or later. >> >> I just realised what the problem is: the entry is waiting on a limbo latch >> inside the NOPify procedure. That's why it never reaches the journal >> (until we re-enable replica3, at least). >> >> I don't know how to wait for this entry's arrival then. >> The current test version looks OK to me. >> >> Vlad, do you have any ideas here? > I think it might worth adding an errinj for the number of blocked > fibers waiting on the limbo latch. Could even expose that to box.info.qsync, > seems like useful info. Would help to measure contention. This might be useful, indeed. Cyrill, let's implement `box.info.synchro.queue.waiters` then and use it in the test. Or any other suitable name if you guys come up with one. -- Serge Petrenko