From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id BE0D76EC55; Mon, 12 Jul 2021 19:49:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org BE0D76EC55 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1626108591; bh=haXM/FAgwgYt6GGzHI5O/r/LKCGmNwaiM01H74lCWnQ=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=nxvi6/JOkuG3eVS0rQxSTHUSZofrTMYtDuFT6+r/8WvcQzYXSyfA0wVOMmBGPoccs w2f1Vw5cQjs3VhZaBIh5XT4vY1jOyxk9OTQV87CRiFkZZ6OkElIQCjp4/0EUEGrWvb 0S2mr3HIbsCQUT6QNtYDndP6FaVu+kTN6tqM8M1E= Received: from smtp48.i.mail.ru (smtp48.i.mail.ru [94.100.177.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 95E4D6EC55 for ; Mon, 12 Jul 2021 19:49:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 95E4D6EC55 Received: by smtp48.i.mail.ru with esmtpa (envelope-from ) id 1m2z7t-0004NJ-T0; Mon, 12 Jul 2021 19:49:50 +0300 To: Cyrill Gorcunov , Vladislav Shpilevoy Cc: tml References: <20210710222803.253251-1-gorcunov@gmail.com> <187d1ae2-99cb-50d4-d5b4-18aa6c5f5546@tarantool.org> Message-ID: Date: Mon, 12 Jul 2021 19:49:49 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD97BB0EF39AD2B33D598226807B8A1E9DC331E3F9997896515182A05F538085040A6748E9665E22467206F9E3DA2B077A61266748B86859603D234DBF727385630 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B05688F02FB57514EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637188F5654332B449D8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8BD684200E3DB57F97F7F9A1A01042FBC117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8B2EE5AD8F952D28FBA471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FCC3C92DCC868ACD353AA81AA40904B5D9CF19DD082D7633A078D18283394535A93AA81AA40904B5D98AA50765F7900637BECDE000F464C10CD81D268191BDAD3D698AB9A7B718F8C4D1B931868CE1C5781A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89FB26E97DCB74E6252262FEC7FBD7D1F5BB5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A56C17BC636A4061D06D1A066C7D2FB4D6210DE6CF746ED828D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA753753CEE10E4ED4A7410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D347215713CF3EEE9D194FF30B1739FFD0CEDCEBB89D9E2AF617DE98884586DF46A76282DEE8EF5CCC11D7E09C32AA3244C985C78D90E6576DAF2132D8B5D986A7DB038C9161EF167A1FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojAZDAgpmGsvaHzjDTJIT9TQ== X-Mailru-Sender: 583F1D7ACE8F49BDCE9F948DA3B7A95332D3455E9CC2B92D37DB97ECDA2BAF7394445843EECF69456BB2E709EA627F343C7DDD459B58856F0E45BC603594F5A135B915D4279FF0579437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH] limbo: introduce request processing hooks X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 12.07.2021 18:48, Cyrill Gorcunov пишет: > Guys, there are some moments even in current code structure which > looks somehow strange so I would like to discuss them. Lets consider > the following case: one replic sends us a promote request (assume > we're sitting in term 2 and max term is 2). > > applier 1 > --------- > applier_apply_tx > (promote term = 3 > current max term = 2) > applier_synchro_filter_tx > apply_synchro_row > journal_write > (sleeping) > > at this moment another applier comes in with obsolete > data and term 2 > applier 2 > --------- > applier_apply_tx > (term 2) > applier_synchro_filter_tx > txn_limbo_is_replica_outdated -> false > journal_write (sleep) > > applier 1 > --------- > journal wakes up > apply_synchro_row_cb > set max term to 3 > return to applier read and > applier 2 could finish its write > and wake up too > > at this moment the data from applier 2 is actually queued > for write as valid but we just wrote the term 3, so if we would > had been updating terms map earlier (before jornal write) the data > from applier 2 should be NOPified. I think there is some problem > because due to journal write lag the data is not as it could be > if terms map updated early. Serge, Vlad, am I right? Which consequences > can be here? IOW, should not we track terms earlier even without > my filter series? Looks like a bug, indeed. We may either introduce a limbo latch or start tracking terms before the WAL write. I'm starting to like the idea with limbo latch more. It's come up a couple of times already for various occasions, so maybe it's time to finally implement it. -- Serge Petrenko