From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 536C26EC5B; Thu, 13 May 2021 16:54:09 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 536C26EC5B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1620914049; bh=IfKeXYQuVDdFjrH/YHCQQtZO/0zgJDdVH3pZdEIU5Sc=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=F3a7pswKdalmejlZTcQJtfRPOfffspFelXMCpnaj4KBn2D2I5N5rkl9T6wEGPgyJ9 btnMEK8IyT0R9294auUki+0lK8rvLZJGyAB6fvBl8Erjqj2NMjnRWSDMC04fQsmbl/ JwtB+dXT/ReDa6WC5hbRH3nbpopt23AC/rQjLT2E= Received: from smtp45.i.mail.ru (smtp45.i.mail.ru [94.100.177.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 7258D6EC5B for ; Thu, 13 May 2021 16:54:07 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7258D6EC5B Received: by smtp45.i.mail.ru with esmtpa (envelope-from ) id 1lhBmw-0006dT-Ia; Thu, 13 May 2021 16:54:06 +0300 Date: Thu, 13 May 2021 13:54:03 +0000 To: Serge Petrenko Message-ID: <20210513135403.53lhd2sglmdciv3a@tarantool.org> References: <20210512112928.12509-1-sergepetrenko@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20210512112928.12509-1-sergepetrenko@tarantool.org> User-Agent: NeoMutt/20170113 (1.7.2) X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD95978C26455E69BE0FB16CA32063744B94D060C3A2F6991A4182A05F5380850402CEA986A5C2A2A81112E434AC923D956D615538EC8EAFAF6D8E71D912D78D6FB X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE73230F712CF4B3924EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637F7C2A16C2A438FE58638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2F6B92FFCE666A312B5DAB013BACB1E8E42232221D1CC1427D2E47CDBA5A96583C09775C1D3CA48CF1E02C13459908652117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE7DEC8C2C8BCD2534D8941B15DA834481F9449624AB7ADAF37BA3038C0950A5D3613377AFFFEAFD269176DF2183F8FC7C005266751EC4C296E7B076A6E789B0E97A8DF7F3B2552694A1E7802607F20496D49FD398EE364050FB1593CA6EC85F86D452896749CDDA0A6B3661434B16C20AC78D18283394535A9E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B9149C560DC76099D75ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: 0D63561A33F958A533586D0FE9A2467A9752EFF782A908CCEE263289BB5DC15ED59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75F04B387B5D7535DE410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3414C91BA6844B96CADEE1CFF1542C8C2436D57CB9AB4BEFCFC89081D8ED1318D75CAE06BAF15D11A31D7E09C32AA3244CB12D9FD23F010E57519361FB19A7E380E3D93501275E802F927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojoybArHp+PQVvIGvMiJgyMw== X-Mailru-Sender: 05EB39F83D09414F9B5D7F52D7CDFDD268B8869888507B192D9F5AA3EC54AFA83D507E560184A85FCA16B95394F0DD5CE99530A0C0F27B5268329DCED823713783C0E760C018FF54112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2] recovery: make it yield when positioning in a WAL X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Kirill Yukhin via Tarantool-patches Reply-To: Kirill Yukhin Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hello, On 12 май 14:29, Serge Petrenko via Tarantool-patches wrote: > We had various places in box.cc and relay.cc which counted processed > rows and yielded every now and then. These yields didn't cover cases, > when recovery has to position inside a long WAL file: > > For example, when tarantool exits without leaving an empty WAL file > which'll be used to recover instance vclock on restart. In this case > the instance freezes while processing the last availabe WAL in order > to recover the vclock. > > Another issue is with replication. If a replica connects and needs data > from the end of a really long WAL, recovery will read up to the needed > position without yields, making relay disconnect by timeout. > > In order to fix the issue, make recovery decide when a yield should > happen. Once recovery decides so, it calls a xstream callback, > schedule_yield. Currently schedule_yield is fired once recovery > processes (either skips or writes) WAL_ROWS_PER_YIELD rows. > > schedule_yield either yields right away, in case of relay, or saves the > yield for later, in case of local recovery, because it might be in the > middle of a transaction. > > Closes #5979 > --- > https://github.com/tarantool/tarantool/tree/sp/gh-5979-recovery-yield > https://github.com/tarantool/tarantool/issues/5979 I've checked your patch into 2.7, 2.8, master. -- Regards, Kirill Yukhin