From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id B0C356F865; Thu, 29 Apr 2021 23:03:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B0C356F865 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1619726637; bh=0nGOQiZ9RDqJpfMAe5TFKjtqfM3PLQAwNLpFOahcqf4=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=L232ypn0a76dnsDk/0GhmxMlXQh4Rrd4t/NoM52BXxzZ5aLWrfMQV/9vc8yRODC0w 7NAYC4gI6HxUZeAemJ3uNP8Ssk9G4Oxomp+a9kqhph6harAQSzV2NQ/Fizubo2IZUX oB9Y4Kjj+vhCxEfMVm1axMksRddtaMYNy6HnrWjU= Received: from smtp57.i.mail.ru (smtp57.i.mail.ru [217.69.128.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id A2C896F865 for ; Thu, 29 Apr 2021 23:03:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A2C896F865 Received: by smtp57.i.mail.ru with esmtpa (envelope-from ) id 1lcCt9-0000zd-ST; Thu, 29 Apr 2021 23:03:56 +0300 To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org References: <20210426165954.46474-1-sergepetrenko@tarantool.org> <738d3030-80cf-8079-1b03-55a7d665dbbf@tarantool.org> <3d0997f3-066f-5b2c-3ae3-22b90583e9bf@tarantool.org> Message-ID: <70224f08-737b-db3c-b809-1472ed603568@tarantool.org> Date: Thu, 29 Apr 2021 22:03:54 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <3d0997f3-066f-5b2c-3ae3-22b90583e9bf@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD95978C26455E69BE0890FB394E039F71AA2C815CBBA0C08E7182A05F53808504043EEF401D1F9BC9432B417BED1E80D169EF5B4DDAB7CE18194E1D47136CF155B X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE781307CBDB76B677BEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063750965CC5CDB672DE8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B215C1760DC1354860AE0F078BC706CDB10FE69BD46C3DD557D2E47CDBA5A96583C09775C1D3CA48CF329DC2EF12373DEA117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE7ECC8AC47CD0EDEFF8941B15DA834481F9449624AB7ADAF37BA3038C0950A5D3613377AFFFEAFD2691661749BA6B977351889BC216C0780097B076A6E789B0E97A8DF7F3B2552694A1E7802607F20496D49FD398EE364050F26055571C92BF10F28765F5520A300B2B3661434B16C20AC6FCAC8D36C5C0985E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B851EDB9C5A93305EEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A24209795067102C07E8F7B195E1C97831064AC8DC0C36F9B166BC45FBD0D174ED X-C1DE0DAB: 0D63561A33F958A556A2EB179C7FD90F86B374044C8AE3AEDD33F19CA3AF695ED59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D345DB600F8E858000FAA447732F048FC1EA32E7474D137E52A84598ED4208FA974F1393AFDF090BFA91D7E09C32AA3244C22D801DE8CAA02D27C504CB165D0D3A0F165894D92D62706FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojVCmWsvwT1HYwatySmpbuAw== X-Mailru-Sender: 504CC1E875BF3E7D9BC0E5172ADA3110B04C22393BAA2F962EE33BA7D3FF44FBDE35B75AAAF117FD07784C02288277CA03E0582D3806FB6A5317862B1921BA260ED6CFD6382C13A6112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH] recovery: make it yield when positioning in a WAL X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thanks for the patch! Technically looks good now, can be pushed. But in case you would find it interesting: now relay->row_count looks odd. Because its comment is not very correct, most of that is moved to the recovery layer. And because it is used only by initial join. And because we still have a single place where we use xstream/recovery, but the yields are not managed by them. Did you think about moving your code to xstream instead of recovery? Then it would be truly not related to recovery. It would have 2 methods: - write(struct xstream *, struct xrow_header *) - skip(struct xstream *, struct xrow_header *) Recovery calls skip() while establishes a position, for each skipped row. Relaying from memory never calls skip, but calls only write() like now. In order to yield you inherit xtream, add a member row_count, and both in skip() and write() increment it like you do. And then yield periodically. It would work for the initial join too. Or add row_count right to the xtream struct, and replace skip() with your function schedule_yield(). This way row_count would be visible to recovery and available for reporting in the logs like '... rows recovered'. And we wouldn't all too many virtual functions for the skipped rows. All that up to you.