Tarantool development patches archive
 help / color / mirror / Atom feed
From: Kirill Yukhin via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Serge Petrenko <sergepetrenko@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org
Subject: Re: [Tarantool-patches] [PATCH v2] recovery: make it yield when positioning in a WAL
Date: Thu, 13 May 2021 13:54:03 +0000	[thread overview]
Message-ID: <20210513135403.53lhd2sglmdciv3a@tarantool.org> (raw)
In-Reply-To: <20210512112928.12509-1-sergepetrenko@tarantool.org>

Hello,

On 12 май 14:29, Serge Petrenko via Tarantool-patches wrote:
> We had various places in box.cc and relay.cc which counted processed
> rows and yielded every now and then. These yields didn't cover cases,
> when recovery has to position inside a long WAL file:
> 
> For example, when tarantool exits without leaving an empty WAL file
> which'll be used to recover instance vclock on restart. In this case
> the instance freezes while processing the last availabe WAL in order
> to recover the vclock.
> 
> Another issue is with replication. If a replica connects and needs data
> from the end of a really long WAL, recovery will read up to the needed
> position without yields, making relay disconnect by timeout.
> 
> In order to fix the issue, make recovery decide when a yield should
> happen. Once recovery decides so, it calls a xstream callback,
> schedule_yield. Currently schedule_yield is fired once recovery
> processes (either skips or writes) WAL_ROWS_PER_YIELD rows.
> 
> schedule_yield either yields right away, in case of relay, or saves the
> yield for later, in case of local recovery, because it might be in the
> middle of a transaction.
> 
> Closes #5979
> ---
> https://github.com/tarantool/tarantool/tree/sp/gh-5979-recovery-yield
> https://github.com/tarantool/tarantool/issues/5979

I've checked your patch into 2.7, 2.8, master.

--
Regards, Kirill Yukhin

      parent reply	other threads:[~2021-05-13 13:54 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-12 11:29 Serge Petrenko via Tarantool-patches
2021-05-12 20:36 ` Vladislav Shpilevoy via Tarantool-patches
2021-05-13 10:11   ` Serge Petrenko via Tarantool-patches
2021-05-13 11:21     ` Vladislav Shpilevoy via Tarantool-patches
2021-05-13 13:37       ` Serge Petrenko via Tarantool-patches
2021-05-13 13:54 ` Kirill Yukhin via Tarantool-patches [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210513135403.53lhd2sglmdciv3a@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=kyukhin@tarantool.org \
    --cc=sergepetrenko@tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v2] recovery: make it yield when positioning in a WAL' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox