Tarantool development patches archive
 help / color / mirror / Atom feed
From: Nikita Pettik <korablev@tarantool.org>
To: Aleksandr Lyapunov <alyapunov@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH] vinyl: bump dump_generation in case scheduler doesn't catch up with DDL
Date: Wed, 10 Jun 2020 21:41:13 +0000	[thread overview]
Message-ID: <20200610214113.GA17509@tarantool.org> (raw)
In-Reply-To: <43322a95-0fa3-fd8a-e77e-fc73a4bfa585@tarantool.org>

On 10 Jun 18:31, Aleksandr Lyapunov wrote:
> Thanks for the patch, lgtm.
> Actually it's a bit complicated, as well vy_scheduler itself.

Pushed to master, 2.4, 2.3 and 1.10. Changelogs are updated
correspondingly. Branch is dropped.
 
> > It may turn out that dump_generation does not catch up with current
> > generation and no other dump tasks are in progress. This may happen
> > dump process is throttled due to errors. In this case generation is
> > bumped but dump_generation is not (since dump is not completed). In
> > turn, throttling opens a window for DDL operations. For instance, index
> > dropping and creation of new one results in mentioned situation:
> > 
> > box.snapshot() -- fails for some reason; next attempt at dumping will be
> >                 -- taken in one second.
> > s:drop() -- drop index to be dumped
> > s = box.schema.space.create('test', {engine = 'vinyl'})
> > -- create new one (its mem generation is greater than scheduler's one)
> > i = s:create_index('pk')
> > 
> > Closes #4821
> > ---
> > Note that current fix is only workaround amd doesn't pretend to be
> > the most proper one. Alternatively, we can rollback scheduler
> > generation in case dump is failed and index to be dumped is dropped.
> > 
> > Branch:https://github.com/tarantool/tarantool/tree/np/gh-4821-scheduler-gen-assert
> > Issue:https://github.com/tarantool/tarantool/issues/4821
> > 
> >   src/box/vy_scheduler.c                        | 14 ++++-
> >   .../gh-4821-ddl-during-throttled-dump.result  | 57 +++++++++++++++++++
> >   ...gh-4821-ddl-during-throttled-dump.test.lua | 22 +++++++
> >   test/vinyl/suite.ini                          |  2 +-
> >   4 files changed, 92 insertions(+), 3 deletions(-)
> >   create mode 100644 test/vinyl/gh-4821-ddl-during-throttled-dump.result
> >   create mode 100644 test/vinyl/gh-4821-ddl-during-throttled-dump.test.lua
> > 
> > diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c
> > index bf4c3fe58..880304b66 100644
> > --- a/src/box/vy_scheduler.c
> > +++ b/src/box/vy_scheduler.c
> > @@ -1846,9 +1846,19 @@ retry:
> >   	/*
> >   	 * Dump is in progress, but all eligible LSM trees are
> >   	 * already being dumped. Wait until the current round
> > -	 * is complete.
> > +	 * is complete. But if there's no any other tasks
> > +	 * in progress, it may mean that dump_generation does
> > +	 * not catch up with current generation. This may happen
> > +	 * due to failed dump process (i.e. generation is bumped
> > +	 * but dump_generation is not). In turn, after dump is failed,
> > +	 * next dump will be throttled which opens a window for DDL
> > +	 * operations. For instance, index dropping and creation of
> > +	 * new one results in mentioned situation.
> >   	 */
> > -	assert(scheduler->dump_task_count > 0);
> > +	if (scheduler->dump_task_count == 0) {
> > +		scheduler->dump_generation = vy_lsm_generation(lsm);
> > +		goto retry;
> > +	}
> >   no_task:
> >   	if (worker != NULL)
> >   		vy_worker_pool_put(worker);
> 

  reply	other threads:[~2020-06-10 21:41 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-10 15:31 Aleksandr Lyapunov
2020-06-10 21:41 ` Nikita Pettik [this message]
2020-06-11  1:40   ` Nikita Pettik
  -- strict thread matches above, loose matches on Subject: below --
2020-03-31  0:47 Nikita Pettik
2020-04-03  0:21 ` Vladislav Shpilevoy
2020-04-03  1:25   ` Nikita Pettik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200610214113.GA17509@tarantool.org \
    --to=korablev@tarantool.org \
    --cc=alyapunov@tarantool.org \
    --cc=tarantool-patches@dev.tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH] vinyl: bump dump_generation in case scheduler doesn'\''t catch up with DDL' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox