[tarantool-patches] Re: [PATCH 05/10] vinyl: don't purge deleted runs from vylog on compaction

[Vladimir Davydov]

On Sat, Jun 01, 2019 at 11:39:24AM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev at gmail.com> [19/05/17 17:54]:
> > After compacting runs, we first mark them as dropped (VY_LOG_DROP_RUN),
> > then try to delete their files unless they are needed for recovery from
> > the checkpoint, and finally mark them as not needed in the vylog
> > (VY_LOG_FORGET_RUN). There's a potential race sitting here: the problem
> > is the garbage collector might kick in after files are dropped, but
> > before they are marked as not needed. If this happens, there will be
> > runs that have two VY_LOG_FORGET_RUN records, which will break recovery:
> > 
> >   Run XX is forgotten, but not registered
> > 
> > The following patches make the race more likely to happen so let's
> > eliminate it by making the garbage collector the only one who can mark
> > runs as not needed (i.e. write VY_LOG_FORGET_RUN record). There will
> > be no warnings, because the garbage collector silently ignores ENOENT
> > errors, see vy_gc().
> > 
> > Another good thing about this patch is that now we never yield inside
> > a vylog transaction, which makes it easier to remove the vylog latch
> > blocking implementation of transactional DDL.
> > ---
> 
> This is also a good one, LGTM.

Pushed to master, 2.1, 1.10.