Re: [tarantool-patches] Re: [PATCH 03/10] vinyl: move vylog recovery to vylog thread

[Konstantin Osipov <kostja@tarantool.org>]

* Vladimir Davydov <vdavydov.dev@gmail.com> [19/06/06 13:24]:
> > > We used coio, because vylog was written from a WAL thread, which
> > > shouldn't be used for such a heavy operation as vylog recovery.
> > > Now, we can move it to the dedicated vylog thread. This allows
> > > us to simplify rotation logic as well: now most of work is done
> > > from the same function (vy_log_rotate_f) executed by vylog thread,
> > > not scattered between coio and WAL, as it used to be.
> > 
> > Why do we need to lock out the scheduler while rotating the log in
> > the first place? 
> 
> We rotate vylog by first reading the old vylog and forming a recovery
> context, then dumping this recovery context to the new vylog. If a new
> record appears in the old vylog in between, it will be missing in the
> new vylog. That's why we lock out writers.

We have two layers of abstractions intermixed here. During
snapshotting, when we really rotate the vylog, no DDL can happen,
it's locked out. So no one can take the problematic latch
anyway. So there is, strictly speaking, no problem at all. But
since we're using a low level latch, and not a centralized
mechanism to lock out writers, we wouldn't know.

One option could be to append the writes to vylog which happen
during checkpointing to the vylog buffer, and not flush them to
the vylog file which is about-to-become-obsolete.

Anyway, I keep thinking that if you want to kill a latch, there is
a dozen of ways of killing it, not an own thread.


-- 
Konstantin Osipov, Moscow, Russia