Re: [tarantool-patches] Re: [PATCH 10/10] vinyl: get rid of the latch protecting vylog buffer

[Vladimir Davydov <vdavydov.dev@gmail.com>]

On Sat, Jun 01, 2019 at 11:44:36AM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [19/05/17 17:54]:
> > The latch is needed solely to synchronize access to the transaction
> > write buffer, which is shared by all transactions. We don't need it to
> > sync vylog readers vs writers as everything is done from the same
> > thread. So to get rid of the latch, which is a prerequisite for
> > transactional DDL, as it makes even simply Vinyl DDL operations
> > yielding, we need to rework the buffer management.
> > 
> > To achieve that, this patch introduces a separate object for each vylog
> > transaction, which stores the list of records to be committed. Once a
> > transaction is ready to be committed, it is sent to the vylog thread,
> > which writes it to vylog and then wakes up the awaiting fiber. For
> > non-discardable transactions, it doesn't wakeup any fiber, the fiber
> > returns immediately and the transaction is freed automatically.
> > 
> > To allow that, transactions are now allocated on lsregion, which acts
> > as a queue: new transactions are appended to the tail, committed
> > transactions are popped from the head.
> 
> The same could be achieved with xlog savepoints that you
> introduced earlier, if you allow multiple "savepoints" to be present 
> in xlog tx, each of which could be rolled back independently (the
> tail of the buffer would have to be moved on rollback).

Before writing a transaction to the xlog buffer, we need to accumulate
its statements in a buffer in the tx thread. We can either malloc() them
and then free(), but I considered it to have too much overhead. So I
decided to allocate all vylog statements using the same allocator -
lsregion - acting as a FIFO. Possibly, I'm wrong about it, and using
malloc() is just fine.

> 
> I mean, honestly, if we're getting rid of the latch, there is no
> point of having adding a separate thread on the way, it just takes
> a little extra effort. And better yet, we could kill entire vy_log
> altogether and store everything in xlog.

We need to discuss that f2f, I think.