[Tarantool-patches] [RFC] Quorum-based synchronous replication

[Konstantin Osipov]

* Sergey Ostanevich <sergos at tarantool.org> [20/05/12 19:43]:

> > 1) It's unclear what happens here if async tx follows a sync tx.
> >    Does it wait for the sync tx? This reduces availability for
> 
> Definitely yes, unless we keep the 'dirty read' as it is at the moment
> in memtx. This is the essence of the design, and it is temporary until 
> the MVCC similar to the vinyl machinery appears. I intentionally didn't
> include this big task into this RFC. 
> 
> It will provide similar capabilities, although it will keep only
> dependent transactions in the undo log. Also, it looks like it will fit
> well into the machinery of this RFC. 

= reduced availability for all who have at least one sync space.

If different spaces have different quorum size = quorum size of
the biggest group is effectively used for all spaces.

Replica-local transactions, e.g. those used by vinyl compaction, 
are rolled back if there is no quorum.

What's the value of this?


> 
> >    async txs - so it's hardly acceptable. Besides, with
> >    group=local spaces, one can quickly run out of memory for undo.
> >    
> > 
> > Then it should be allowed to proceed and commit.
> > 
> > Then mixing sync and async tables in a single transaction
> > shouldn't be allowed.
> > 
> > Imagine t1 is sync and t2 is async. tx1 changes t1 and t2, tx2
> > changes t2. tx1 is not confirmed and must be rolled back. But it can
> > not revert changes of tx2.
> > 
> > The spec should clarify that.

You conveniently skip this explanation of the problem - meaning
you don't intend to address it?

> > 
> > 3) One can quickly run out of memory for undo. Any sync
> >    transaction should be capped with a timeout to avoid OOMs. I
> >    don't know how many times I should repeat it. The only good
> >    solution for load control is in-memory WAL, which will allow to
> >    rollback all transactions as soon as network partitioning is
> >    detected.
> 
> How in-memry WAL can help save on _undo_ memory? 
> To rollback whatever amount of transactions one need to store the undo. 

I wrote earlier that it works as a natural failure detector and
throttling mechanism. If
there is no quorum, we can see it immediately by looking at the
number of active subscribers of the in-memory WAL, so do not
accumulate undo.

-- 
Konstantin Osipov, Moscow, Russia