Re: [Tarantool-patches] [PATCH] limbo: introduce request processing hooks

[Cyrill Gorcunov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

On Mon, Jul 12, 2021 at 11:01:29AM +0300, Serge Petrenko wrote:
> > Well, I still don't understand what the issue is. We discussed it
> > privately already. You simply should not apply anything until WAL
> > write is done. And it is not happening now on the master. The
> > terms vclock is updated only **after** WAL write.
> > 
> > Why do you need all these new vclocks if you should not apply
> > anything before WAL write in the first place?
> 
> If I understand correctly, the issue is that if we filter (and check for
> the split brain) after the WAL write, we will end up with a conflicting
> PROMOTE in our WAL. Cyrill is trying to avoid this, that's why
> he's separating the filter stage. This way the error will reach
> the remote peer before any WAL write, and the WAL write won't happen.
> 
> And if we filter before the WAL write, we need the second vclock, which
> Cyrill has introduced.
> 
> We may leave confligting PROMOTEs in WAL (first write them and only
> then check for conflicts). In this case this whole patch isn't
> needed. But I personally don't like such an approach.

Exactly. What is more interesting is that each term is involved
into maximal term calculation.

Imagine we have several appliers running (with early filtering
of incoming packets)

applier 1                       applier 2
---------                       ---------
applier_apply_tx
  apply_synchro_row
    filter
      - update term
      - calculate term_max
    journal_write
      (sleeping)
                                applier_apply_tx
                                  applier_synchro_filter_tx(rows);
                                      - the term_max should be already
                                        updated so we could NOP data
                                    journal_write
                                      (will be woken after applier 1
                                       finished its write)

Actually I start to think that we might not NOP data until applier's 1
write finished...