On Monday, March 11, 2019 12:59:26 PM MSK Konstantin Osipov wrote:
> * Georgy Kirichenko <georgy@tarantool.org> [19/03/11 09:55]:
> > Form a separate transaction with local changes in case of replication.
> > This is important because we should be able to replicate such changes
> > (e.g. made within an on_replace trigger) back. In the opposite case
> > local changes will be incorporated into originating transaction and
> > would be skipped by the originator replica.
> 
> I wonder will we possibly have some recovery issues, since in fact
> we're performing a reordering of execution here?
> 
> Imagine local and remote statements change the same set of rows.
> During initial execution these changes are intermixed, during
> recovery they are serialized.
If you remember we were agreed that only local spaces are allowed to change in 
case of replication triggers.
> 
> It seems we clearly have a problem here. We can either open a bug,
> support multiple txn ids in the same stream, support multiple
> server ids in the same transaction, ban triggers in
> multi-statement transaction?
You pushed me to remove txn_replica_id but it was one of the instruments I 
planed to use in order to support distributed transactions (with multiple 
replica ids in the same transaction) in the future. So I would prefer if we 
just disable changing of non-local spaces during replication. In such case we 
won't have any issues with reordering.
> 
> Can we attribute local changes to the same server id?
It is impossible because of lsn
> We don't have to replicate them back - this is a gray zone and we can do it
> in any way we want.
I'm afraid no because we already have this functionality and it is even 
covered with tests.

So we have to make a high level decision: what is expected behavior.
In any case I will be agreed with your decision what we should to do: disable 
non-local replication changes, change behavior of replication for such changes 
or start further distributed transaction investigation.