Re: [Tarantool-patches] [PATCH v19 2/3] qsync: order access to the limbo terms

[Serge Petrenko via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

05.10.2021 00:53, Cyrill Gorcunov пишет:
> On Fri, Oct 01, 2021 at 03:37:18PM +0300, Serge Petrenko wrote:
>> Shouldn't we guard limbo->owner as well?
>> Otherwise you may start writing confirm for an old leader once promote for a
>> new one is already in progress.
>>
>> I don't remember us discussing this before, so, maybe I'm just confused.
> Seems I still don't follow. Look, currently limbo::owner_id is assigned
> from the one place:
>
> txn_limbo_process_core
>    txn_limbo_process_begin(limbo)
>      txn_limbo_read_demote | txn_limbo_read_promote
>        ...
>        limbo->owner_id = replica_id;
>
> The txn_limbo_process_core is called from console (via box.ctl.promote|demote
> commands) or via applier. When it comes from console then we still take the
> lock before write into the journal
>
> static void
> box_issue_promote(uint32_t prev_leader_id, int64_t promote_lsn)
> {
> 	struct raft *raft = box_raft();
> 	assert(raft->volatile_term == raft->term);
> 	assert(promote_lsn >= 0);
> 	struct synchro_request req = {
> 		.type = IPROTO_RAFT_PROMOTE,
> 		.replica_id = prev_leader_id,
> 		.origin_id = instance_id,
> 		.lsn = promote_lsn,
> 		.term = raft->term,
> 	};
> -->	txn_limbo_process_begin(&txn_limbo);
> -->	txn_limbo_write_promote(&txn_limbo, req.lsn, req.term);
> 	txn_limbo_process_core(&txn_limbo, &req);
> 	assert(txn_limbo_is_empty(&txn_limbo));
> 	txn_limbo_process_commit(&txn_limbo);
> }
>
> thus console commands and applier are serialized. Or I miss something
> obvious?

Here's what can happen. You're the old leader, and you've received a
promote from someone else already, it's being written now.
Now some replica sends you an ack. You enter txn_limbo_ack
(it doesn't check the latch). txn_limbo_ack() may result in writing a 
CONFIRM.

The PROMOTE is already in progress, so you'll end up writing CONFIRM for
a wrong leader id (your own) after PROMOTE is written.

-- 
Serge Petrenko