Re: [Tarantool-patches] [PATCH v3 07/12] box: introduce `box.ctl.demote`

[Vladislav Shpilevoy via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Hi! Thanks for the patch!

I think we are on the finish line here, see 4 small
comments below. After them and after you fix the failing
vinyl test, the patchset will probably be finished!

There are only 2 things which bother me. They are not bugs
and we can work on them in the next quarter.

1) Assume you have election_mode = 'manual'. And you are a
leader. You call box.ctl.demote() and stop being a leader.
But the limbo is still yours. If now you switch election_mode to
'off', you need to call box.ctl.demote() again to free the
limbo.

2) In the last commit I see we make too much actions to ensure
we are a writable leader. Perhaps in the future we should not
report box.info.election.state == 'leader' until promote is
written and should not say the instance is writable.

I don't have a plan for either of these ideas yet.

> diff --git a/src/box/box.cc b/src/box/box.cc
> index 41f665e38..a34e05e94 100644
> --- a/src/box/box.cc
> +++ b/src/box/box.cc
> @@ -1679,20 +1679,44 @@ box_issue_promote(uint32_t prev_leader_id, int64_t promote_lsn)
>  	assert(txn_limbo_is_empty(&txn_limbo));
>  }
>  
> +/* A guard to block multiple simultaneous box_promote() invocations. */

1. For out of function comments we usually use /** as an opening.

> +static bool in_box_promote = false;

2. Could you please use `is_` prefix here? `is_in_box_promote`.

> +
> +int
> +box_promote_qsync(void)
> +{
> +	assert(!in_box_promote);
> +	assert(is_box_configured);
> +	struct raft *raft = box_raft();
> +	in_box_promote = true;
> +	auto promote_guard = make_scoped_guard([&] {
> +		in_box_promote = false;
> +	});
> +	if (raft->state != RAFT_STATE_LEADER)
> +		return 0;

3. This condition is not reachable, according to what I see in
box_raft_worker_f().

> +	assert(box_election_mode == ELECTION_MODE_MANUAL ||
> +	       box_election_mode == ELECTION_MODE_CANDIDATE);
> +	if (txn_limbo_replica_term(&txn_limbo, instance_id) == raft->term)
> +		return 0;
> +	int64_t wait_lsn = box_wait_limbo_acked(TIMEOUT_INFINITY);
> +	if (wait_lsn < 0)
> +		return -1;

4. Perhaps this better be panic? Because infinity timeout should
not ever be reached. And then the function becomes void, because
would not be able to fail anymore.

> +	box_issue_promote(txn_limbo.owner_id, wait_lsn);
> +	return 0;
> +}
> commit 7980cb3096f2616a2851f8d97db8091f0d82879c
> Author: Serge Petrenko <sergepetrenko@tarantool.org>
> Date:   Mon Jun 28 11:52:44 2021 +0300
> 
>     box: allow calling promote on a candidate
>     
>     Part of #6034
> 
> diff --git a/test/replication/gh-6034-election-candidate-promote.result b/test/replication/gh-6034-election-candidate-promote.result
> new file mode 100644
> index 000000000..2b4bc0213
> --- /dev/null
> +++ b/test/replication/gh-6034-election-candidate-promote.result

5. The test name format `gh-####-...` is obligatory only for bug tests.
This patch seems to be adding a feature.