Re: [Tarantool-patches] [PATCH 1/2] replication: introduce ballot.can_be_leader

[Konstantin Osipov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

* Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [21/07/20 01:09]:
> >>> Curious why did you add this feature in the first place, I mean
> >>> "eligibility"? Each voter has to be able to become a leader,
> >>> otherwise raft liveness guarantees are violated. Raft has
> >>> learners, but learners neither vote nor can become leaders.
> >>
> >> Voters are nodes which an admin does not want to be a leader. For
> >> instance, they are too far away physically. As voters, they might
> >> help to elect a leader, for example, if there are just 3 nodes one
> >> of which is a voter.
> >>
> >> Another application is when you specifically start 1 node as a
> >> voter and 2 candidates. The voter might skip all the replication
> >> data and work on a slow small machine.
> >>
> >> It can help to form a majority. We are planning to make this
> >> feature even easier to use by adding dataless nodes just for
> >> voting.
> >>
> >> As for Raft, it should not bring any problems. In Raft you can
> >> say that all nodes are candidates, but some of them are so slow,
> >> that they can never vote for themselves in time. Raft still works,
> >> and you essentially have 'voters'.
> > 
> > Imagine there are nodes A, B, C, D, E.
> > A is a leader, E is a voter which can not become a leader.
> > 
> > Imagine A's log index is 5, B = 4, C = 3, D = 2, E = 5.
> > 
> > The majority's log index is 4, so entry 4 is committed. A dies, B
> > is partitioned away. The cluster is stuck, because neither C nor B
> > can get a quorum (3 votes).
> 
> But how is it different from the real Raft? In normal Raft I can say
> E simply is too slow to make any actions. It is just stuck or died.
> The cluster will be stuck then, yes. Not much you can do here.

In a real raft:
- liveness is guaranteed if quorum is present; this guarantee here
  is not held
- you never sacrifice safety for liveness; you never lose
  committed entries if quorum is present; and you never lose it
  unnoticed! here you can lose a committed entry and not notice
  it.
> 
> You can think of a voter as of almost a permanently broken node which
> sometimes manages to vote but never manages to become a candidate in
> time. I suppose Raft can withstand that behaviour.
> 
> > Worse yet, if E's (voter) commit index is low, not high, it can vote for a
> > node which doesn't have a committed entry. In that case you can
> > lose a committed entry.
> 
> Could you provide an example? Because I still do not see how is it
> different from the classic Raft in which one node either is always too
> late to become a candidate or is turned off when there are no better
> candidates.

-- 
Konstantin Osipov, Moscow, Russia
https://scylladb.com