From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 0C7D16EC55; Tue, 20 Jul 2021 11:49:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0C7D16EC55 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1626770992; bh=xS+HUj7y7efajIwLU4ohIrzCKNqSuG+21TNMqMqGaUc=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rb60J8uWymtr6koxx8yHeoWiDlg0pmDYNWWXVI9Pp7uVjbqtP7M88qz0tgwHjM1rM JiYNzYwgRZOFqrMh2DzWBUxucY/7fy6rGYF///oGRgtEsDtKdx3kvbLPEb6qxQmGZs EnlsNjkDIZHffNZkALSzCXQbdNNWtV6yOQ/ciW/4= Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id EB5AF6EC55 for ; Tue, 20 Jul 2021 11:49:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org EB5AF6EC55 Received: by mail-lf1-f54.google.com with SMTP id q16so34706698lfa.5 for ; Tue, 20 Jul 2021 01:49:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=4kLUJQnRNNk9qoCfjBru6510DS7Hmje8KxdnjDLEyaA=; b=lxTXGSYgIin7TZPiciZYCzqhqBtwVTA8hkFvIX7zlkbSTwfiH1klyHo4tx2/co6qZQ /RAVql/CZpatOUkZHo8TYgDvRs7GNWDiCt79OgWW8lwJkA83zVIXi3wzaQCqGRzCUCCb rO7c92g4RdIyYODAipEykvo3TnmG4UGQp4gw7d3QmBxWGmXhjCF3UFJGmG/5E/gCtiqx quAgcpOwcaTVshcgpG67TXgNWADJ4vbb7Zrbq3CR3q05VeNgojd1cwvglnxZtIY4a56P yd1kWGrf9AKr5E7wRjOuoNUBdV/XrWEgxxvXFhU+UY/Fq0zoIZLMybRLj8kUy5cPTYjb JJRA== X-Gm-Message-State: AOAM533LgZYHIGOi8d1OSF4t8eEqEldnikmqzbuHgfO2kyPuGIbcZIDJ cMv9/w79o+Z4ZdnDCVON5S9BkeOEhA== X-Google-Smtp-Source: ABdhPJx+R4uwxaxR1ifyqEjDIh+lBk4XrfHtq1HFB5CfOjviyfg5+8KbR835QQ8B9i/Bo/vvDVHVTQ== X-Received: by 2002:a19:7514:: with SMTP id y20mr21099245lfe.623.1626770990096; Tue, 20 Jul 2021 01:49:50 -0700 (PDT) Received: from sterling.local ([46.188.68.12]) by smtp.gmail.com with ESMTPSA id q11sm1486055lfo.237.2021.07.20.01.49.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jul 2021 01:49:48 -0700 (PDT) Received: by sterling.local (Postfix, from userid 1000) id E85F1E6386B; Tue, 20 Jul 2021 11:49:47 +0300 (MSK) Date: Tue, 20 Jul 2021 11:49:47 +0300 To: Vladislav Shpilevoy Message-ID: <20210720084947.GA58808@starling> Mail-Followup-To: Konstantin Osipov , Vladislav Shpilevoy , tarantool-patches@dev.tarantool.org, gorcunov@gmail.com, sergepetrenko@tarantool.org References: <0c92a88ff1d392f8b03de59be8cb19a162bf78f8.1626392372.git.v.shpilevoy@tarantool.org> <20210716142959.GC146960@starling> <20210719091248.GA4257@starling> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH 1/2] replication: introduce ballot.can_be_leader X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Konstantin Osipov via Tarantool-patches Reply-To: Konstantin Osipov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * Vladislav Shpilevoy [21/07/20 01:09]: > >>> Curious why did you add this feature in the first place, I mean > >>> "eligibility"? Each voter has to be able to become a leader, > >>> otherwise raft liveness guarantees are violated. Raft has > >>> learners, but learners neither vote nor can become leaders. > >> > >> Voters are nodes which an admin does not want to be a leader. For > >> instance, they are too far away physically. As voters, they might > >> help to elect a leader, for example, if there are just 3 nodes one > >> of which is a voter. > >> > >> Another application is when you specifically start 1 node as a > >> voter and 2 candidates. The voter might skip all the replication > >> data and work on a slow small machine. > >> > >> It can help to form a majority. We are planning to make this > >> feature even easier to use by adding dataless nodes just for > >> voting. > >> > >> As for Raft, it should not bring any problems. In Raft you can > >> say that all nodes are candidates, but some of them are so slow, > >> that they can never vote for themselves in time. Raft still works, > >> and you essentially have 'voters'. > > > > Imagine there are nodes A, B, C, D, E. > > A is a leader, E is a voter which can not become a leader. > > > > Imagine A's log index is 5, B = 4, C = 3, D = 2, E = 5. > > > > The majority's log index is 4, so entry 4 is committed. A dies, B > > is partitioned away. The cluster is stuck, because neither C nor B > > can get a quorum (3 votes). > > But how is it different from the real Raft? In normal Raft I can say > E simply is too slow to make any actions. It is just stuck or died. > The cluster will be stuck then, yes. Not much you can do here. In a real raft: - liveness is guaranteed if quorum is present; this guarantee here is not held - you never sacrifice safety for liveness; you never lose committed entries if quorum is present; and you never lose it unnoticed! here you can lose a committed entry and not notice it. > > You can think of a voter as of almost a permanently broken node which > sometimes manages to vote but never manages to become a candidate in > time. I suppose Raft can withstand that behaviour. > > > Worse yet, if E's (voter) commit index is low, not high, it can vote for a > > node which doesn't have a committed entry. In that case you can > > lose a committed entry. > > Could you provide an example? Because I still do not see how is it > different from the classic Raft in which one node either is always too > late to become a candidate or is turned off when there are no better > candidates. -- Konstantin Osipov, Moscow, Russia https://scylladb.com