From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 3DD2245C304 for ; Fri, 27 Nov 2020 00:01:08 +0300 (MSK) References: <20201124131820.32981-1-sergepetrenko@tarantool.org> From: Vladislav Shpilevoy Message-ID: Date: Thu, 26 Nov 2020 22:01:06 +0100 MIME-Version: 1.0 In-Reply-To: <20201124131820.32981-1-sergepetrenko@tarantool.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [Tarantool-patches] [PATCH] raft: make sure the leader stays ro till it clears the limbo List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Serge Petrenko , gorcunov@gmail.com Cc: tarantool-patches@dev.tarantool.org Hi! Thanks for the patch! On 24.11.2020 14:18, Serge Petrenko wrote: > When running a cluster with leader election, its useful to wait till the > instance is writeable to determine that it has become a leader. However, > sometimes the instance fails to write data right after transitioning to > leader because its limbo still contains pending transactions from the > old leader. Make sure the instance deals with pending transactions first > and becomes writeable only once the limbo is empty. I just realized one thing. We can add a function txn_limbo_is_ro(), like we did with raft_is_ro(), account it in box_update_ro_summary(), and call box_update_ro_summary() when we see that the limbo is emptied, or when its ownership changes to a different instance. Probably would be simpler, and also we could make it work with manual election! So users could call box.ctl.wait_rw() even without using raft! To show concrete error if somebody still tries to write, we could patch box_check_writable() to show the reason why the instance is not writable. We will do it anyway for raft, to tell the users the real leader in case they are trying to write on a replica. In scope of https://github.com/tarantool/tarantool/issues/5568. Your version of the patch also looks good. What do you think?