From: Vladislav Shpilevoy via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: Serge Petrenko <sergepetrenko@tarantool.org>, tarantool-patches@dev.tarantool.org, gorcunov@gmail.com Subject: Re: [Tarantool-patches] [PATCH 1/1] replication: prevent boot when rs uuid mismatches Date: Sat, 5 Jun 2021 01:48:59 +0200 [thread overview] Message-ID: <b8f96750-6857-fde9-83ec-19c7f58cd5c1@tarantool.org> (raw) In-Reply-To: <92afec1a-8781-52ca-14a2-72f3378e4f6e@tarantool.org> Hi! Thanks for the review! > First of all, 5613 is about 3rd replica bootstrapping a separate cluster, > even when it sees that the 2 other nodes have already bootstrapped. > > This patch doesn't actually fix 5613. The 3rd node shows a different error now, > but it still bootstraps its own cluster with a separate uuid. I see now. I thought that in the issue description the first 2 nodes were bootstrapped separately. > I propose to change replicaset_round() somehow, so that it never chooses > non-bootstrapped instances over bootstrapped ones. Even when bootstrapped > instances are read-only. I did it in the new version, see another email thread. > Looks like you don't even have to change ballot for this purpose. There's > already the 'is_loading' field. We just have to assign higher priority to > `is_loading = false` rather than `read_only = false`. > > P.S. I've checked, and looks like is_loading is not that useful now. > It's equal to instance's is_ro flag (not the one passed in ballot, but actual is_ro). > Still, it's easier to change is_loading encoding than introduce a whole new field. Yes, indeed, is_loading has little to do with actual loading. It is more like "box.cfg() is finished and box.cfg{read_only=true} was set". I did several changes to the ballot to make it work. Renamed field is_ro, renamed + slightly changed behaviour of is_loading, and added a new field. Only is_loading is not enough, because I still need to know who is really read-only. Not just by read_only=false, but who is actually writable. There can be orphans who has finished bootstrap/recovery, but are not writable yet. Some replication tests starts failing if we only look at read_only=false and finished bootstrap/recovery. For instance, assume 1 node is started and booted fine, it is writable. Then 2 other nodes are started: node2 and node3. They connect to node1 first, get its ballot, vclock. Then node2 registers on node1. Node3 connects to node2 now and gets its ballot. It sees node3 has higher vclock than node1 (because node2 connected to node3 later). If it does not look at it being read-only (because it is an orphan), it tries to boot from node3 (because its vclock looks like > node1) and fails because node3 can't write to _cluster yet. That error I got on replication/bootstrap_leader.test.lua until I decided to keep both properties of being booted and of being read-only.
prev parent reply other threads:[~2021-06-04 23:49 UTC|newest] Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-03 17:15 Vladislav Shpilevoy via Tarantool-patches 2021-06-04 11:20 ` Serge Petrenko via Tarantool-patches 2021-06-04 23:48 ` Vladislav Shpilevoy via Tarantool-patches [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=b8f96750-6857-fde9-83ec-19c7f58cd5c1@tarantool.org \ --to=tarantool-patches@dev.tarantool.org \ --cc=gorcunov@gmail.com \ --cc=sergepetrenko@tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH 1/1] replication: prevent boot when rs uuid mismatches' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox