From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH 1/2] Revert "replication: update replica gc state on subscribe" Date: Tue, 9 Apr 2019 20:07:10 +0300 [thread overview] Message-ID: <24d1ce5202869981a2182ff1c7947d7d4914bb5b.1554829366.git.vdavydov.dev@gmail.com> (raw) This reverts commit b5b4809cf2e6d48230eb9e4301eac188b080e0f4. The commit reverted by this patch made relay advance the consumer associated with a replica right on subscribe. This is wrong, because the current implementation of the garbage collector operates on vclock signatures so that if a replica reconnects with a greater signature than it had when it last disconnected (e.g. due to replica local changes or changes pulled from other members of the cluster), the garbage collector may delete WAL files still needed by the replica, breaking replication. There are two ways to fix this problem. The first and the most difficult way is to teach the garbage collector to work with vclocks, i.e. rather than simply sorting all consumers by signature and using the smallest signature for garbage collection, maintain a vclock each component of which is the minimum among corresponding components of all registered consumers. The second (easy) way is to advance a consumer only if its acknowledged vclock is greater than or equal to the vclock of a WAL fed to it. This way the garbage collector still works with vclock signatures and it's a responsibility of the caller (i.e. relay) to ensure that consumers are advanced correctly. I took on the second way for now, because I couldn't figure out an efficient way to implement the first. This implies reverting the above mentioned commit and reopening #4034 - sporadic replication/gc.test.lua failure - which will have to be fixed some other way. See the next patch for the rest of the fix and the test. Needed for #4106 --- https://github.com/tarantool/tarantool/issues/4034 https://github.com/tarantool/tarantool/issues/4106 https://github.com/tarantool/tarantool/commits/dv/gh-4106-replica-gc-fix src/box/relay.cc | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/src/box/relay.cc b/src/box/relay.cc index 45cd780a..29570323 100644 --- a/src/box/relay.cc +++ b/src/box/relay.cc @@ -643,16 +643,13 @@ relay_subscribe(struct replica *replica, int fd, uint64_t sync, /* * Register the replica with the garbage collector * unless it has already been registered by initial - * join or subscribe. Otherwise update the consumer - * state with the current replica vclock. + * join. */ if (replica->gc == NULL) { replica->gc = gc_consumer_register(replica_clock, "replica %s", tt_uuid_str(&replica->uuid)); if (replica->gc == NULL) diag_raise(); - } else { - gc_consumer_advance(replica->gc, replica_clock); } relay_start(relay, fd, sync, relay_send_row); -- 2.11.0
next reply other threads:[~2019-04-09 17:07 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-09 17:07 Vladimir Davydov [this message] 2019-04-09 17:07 ` [PATCH 2/2] replication: fix garbage collection logic Vladimir Davydov 2019-04-11 7:32 ` [tarantool-patches] " Konstantin Osipov 2019-04-11 8:25 ` Vladimir Davydov 2019-04-11 11:01 ` Vladimir Davydov 2019-04-11 10:25 ` [tarantool-patches] Re: [PATCH 1/2] Revert "replication: update replica gc state on subscribe" Konstantin Osipov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=24d1ce5202869981a2182ff1c7947d7d4914bb5b.1554829366.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH 1/2] Revert "replication: update replica gc state on subscribe"' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox