From: Konstantin Osipov <kostja.osipov@gmail.com>
To: Serge Petrenko <sergepetrenko@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org,
Vladislav Shpilevoy <v.shpilevoy@tarantool.org>,
kirichenkoga@gmail.com
Subject: Re: [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock
Date: Thu, 19 Mar 2020 14:56:24 +0300 [thread overview]
Message-ID: <20200319115624.GA17950@atlas> (raw)
In-Reply-To: <022BA1E5-1F84-400F-BF09-D363338CC296@tarantool.org>
* Serge Petrenko <sergepetrenko@tarantool.org> [20/03/19 14:29]:
> First of all, let me describe the issue with the current WAL GC:
> it tracks consumers (i.e. remote replicas) by their vclock signature,
> which is the sum of all vclock components.
> It has always been wrong (since the introduction of vclocks, at least):
> Say, you have 2 masters, A and B with ids 1 and 2 respectively, and a replica C with id 3.
> The example will be a little synthetic, but it illustrates the problem:
> Say С replicates from both A and B, and there is no replication between A and B (say, the
> instances were reconfigured to not replicate from each other).
> Now, say replica C has followed A and B to vclock {1:5, 2:13}. At the same time, A has lsn 10
> and B has lsn 15. A and B do not know about each other’s changes, so A’s vclock is {1:10} and
> B’s vclock is {2:15}. Now imagine A does a snapshot and creates a new xlog with signature 10.
> A’s directory will look like: 00…000.xlog 00…010.snap 00….010.xlog
> Replica C reports its vclock {1:5, 2:13} to A, A uses the vclock to update the corresponding GC
> consumer. Since signatures are used, GC consumer is assigned a signature = 13 + 5 = 18.
> This is greater than the signature of the last xlog on A (10), so the previous xlog (00…00.xlog) can be
> deleted (at least A assumes it can be). Actually, replica still needs 00…00.xlog, because it contains
> rows corresponding to vclocks {1:6} - {1:10}, which haven’t been replicated yet.
>
> If instead of using vclock signatures, gc consumers used vclocks, such a problem wouldn’t arise.
> Replia would report its vclock {1:5, 2:13}. The vclock is NOT strictly greater than A’s most recent
> xlog vclock ({1:10}), so the previous log is kept until replica reports a vclock {1:10, 2:something}.
> (or {1:11, …} and so on).
This explanation belongs to the commit comment.
--
Konstantin Osipov, Moscow, Russia
next prev parent reply other threads:[~2020-03-19 11:56 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-18 19:47 [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock Serge Petrenko
2020-03-18 20:08 ` Konstantin Osipov
2020-03-19 8:11 ` Timur Safin
2020-03-19 8:41 ` 'Konstantin Osipov'
2020-03-19 9:17 ` Sergey Ostanevich
2020-03-19 11:28 ` Serge Petrenko
2020-03-19 11:56 ` Konstantin Osipov [this message]
2020-03-19 11:59 ` Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 2/5] wal: track consumer vclock and collect logs in wal thread Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 3/5] vclock: add an ability to set individual clock components Serge Petrenko
2020-03-18 20:10 ` Konstantin Osipov
2020-03-19 11:31 ` Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 4/5] replication: hide 0-th vclock components in replication responses Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 5/5] box: start counting local space requests separately Serge Petrenko
2020-03-18 21:12 ` [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking Vladislav Shpilevoy
2020-03-19 8:17 ` Konstantin Osipov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200319115624.GA17950@atlas \
--to=kostja.osipov@gmail.com \
--cc=kirichenkoga@gmail.com \
--cc=sergepetrenko@tarantool.org \
--cc=tarantool-patches@dev.tarantool.org \
--cc=v.shpilevoy@tarantool.org \
--subject='Re: [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox