Tarantool development patches archive
 help / color / mirror / Atom feed
From: Serge Petrenko <sergepetrenko@tarantool.org>
To: Konstantin Osipov <kostja.osipov@gmail.com>
Cc: tarantool-patches@dev.tarantool.org,
	Vladislav Shpilevoy <v.shpilevoy@tarantool.org>,
	kirichenkoga@gmail.com
Subject: Re: [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock
Date: Thu, 19 Mar 2020 14:59:24 +0300	[thread overview]
Message-ID: <A4CC4C33-2FC1-463E-B261-A46EA6ED13D1@tarantool.org> (raw)
In-Reply-To: <20200319115624.GA17950@atlas>


> 19 марта 2020 г., в 14:56, Konstantin Osipov <kostja.osipov@gmail.com> написал(а):
> 
> * Serge Petrenko <sergepetrenko@tarantool.org> [20/03/19 14:29]:
>> First of all, let me describe the issue with the current WAL GC:
>> it tracks consumers (i.e. remote replicas) by their vclock signature,
>> which is the sum of all vclock components.
>> It has always been wrong (since the introduction of vclocks, at least):
>> Say, you have 2 masters, A and B with ids 1 and 2 respectively, and a replica C with id 3.
>> The example will be a little synthetic, but it illustrates the problem:
>> Say С replicates from both A and B, and there is no replication between A and B (say, the
>> instances were reconfigured to not replicate from each other).
>> Now, say replica C has followed A and B to vclock {1:5, 2:13}. At the same time, A has lsn 10
>> and B has lsn 15. A and B do not know about each other’s changes, so A’s vclock is {1:10} and
>> B’s vclock is {2:15}. Now imagine A does a snapshot and creates a new xlog with signature 10.
>> A’s directory will look like: 00…000.xlog 00…010.snap 00….010.xlog
>> Replica C reports its vclock {1:5, 2:13} to A, A uses the vclock to update the corresponding GC
>> consumer. Since signatures are used, GC consumer is assigned a signature = 13 + 5 = 18.
>> This is greater than the signature of the last xlog on A (10), so the previous xlog (00…00.xlog) can be
>> deleted (at least A assumes it can be). Actually, replica still needs 00…00.xlog, because it contains
>> rows corresponding to vclocks {1:6} - {1:10}, which haven’t been replicated yet.
>> 
>> If instead of using vclock signatures, gc consumers used vclocks, such a problem wouldn’t arise.
>> Replia would report its vclock {1:5, 2:13}. The vclock is NOT strictly greater than A’s most recent
>> xlog vclock ({1:10}), so the previous log is kept until replica reports a vclock {1:10, 2:something}.
>> (or {1:11, …} and so on).
> 
> This explanation belongs to the commit comment.
> 

Ok, will amend.

> 
> 
> -- 
> Konstantin Osipov, Moscow, Russia

--
Serge Petrenko
sergepetrenko@tarantool.org

  reply	other threads:[~2020-03-19 11:59 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-18 19:47 [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock Serge Petrenko
2020-03-18 20:08   ` Konstantin Osipov
2020-03-19  8:11     ` Timur Safin
2020-03-19  8:41       ` 'Konstantin Osipov'
2020-03-19  9:17         ` Sergey Ostanevich
2020-03-19 11:28           ` Serge Petrenko
2020-03-19 11:56             ` Konstantin Osipov
2020-03-19 11:59               ` Serge Petrenko [this message]
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 2/5] wal: track consumer vclock and collect logs in wal thread Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 3/5] vclock: add an ability to set individual clock components Serge Petrenko
2020-03-18 20:10   ` Konstantin Osipov
2020-03-19 11:31     ` Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 4/5] replication: hide 0-th vclock components in replication responses Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 5/5] box: start counting local space requests separately Serge Petrenko
2020-03-18 21:12 ` [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking Vladislav Shpilevoy
2020-03-19  8:17 ` Konstantin Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=A4CC4C33-2FC1-463E-B261-A46EA6ED13D1@tarantool.org \
    --to=sergepetrenko@tarantool.org \
    --cc=kirichenkoga@gmail.com \
    --cc=kostja.osipov@gmail.com \
    --cc=tarantool-patches@dev.tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox