Tarantool development patches archive
 help / color / mirror / Atom feed
From: Serge Petrenko <sergepetrenko@tarantool.org>
To: v.shpilevoy@tarantool.org, kostja.osipov@gmail.com, georgy@tarantool.org
Cc: tarantool-patches@dev.tarantool.org
Subject: [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking
Date: Wed, 18 Mar 2020 22:47:43 +0300	[thread overview]
Message-ID: <cover.1584558067.git.sergepetrenko@tarantool.org> (raw)

https://github.com/tarantool/tarantool/issues/4114
https://github.com/tarantool/tarantool/tree/sp/gh-4114-local-space-replication

The patchset contains 2 of Georgy's patches perorming WAL gc rework.
The first patch introduces a matrix clock structure, which contains the latest
known vclocks of all the cluster members and can be used to build a vclock,
which is less than or equal to all the member vclocks.
Such a vclock is used then in the second patch, where gc is rewritten to stop
tracking needed WAL files by vclock signature. Now replica vclocks are used to
define which files are still needed.

These two patches allow to start using a  0 vclock component to track local
space changes and ignore this component when finding WALs needed by a replica.

Patches 3 and 4 are preparation for the patch 5, which performs the task itself.
Now every replica has its own 0-th vclock component, which doesn't match any of
the other instance components and indicates the number of local space requests
executed.

Georgy Kirichenko (2):
  box: introduce matrix clock
  wal: track consumer vclock and collect logs in wal thread

Serge Petrenko (3):
  vclock: add an ability to set individual clock components
  replication: hide 0-th vclock components in replication responses
  box: start counting local space requests separately

 src/box/CMakeLists.txt                        |   4 +
 src/box/applier.cc                            |   5 +-
 src/box/box.cc                                |  70 ++--
 src/box/gc.c                                  | 216 +++-------
 src/box/gc.h                                  |  95 +----
 src/box/lua/info.c                            |  29 +-
 src/box/mclock.c                              | 394 ++++++++++++++++++
 src/box/mclock.h                              | 151 +++++++
 src/box/relay.cc                              | 157 ++-----
 src/box/replication.cc                        |  47 +--
 src/box/replication.h                         |   3 -
 src/box/vclock.c                              |  15 +
 src/box/vclock.h                              |  11 +
 src/box/wal.c                                 | 286 +++++++++++--
 src/box/wal.h                                 |  17 +-
 test/replication/anon.result                  |   5 +
 test/replication/anon.test.lua                |   2 +
 test/replication/autobootstrap.result         |   6 +
 test/replication/autobootstrap.test.lua       |   2 +
 test/replication/before_replace.result        |   8 +-
 test/replication/before_replace.test.lua      |   4 +-
 test/replication/gc_no_space.result           |  30 +-
 test/replication/gc_no_space.test.lua         |  12 +-
 .../gh-4114-local-space-replication.result    | 125 ++++++
 .../gh-4114-local-space-replication.test.lua  |  48 +++
 test/replication/local_spaces.result          |   4 +
 test/replication/local_spaces.test.lua        |   3 +
 test/replication/misc.result                  |   6 +
 test/replication/misc.test.lua                |   2 +
 test/replication/quorum.result                |   6 +
 test/replication/quorum.test.lua              |   2 +
 test/replication/replica_rejoin.result        |   9 +
 test/replication/replica_rejoin.test.lua      |   3 +
 test/replication/skip_conflict_row.result     |   3 +
 test/replication/skip_conflict_row.test.lua   |   1 +
 test/replication/suite.cfg                    |   1 +
 test/unit/CMakeLists.txt                      |   2 +
 test/unit/mclock.result                       |  18 +
 test/unit/mclock.test.c                       | 160 +++++++
 test/vinyl/errinj.result                      |   5 +
 test/vinyl/errinj.test.lua                    |   4 +
 41 files changed, 1430 insertions(+), 541 deletions(-)
 create mode 100644 src/box/mclock.c
 create mode 100644 src/box/mclock.h
 create mode 100644 test/replication/gh-4114-local-space-replication.result
 create mode 100644 test/replication/gh-4114-local-space-replication.test.lua
 create mode 100644 test/unit/mclock.result
 create mode 100644 test/unit/mclock.test.c

-- 
2.21.1 (Apple Git-122.3)

             reply	other threads:[~2020-03-18 19:48 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-18 19:47 Serge Petrenko [this message]
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 1/5] box: introduce matrix clock Serge Petrenko
2020-03-18 20:08   ` Konstantin Osipov
2020-03-19  8:11     ` Timur Safin
2020-03-19  8:41       ` 'Konstantin Osipov'
2020-03-19  9:17         ` Sergey Ostanevich
2020-03-19 11:28           ` Serge Petrenko
2020-03-19 11:56             ` Konstantin Osipov
2020-03-19 11:59               ` Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 2/5] wal: track consumer vclock and collect logs in wal thread Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 3/5] vclock: add an ability to set individual clock components Serge Petrenko
2020-03-18 20:10   ` Konstantin Osipov
2020-03-19 11:31     ` Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 4/5] replication: hide 0-th vclock components in replication responses Serge Petrenko
2020-03-18 19:47 ` [Tarantool-patches] [PATCH v2 5/5] box: start counting local space requests separately Serge Petrenko
2020-03-18 21:12 ` [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking Vladislav Shpilevoy
2020-03-19  8:17 ` Konstantin Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1584558067.git.sergepetrenko@tarantool.org \
    --to=sergepetrenko@tarantool.org \
    --cc=georgy@tarantool.org \
    --cc=kostja.osipov@gmail.com \
    --cc=tarantool-patches@dev.tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v2 0/5] replication: fix local space tracking' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox