Tarantool development patches archive
 help / color / mirror / Atom feed
* [tarantool-patches] [PATCH 0/7] Replication: In-memory replication
@ 2019-08-13  6:27 Georgy Kirichenko
  2019-08-13  6:27 ` [tarantool-patches] [PATCH 1/7] Refactoring: wal writer fiber and queue Georgy Kirichenko
                   ` (7 more replies)
  0 siblings, 8 replies; 18+ messages in thread
From: Georgy Kirichenko @ 2019-08-13  6:27 UTC (permalink / raw)
  To: tarantool-patches; +Cc: Georgy Kirichenko

This patchset introduces from-wal-memory-replication and consist of 4
preparation patches, one patch that introduces memory-replication and 2
follow ups that clear unused facility from tarantool code.

First patch introduces a wal writer fiber and write queue what is needed
in order to have an ability to yield during wal writes. Without this
functionality there is no possibility to make a cbus call from wal
writer functions.

Second patch moves wal dir tracking into gc state. This is required to
simplify relay state tracking as in-memory relay would not have any glue
about wal files boundaries.

Third patch removes wal boundaries tracking from relay

Forth patch prepares wal to write all data in a memory buffer and then
write the data to a file. This memory buffer contains raw encoded data
and xrow_header index to support navigation thorough the data.

The next patch allows a relay to attach to relay and call send callback
on each row stored in a wal memory buffer.

Two subsequent patches removes wal_watcher and on_close_log triggers as
unused.

Branch: https://github.com/tarantool/tarantool/tree/g.kirichenko/gh-3794-memory-replication
Issue: https://github.com/tarantool/tarantool/issues/3794

Georgy Kirichenko (7):
  Refactoring: wal writer fiber and queue
  Refactoring: Track wal files using gc state.
  Replication: Relay does not rely on xlog boundaries
  Replication: wal memory buffer
  Replication: in memory replication
  Refactoring: remove wal_watcher routines
  Refactoring: get rid of on_close_log

 src/box/CMakeLists.txt                |   1 +
 src/box/box.cc                        |  25 +-
 src/box/gc.c                          | 131 ++++-
 src/box/gc.h                          |  34 +-
 src/box/lua/info.c                    |  25 +-
 src/box/memtx_engine.c                |   2 +-
 src/box/recovery.cc                   |  10 +-
 src/box/recovery.h                    |   5 +-
 src/box/relay.cc                      | 263 +++------
 src/box/vy_log.c                      |   4 +-
 src/box/wal.c                         | 742 +++++++++++++++-----------
 src/box/wal.h                         |  97 +---
 src/box/wal_mem.c                     | 273 ++++++++++
 src/box/wal_mem.h                     | 166 ++++++
 src/box/xlog.c                        |  87 ++-
 src/box/xlog.h                        |  11 +-
 test/replication/gc_no_space.result   |   4 +-
 test/replication/gc_no_space.test.lua |   4 +-
 18 files changed, 1208 insertions(+), 676 deletions(-)
 create mode 100644 src/box/wal_mem.c
 create mode 100644 src/box/wal_mem.h

-- 
2.22.0

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2019-08-21 13:52 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-13  6:27 [tarantool-patches] [PATCH 0/7] Replication: In-memory replication Georgy Kirichenko
2019-08-13  6:27 ` [tarantool-patches] [PATCH 1/7] Refactoring: wal writer fiber and queue Georgy Kirichenko
2019-08-16 13:53   ` [tarantool-patches] " Konstantin Osipov
2019-08-20 10:57     ` Георгий Кириченко
2019-08-21 10:18   ` [tarantool-patches] " Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 2/7] Refactoring: Track wal files using gc state Georgy Kirichenko
2019-08-21 10:44   ` Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 3/7] Replication: Relay does not rely on xlog boundaries Georgy Kirichenko
2019-08-21 11:35   ` Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 4/7] Replication: wal memory buffer Georgy Kirichenko
2019-08-21 11:57   ` Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 5/7] Replication: in memory replication Georgy Kirichenko
2019-08-21 13:52   ` Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 6/7] Refactoring: remove wal_watcher routines Georgy Kirichenko
2019-08-21 13:52   ` Vladimir Davydov
2019-08-13  6:27 ` [tarantool-patches] [PATCH 7/7] Refactoring: get rid of on_close_log Georgy Kirichenko
2019-08-21 13:52   ` Vladimir Davydov
2019-08-16 13:47 ` [tarantool-patches] Re: [PATCH 0/7] Replication: In-memory replication Konstantin Osipov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox