[tarantool-patches] [PATCH v3 0/4] From memory wal replication

Georgy Kirichenko georgy at tarantool.org
Wed Oct 9 19:45:42 MSK 2019


This patchset introduces from-wal-memory-replication and consist of 4
patches.

The first patch purges distinct relay facilities to advanced gc consumer
state. Also it changes gc behaviour - now gc compares consumers using
lsns corresponding to local instance id instead of vclock signature.
Howevers this changes gc behavior, its look like we could restore the
gc behavior after relay would be migrated to the wal thread. The whole
patch is essential because in relay switched to memory relaying there is
no more file boundaries and on_close_log trigger invocation now.

The second patch introduces xrow buffer structure to be used by wal
while logging. Wal encodes rows to a memory buffer (xrow_buffer
structure) and then writes encoded data to a xlog file.

The next patch implements xrow_buf_cursor to have an ability to found
a position in the buffer by vclock and then fetch all rows one-by-one
towards the last one.

The last patch makes a relay able to read from wal memory using
a xrow_buffer cursor. When relay went out of wal memory window it turns
back to file mode. After last known xlog file is relayed then relay
tries to return to wal memory mode.

Changes in v3:
 - A lot of commentary fixes and renaming according to review
 - Slight refactoring
 - On-stack heartbeat message issue fixed

Changes in v2:
 - Get rid of gc refactoring
 - Rename wal_memory to xrow_buffer according to review
 - Commentaries
 - Bug fixes

Branch: http://github.com/tarantool/tarantool/tree/g.kirichenko/gh-3794-memory-replication
Issue: https://github.com/tarantool/tarantool/issues/3794

Georgy Kirichenko (4):
  relay: adjust gc state on relay status update
  wal: xrow buffer structure
  wal: xrow buffer cursor
  replication: use wal memory buffer to fetch rows

 src/box/CMakeLists.txt                        |   1 +
 src/box/gc.c                                  |   7 +-
 src/box/relay.cc                              | 286 +++++--------
 src/box/wal.c                                 | 205 +++++++++-
 src/box/wal.h                                 |  60 +++
 src/box/xlog.c                                |  57 ++-
 src/box/xlog.h                                |  16 +-
 src/box/xrow_buf.c                            | 382 ++++++++++++++++++
 src/box/xrow_buf.h                            | 198 +++++++++
 src/lib/core/cbus.c                           |   4 +
 src/lib/core/errinj.h                         |   1 +
 test/box/errinj.result                        |   2 +
 test/replication/force_recovery.result        |   8 +
 test/replication/force_recovery.test.lua      |   2 +
 test/replication/replica_rejoin.result        |   8 +
 test/replication/replica_rejoin.test.lua      |   2 +
 .../show_error_on_disconnect.result           |   8 +
 .../show_error_on_disconnect.test.lua         |   2 +
 test/replication/suite.ini                    |   2 +-
 test/xlog/panic_on_wal_error.result           |  12 +
 test/xlog/panic_on_wal_error.test.lua         |   3 +
 test/xlog/suite.ini                           |   2 +-
 22 files changed, 1045 insertions(+), 223 deletions(-)
 create mode 100644 src/box/xrow_buf.c
 create mode 100644 src/box/xrow_buf.h

-- 
2.23.0





More information about the Tarantool-patches mailing list