Re: [Tarantool-patches] [tarantool-patches] [PATCH v3 0/4] From memory wal replication

[Vladislav Shpilevoy <v.shpilevoy@tarantool.org>]

Hi! Thanks for the patchset!

- I see that Travis is still red on your branch:
https://travis-ci.org/tarantool/tarantool/jobs/595713541.

- Please, rebase your branch on the latest master. It
is very outdated. Probably it also will help to fix Travis.

On 09/10/2019 18:45, Georgy Kirichenko wrote:
> This patchset introduces from-wal-memory-replication and consist of 4
> patches.
> 
> The first patch purges distinct relay facilities to advanced gc consumer
> state. Also it changes gc behaviour - now gc compares consumers using
> lsns corresponding to local instance id instead of vclock signature.
> Howevers this changes gc behavior, its look like we could restore the
> gc behavior after relay would be migrated to the wal thread. The whole
> patch is essential because in relay switched to memory relaying there is
> no more file boundaries and on_close_log trigger invocation now.> 
> The second patch introduces xrow buffer structure to be used by wal
> while logging. Wal encodes rows to a memory buffer (xrow_buffer
> structure) and then writes encoded data to a xlog file.
> 
> The next patch implements xrow_buf_cursor to have an ability to found
> a position in the buffer by vclock and then fetch all rows one-by-one
> towards the last one.
> 
> The last patch makes a relay able to read from wal memory using
> a xrow_buffer cursor. When relay went out of wal memory window it turns
> back to file mode. After last known xlog file is relayed then relay
> tries to return to wal memory mode.
> 
> Changes in v3:
>  - A lot of commentary fixes and renaming according to review
>  - Slight refactoring
>  - On-stack heartbeat message issue fixed
> 
> Changes in v2:
>  - Get rid of gc refactoring
>  - Rename wal_memory to xrow_buffer according to review
>  - Commentaries
>  - Bug fixes
> 
> Branch: http://github.com/tarantool/tarantool/tree/g.kirichenko/gh-3794-memory-replication
> Issue: https://github.com/tarantool/tarantool/issues/3794
> 
> Georgy Kirichenko (4):
>   relay: adjust gc state on relay status update
>   wal: xrow buffer structure
>   wal: xrow buffer cursor
>   replication: use wal memory buffer to fetch rows
> 
>  src/box/CMakeLists.txt                        |   1 +
>  src/box/gc.c                                  |   7 +-
>  src/box/relay.cc                              | 286 +++++--------
>  src/box/wal.c                                 | 205 +++++++++-
>  src/box/wal.h                                 |  60 +++
>  src/box/xlog.c                                |  57 ++-
>  src/box/xlog.h                                |  16 +-
>  src/box/xrow_buf.c                            | 382 ++++++++++++++++++
>  src/box/xrow_buf.h                            | 198 +++++++++
>  src/lib/core/cbus.c                           |   4 +
>  src/lib/core/errinj.h                         |   1 +
>  test/box/errinj.result                        |   2 +
>  test/replication/force_recovery.result        |   8 +
>  test/replication/force_recovery.test.lua      |   2 +
>  test/replication/replica_rejoin.result        |   8 +
>  test/replication/replica_rejoin.test.lua      |   2 +
>  .../show_error_on_disconnect.result           |   8 +
>  .../show_error_on_disconnect.test.lua         |   2 +
>  test/replication/suite.ini                    |   2 +-
>  test/xlog/panic_on_wal_error.result           |  12 +
>  test/xlog/panic_on_wal_error.test.lua         |   3 +
>  test/xlog/suite.ini                           |   2 +-
>  22 files changed, 1045 insertions(+), 223 deletions(-)
>  create mode 100644 src/box/xrow_buf.c
>  create mode 100644 src/box/xrow_buf.h
>