[tarantool-patches] Re: [PATCH v2 0/4] In-memory wal replication

[Vladislav Shpilevoy]

Hi! Thanks for the patchset!

[NOTE: review is not finished yet, I will send more emails]

Please, don't omit branch and issue links. You have 4 branches
having 3794 in their name. You could save a little bit of
reviewer's time by specifying a branch here.

Looks like your branch fails on travis. Please, fix it.

On 18/09/2019 11:36, Georgy Kirichenko wrote:
> This patchset introduces from-wal-memory-replication and consist of 4
> patches.
> 
> The first patch purges distinct relay facilities to advanced gc consumer
> state. Also it changes gc behaviour - now gc compares consumers using
> lsns corresponding to local instance id instead of vclock signature.
> Howevers this changes gc behavior, its look like we could restore the
> gc behavior after relay would be migrated to the wal thread. The whole
> patch is essential because in relay switched to memory relaying there is
> no more file boundaries and on_close_log trigger invocation now.

So you've broken GC and promise to fix it or what? Not sure if I
understand.

> 
> The second patch introduces xrow buffer structure to be used by wal
> while logging. Wal encodes rows to a memory buffer (xrow_buffer
> structure) and then writes encoded data to a xlog file.
> 
> The next patch implements xrow_buffer_cursor to have an ability to found a
> position in the buffer by vclock and then fetch all rows one-by-one
> until the last row.
> 
> The last patch makes a relay able to read from wal memory using
> a xrow_buffer cursor. When relay went out of wal memory window it turns
> back to file mode. After last known xlog file is relayed the a relay
> returns to wal memory.

Please, ask Alexander to bench your branch to check if there are no
performance problems.

> 
> Changes in v2:
>   - Get rid of gc refactoring
>   - Rename wal_memory to xrow_buffer according to review
>   - Commentaries
>   - Bug fixes
> 
> Georgy Kirichenko (4):
>   relay: adjust gc state on relay status update
>   wal: wal memory buffer
>   wal: xrow buffer cursor
>   replication: use wal memory buffer to fetch rows
> 
>  src/box/CMakeLists.txt                        |   1 +
>  src/box/gc.c                                  |   7 +-
>  src/box/relay.cc                              | 275 ++++++-----------
>  src/box/wal.c                                 | 193 +++++++++++-
>  src/box/wal.h                                 |  36 +++
>  src/box/xlog.c                                |  77 +++--
>  src/box/xlog.h                                |   9 +
>  src/box/xrow_buf.c                            | 279 ++++++++++++++++++
>  src/box/xrow_buf.h                            | 160 ++++++++++
>  src/lib/core/cbus.c                           |   4 +
>  src/lib/core/errinj.h                         |   1 +
>  test/box/errinj.result                        |   2 +
>  test/replication/force_recovery.result        |   8 +
>  test/replication/force_recovery.test.lua      |   2 +
>  test/replication/replica_rejoin.result        |   8 +
>  test/replication/replica_rejoin.test.lua      |   2 +
>  .../show_error_on_disconnect.result           |   8 +
>  .../show_error_on_disconnect.test.lua         |   2 +
>  test/replication/suite.ini                    |   2 +-
>  test/xlog/panic_on_wal_error.result           |  12 +
>  test/xlog/panic_on_wal_error.test.lua         |   3 +
>  test/xlog/suite.ini                           |   2 +-
>  22 files changed, 874 insertions(+), 219 deletions(-)
>  create mode 100644 src/box/xrow_buf.c
>  create mode 100644 src/box/xrow_buf.h
>