[Tarantool-patches] [PATCH 0/4] replication: introduce applier thread

[Serge Petrenko via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Patches #1 and #2 rework applier and coio_read_xrow to keep parsed input.
This allows us to get rid of copying row bodies in applier right after parsing
them.

The patches are left after a failed applier-in-thread approach. They are not
needed anymore, strictly speaking, so we may drop them. Or use them. After all,
they make usual applier (fiber in tx thread) better. I'm still sending them for
review.

Patch #3 is a small refactoring, needed for patch #4.

Patch #4 is the applier in thread itself. Here's a brief idea:

There's a separate thread which decodes the incoming rows. This way the applier
fiber (in tx thread) may deal with already decoded requests and doesn't waste
its time on decoding them. This should improve replication performance quite a
bit.

We introduce a new config option: `replication_num_threads`. There may be not
more than `replication_num_threads` applier threads (default is 1). And when
there are more appliers than threads, each thread handles multiple appliers.

The thread has 2 fibers per each applier: a reader and a writer.

The writer simply copies applier_writer_f behaviour. Nothing to see here.

The reader reads ahead as much data as possible and decodes it.
As soon as there is enough data read to decode a full transaction, the reader
decodes the transaction (or multiple transactions) and sends the result to the
tx thread.

I wasted quite some time to make the approach with coio_read_xrow() work. (It
was the first one that came to mind, and looked the simplest, but failed).
Everything came together once I dumped the coio_read_xrow() approach, and
started using readahead.

Some of the tests are still failing in CI and local runs, so I'm sending the
branch without tests now. I'm going to fix the tests now.

Branch:
https://github.com/tarantool/tarantool/tree/sp/gh-6329-applier-in-thread-notest
Issue:
https://github.com/tarantool/tarantool/issues/6329


Serge Petrenko (4):
  xrow: rework coio_read_xrow to keep parsed input
  applier: reuse input buffer to store row bodies
  applier: factor replication stream processing out of subscribe()
  Introduce applier thread

 .../unreleased/gh-6329-applier-in-thread.md   |    5 +
 src/box/applier.cc                            | 1014 +++++++++++++++--
 src/box/applier.h                             |    4 +
 src/box/box.cc                                |    2 +-
 src/box/lua/load_cfg.lua                      |    2 +
 src/box/replication.cc                        |    5 +-
 src/box/replication.h                         |    7 +-
 src/box/xrow_io.cc                            |   28 +-
 src/box/xrow_io.h                             |   18 +-
 src/lib/small                                 |    2 +-
 src/lua/buffer.lua                            |    2 +
 11 files changed, 955 insertions(+), 134 deletions(-)
 create mode 100644 changelogs/unreleased/gh-6329-applier-in-thread.md

-- 
2.30.1 (Apple Git-130)