[PATCH 00/10] vinyl: don't yield in DDL on_commit triggers

[Vladimir Davydov <vdavydov.dev@gmail.com>]

DDL on_commit triggers may yield on vylog write, even though they don't
really need to wait for the transaction to be flushed to disk, as we can
recover it after restart. This blocks implementation of transactional
DDL (see #4083), because even an empty index creation may yield and
hence can't be executed transactionally along with other lightweight
operations, such as space creation.

This patch reworks vylog implementation so that we don't need to yield
in case the vylog transaction is non-discardable (i.e. stays in the
memory buffer in case of failure). Since on_commit triggers issue only
non-discardable transactions, this makes them non-yielding, as they
should be.

https://github.com/tarantool/tarantool/issues/4218
https://github.com/tarantool/tarantool/commits/dv/gh-4218-vy-remove-yields-from-ddl-commit

Vladimir Davydov (10):
  box: zap atfork callback
  vinyl: add a separate thread for vylog
  vinyl: move vylog recovery to vylog thread
  vinyl: rework vylog transaction backlog implementation
  vinyl: don't purge deleted runs from vylog on compaction
  vinyl: lock out compaction while checkpointing is in progress
  vinyl: don't access last vylog signature outside vylog implementation
  vinyl: zap ERRINJ_VY_LOG_FLUSH_DELAY
  key_def: pass alloc callback to key_def_dump_parts
  vinyl: get rid of the latch protecting vylog buffer

 src/box/box.cc             |  10 -
 src/box/box.h              |   6 -
 src/box/key_def.c          |   6 +-
 src/box/key_def.h          |   4 +-
 src/box/sql/select.c       |   2 +-
 src/box/vinyl.c            |   2 +-
 src/box/vy_log.c           | 618 ++++++++++++++++++++++++++++-----------------
 src/box/vy_log.h           |  23 +-
 src/box/vy_scheduler.c     |  44 +++-
 src/box/wal.c              |  95 -------
 src/box/wal.h              |  18 --
 src/box/xlog.c             |  28 +-
 src/box/xlog.h             |  38 ++-
 src/lib/core/errinj.h      |   1 -
 src/main.cc                |   1 -
 test/box/errinj.result     |  44 ++--
 test/vinyl/errinj.result   |  38 ++-
 test/vinyl/errinj.test.lua |  20 +-
 18 files changed, 519 insertions(+), 479 deletions(-)

-- 
2.11.0