[PATCH v2 0/6] Create empty xlog on shutdown

[Vladimir Davydov <vdavydov.dev@gmail.com>]

In order to determine if the instance needs to be rebootstrapped,
we need to know its vclock and we need to know it before we start
recovery (see #461). We are planning to retrieve the vclock by
rescanning the last xlog. In order not to degrade recovery time
in case rebootstrap is not needed, let's create a new empty xlog
on shutdown.

Patches 1-4 of the series improve the LSN gap check so that it
doesn't trigger in case an xlog has a gap at the end due to write
errors. They are needed to make the tests pass after patch 5 is
applied.

Patch 5 makes WAL thread create an empty xlog on shutdown and
reopen it on restart.

Patch 6 is optional: it moves XlogGapError to box/error.h, where
other exceptions are defined.

https://github.com/tarantool/tarantool/issues/461
https://github.com/tarantool/tarantool/commits/wal-make-new-xlog-on-shutdown

Changes in v2:
 - Drop the patch that makes WAL rollback LSN on write failure.
   Instead add PrevVclock to xlog header to improve gap check,
   as suggested by Kostja.
 - Improve comments and refactor the code as per review.

v1: https://www.freelists.org/post/tarantool-patches/PATCH-03-Speed-up-recovery-in-case-rebootstrap-is-not-needed

Vladimir Davydov (6):
  xlog: store prev vclock in xlog header
  xlog: differentiate between closed and never opened cursor
  recovery: make LSN gap check more thorough
  recovery: promote recovery clock even if the WAL is empty
  wal: create empty xlog on shutdown
  error: move XlogGapError to box/error.h

 src/box/CMakeLists.txt                |   2 +-
 src/box/error.cc                      |  36 ++++++++--
 src/box/error.h                       |  16 +++++
 src/box/recovery.cc                   | 118 +++++++++++++++-----------------
 src/box/wal.c                         |  94 ++++++++++++++++++++++++-
 src/box/xlog.c                        |  70 ++++++++++++++-----
 src/box/xlog.h                        |  17 ++++-
 src/diag.h                            |   2 -
 test/replication/gc.result            |  23 +------
 test/replication/gc.test.lua          |   8 +--
 test/replication/hot_standby.result   |  12 ++--
 test/replication/hot_standby.test.lua |   4 +-
 test/xlog-py/dup_key.result           |  20 ++----
 test/xlog-py/dup_key.test.py          |  29 +++-----
 test/xlog/errinj.result               |   1 +
 test/xlog/header.result               |  41 +++++++++++
 test/xlog/header.test.lua             |  13 ++++
 test/xlog/panic_on_lsn_gap.result     | 125 ++++++++++++++++++++++++++++------
 test/xlog/panic_on_lsn_gap.test.lua   |  33 +++++++--
 test/xlog/panic_on_wal_error.result   |  23 +------
 test/xlog/panic_on_wal_error.test.lua |   9 +--
 21 files changed, 480 insertions(+), 216 deletions(-)

-- 
2.11.0