From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH v2 0/6] Create empty xlog on shutdown Date: Fri, 29 Jun 2018 19:48:27 +0300 Message-Id: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: In order to determine if the instance needs to be rebootstrapped, we need to know its vclock and we need to know it before we start recovery (see #461). We are planning to retrieve the vclock by rescanning the last xlog. In order not to degrade recovery time in case rebootstrap is not needed, let's create a new empty xlog on shutdown. Patches 1-4 of the series improve the LSN gap check so that it doesn't trigger in case an xlog has a gap at the end due to write errors. They are needed to make the tests pass after patch 5 is applied. Patch 5 makes WAL thread create an empty xlog on shutdown and reopen it on restart. Patch 6 is optional: it moves XlogGapError to box/error.h, where other exceptions are defined. https://github.com/tarantool/tarantool/issues/461 https://github.com/tarantool/tarantool/commits/wal-make-new-xlog-on-shutdown Changes in v2: - Drop the patch that makes WAL rollback LSN on write failure. Instead add PrevVclock to xlog header to improve gap check, as suggested by Kostja. - Improve comments and refactor the code as per review. v1: https://www.freelists.org/post/tarantool-patches/PATCH-03-Speed-up-recovery-in-case-rebootstrap-is-not-needed Vladimir Davydov (6): xlog: store prev vclock in xlog header xlog: differentiate between closed and never opened cursor recovery: make LSN gap check more thorough recovery: promote recovery clock even if the WAL is empty wal: create empty xlog on shutdown error: move XlogGapError to box/error.h src/box/CMakeLists.txt | 2 +- src/box/error.cc | 36 ++++++++-- src/box/error.h | 16 +++++ src/box/recovery.cc | 118 +++++++++++++++----------------- src/box/wal.c | 94 ++++++++++++++++++++++++- src/box/xlog.c | 70 ++++++++++++++----- src/box/xlog.h | 17 ++++- src/diag.h | 2 - test/replication/gc.result | 23 +------ test/replication/gc.test.lua | 8 +-- test/replication/hot_standby.result | 12 ++-- test/replication/hot_standby.test.lua | 4 +- test/xlog-py/dup_key.result | 20 ++---- test/xlog-py/dup_key.test.py | 29 +++----- test/xlog/errinj.result | 1 + test/xlog/header.result | 41 +++++++++++ test/xlog/header.test.lua | 13 ++++ test/xlog/panic_on_lsn_gap.result | 125 ++++++++++++++++++++++++++++------ test/xlog/panic_on_lsn_gap.test.lua | 33 +++++++-- test/xlog/panic_on_wal_error.result | 23 +------ test/xlog/panic_on_wal_error.test.lua | 9 +-- 21 files changed, 480 insertions(+), 216 deletions(-) -- 2.11.0