From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH] vinyl: fix index build not working after recovery Date: Mon, 24 Dec 2018 16:54:29 +0300 Message-Id: <6351d916f0162fa17a73ed4a30dd35f9843df5af.1545658557.git.vdavydov.dev@gmail.com> To: tarantool-patches@freelists.org List-ID: While building a secondary index for a Vinyl space, we use xm->lsn to skip statements inserted after build began (as those statements are propagated by the on_replace trigger callback anyway). The xm->lsn is advanced every time we insert something into a Vinyl space. The problem is during recovery we skip statements that have been dumped to disk so xm->lsn may lag behind the instance vclock once recovery is complete. In this case if we try to create a new Vinyl index right after recovery completion, before any DML operation is executed on a Vinyl space, we will skip statements that would otherwise get into the new index. Fix this issue by resetting xm->lsn to the instance vclock upon recovery completion. Closes #3903 --- https://github.com/tarantool/tarantool/issues/3903 https://github.com/tarantool/tarantool/commits/dv/gh-3903-vy-fix-index-build-after-recovery src/box/vinyl.c | 13 ++++++++++++- test/engine/ddl.result | 18 ++++++++++++++++++ test/engine/ddl.test.lua | 7 +++++++ 3 files changed, 37 insertions(+), 1 deletion(-) diff --git a/src/box/vinyl.c b/src/box/vinyl.c index f5b36ce1..ca987134 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -2725,7 +2725,6 @@ vinyl_engine_begin_initial_recovery(struct engine *engine, struct vy_env *e = vy_env(engine); assert(e->status == VINYL_OFFLINE); if (recovery_vclock != NULL) { - e->xm->lsn = vclock_sum(recovery_vclock); e->recovery_vclock = recovery_vclock; e->recovery = vy_log_begin_recovery(recovery_vclock); if (e->recovery == NULL) @@ -2787,6 +2786,18 @@ vinyl_engine_end_recovery(struct engine *engine) vy_gc(e, e->recovery, VY_GC_INCOMPLETE, INT64_MAX); vy_recovery_delete(e->recovery); e->recovery = NULL; + /* + * During recovery we skip statements that have + * been dumped to disk - see vy_is_committed() - + * so it may turn out that tx_manager::lsn stays + * behind the instance vclock while we need it + * to be up-to-date once recovery is complete, + * because we use it while building an index to + * skip statements inserted after build began - + * see vinyl_space_build_index() - so we reset + * it upon recovery completion. + */ + e->xm->lsn = vclock_sum(e->recovery_vclock); e->recovery_vclock = NULL; vy_env_complete_recovery(e); break; diff --git a/test/engine/ddl.result b/test/engine/ddl.result index 3c84e942..e3cff7f4 100644 --- a/test/engine/ddl.result +++ b/test/engine/ddl.result @@ -2114,6 +2114,24 @@ s.index.i3:select() - [4, 'zzz', 'ddd', -666] - [1, 'zzz', 'aaa', 999, 'fff'] ... +-- +-- gh-3903: index build doesn't work after recovery. +-- +s.index.i1:drop() +--- +... +_ = s:create_index('i1', {parts = {2, 'string'}, unique = false}) +--- +... +s.index.i1:select() +--- +- - [3, 'xxx', null, 777, 'ggg'] + - [5, 'xxx', 'eee', 555, 'hhh'] + - [2, 'yyy', 'bbb', -888] + - [6, 'yyy', null, -444] + - [1, 'zzz', 'aaa', 999, 'fff'] + - [4, 'zzz', 'ddd', -666] +... box.snapshot() --- - ok diff --git a/test/engine/ddl.test.lua b/test/engine/ddl.test.lua index 563a20fa..cdaf7a5b 100644 --- a/test/engine/ddl.test.lua +++ b/test/engine/ddl.test.lua @@ -765,6 +765,13 @@ s.index.i1:select() s.index.i2:select() s.index.i3:select() +-- +-- gh-3903: index build doesn't work after recovery. +-- +s.index.i1:drop() +_ = s:create_index('i1', {parts = {2, 'string'}, unique = false}) +s.index.i1:select() + box.snapshot() s:drop() -- 2.11.0