From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH 02/13] vinyl: force deletion of runs left from unfinished indexes on restart
Date: Thu, 4 Oct 2018 20:20:04 +0300 [thread overview]
Message-ID: <e4d8be2fb5a56ad9f6690fb54220392aaf102f5f.1538671546.git.vdavydov.dev@gmail.com> (raw)
In-Reply-To: <cover.1538671546.git.vdavydov.dev@gmail.com>
In-Reply-To: <cover.1538671546.git.vdavydov.dev@gmail.com>
If an instance is restarted while building a new vinyl index, there will
probably be some run files left. Currently, we won't delete such files
until box.snapshot() is called, even though there's no point in keeping
them around. Let's tweak vy_gc_lsm() so that it marks all runs that
belong to an unfinished index as incomplete to force vy_gc() to remove
them immediately after recovery is complete.
This also removes files left from a failed rebootstrap attempt so we can
remove a call to box.snapshot() from vinyl/replica_rejoin.test.lua.
---
src/box/vinyl.c | 2 ++
test/vinyl/errinj_gc.result | 4 ----
test/vinyl/errinj_gc.test.lua | 1 -
test/vinyl/replica_rejoin.result | 7 -------
test/vinyl/replica_rejoin.test.lua | 2 --
5 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index 0cd7b188..fd24f9b5 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -3246,6 +3246,8 @@ vy_gc_lsm(struct vy_lsm_recovery_info *lsm_info)
}
struct vy_run_recovery_info *run_info;
rlist_foreach_entry(run_info, &lsm_info->runs, in_lsm) {
+ if (lsm_info->create_lsn < 0)
+ run_info->is_incomplete = true;
if (!run_info->is_dropped) {
run_info->is_dropped = true;
run_info->gc_lsn = lsm_info->drop_lsn;
diff --git a/test/vinyl/errinj_gc.result b/test/vinyl/errinj_gc.result
index 7c963103..13a43171 100644
--- a/test/vinyl/errinj_gc.result
+++ b/test/vinyl/errinj_gc.result
@@ -254,10 +254,6 @@ ch:get()
- true
...
test_run:cmd('restart server default')
-box.snapshot()
----
-- ok
-...
fio = require('fio')
---
...
diff --git a/test/vinyl/errinj_gc.test.lua b/test/vinyl/errinj_gc.test.lua
index 4c3d1bb3..aaa499a0 100644
--- a/test/vinyl/errinj_gc.test.lua
+++ b/test/vinyl/errinj_gc.test.lua
@@ -120,7 +120,6 @@ ch:get()
#fio.listdir(fio.pathjoin(box.cfg.vinyl_dir, s.id, 1)) > 0
test_run:cmd('restart server default')
-box.snapshot()
fio = require('fio')
s = box.space.test
diff --git a/test/vinyl/replica_rejoin.result b/test/vinyl/replica_rejoin.result
index bd5d1ed3..d153e346 100644
--- a/test/vinyl/replica_rejoin.result
+++ b/test/vinyl/replica_rejoin.result
@@ -164,13 +164,6 @@ test_run:cmd("switch replica")
---
- true
...
-box.cfg{checkpoint_count = 1}
----
-...
-box.snapshot()
----
-- ok
-...
fio = require('fio')
---
...
diff --git a/test/vinyl/replica_rejoin.test.lua b/test/vinyl/replica_rejoin.test.lua
index 972b04e5..8226fb94 100644
--- a/test/vinyl/replica_rejoin.test.lua
+++ b/test/vinyl/replica_rejoin.test.lua
@@ -59,8 +59,6 @@ test_run:cmd("start server replica with crash_expected=True") -- fail
test_run:cmd("start server replica with crash_expected=True") -- fail again
test_run:cmd("start server replica with args='disable_replication'")
test_run:cmd("switch replica")
-box.cfg{checkpoint_count = 1}
-box.snapshot()
fio = require('fio')
fio.chdir(box.cfg.vinyl_dir)
fio.glob(fio.pathjoin(box.space.test.id, 0, '*'))
--
2.11.0
next prev parent reply other threads:[~2018-10-04 17:20 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-04 17:20 [PATCH 00/13] box: garbage collection refactoring and fixes Vladimir Davydov
2018-10-04 17:20 ` [PATCH 01/13] vinyl: fix master crash on replica join failure Vladimir Davydov
2018-10-04 21:43 ` Konstantin Osipov
2018-10-04 17:20 ` Vladimir Davydov [this message]
2018-10-04 21:44 ` [PATCH 02/13] vinyl: force deletion of runs left from unfinished indexes on restart Konstantin Osipov
2018-10-04 17:20 ` [PATCH 03/13] gc: make gc_consumer and gc_state structs transparent Vladimir Davydov
2018-10-04 21:47 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 04/13] gc: use fixed length buffer for storing consumer name Vladimir Davydov
2018-10-04 21:47 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 05/13] gc: fold gc_consumer_new and gc_consumer_delete Vladimir Davydov
2018-10-04 21:50 ` Konstantin Osipov
2018-10-05 8:56 ` Vladimir Davydov
2018-10-04 17:20 ` [PATCH 06/13] gc: format consumer name in gc_consumer_register Vladimir Davydov
2018-10-04 21:50 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 07/13] gc: rename checkpoint_count to min_checkpoint_count Vladimir Davydov
2018-10-04 21:51 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 08/13] gc: keep track of available checkpoints Vladimir Davydov
2018-10-04 21:59 ` Konstantin Osipov
2018-10-05 8:50 ` Vladimir Davydov
2018-10-04 17:20 ` [PATCH 09/13] gc: cleanup garbage collection procedure Vladimir Davydov
2018-10-04 22:00 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 10/13] gc: improve box.info.gc output Vladimir Davydov
2018-10-04 22:01 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 11/13] gc: separate checkpoint references from wal consumers Vladimir Davydov
2018-10-04 22:05 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 12/13] gc: call gc_run unconditionally when consumer is advanced Vladimir Davydov
2018-10-04 22:26 ` Konstantin Osipov
2018-10-04 17:20 ` [PATCH 13/13] replication: ref checkpoint needed to join replica Vladimir Davydov
2018-10-04 22:27 ` Konstantin Osipov
2018-10-05 17:03 ` [PATCH 00/13] box: garbage collection refactoring and fixes Vladimir Davydov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e4d8be2fb5a56ad9f6690fb54220392aaf102f5f.1538671546.git.vdavydov.dev@gmail.com \
--to=vdavydov.dev@gmail.com \
--cc=kostja@tarantool.org \
--cc=tarantool-patches@freelists.org \
--subject='Re: [PATCH 02/13] vinyl: force deletion of runs left from unfinished indexes on restart' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox