From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH 05/10] vinyl: don't purge deleted runs from vylog on compaction Date: Fri, 17 May 2019 17:52:39 +0300 Message-Id: <67424f76249971cf6c4fc5a361f294a0b3c88283.1558103547.git.vdavydov.dev@gmail.com> In-Reply-To: References: In-Reply-To: References: To: tarantool-patches@freelists.org List-ID: After compacting runs, we first mark them as dropped (VY_LOG_DROP_RUN), then try to delete their files unless they are needed for recovery from the checkpoint, and finally mark them as not needed in the vylog (VY_LOG_FORGET_RUN). There's a potential race sitting here: the problem is the garbage collector might kick in after files are dropped, but before they are marked as not needed. If this happens, there will be runs that have two VY_LOG_FORGET_RUN records, which will break recovery: Run XX is forgotten, but not registered The following patches make the race more likely to happen so let's eliminate it by making the garbage collector the only one who can mark runs as not needed (i.e. write VY_LOG_FORGET_RUN record). There will be no warnings, because the garbage collector silently ignores ENOENT errors, see vy_gc(). Another good thing about this patch is that now we never yield inside a vylog transaction, which makes it easier to remove the vylog latch blocking implementation of transactional DDL. --- src/box/vy_scheduler.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c index fabb4bb4..0180331e 100644 --- a/src/box/vy_scheduler.c +++ b/src/box/vy_scheduler.c @@ -1523,16 +1523,17 @@ vy_task_compaction_complete(struct vy_task *task) * Remove compacted run files that were created after * the last checkpoint (and hence are not referenced * by any checkpoint) immediately to save disk space. + * + * Don't bother logging it to avoid a potential race + * with a garbage collection task, which may be cleaning + * up concurrently. The log will be cleaned up on the + * next checkpoint. */ - vy_log_tx_begin(); rlist_foreach_entry(run, &unused_runs, in_unused) { - if (run->dump_lsn > gc_lsn && - vy_run_remove_files(lsm->env->path, lsm->space_id, - lsm->index_id, run->id) == 0) { - vy_log_forget_run(run->id); - } + if (run->dump_lsn > gc_lsn) + vy_run_remove_files(lsm->env->path, lsm->space_id, + lsm->index_id, run->id); } - vy_log_tx_try_commit(); /* * Account the new run if it is not empty, -- 2.11.0