[PATCH 05/10] vinyl: don't purge deleted runs from vylog on compaction

[Vladimir Davydov <vdavydov.dev@gmail.com>]

After compacting runs, we first mark them as dropped (VY_LOG_DROP_RUN),
then try to delete their files unless they are needed for recovery from
the checkpoint, and finally mark them as not needed in the vylog
(VY_LOG_FORGET_RUN). There's a potential race sitting here: the problem
is the garbage collector might kick in after files are dropped, but
before they are marked as not needed. If this happens, there will be
runs that have two VY_LOG_FORGET_RUN records, which will break recovery:

  Run XX is forgotten, but not registered

The following patches make the race more likely to happen so let's
eliminate it by making the garbage collector the only one who can mark
runs as not needed (i.e. write VY_LOG_FORGET_RUN record). There will
be no warnings, because the garbage collector silently ignores ENOENT
errors, see vy_gc().

Another good thing about this patch is that now we never yield inside
a vylog transaction, which makes it easier to remove the vylog latch
blocking implementation of transactional DDL.
---
 src/box/vy_scheduler.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c
index fabb4bb4..0180331e 100644
--- a/src/box/vy_scheduler.c
+++ b/src/box/vy_scheduler.c
@@ -1523,16 +1523,17 @@ vy_task_compaction_complete(struct vy_task *task)
 	 * Remove compacted run files that were created after
 	 * the last checkpoint (and hence are not referenced
 	 * by any checkpoint) immediately to save disk space.
+	 *
+	 * Don't bother logging it to avoid a potential race
+	 * with a garbage collection task, which may be cleaning
+	 * up concurrently. The log will be cleaned up on the
+	 * next checkpoint.
 	 */
-	vy_log_tx_begin();
 	rlist_foreach_entry(run, &unused_runs, in_unused) {
-		if (run->dump_lsn > gc_lsn &&
-		    vy_run_remove_files(lsm->env->path, lsm->space_id,
-					lsm->index_id, run->id) == 0) {
-			vy_log_forget_run(run->id);
-		}
+		if (run->dump_lsn > gc_lsn)
+			vy_run_remove_files(lsm->env->path, lsm->space_id,
+					    lsm->index_id, run->id);
 	}
-	vy_log_tx_try_commit();
 
 	/*
 	 * Account the new run if it is not empty,
-- 
2.11.0