Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH v2 2/8] vinyl: update recovery context with records written during recovery
Date: Sun, 27 May 2018 22:05:50 +0300	[thread overview]
Message-ID: <958a42889026e5ed70ac64d732b51d3ff9accca2.1527446023.git.vdavydov.dev@gmail.com> (raw)
In-Reply-To: <cover.1527446023.git.vdavydov.dev@gmail.com>
In-Reply-To: <cover.1527446023.git.vdavydov.dev@gmail.com>

During recovery, we may write VY_LOG_CREATE_LSM and VY_LOG_DROP_LSM
records we failed to write before restart (because those records are
written after WAL and hence may not make it to vylog). Right after
recovery we invoke garbage collection to drop incomplete runs. Once
VY_LOG_PREPARE_LSM record is introduced, we will also collect incomplete
LSM trees there (those we failed to build). However, there may be LSM
trees we managed to build but failed to write VY_LOG_CREATE_LSM for.
This is OK as we will retry vylog write, but currenntly it isn't
reflected in the recovery context used for garbage collection. To avoid
purging such LSM trees, let's update the recovery context with records
written during recovery.

Needed for #1653
---
 src/box/vy_log.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/src/box/vy_log.c b/src/box/vy_log.c
index bdf9eee3..069cd528 100644
--- a/src/box/vy_log.c
+++ b/src/box/vy_log.c
@@ -177,6 +177,10 @@ static struct vy_log vy_log;
 static struct vy_recovery *
 vy_recovery_new_locked(int64_t signature, bool only_checkpoint);
 
+static int
+vy_recovery_process_record(struct vy_recovery *recovery,
+			   const struct vy_log_record *record);
+
 /**
  * Return the name of the vylog file that has the given signature.
  */
@@ -890,6 +894,14 @@ vy_log_end_recovery(void)
 {
 	assert(vy_log.recovery != NULL);
 
+	/*
+	 * Update the recovery context with records written during
+	 * recovery - we will need them for garbage collection.
+	 */
+	struct vy_log_record *record;
+	stailq_foreach_entry(record, &vy_log.tx, in_tx)
+		vy_recovery_process_record(vy_log.recovery, record);
+
 	/* Flush all pending records. */
 	if (vy_log_flush() < 0) {
 		diag_log();
-- 
2.11.0

  parent reply	other threads:[~2018-05-27 19:05 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-27 19:05 [PATCH v2 0/8] Allow to build indexes for vinyl spaces Vladimir Davydov
2018-05-27 19:05 ` [PATCH v2 1/8] vinyl: allocate key parts in vy_recovery_do_create_lsm Vladimir Davydov
2018-05-30 11:51   ` Konstantin Osipov
2018-05-27 19:05 ` Vladimir Davydov [this message]
2018-05-30 11:51   ` [PATCH v2 2/8] vinyl: update recovery context with records written during recovery Konstantin Osipov
2018-05-27 19:05 ` [PATCH v2 3/8] vinyl: log new index before WAL write on DDL Vladimir Davydov
2018-06-06 18:01   ` Konstantin Osipov
2018-05-27 19:05 ` [PATCH v2 4/8] vinyl: bump mem version after committing statement Vladimir Davydov
2018-06-07  5:41   ` Konstantin Osipov
2018-05-27 19:05 ` [PATCH v2 5/8] vinyl: allow to commit statements to mem in arbitrary order Vladimir Davydov
2018-06-07  5:41   ` Konstantin Osipov
2018-05-27 19:05 ` [PATCH v2 6/8] vinyl: relax limitation imposed on run min/max lsn Vladimir Davydov
2018-05-27 19:05 ` [PATCH v2 7/8] vinyl: factor out vy_check_is_unique_secondary Vladimir Davydov
2018-05-27 19:05 ` [PATCH v2 8/8] vinyl: allow to build secondary index for non-empty space Vladimir Davydov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=958a42889026e5ed70ac64d732b51d3ff9accca2.1527446023.git.vdavydov.dev@gmail.com \
    --to=vdavydov.dev@gmail.com \
    --cc=kostja@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH v2 2/8] vinyl: update recovery context with records written during recovery' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox