From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH v2 09/11] vinyl: simplify vylog recovery from backup Date: Fri, 8 Jun 2018 20:34:27 +0300 [thread overview] Message-ID: <459b07548e8f08792727a4774cecd2493baaf1ac.1528478913.git.vdavydov.dev@gmail.com> (raw) In-Reply-To: <cover.1528478913.git.vdavydov.dev@gmail.com> In-Reply-To: <cover.1528478913.git.vdavydov.dev@gmail.com> Since we don't create snapshot files for vylog, but instead append records written after checkpoint to the same file, we have to use the previous vylog file for backup (see vy_log_backup_path()). So when recovering from a backup we need to rotate the last vylog to keep vylog and checkpoint signatures in sync. Currently, we do it on recovery completion and we use vy_log_create() instead of vy_log_rotate() for it. This is done so that we can reuse the context that was used for recovery instead of rereading vylog for rotation. Actually, there's no point in this micro-optimization, because we rotate vylog only when recovering from a backup. Let's remove it and use vy_log_rotate() for this. Needed for #461 --- src/box/vy_log.c | 60 +++++++++++++++++++++----------------------------------- 1 file changed, 22 insertions(+), 38 deletions(-) diff --git a/src/box/vy_log.c b/src/box/vy_log.c index 6556dd37..8330a26c 100644 --- a/src/box/vy_log.c +++ b/src/box/vy_log.c @@ -182,6 +182,12 @@ static int vy_recovery_process_record(struct vy_recovery *recovery, const struct vy_log_record *record); +static int +vy_log_create(const struct vclock *vclock, struct vy_recovery *recovery); + +int +vy_log_rotate(const struct vclock *vclock); + /** * Return the name of the vylog file that has the given signature. */ @@ -866,10 +872,11 @@ vy_log_begin_recovery(const struct vclock *vclock) if (xdir_scan(&vy_log.dir) < 0 && errno != ENOENT) return NULL; - struct vclock vy_log_vclock; - vclock_create(&vy_log_vclock); - if (xdir_last_vclock(&vy_log.dir, &vy_log_vclock) >= 0 && - vclock_compare(&vy_log_vclock, vclock) > 0) { + if (xdir_last_vclock(&vy_log.dir, &vy_log.last_checkpoint) < 0) + vclock_copy(&vy_log.last_checkpoint, vclock); + + int cmp = vclock_compare(&vy_log.last_checkpoint, vclock); + if (cmp > 0) { /* * Last vy_log log is newer than the last snapshot. * This can't normally happen, as vy_log is rotated @@ -879,21 +886,27 @@ vy_log_begin_recovery(const struct vclock *vclock) diag_set(ClientError, ER_MISSING_SNAPSHOT); return NULL; } + if (cmp < 0) { + /* + * Last vy_log log is older than the last snapshot. + * This happens if we are recovering from a backup. + * Rotate the log to keep its signature in sync with + * checkpoint. + */ + if (vy_log_rotate(vclock) != 0) + return NULL; + } struct vy_recovery *recovery; - recovery = vy_recovery_new(vclock_sum(&vy_log_vclock), false); + recovery = vy_recovery_new(vclock_sum(&vy_log.last_checkpoint), false); if (recovery == NULL) return NULL; vy_log.next_id = recovery->max_id + 1; vy_log.recovery = recovery; - vclock_copy(&vy_log.last_checkpoint, vclock); return recovery; } -static int -vy_log_create(const struct vclock *vclock, struct vy_recovery *recovery); - int vy_log_end_recovery(void) { @@ -914,35 +927,6 @@ vy_log_end_recovery(void) return -1; } - /* - * On backup we copy files corresponding to the most recent - * checkpoint. Since vy_log does not create snapshots of its log - * files, but instead appends records written after checkpoint - * to the most recent log file, the signature of the vy_log file - * corresponding to the last checkpoint equals the signature - * of the previous checkpoint. So upon successful recovery - * from a backup we need to rotate the log to keep checkpoint - * and vy_log signatures in sync. - */ - struct vclock *vclock = vclockset_last(&vy_log.dir.index); - if (vclock == NULL || - vclock_compare(vclock, &vy_log.last_checkpoint) != 0) { - vclock = malloc(sizeof(*vclock)); - if (vclock == NULL) { - diag_set(OutOfMemory, sizeof(*vclock), - "malloc", "struct vclock"); - return -1; - } - vclock_copy(vclock, &vy_log.last_checkpoint); - xdir_add_vclock(&vy_log.dir, vclock); - if (vy_log_create(vclock, vy_log.recovery) < 0) { - diag_log(); - say_error("failed to write `%s'", - vy_log_filename(vclock_sum(vclock))); - return -1; - } - } - vy_log.recovery = NULL; return 0; } -- 2.11.0
next prev parent reply other threads:[~2018-06-08 17:34 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-06-08 17:34 [PATCH v2 00/11] Replica rejoin Vladimir Davydov 2018-06-08 17:34 ` [PATCH v2 01/11] box: retrieve instance uuid before starting local recovery Vladimir Davydov 2018-06-08 17:51 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 02/11] box: refactor hot standby recovery Vladimir Davydov 2018-06-08 17:34 ` [PATCH v2 03/11] box: retrieve end vclock before starting local recovery Vladimir Davydov 2018-06-14 12:58 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 04/11] box: open the port " Vladimir Davydov 2018-06-13 20:43 ` Konstantin Osipov 2018-06-14 8:31 ` Vladimir Davydov 2018-06-14 12:59 ` Konstantin Osipov 2018-06-15 15:48 ` [PATCH 0/3] Speed up recovery in case rebootstrap is not needed Vladimir Davydov 2018-06-15 15:48 ` [PATCH 1/3] xlog: erase eof marker when reopening existing file for writing Vladimir Davydov 2018-06-27 17:09 ` Konstantin Osipov 2018-06-15 15:48 ` [PATCH 2/3] wal: rollback vclock on write failure Vladimir Davydov 2018-06-27 17:22 ` Konstantin Osipov 2018-06-15 15:48 ` [PATCH 3/3] wal: create empty xlog on shutdown Vladimir Davydov 2018-06-27 17:29 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 05/11] box: connect to remote peers before starting local recovery Vladimir Davydov 2018-06-13 20:45 ` Konstantin Osipov 2018-06-14 8:34 ` Vladimir Davydov 2018-06-14 12:59 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 06/11] box: factor out local recovery function Vladimir Davydov 2018-06-13 20:50 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 07/11] applier: inquire oldest vclock on connect Vladimir Davydov 2018-06-13 20:51 ` Konstantin Osipov 2018-06-14 8:40 ` Vladimir Davydov 2018-06-08 17:34 ` [PATCH v2 08/11] replication: rebootstrap instance on startup if it fell behind Vladimir Davydov 2018-06-13 20:55 ` Konstantin Osipov 2018-06-14 8:58 ` Vladimir Davydov 2018-06-08 17:34 ` Vladimir Davydov [this message] 2018-06-08 17:34 ` [PATCH v2 10/11] vinyl: pass flags to vy_recovery_new Vladimir Davydov 2018-06-13 20:56 ` Konstantin Osipov 2018-06-08 17:34 ` [PATCH v2 11/11] vinyl: implement rebootstrap support Vladimir Davydov 2018-06-10 12:02 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=459b07548e8f08792727a4774cecd2493baaf1ac.1528478913.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH v2 09/11] vinyl: simplify vylog recovery from backup' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox