From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH v3 2/3] vinyl: relax limitation imposed on run min/max lsn Date: Thu, 7 Jun 2018 13:56:17 +0300 Message-Id: <6d38ab6a6748d2f80d9e48ff28627c87a1efeec5.1528368754.git.vdavydov.dev@gmail.com> In-Reply-To: References: In-Reply-To: References: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: Currently, we assume that no two runs of the same range intersect by LSN. This holds, because LSNs grow strictly monotonically, and no transaction may be split between two runs (as we pin each affected vy_mem until the transaction is complete). We ensure this with an assertion in vy_task_dump_complete. However, tuples inserted during index build will have arbitrary (not monotonically growing) LSNs. This is OK as for each particular key, two statements will still have different LSNs, but this may break the assertion in vy_task_dump_complete in case dump occurs while build is in progress. To avoid that, let's relax this limitation and assume that a dumped run may intersect by LSN with runs dumped before. Moreover, let's assume that it may have max LSN less than the max LSN stored on disk so that we should update vy_lsm::dump_lsn only if the dumped run has newer data. Needed for #1653 --- src/box/vy_log.c | 2 +- src/box/vy_log.h | 4 ++-- src/box/vy_lsm.h | 2 +- src/box/vy_scheduler.c | 3 +-- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/src/box/vy_log.c b/src/box/vy_log.c index 8da457a6..6556dd37 100644 --- a/src/box/vy_log.c +++ b/src/box/vy_log.c @@ -1485,7 +1485,7 @@ vy_recovery_dump_lsm(struct vy_recovery *recovery, (long long)id)); return -1; } - lsm->dump_lsn = dump_lsn; + lsm->dump_lsn = MAX(lsm->dump_lsn, dump_lsn); return 0; } diff --git a/src/box/vy_log.h b/src/box/vy_log.h index 442563f0..0a216de8 100644 --- a/src/box/vy_log.h +++ b/src/box/vy_log.h @@ -136,7 +136,7 @@ enum vy_log_record_type { */ VY_LOG_DELETE_SLICE = 9, /** - * Update LSN of the last LSM tree dump. + * Log LSM tree dump. Used to update max LSN stored on disk. * Requires vy_log_record::lsm_id, dump_lsn. */ VY_LOG_DUMP_LSM = 10, @@ -303,7 +303,7 @@ struct vy_lsm_recovery_info { * if the tree is still active. */ int64_t drop_lsn; - /** LSN of the last LSM tree dump. */ + /** Max LSN stored on disk. */ int64_t dump_lsn; /** * List of all ranges in the LSM tree, linked by diff --git a/src/box/vy_lsm.h b/src/box/vy_lsm.h index 2e99e4d0..90ccb534 100644 --- a/src/box/vy_lsm.h +++ b/src/box/vy_lsm.h @@ -253,7 +253,7 @@ struct vy_lsm { */ uint32_t range_tree_version; /** - * LSN of the last dump or -1 if the LSM tree has not + * Max LSN stored on disk or -1 if the LSM tree has not * been dumped yet. */ int64_t dump_lsn; diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c index f4746d68..e3f2e223 100644 --- a/src/box/vy_scheduler.c +++ b/src/box/vy_scheduler.c @@ -721,7 +721,6 @@ vy_task_dump_complete(struct vy_scheduler *scheduler, struct vy_task *task) goto delete_mems; } - assert(new_run->info.min_lsn > lsm->dump_lsn); assert(new_run->info.max_lsn <= dump_lsn); /* @@ -828,7 +827,7 @@ delete_mems: vy_stmt_counter_add(&lsm->stat.disk.dump.in, &mem->count); vy_lsm_delete_mem(lsm, mem); } - lsm->dump_lsn = dump_lsn; + lsm->dump_lsn = MAX(lsm->dump_lsn, dump_lsn); lsm->stat.disk.dump.count++; /* The iterator has been cleaned up in a worker thread. */ -- 2.11.0