From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH v2 5/5] alter: rewrite space truncation using alter infrastructure Date: Tue, 20 Mar 2018 14:29:05 +0300 [thread overview] Message-ID: <c77b8c830ecbbfce722d16f206c33948af321c83.1521545062.git.vdavydov.dev@gmail.com> (raw) In-Reply-To: <cover.1521545062.git.vdavydov.dev@gmail.com> In-Reply-To: <cover.1521545062.git.vdavydov.dev@gmail.com> Truncation of a space is equivalent to recreation of all space indexes with the same definition. The reason why we use a special system space to trigger space truncation (_truncate) is that we don't have transactional DDL while space truncation has to be done atomically. However, apart from the new system space, implementation of truncation entailed a new vylog record (VY_LOG_TRUNCATE_INDEX) and quite a few lines of code to handle it. So why couldn't we just invoke ALTER that would recreate all indexes? To answer this question, one needs to recall that back then vinyl used LSN to identify indexes in vylog. As a result, we couldn't recreate more than one index in one operation - if we did that, they would all have the same LSN and hence wouldn't be distinguishable in vylog. So we had to introduce a special vylog operation (VY_LOG_TRUNCATE_INDEX) that bump the truncation counter of an index instead of just dropping and recreating it. We also had to introduce a pair of new virtual space methods, prepare_truncate and commit_truncate so that we could write this new command to vylog in vinyl. Putting it all together, it becomes obvious why we couldn't reuse ALTER code for space truncation. Fortunately, things have changed since then. Now, vylog identifies indexes by space_id/index_id. That means that now we can simplify space truncation implementation a great deal by - reusing alter_space_do() for space truncation, - dropping space_vtab::prepare_truncate and commit_truncate, - removing truncate_count from space, index, and vylog. --- src/box/alter.cc | 107 ++++--------------------------- src/box/memtx_space.c | 29 +++------ src/box/space.h | 44 ------------- src/box/sysview_engine.c | 17 ----- src/box/vinyl.c | 160 ++++------------------------------------------- src/box/vy_index.c | 25 +------- src/box/vy_index.h | 22 ------- src/box/vy_log.c | 56 +---------------- src/box/vy_log.h | 29 ++++----- 9 files changed, 48 insertions(+), 441 deletions(-) diff --git a/src/box/alter.cc b/src/box/alter.cc index 8455373b..54db02e3 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -826,7 +826,6 @@ alter_space_do(struct txn *txn, struct alter_space *alter) space_prepare_alter_xc(alter->old_space, alter->new_space); alter->new_space->sequence = alter->old_space->sequence; - alter->new_space->truncate_count = alter->old_space->truncate_count; memcpy(alter->new_space->access, alter->old_space->access, sizeof(alter->old_space->access)); @@ -1803,48 +1802,6 @@ on_replace_dd_index(struct trigger * /* trigger */, void *event) scoped_guard.is_active = false; } -/* {{{ space truncate */ - -struct truncate_space { - /** Space being truncated. */ - struct space *old_space; - /** Space created as a result of truncation. */ - struct space *new_space; - /** Trigger executed to commit truncation. */ - struct trigger on_commit; - /** Trigger executed to rollback truncation. */ - struct trigger on_rollback; -}; - -/** - * Call the engine specific method to commit truncation - * and delete the old space. - */ -static void -truncate_space_commit(struct trigger *trigger, void * /* event */) -{ - struct truncate_space *truncate = - (struct truncate_space *) trigger->data; - space_commit_truncate(truncate->old_space, truncate->new_space); - space_delete(truncate->old_space); -} - -/** - * Move the old space back to the cache and delete - * the new space. - */ -static void -truncate_space_rollback(struct trigger *trigger, void * /* event */) -{ - struct truncate_space *truncate = - (struct truncate_space *) trigger->data; - if (space_cache_replace(truncate->old_space) != truncate->new_space) - unreachable(); - - space_swap_triggers(truncate->new_space, truncate->old_space); - space_delete(truncate->new_space); -} - /** * A trigger invoked on replace in space _truncate. * @@ -1871,16 +1828,13 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) uint32_t space_id = tuple_field_u32_xc(new_tuple, BOX_TRUNCATE_FIELD_SPACE_ID); - uint64_t truncate_count = - tuple_field_u64_xc(new_tuple, BOX_TRUNCATE_FIELD_COUNT); struct space *old_space = space_cache_find_xc(space_id); if (stmt->row->type == IPROTO_INSERT) { /* * Space creation during initial recovery - - * initialize truncate_count. + * nothing to do. */ - old_space->truncate_count = truncate_count; return; } @@ -1898,59 +1852,24 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) */ access_check_space_xc(old_space, PRIV_W); - /* - * Truncate counter is updated - truncate the space. - */ - struct truncate_space *truncate = - region_calloc_object_xc(&fiber()->gc, struct truncate_space); - - /* Create an empty copy of the old space. */ - struct rlist key_list; - space_dump_def(old_space, &key_list); - struct space *new_space = space_new_xc(old_space->def, &key_list); - new_space->truncate_count = truncate_count; - auto space_guard = make_scoped_guard([=] { space_delete(new_space); }); - - /* Notify the engine about upcoming space truncation. */ - space_prepare_truncate_xc(old_space, new_space); - - space_guard.is_active = false; - - /* Preserve the access control lists during truncate. */ - memcpy(new_space->access, old_space->access, sizeof(old_space->access)); - - /* Truncate does not affect space sequence. */ - new_space->sequence = old_space->sequence; - - /* - * Replace the old space with the new one in the space - * cache. Requests processed after this point will see - * the space as truncated. - */ - if (space_cache_replace(new_space) != old_space) - unreachable(); + struct alter_space *alter = alter_space_new(old_space); + auto scoped_guard = + make_scoped_guard([=] { alter_space_delete(alter); }); /* - * Register the trigger that will commit or rollback - * truncation depending on whether WAL write succeeds - * or fails. + * Recreate all indexes of the truncated space. */ - truncate->old_space = old_space; - truncate->new_space = new_space; - - trigger_create(&truncate->on_commit, - truncate_space_commit, truncate, NULL); - txn_on_commit(txn, &truncate->on_commit); - - trigger_create(&truncate->on_rollback, - truncate_space_rollback, truncate, NULL); - txn_on_rollback(txn, &truncate->on_rollback); + for (uint32_t i = 0; i < old_space->index_count; i++) { + struct index *old_index = old_space->index[i]; + (void) new DropIndex(alter, old_index->def); + auto create_index = new CreateIndex(alter); + create_index->new_index_def = index_def_dup_xc(old_index->def); + } - space_swap_triggers(truncate->new_space, truncate->old_space); + alter_space_do(txn, alter); + scoped_guard.is_active = false; } -/* }}} */ - /* {{{ access control */ bool diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index 2d94597a..c7e58946 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -818,16 +818,6 @@ memtx_space_build_secondary_key(struct space *old_space, return rc; } -static int -memtx_space_prepare_truncate(struct space *old_space, - struct space *new_space) -{ - struct memtx_space *old_memtx_space = (struct memtx_space *)old_space; - struct memtx_space *new_memtx_space = (struct memtx_space *)new_space; - new_memtx_space->replace = old_memtx_space->replace; - return 0; -} - static void memtx_space_prune(struct space *space) { @@ -858,14 +848,6 @@ fail: panic("failed to prune space"); } -static void -memtx_space_commit_truncate(struct space *old_space, - struct space *new_space) -{ - (void)new_space; - memtx_space_prune(old_space); -} - static int memtx_space_prepare_alter(struct space *old_space, struct space *new_space) { @@ -883,9 +865,14 @@ memtx_space_commit_alter(struct space *old_space, struct space *new_space) { struct memtx_space *old_memtx_space = (struct memtx_space *)old_space; struct memtx_space *new_memtx_space = (struct memtx_space *)new_space; + bool is_empty = new_space->index_count == 0 || + index_size(new_space->index[0]) == 0; - /* Delete all tuples when the last index is dropped. */ - if (new_space->index_count == 0) + /* + * Delete all tuples when the last index is dropped + * or the space is truncated. + */ + if (is_empty) memtx_space_prune(old_space); else new_memtx_space->bsize = old_memtx_space->bsize; @@ -908,8 +895,6 @@ static const struct space_vtab memtx_space_vtab = { /* .drop_primary_key = */ memtx_space_drop_primary_key, /* .check_format = */ memtx_space_check_format, /* .build_secondary_key = */ memtx_space_build_secondary_key, - /* .prepare_truncate = */ memtx_space_prepare_truncate, - /* .commit_truncate = */ memtx_space_commit_truncate, /* .prepare_alter = */ memtx_space_prepare_alter, /* .commit_alter = */ memtx_space_commit_alter, }; diff --git a/src/box/space.h b/src/box/space.h index 6408eedc..65f1531d 100644 --- a/src/box/space.h +++ b/src/box/space.h @@ -104,23 +104,6 @@ struct space_vtab { struct space *new_space, struct index *new_index); /** - * Notify the enigne about upcoming space truncation - * so that it can prepare new_space object. - */ - int (*prepare_truncate)(struct space *old_space, - struct space *new_space); - /** - * Commit space truncation. Called after space truncate - * record was written to WAL hence must not fail. - * - * The old_space is the space that was replaced with the - * new_space as a result of truncation. The callback is - * supposed to release resources associated with the - * old_space and commit the new_space. - */ - void (*commit_truncate)(struct space *old_space, - struct space *new_space); - /** * Notify the engine about the changed space, * before it's done, to prepare 'new_space' object. */ @@ -167,12 +150,6 @@ struct space { struct space_def *def; /** Sequence attached to this space or NULL. */ struct sequence *sequence; - /** - * Number of times the space has been truncated. - * Updating this counter via _truncate space triggers - * space truncation. - */ - uint64_t truncate_count; /** Enable/disable triggers. */ bool run_triggers; /** @@ -354,20 +331,6 @@ space_build_secondary_key(struct space *old_space, } static inline int -space_prepare_truncate(struct space *old_space, struct space *new_space) -{ - assert(old_space->vtab == new_space->vtab); - return new_space->vtab->prepare_truncate(old_space, new_space); -} - -static inline void -space_commit_truncate(struct space *old_space, struct space *new_space) -{ - assert(old_space->vtab == new_space->vtab); - new_space->vtab->commit_truncate(old_space, new_space); -} - -static inline int space_prepare_alter(struct space *old_space, struct space *new_space) { assert(old_space->vtab == new_space->vtab); @@ -525,13 +488,6 @@ space_build_secondary_key_xc(struct space *old_space, } static inline void -space_prepare_truncate_xc(struct space *old_space, struct space *new_space) -{ - if (space_prepare_truncate(old_space, new_space) != 0) - diag_raise(); -} - -static inline void space_prepare_alter_xc(struct space *old_space, struct space *new_space) { if (space_prepare_alter(old_space, new_space) != 0) diff --git a/src/box/sysview_engine.c b/src/box/sysview_engine.c index 27d9263a..f6122645 100644 --- a/src/box/sysview_engine.c +++ b/src/box/sysview_engine.c @@ -147,21 +147,6 @@ sysview_space_build_secondary_key(struct space *old_space, } static int -sysview_space_prepare_truncate(struct space *old_space, struct space *new_space) -{ - (void)old_space; - (void)new_space; - return 0; -} - -static void -sysview_space_commit_truncate(struct space *old_space, struct space *new_space) -{ - (void)old_space; - (void)new_space; -} - -static int sysview_space_prepare_alter(struct space *old_space, struct space *new_space) { (void)old_space; @@ -200,8 +185,6 @@ static const struct space_vtab sysview_space_vtab = { /* .drop_primary_key = */ sysview_space_drop_primary_key, /* .check_format = */ sysview_space_check_format, /* .build_secondary_key = */ sysview_space_build_secondary_key, - /* .prepare_truncate = */ sysview_space_prepare_truncate, - /* .commit_truncate = */ sysview_space_commit_truncate, /* .prepare_alter = */ sysview_space_prepare_alter, /* .commit_alter = */ sysview_space_commit_alter, }; diff --git a/src/box/vinyl.c b/src/box/vinyl.c index 1908d5fc..f13a9fe9 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -904,137 +904,6 @@ vinyl_init_system_space(struct space *space) } static int -vinyl_space_prepare_truncate(struct space *old_space, struct space *new_space) -{ - struct vy_env *env = vy_env(old_space->engine); - - if (vinyl_check_wal(env, "DDL") != 0) - return -1; - - assert(old_space->index_count == new_space->index_count); - uint32_t index_count = new_space->index_count; - if (index_count == 0) - return 0; - - struct vy_index *pk = vy_index(old_space->index[0]); - - /* - * On local recovery, we need to handle the following - * scenarios: - * - * - Space truncation was successfully logged before restart. - * In this case indexes of the old space contain data added - * after truncation (recovered by vy_index_recover()) and - * hence we just need to swap contents between old and new - * spaces. - * - * - We failed to log space truncation before restart. - * In this case we have to replay space truncation the - * same way we handle it during normal operation. - * - * See also vy_commit_truncate_space(). - */ - bool truncate_done = (env->status == VINYL_FINAL_RECOVERY_LOCAL && - pk->truncate_count > old_space->truncate_count); - - for (uint32_t i = 0; i < index_count; i++) { - struct vy_index *old_index = vy_index(old_space->index[i]); - struct vy_index *new_index = vy_index(new_space->index[i]); - - new_index->id = old_index->id; - new_index->is_committed = old_index->is_committed; - - if (truncate_done) { - /* - * We are replaying truncate from WAL and the - * old space already contains data added after - * truncate (recovered from vylog). Avoid - * reloading the space content from vylog, - * simply swap the contents of old and new - * spaces instead. - */ - vy_index_swap(old_index, new_index); - new_index->is_dropped = old_index->is_dropped; - new_index->truncate_count = old_index->truncate_count; - vy_scheduler_remove_index(&env->scheduler, old_index); - vy_scheduler_add_index(&env->scheduler, new_index); - continue; - } - - if (vy_index_init_range_tree(new_index) != 0) - return -1; - - new_index->truncate_count = new_space->truncate_count; - } - return 0; -} - -static void -vinyl_space_commit_truncate(struct space *old_space, struct space *new_space) -{ - struct vy_env *env = vy_env(old_space->engine); - - assert(old_space->index_count == new_space->index_count); - uint32_t index_count = new_space->index_count; - if (index_count == 0) - return; - - struct vy_index *pk = vy_index(old_space->index[0]); - - /* - * See the comment in vy_prepare_truncate_space(). - */ - if (env->status == VINYL_FINAL_RECOVERY_LOCAL && - pk->truncate_count > old_space->truncate_count) - return; - - /* - * Mark old indexes as dropped and remove them from the scheduler. - * After this point no task can be scheduled or completed for any - * of them (only aborted). - */ - for (uint32_t i = 0; i < index_count; i++) { - struct vy_index *index = vy_index(old_space->index[i]); - index->is_dropped = true; - vy_scheduler_remove_index(&env->scheduler, index); - } - - /* - * Log change in metadata. - * - * Since we can't fail here, in case of vylog write failure - * we leave records we failed to write in vylog buffer so - * that they get flushed along with the next write. If they - * don't, we will replay them during WAL recovery. - */ - vy_log_tx_begin(); - int64_t gc_lsn = checkpoint_last(NULL); - for (uint32_t i = 0; i < index_count; i++) { - struct vy_index *old_index = vy_index(old_space->index[i]); - struct vy_index *new_index = vy_index(new_space->index[i]); - struct vy_range *range = vy_range_tree_first(new_index->tree); - - assert(!new_index->is_dropped); - assert(new_index->truncate_count == new_space->truncate_count); - assert(new_index->range_count == 1); - - vy_log_index_prune(old_index, gc_lsn); - vy_log_insert_range(new_index->id, range->id, NULL, NULL); - vy_log_truncate_index(new_index->id, new_index->truncate_count); - } - vy_log_tx_try_commit(); - - /* - * After we committed space truncation in the metadata log, - * we can make new indexes eligible for dump and compaction. - */ - for (uint32_t i = 0; i < index_count; i++) { - struct vy_index *index = vy_index(new_space->index[i]); - vy_scheduler_add_index(&env->scheduler, index); - } -} - -static int vinyl_space_prepare_alter(struct space *old_space, struct space *new_space) { struct vy_env *env = vy_env(old_space->engine); @@ -1329,15 +1198,12 @@ vinyl_index_bsize(struct index *base) * either. */ static inline bool -vy_is_committed_one(struct vy_env *env, struct space *space, - struct vy_index *index) +vy_is_committed_one(struct vy_env *env, struct vy_index *index) { if (likely(env->status != VINYL_FINAL_RECOVERY_LOCAL)) return false; if (index->is_dropped) return true; - if (index->truncate_count > space->truncate_count) - return true; if (vclock_sum(env->recovery_vclock) <= index->dump_lsn) return true; return false; @@ -1354,7 +1220,7 @@ vy_is_committed(struct vy_env *env, struct space *space) return false; for (uint32_t iid = 0; iid < space->index_count; iid++) { struct vy_index *index = vy_index(space->index[iid]); - if (!vy_is_committed_one(env, space, index)) + if (!vy_is_committed_one(env, index)) return false; } return true; @@ -1591,7 +1457,7 @@ vy_replace_impl(struct vy_env *env, struct vy_tx *tx, struct space *space, if (pk == NULL) /* space has no primary key */ return -1; /* Primary key is dumped last. */ - assert(!vy_is_committed_one(env, space, pk)); + assert(!vy_is_committed_one(env, pk)); assert(pk->index_id == 0); if (tuple_validate_raw(pk->mem_format, request->tuple)) return -1; @@ -1630,7 +1496,7 @@ vy_replace_impl(struct vy_env *env, struct vy_tx *tx, struct space *space, for (uint32_t iid = 1; iid < space->index_count; ++iid) { struct vy_index *index; index = vy_index(space->index[iid]); - if (vy_is_committed_one(env, space, index)) + if (vy_is_committed_one(env, index)) continue; /* * Delete goes first, so if old and new keys @@ -1763,7 +1629,7 @@ vy_delete_impl(struct vy_env *env, struct vy_tx *tx, struct space *space, if (pk == NULL) return -1; /* Primary key is dumped last. */ - assert(!vy_is_committed_one(env, space, pk)); + assert(!vy_is_committed_one(env, pk)); struct tuple *delete = vy_stmt_new_surrogate_delete(pk->mem_format, tuple); if (delete == NULL) @@ -1775,7 +1641,7 @@ vy_delete_impl(struct vy_env *env, struct vy_tx *tx, struct space *space, struct vy_index *index; for (uint32_t i = 1; i < space->index_count; ++i) { index = vy_index(space->index[i]); - if (vy_is_committed_one(env, space, index)) + if (vy_is_committed_one(env, index)) continue; if (vy_tx_set(tx, index, delete) != 0) goto error; @@ -1924,7 +1790,7 @@ vy_update(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, assert(pk != NULL); assert(pk->index_id == 0); /* Primary key is dumped last. */ - assert(!vy_is_committed_one(env, space, pk)); + assert(!vy_is_committed_one(env, pk)); uint64_t column_mask = 0; const char *new_tuple, *new_tuple_end; uint32_t new_size, old_size; @@ -1978,7 +1844,7 @@ vy_update(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, for (uint32_t i = 1; i < space->index_count; ++i) { index = vy_index(space->index[i]); - if (vy_is_committed_one(env, space, index)) + if (vy_is_committed_one(env, index)) continue; if (vy_tx_set(tx, index, delete) != 0) goto error; @@ -2162,7 +2028,7 @@ vy_upsert(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, if (pk == NULL) return -1; /* Primary key is dumped last. */ - assert(!vy_is_committed_one(env, space, pk)); + assert(!vy_is_committed_one(env, pk)); if (tuple_validate_raw(pk->mem_format, tuple)) return -1; @@ -2258,7 +2124,7 @@ vy_upsert(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, for (uint32_t i = 1; i < space->index_count; ++i) { index = vy_index(space->index[i]); - if (vy_is_committed_one(env, space, index)) + if (vy_is_committed_one(env, index)) continue; if (vy_tx_set(tx, index, delete) != 0) goto error; @@ -2298,7 +2164,7 @@ vy_insert(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, return -1; assert(pk->index_id == 0); /* Primary key is dumped last. */ - assert(!vy_is_committed_one(env, space, pk)); + assert(!vy_is_committed_one(env, pk)); if (tuple_validate_raw(pk->mem_format, request->tuple)) return -1; /* First insert into the primary index. */ @@ -2311,7 +2177,7 @@ vy_insert(struct vy_env *env, struct vy_tx *tx, struct txn_stmt *stmt, for (uint32_t iid = 1; iid < space->index_count; ++iid) { struct vy_index *index = vy_index(space->index[iid]); - if (vy_is_committed_one(env, space, index)) + if (vy_is_committed_one(env, index)) continue; if (vy_insert_secondary(env, tx, space, index, stmt->new_tuple) != 0) @@ -4080,8 +3946,6 @@ static const struct space_vtab vinyl_space_vtab = { /* .drop_primary_key = */ vinyl_space_drop_primary_key, /* .check_format = */ vinyl_space_check_format, /* .build_secondary_key = */ vinyl_space_build_secondary_key, - /* .prepare_truncate = */ vinyl_space_prepare_truncate, - /* .commit_truncate = */ vinyl_space_commit_truncate, /* .prepare_alter = */ vinyl_space_prepare_alter, /* .commit_alter = */ vinyl_space_commit_alter, }; diff --git a/src/box/vy_index.c b/src/box/vy_index.c index de8c5f1e..59c79910 100644 --- a/src/box/vy_index.c +++ b/src/box/vy_index.c @@ -318,23 +318,8 @@ vy_index_delete(struct vy_index *index) free(index); } -void -vy_index_swap(struct vy_index *old_index, struct vy_index *new_index) -{ - assert(old_index->stat.memory.count.rows == 0); - assert(new_index->stat.memory.count.rows == 0); - - SWAP(old_index->dump_lsn, new_index->dump_lsn); - SWAP(old_index->range_count, new_index->range_count); - SWAP(old_index->run_count, new_index->run_count); - SWAP(old_index->stat, new_index->stat); - SWAP(old_index->run_hist, new_index->run_hist); - SWAP(old_index->tree, new_index->tree); - SWAP(old_index->range_heap, new_index->range_heap); - rlist_swap(&old_index->runs, &new_index->runs); -} - -int +/** Initialize the range tree of a new index. */ +static int vy_index_init_range_tree(struct vy_index *index) { struct vy_range *range = vy_range_new(vy_log_next_id(), NULL, NULL, @@ -606,11 +591,6 @@ vy_index_recover(struct vy_index *index, struct vy_recovery *recovery, */ index->is_dropped = true; /* - * If the index was dropped, we don't need to replay - * truncate (see vinyl_space_prepare_truncate()). - */ - index->truncate_count = UINT64_MAX; - /* * We need range tree initialized for all indexes, * even for dropped ones. */ @@ -621,7 +601,6 @@ vy_index_recover(struct vy_index *index, struct vy_recovery *recovery, * Loading the last incarnation of the index from vylog. */ index->dump_lsn = index_info->dump_lsn; - index->truncate_count = index_info->truncate_count; int rc = 0; struct vy_range_recovery_info *range_info; diff --git a/src/box/vy_index.h b/src/box/vy_index.h index 417f2d36..33d1da4a 100644 --- a/src/box/vy_index.h +++ b/src/box/vy_index.h @@ -267,15 +267,6 @@ struct vy_index { */ bool is_dropped; /** - * Number of times the index was truncated. - * - * After recovery is complete, it equals space->truncate_count. - * On local recovery, it is loaded from the metadata log and may - * be greater than space->truncate_count, which indicates that - * the space is truncated in WAL. - */ - uint64_t truncate_count; - /** * If pin_count > 0 the index can't be scheduled for dump. * Used to make sure that the primary index is dumped last. */ @@ -346,19 +337,6 @@ vy_index_unref(struct vy_index *index) } /** - * Swap disk contents (ranges, runs, and corresponding stats) - * between two indexes. Used only on recovery, to skip reloading - * indexes of a truncated space. The in-memory tree of the index - * can't be populated - see vy_is_committed_one(). - */ -void -vy_index_swap(struct vy_index *old_index, struct vy_index *new_index); - -/** Initialize the range tree of a new index. */ -int -vy_index_init_range_tree(struct vy_index *index); - -/** * Create a new vinyl index. * * This function is called when an index is created after recovery diff --git a/src/box/vy_log.c b/src/box/vy_log.c index 9c8dd631..0a5dd26e 100644 --- a/src/box/vy_log.c +++ b/src/box/vy_log.c @@ -264,10 +264,6 @@ vy_log_record_snprint(char *buf, int size, const struct vy_log_record *record) SNPRINT(total, snprintf, buf, size, "%s=%"PRIi64", ", vy_log_key_name[VY_LOG_KEY_GC_LSN], record->gc_lsn); - if (record->truncate_count > 0) - SNPRINT(total, snprintf, buf, size, "%s=%"PRIi64", ", - vy_log_key_name[VY_LOG_KEY_TRUNCATE_COUNT], - record->truncate_count); SNPRINT(total, snprintf, buf, size, "}"); return total; } @@ -379,11 +375,6 @@ vy_log_record_encode(const struct vy_log_record *record, size += mp_sizeof_uint(record->gc_lsn); n_keys++; } - if (record->truncate_count > 0) { - size += mp_sizeof_uint(VY_LOG_KEY_TRUNCATE_COUNT); - size += mp_sizeof_uint(record->truncate_count); - n_keys++; - } size += mp_sizeof_map(n_keys); /* @@ -454,10 +445,6 @@ vy_log_record_encode(const struct vy_log_record *record, pos = mp_encode_uint(pos, VY_LOG_KEY_GC_LSN); pos = mp_encode_uint(pos, record->gc_lsn); } - if (record->truncate_count > 0) { - pos = mp_encode_uint(pos, VY_LOG_KEY_TRUNCATE_COUNT); - pos = mp_encode_uint(pos, record->truncate_count); - } assert(pos == tuple + size); /* @@ -577,7 +564,7 @@ vy_log_record_decode(struct vy_log_record *record, record->gc_lsn = mp_decode_uint(&pos); break; case VY_LOG_KEY_TRUNCATE_COUNT: - record->truncate_count = mp_decode_uint(&pos); + /* Not used anymore, ignore. */ break; default: diag_set(ClientError, ER_INVALID_VYLOG_FILE, @@ -1277,7 +1264,6 @@ vy_recovery_create_index(struct vy_recovery *recovery, int64_t id, index->is_dropped = false; index->commit_lsn = commit_lsn; index->dump_lsn = -1; - index->truncate_count = 0; /* * Add the index to the hash. @@ -1374,33 +1360,6 @@ vy_recovery_dump_index(struct vy_recovery *recovery, } /** - * Handle a VY_LOG_TRUNCATE_INDEX log record. - * This function updates truncate_count of the index with ID @id. - * Returns 0 on success, -1 if ID not found or index is dropped. - */ -static int -vy_recovery_truncate_index(struct vy_recovery *recovery, - int64_t id, int64_t truncate_count) -{ - struct vy_index_recovery_info *index; - index = vy_recovery_lookup_index(recovery, id); - if (index == NULL) { - diag_set(ClientError, ER_INVALID_VYLOG_FILE, - tt_sprintf("Truncation of unregistered index %lld", - (long long)id)); - return -1; - } - if (index->is_dropped) { - diag_set(ClientError, ER_INVALID_VYLOG_FILE, - tt_sprintf("Truncation of deleted index %lld", - (long long)id)); - return -1; - } - index->truncate_count = truncate_count; - return 0; -} - -/** * Allocate a vinyl run with ID @run_id and insert it to the hash. * Return the new run on success, NULL on OOM. */ @@ -1842,8 +1801,8 @@ vy_recovery_process_record(struct vy_recovery *recovery, record->dump_lsn); break; case VY_LOG_TRUNCATE_INDEX: - rc = vy_recovery_truncate_index(recovery, record->index_id, - record->truncate_count); + /* Not used anymore, ignore. */ + rc = 0; break; default: unreachable(); @@ -2056,15 +2015,6 @@ vy_log_append_index(struct xlog *xlog, struct vy_index_recovery_info *index) if (vy_log_append_record(xlog, &record) != 0) return -1; - if (index->truncate_count > 0) { - vy_log_record_init(&record); - record.type = VY_LOG_TRUNCATE_INDEX; - record.index_id = index->id; - record.truncate_count = index->truncate_count; - if (vy_log_append_record(xlog, &record) != 0) - return -1; - } - if (index->dump_lsn >= 0) { vy_log_record_init(&record); record.type = VY_LOG_DUMP_INDEX; diff --git a/src/box/vy_log.h b/src/box/vy_log.h index 19987c61..8e1a2a1d 100644 --- a/src/box/vy_log.h +++ b/src/box/vy_log.h @@ -151,8 +151,17 @@ enum vy_log_record_type { */ VY_LOG_SNAPSHOT = 11, /** - * Update truncate count of a vinyl index. - * Requires vy_log_record::index_id, truncate_count. + * When we used LSN for identifying indexes in vylog, we + * couldn't simply recreate an index on space truncation, + * because in case the space had more than one index, we + * wouldn't be able to distinguish them after truncation. + * So we wrote special 'truncate' record. + * + * Now, we assign a unique id to each index and so we don't + * need a special record type for space truncation. If we + * are recovering from an old vylog, we simply ignore all + * 'truncate' records - this will result in replay of all + * WAL records written after truncation. */ VY_LOG_TRUNCATE_INDEX = 12, @@ -200,8 +209,6 @@ struct vy_log_record { * that uses this run. */ int64_t gc_lsn; - /** Index truncate count. */ - int64_t truncate_count; /** Link in vy_log::tx. */ struct stailq_entry in_tx; }; @@ -250,8 +257,6 @@ struct vy_index_recovery_info { int64_t commit_lsn; /** LSN of the last index dump. */ int64_t dump_lsn; - /** Truncate count. */ - int64_t truncate_count; /** * List of all ranges in the index, linked by * vy_range_recovery_info::in_index. @@ -635,18 +640,6 @@ vy_log_dump_index(int64_t id, int64_t dump_lsn) vy_log_write(&record); } -/** Helper to log index truncation. */ -static inline void -vy_log_truncate_index(int64_t id, int64_t truncate_count) -{ - struct vy_log_record record; - vy_log_record_init(&record); - record.type = VY_LOG_TRUNCATE_INDEX; - record.index_id = id; - record.truncate_count = truncate_count; - vy_log_write(&record); -} - #if defined(__cplusplus) } /* extern "C" */ #endif /* defined(__cplusplus) */ -- 2.11.0
prev parent reply other threads:[~2018-03-20 11:29 UTC|newest] Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-03-20 11:29 [PATCH v2 0/5] Prepare vylog for space alter Vladimir Davydov 2018-03-20 11:29 ` [PATCH v2 1/5] vinyl: refactor vylog recovery Vladimir Davydov 2018-03-20 11:29 ` [PATCH v2 2/5] vinyl: rename vy_index::id to index_id Vladimir Davydov 2018-03-20 11:29 ` [PATCH v2 3/5] vinyl: rename vy_log_record::index_id/space_id to index_def_id/space_def_id Vladimir Davydov 2018-03-20 11:29 ` [PATCH v2 4/5] vinyl: do not use index lsn to identify indexes in vylog Vladimir Davydov 2018-03-22 15:08 ` Vladimir Davydov 2018-03-20 11:29 ` Vladimir Davydov [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=c77b8c830ecbbfce722d16f206c33948af321c83.1521545062.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH v2 5/5] alter: rewrite space truncation using alter infrastructure' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox