From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [RFC PATCH 00/23] vinyl: eliminate read on REPLACE/DELETE Date: Sun, 8 Jul 2018 19:48:31 +0300 Message-Id: In-Reply-To: <1c13a7ce54e2ba625965212fb538725cd974699b.1531065648.git.vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: This patch set optimizes REPLACE and DELETE operations in vinyl in presence of secondary indexes: now they don't need to read the primary key in order to delete the overwritten/deleted tuple from secondary indexes, instead this job is handed over to primary index compaction task, while read iterator filters out overwritten tuples that haven't been purged yet. The patch set still has a few serious problems (deferred DELETEs generated by compaction task may be lost on restart; generation of deferred DELETEs may cause OOM in tx thread) and needs some efforts to be put in it, but it passes all functional tests should be suitable for testing and the first round of review. Most patches will stay the same anyways. https://github.com/tarantool/tarantool/issues/2129 https://github.com/tarantool/tarantool/commits/dv/gh-2129-vy-eliminate-read-on-replace-delete Vladimir Davydov (23): vinyl: do not turn REPLACE into INSERT when processing DML request vinyl: always get full tuple from pk after reading from secondary index vinyl: use vy_mem_iterator for point lookup vinyl: make point lookup always return the latest tuple version vinyl: fold vy_replace_one and vy_replace_impl vinyl: fold vy_delete_impl vinyl: refactor unique check vinyl: check key uniqueness before modifying tx write set vinyl: remove env argument of vy_check_is_unique_{primary,secondary} vinyl: store full tuples in secondary index cache xrow: allow to store flags in DML requests vinyl: do not pass region explicitly to write iterator functions vinyl: fix potential use-after-free in vy_read_view_merge test: unit/vy_write_iterator: minor refactoring vinyl: teach write iterator to return overwritten tuples vinyl: allow to skip certain statements on read vinyl: do not free pending tasks on shutdown vinyl: store pointer to scheduler in struct vy_task vinyl: rename some members of vy_scheduler and vy_task struct vinyl: use cbus for communication between scheduler and worker threads vinyl: zap vy_scheduler::is_worker_pool_running vinyl: rename vy_task::status to is_failed vinyl: eliminate read on REPLACE/DELETE src/box/iproto_constants.c | 4 +- src/box/iproto_constants.h | 3 +- src/box/vinyl.c | 792 +++++++++++++++++++------------------ src/box/vy_mem.c | 19 +- src/box/vy_point_lookup.c | 87 ++-- src/box/vy_point_lookup.h | 9 +- src/box/vy_read_iterator.c | 61 ++- src/box/vy_read_iterator.h | 24 ++ src/box/vy_run.c | 7 +- src/box/vy_scheduler.c | 563 ++++++++++++++++++-------- src/box/vy_scheduler.h | 41 +- src/box/vy_stmt.c | 4 + src/box/vy_stmt.h | 44 +++ src/box/vy_tx.c | 26 ++ src/box/vy_write_iterator.c | 173 ++++++-- src/box/vy_write_iterator.h | 27 +- src/box/xrow.c | 8 + src/box/xrow.h | 2 + test/unit/vy_iterators_helper.c | 5 + test/unit/vy_iterators_helper.h | 12 +- test/unit/vy_point_lookup.c | 4 +- test/unit/vy_write_iterator.c | 319 ++++++++++++--- test/unit/vy_write_iterator.result | 23 +- test/vinyl/info.result | 5 + test/vinyl/info.test.lua | 3 + test/vinyl/layout.result | 166 +++++--- test/vinyl/tx_gap_lock.result | 16 +- test/vinyl/tx_gap_lock.test.lua | 10 +- test/vinyl/write_iterator.result | 11 +- test/vinyl/write_iterator.test.lua | 5 +- 30 files changed, 1615 insertions(+), 858 deletions(-) -- 2.11.0