From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH v2 0/7] vinyl: eliminate disk read on REPLACE/DELETE
Date: Tue, 21 Aug 2018 14:15:33 +0300 [thread overview]
Message-ID: <cover.1534847663.git.vdavydov.dev@gmail.com> (raw)
This patch set optimizes REPLACE and DELETE operations in vinyl in
presence of secondary indexes: now they don't need to read the primary
key in order to delete the overwritten/deleted tuple from secondary
indexes, instead this job is handed over to primary index compaction
task, while the read iterator is tought to filter out overwritten
tuples that haven't been purged yet.
For more information about the algorithm and implementation details,
please see the final patch of the series.
https://github.com/tarantool/tarantool/issues/2129
https://github.com/tarantool/tarantool/commits/dv/gh-2129-vy-eliminate-read-on-replace-delete
Changes in v2:
- Remove merged patches and rebase on the latest 1.10.
- Use id 257 for _vinyl_deferred_delete space, as it doesn't depend on
other system spaces.
- Refactor _vinyl_deferred_delete space proto creation code.
- Use interface instead of callback for returning deferred DELETEs from
write iterator.
- Get rid of vy_mem::wal_deferred_delete_max_lsn. Account deferred
DELETE WAL LSN in vy_mem::dump_lsn.
- Don't encode flags for secondary index statements (so that there's no
need to pass deferred DELETE handler to the write iterator on dump).
v1: https://www.freelists.org/post/tarantool-patches/PATCH-0025-vinyl-eliminate-disk-read-on-REPLACEDELETE
v0: https://www.freelists.org/post/tarantool-patches/RFC-PATCH-0023-vinyl-eliminate-read-on-REPLACEDELETE
Vladimir Davydov (7):
vinyl: do not store meta in secondary index runs
vinyl: teach write iterator to return overwritten tuples
vinyl: prepare write iterator heap comparator for deferred DELETEs
vinyl: allow to skip certain statements on read
Introduce _vinyl_deferred_delete system space
vinyl: zap vy_mem::min_lsn and rename max_lsn to dump_lsn
vinyl: eliminate disk read on REPLACE/DELETE
src/box/bootstrap.snap | Bin 1540 -> 1607 bytes
src/box/lua/space.cc | 2 +
src/box/lua/upgrade.lua | 21 ++
src/box/schema.cc | 27 +-
src/box/schema_def.h | 2 +
src/box/vinyl.c | 261 ++++++++++++--
src/box/vy_lsm.h | 5 +
src/box/vy_mem.c | 27 +-
src/box/vy_mem.h | 16 +-
src/box/vy_point_lookup.c | 32 ++
src/box/vy_point_lookup.h | 18 +
src/box/vy_run.c | 7 +-
src/box/vy_scheduler.c | 310 ++++++++++++++++-
src/box/vy_stmt.c | 7 +-
src/box/vy_stmt.h | 29 ++
src/box/vy_tx.c | 133 +++++++
src/box/vy_write_iterator.c | 153 +++++++-
src/box/vy_write_iterator.h | 45 ++-
test/app-tap/tarantoolctl.test.lua | 2 +-
test/box-py/bootstrap.result | 7 +-
test/box/access_misc.result | 5 +-
test/box/access_sysview.result | 2 +-
test/unit/vy_iterators_helper.c | 5 +
test/unit/vy_iterators_helper.h | 12 +-
test/unit/vy_mem.c | 11 +-
test/unit/vy_mem.result | 22 +-
test/unit/vy_point_lookup.c | 6 +-
test/unit/vy_write_iterator.c | 254 +++++++++++++-
test/unit/vy_write_iterator.result | 22 +-
test/vinyl/deferred_delete.result | 677 ++++++++++++++++++++++++++++++++++++
test/vinyl/deferred_delete.test.lua | 261 ++++++++++++++
test/vinyl/info.result | 18 +-
test/vinyl/info.test.lua | 9 +-
test/vinyl/layout.result | 46 ++-
test/vinyl/quota.result | 2 +-
test/vinyl/tx_gap_lock.result | 16 +-
test/vinyl/tx_gap_lock.test.lua | 10 +-
test/vinyl/write_iterator.result | 5 +
test/vinyl/write_iterator.test.lua | 3 +
test/wal_off/alter.result | 2 +-
test/xlog/upgrade.result | 7 +-
41 files changed, 2344 insertions(+), 155 deletions(-)
create mode 100644 test/vinyl/deferred_delete.result
create mode 100644 test/vinyl/deferred_delete.test.lua
--
2.11.0
next reply other threads:[~2018-08-21 11:15 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-08-21 11:15 Vladimir Davydov [this message]
2018-08-21 11:15 ` [PATCH v2 1/7] vinyl: do not store meta in secondary index runs Vladimir Davydov
2018-08-21 15:08 ` Konstantin Osipov
2018-08-21 11:15 ` [PATCH v2 2/7] vinyl: teach write iterator to return overwritten tuples Vladimir Davydov
2018-08-21 15:14 ` Konstantin Osipov
2018-08-21 15:37 ` Vladimir Davydov
2018-08-21 11:15 ` [PATCH v2 3/7] vinyl: prepare write iterator heap comparator for deferred DELETEs Vladimir Davydov
2018-08-21 15:38 ` Konstantin Osipov
2018-08-21 11:15 ` [PATCH v2 4/7] vinyl: allow to skip certain statements on read Vladimir Davydov
2018-08-21 15:39 ` Konstantin Osipov
2018-08-21 11:15 ` [PATCH v2 5/7] Introduce _vinyl_deferred_delete system space Vladimir Davydov
2018-08-21 15:42 ` Konstantin Osipov
2018-08-22 17:04 ` Vladimir Davydov
2018-08-21 11:15 ` [PATCH v2 6/7] vinyl: zap vy_mem::min_lsn and rename max_lsn to dump_lsn Vladimir Davydov
2018-08-21 15:44 ` Konstantin Osipov
2018-08-22 13:00 ` Vladimir Davydov
2018-08-21 11:15 ` [PATCH v2 7/7] vinyl: eliminate disk read on REPLACE/DELETE Vladimir Davydov
2018-08-21 16:13 ` Konstantin Osipov
2018-08-22 17:08 ` Vladimir Davydov
2018-08-22 17:50 ` [PATCH v2 0/7] " Vladimir Davydov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1534847663.git.vdavydov.dev@gmail.com \
--to=vdavydov.dev@gmail.com \
--cc=kostja@tarantool.org \
--cc=tarantool-patches@freelists.org \
--subject='Re: [PATCH v2 0/7] vinyl: eliminate disk read on REPLACE/DELETE' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox