[PATCH 0/9] vinyl: compaction randomization and throttling
Vladimir Davydov
vdavydov.dev at gmail.com
Mon Jan 21 00:16:59 MSK 2019
This patch set achieves two goals. The first one is randomizing
compaction pace among ranges so as to avoid IO load bursts, which badly
affect read and space amplification and complicate implementation of
transaction throttling. The second goal is making sure that compaction
always keeps up with dumps and throttling transactions if it doesn't.
For more details, see comments to the individual patches and the GitHub
issues.
https://github.com/tarantool/tarantool/issues/3944
https://github.com/tarantool/tarantool/issues/3721
https://github.com/tarantool/tarantool/commits/dv/gh-3721-3944-vy-compaction-randomization-and-throttling
Vladimir Davydov (9):
vinyl: update lsm->range_heap in one go on dump completion
vinyl: ignore unknown .run, .index and .vylog keys
vinyl: use uncompressed run size for range split/coalesce/compaction
vinyl: rename lsm->range_heap to max_compaction_priority
vinyl: keep track of dumps per compaction for each LSM tree
vinyl: set range size automatically
vinyl: randomize range compaction to avoid IO load spikes
vinyl: introduce quota consumer types
vinyl: throttle tx to ensure compaction keeps up with dumps
src/box/alter.cc | 8 +-
src/box/box.cc | 6 +-
src/box/index_def.c | 2 +-
src/box/lua/load_cfg.lua | 2 +-
src/box/lua/space.cc | 6 +-
src/box/vinyl.c | 39 +++++--
src/box/vy_log.c | 33 ++++--
src/box/vy_log.h | 10 +-
src/box/vy_lsm.c | 83 ++++++++++++--
src/box/vy_lsm.h | 12 +-
src/box/vy_quota.c | 132 +++++++++++++++++-----
src/box/vy_quota.h | 97 ++++++++++++++---
src/box/vy_range.c | 55 ++++++++--
src/box/vy_range.h | 60 +++++++---
src/box/vy_regulator.c | 96 ++++++++++++++--
src/box/vy_regulator.h | 27 +++++
src/box/vy_run.c | 13 +--
src/box/vy_run.h | 20 ++++
src/box/vy_scheduler.c | 40 ++++---
test/app-tap/init_script.result | 21 ++--
test/box-tap/cfg.test.lua | 3 +-
test/box/admin.result | 2 -
test/box/cfg.result | 4 -
test/vinyl/ddl.result | 5 -
test/vinyl/ddl.test.lua | 1 -
test/vinyl/layout.result | 8 +-
test/vinyl/misc.result | 78 +++++++++++++
test/vinyl/misc.test.lua | 26 +++++
test/vinyl/stat.result | 217 +++++++++++++++++++++++++++++--------
test/vinyl/stat.test.lua | 57 ++++++++++
test/vinyl/write_iterator.result | 16 +++
test/vinyl/write_iterator.test.lua | 6 +
32 files changed, 962 insertions(+), 223 deletions(-)
--
2.11.0
More information about the Tarantool-patches
mailing list