[PATCH 00/18] Implement write throttling for vinyl
Vladimir Davydov
vdavydov.dev at gmail.com
Thu Aug 16 19:11:54 MSK 2018
If vinyl doesn't keep up with dumps, transactions can consume all
available memory and stall until the last dump is complete, which may
take quite a while (seconds, even minutes sometimes). To avoid
unpredictably long stalls, this patch set introduces transaction
throttling. Now, once a dump is started, vinyl will limit the rate at
which transactions can write to the database so that they won't hit the
limit before the dump is complete.
https://github.com/tarantool/tarantool/issues/1862
https://github.com/tarantool/tarantool/commits/dv/gh-1862-vy-throttling
Vladimir Davydov (18):
vinyl: rework internal quota API
vinyl: move quota methods implementation to vy_quota.c
vinyl: move quota related methods and variables from vy_env to
vy_quota
vinyl: implement vy_quota_wait using vy_quota_try_use
vinyl: wake up fibers waiting for quota one by one
vinyl: do not wake up fibers waiting for quota if quota is unavailable
vinyl: tune dump bandwidth histogram buckets
vinyl: rename vy_quota::dump_bw to dump_bw_hist
vinyl: cache dump bandwidth for timer invocation
vinyl: do not add initial guess to dump bandwidth histogram
vinyl: use snap_io_rate_limit for initial dump bandwidth estimate
histogram: add function for computing lower bound percentile estimate
vinyl: use lower bound percentile estimate for dump bandwidth
vinyl: do not try to trigger dump if it is already in progress
vinyl: improve dump start/stop logging
vinyl: confine quota watermark within sane value range
vinyl: set quota timer period to 100 ms
vinyl: throttle tx rate if dump does not catch up
src/box/CMakeLists.txt | 1 +
src/box/vinyl.c | 180 +++------------------
src/box/vy_quota.c | 362 +++++++++++++++++++++++++++++++++++++++++++
src/box/vy_quota.h | 193 +++++++++++++----------
src/histogram.c | 13 ++
src/histogram.h | 7 +
test/unit/histogram.c | 4 +
test/vinyl/suite.ini | 2 +-
test/vinyl/throttle.result | 95 ++++++++++++
test/vinyl/throttle.test.lua | 47 ++++++
10 files changed, 664 insertions(+), 240 deletions(-)
create mode 100644 src/box/vy_quota.c
create mode 100644 test/vinyl/throttle.result
create mode 100644 test/vinyl/throttle.test.lua
--
2.11.0
More information about the Tarantool-patches
mailing list