[PATCH 00/18] Implement write throttling for vinyl

[Vladimir Davydov <vdavydov.dev@gmail.com>]

If vinyl doesn't keep up with dumps, transactions can consume all
available memory and stall until the last dump is complete, which may
take quite a while (seconds, even minutes sometimes). To avoid
unpredictably long stalls, this patch set introduces transaction
throttling. Now, once a dump is started, vinyl will limit the rate at
which transactions can write to the database so that they won't hit the
limit before the dump is complete.

https://github.com/tarantool/tarantool/issues/1862
https://github.com/tarantool/tarantool/commits/dv/gh-1862-vy-throttling

Vladimir Davydov (18):
  vinyl: rework internal quota API
  vinyl: move quota methods implementation to vy_quota.c
  vinyl: move quota related methods and variables from vy_env to
    vy_quota
  vinyl: implement vy_quota_wait using vy_quota_try_use
  vinyl: wake up fibers waiting for quota one by one
  vinyl: do not wake up fibers waiting for quota if quota is unavailable
  vinyl: tune dump bandwidth histogram buckets
  vinyl: rename vy_quota::dump_bw to dump_bw_hist
  vinyl: cache dump bandwidth for timer invocation
  vinyl: do not add initial guess to dump bandwidth histogram
  vinyl: use snap_io_rate_limit for initial dump bandwidth estimate
  histogram: add function for computing lower bound percentile estimate
  vinyl: use lower bound percentile estimate for dump bandwidth
  vinyl: do not try to trigger dump if it is already in progress
  vinyl: improve dump start/stop logging
  vinyl: confine quota watermark within sane value range
  vinyl: set quota timer period to 100 ms
  vinyl: throttle tx rate if dump does not catch up

 src/box/CMakeLists.txt       |   1 +
 src/box/vinyl.c              | 180 +++------------------
 src/box/vy_quota.c           | 362 +++++++++++++++++++++++++++++++++++++++++++
 src/box/vy_quota.h           | 193 +++++++++++++----------
 src/histogram.c              |  13 ++
 src/histogram.h              |   7 +
 test/unit/histogram.c        |   4 +
 test/vinyl/suite.ini         |   2 +-
 test/vinyl/throttle.result   |  95 ++++++++++++
 test/vinyl/throttle.test.lua |  47 ++++++
 10 files changed, 664 insertions(+), 240 deletions(-)
 create mode 100644 src/box/vy_quota.c
 create mode 100644 test/vinyl/throttle.result
 create mode 100644 test/vinyl/throttle.test.lua

-- 
2.11.0