[PATCH v3 0/4] vinyl: compaction randomization and throttling

[Vladimir Davydov <vdavydov.dev@gmail.com>]

This patch set achieves two goals. The first one is randomizing
compaction pace among ranges so as to avoid IO load bursts, which badly
affect read and space amplification and complicate implementation of
transaction throttling. The second goal is making sure that compaction
always keeps up with dumps and throttling transactions if it doesn't.
For more details, see comments to the individual patches and the GitHub
issues.

https://github.com/tarantool/tarantool/issues/3944
https://github.com/tarantool/tarantool/issues/3721
https://github.com/tarantool/tarantool/commits/dv/gh-3721-3944-vy-compaction-randomization-and-throttling

Changes in v3:
 - Pushed all the patches related to compaction randomization. Now this
   patch set is only about transaction throttling.
 - Removed vy_quota_consumer_for_each_rate_limit (inlined it) as
   requested by Kostja.
 - Added a couple of patches addressing minor issues with vy_quota:
   fairness and extra check.
 - Changed the rate limit coefficient to 0.75 and updated the comment
   as suggested by Kostja.

v2: https://www.freelists.org/post/tarantool-patches/PATCH-v2-08-vinyl-compaction-randomization-and-throttling

Changes in v2:
 - Fixed compaction priority calculation that resulted in unstable
   compaction behavior and prevented randomization from smoothing out
   IO load generated by compaction.
 - Tuned automatic range sizing: don't limit the total number of ranges
   as this isn't going to play nicely with time series workloads; set
   max range size to 2 GB as we don't want compaction to take eternity
   no matter what; use 4 * dumps_per_compaction for range_count - better
   safe than sorry.
 - Fixed dumps_per_compaction update in case compaction was forced
   manually using index.compact(). We don't want to update this metric
   in such a case to avoid undesired side effects, such as splitting or
   coalescing ranges for no good reason.
 - Pushed the first two trivial patches and rebased.
 - Fixed a few failing tests.

v1: https://www.freelists.org/post/tarantool-patches/PATCH-09-vinyl-compaction-randomization-and-throttling

Vladimir Davydov (4):
  vinyl: introduce quota consumer types
  vinyl: remove extra quota check from vy_quota_use
  vinyl: don't consume quota if wait queue isn't empty
  vinyl: throttle tx to ensure compaction keeps up with dumps

 src/box/vinyl.c        |  36 +++++++----
 src/box/vy_quota.c     | 168 +++++++++++++++++++++++++++++++++++++------------
 src/box/vy_quota.h     |  97 +++++++++++++++++++++++-----
 src/box/vy_regulator.c | 155 ++++++++++++++++++++++++++++++++++++++++++---
 src/box/vy_regulator.h |  27 ++++++++
 5 files changed, 410 insertions(+), 73 deletions(-)

-- 
2.11.0