From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH v2 00/11] vinyl: transaction throttling infrastructure Date: Fri, 28 Sep 2018 20:39:58 +0300 Message-Id: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: This patch set contains preparatory patches necessary for implementing transaction throttling. It does some refactoring, implements basic rate limiting based on dump bandwidth, and introduces quota consumer priorities, which is necessary for compaction based throttling. The only thing left is invent such a throttling policy that guarantees that compaction always keeps up with dumps. https://github.com/tarantool/tarantool/issues/1862 https://github.com/tarantool/tarantool/tree/dv/gh-1862-vy-throttling Changes in v2: - Factor out load regulator logic out of quota and do some renames, as suggested by Kostja. v1: https://www.freelists.org/post/tarantool-patches/PATCH-0011-vinyl-prepare-for-transaction-throttling Vladimir Davydov (11): vinyl: add helper to start scheduler and enable quota on startup vinyl: factor load regulator out of quota vinyl: minor refactoring of quota methods vinyl: move transaction size sanity check to quota vinyl: implement quota wait queue without fiber_cond vinyl: enable quota upon recovery completion explicitly vinyl: zap vy_env::memory, read_threads, and write_threads vinyl: do not try to trigger dump in regulator if already in progress vinyl: do not account zero dump bandwidth vinyl: implement basic transaction throttling vinyl: introduce quota consumer priorities src/box/CMakeLists.txt | 1 + src/box/vinyl.c | 133 +++++++------- src/box/vy_quota.c | 357 +++++++++++++++++++++---------------- src/box/vy_quota.h | 231 ++++++++++++++++++------ src/box/vy_regulator.c | 268 ++++++++++++++++++++++++++++ src/box/vy_regulator.h | 153 ++++++++++++++++ src/box/vy_run.c | 12 +- src/box/vy_run.h | 17 +- test/unit/vy_point_lookup.c | 2 +- test/vinyl/errinj.result | 4 +- test/vinyl/errinj.test.lua | 4 +- test/vinyl/info.result | 14 +- test/vinyl/info.test.lua | 8 +- test/vinyl/quota.result | 26 +-- test/vinyl/quota.test.lua | 26 +-- test/vinyl/quota_timeout.result | 6 +- test/vinyl/quota_timeout.test.lua | 6 +- test/vinyl/recovery_quota.result | 6 +- test/vinyl/recovery_quota.test.lua | 6 +- test/vinyl/suite.ini | 2 +- test/vinyl/throttle.result | 102 +++++++++++ test/vinyl/throttle.test.lua | 54 ++++++ 22 files changed, 1093 insertions(+), 345 deletions(-) create mode 100644 src/box/vy_regulator.c create mode 100644 src/box/vy_regulator.h create mode 100644 test/vinyl/throttle.result create mode 100644 test/vinyl/throttle.test.lua -- 2.11.0