From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH v2 00/11] vinyl: transaction throttling infrastructure
Date: Fri, 28 Sep 2018 20:39:58 +0300 [thread overview]
Message-ID: <cover.1538155645.git.vdavydov.dev@gmail.com> (raw)
This patch set contains preparatory patches necessary for implementing
transaction throttling. It does some refactoring, implements basic
rate limiting based on dump bandwidth, and introduces quota consumer
priorities, which is necessary for compaction based throttling. The only
thing left is invent such a throttling policy that guarantees that
compaction always keeps up with dumps.
https://github.com/tarantool/tarantool/issues/1862
https://github.com/tarantool/tarantool/tree/dv/gh-1862-vy-throttling
Changes in v2:
- Factor out load regulator logic out of quota and do some renames,
as suggested by Kostja.
v1: https://www.freelists.org/post/tarantool-patches/PATCH-0011-vinyl-prepare-for-transaction-throttling
Vladimir Davydov (11):
vinyl: add helper to start scheduler and enable quota on startup
vinyl: factor load regulator out of quota
vinyl: minor refactoring of quota methods
vinyl: move transaction size sanity check to quota
vinyl: implement quota wait queue without fiber_cond
vinyl: enable quota upon recovery completion explicitly
vinyl: zap vy_env::memory, read_threads, and write_threads
vinyl: do not try to trigger dump in regulator if already in progress
vinyl: do not account zero dump bandwidth
vinyl: implement basic transaction throttling
vinyl: introduce quota consumer priorities
src/box/CMakeLists.txt | 1 +
src/box/vinyl.c | 133 +++++++-------
src/box/vy_quota.c | 357 +++++++++++++++++++++----------------
src/box/vy_quota.h | 231 ++++++++++++++++++------
src/box/vy_regulator.c | 268 ++++++++++++++++++++++++++++
src/box/vy_regulator.h | 153 ++++++++++++++++
src/box/vy_run.c | 12 +-
src/box/vy_run.h | 17 +-
test/unit/vy_point_lookup.c | 2 +-
test/vinyl/errinj.result | 4 +-
test/vinyl/errinj.test.lua | 4 +-
test/vinyl/info.result | 14 +-
test/vinyl/info.test.lua | 8 +-
test/vinyl/quota.result | 26 +--
test/vinyl/quota.test.lua | 26 +--
test/vinyl/quota_timeout.result | 6 +-
test/vinyl/quota_timeout.test.lua | 6 +-
test/vinyl/recovery_quota.result | 6 +-
test/vinyl/recovery_quota.test.lua | 6 +-
test/vinyl/suite.ini | 2 +-
test/vinyl/throttle.result | 102 +++++++++++
test/vinyl/throttle.test.lua | 54 ++++++
22 files changed, 1093 insertions(+), 345 deletions(-)
create mode 100644 src/box/vy_regulator.c
create mode 100644 src/box/vy_regulator.h
create mode 100644 test/vinyl/throttle.result
create mode 100644 test/vinyl/throttle.test.lua
--
2.11.0
next reply other threads:[~2018-09-28 17:39 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-28 17:39 Vladimir Davydov [this message]
2018-09-28 17:39 ` [PATCH v2 01/11] vinyl: add helper to start scheduler and enable quota on startup Vladimir Davydov
2018-09-29 4:37 ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 02/11] vinyl: factor load regulator out of quota Vladimir Davydov
2018-09-29 5:00 ` [tarantool-patches] " Konstantin Osipov
2018-09-29 11:36 ` Vladimir Davydov
[not found] ` <20180929114308.GA19162@chai>
2018-10-01 10:27 ` Vladimir Davydov
2018-10-01 10:31 ` Vladimir Davydov
2018-10-02 18:16 ` [tarantool-patches] " Konstantin Osipov
2018-10-03 8:49 ` Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 03/11] vinyl: minor refactoring of quota methods Vladimir Davydov
2018-09-29 5:01 ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 04/11] vinyl: move transaction size sanity check to quota Vladimir Davydov
2018-09-29 5:02 ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 05/11] vinyl: implement quota wait queue without fiber_cond Vladimir Davydov
2018-09-29 5:05 ` [tarantool-patches] " Konstantin Osipov
2018-09-29 11:44 ` Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 06/11] vinyl: enable quota upon recovery completion explicitly Vladimir Davydov
2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 07/11] vinyl: zap vy_env::memory, read_threads, and write_threads Vladimir Davydov
2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 08/11] vinyl: do not try to trigger dump in regulator if already in progress Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 09/11] vinyl: do not account zero dump bandwidth Vladimir Davydov
2018-10-12 13:27 ` Vladimir Davydov
2018-10-16 18:25 ` [tarantool-patches] " Konstantin Osipov
2018-10-17 8:44 ` Vladimir Davydov
2018-10-23 7:02 ` Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 10/11] vinyl: implement basic transaction throttling Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 11/11] vinyl: introduce quota consumer priorities Vladimir Davydov
2018-10-06 13:24 ` Konstantin Osipov
2018-10-08 11:10 ` Vladimir Davydov
2018-10-09 13:25 ` Vladimir Davydov
2018-10-11 7:02 ` Konstantin Osipov
2018-10-11 8:29 ` Vladimir Davydov
2018-10-03 9:06 ` [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1538155645.git.vdavydov.dev@gmail.com \
--to=vdavydov.dev@gmail.com \
--cc=kostja@tarantool.org \
--cc=tarantool-patches@freelists.org \
--subject='Re: [PATCH v2 00/11] vinyl: transaction throttling infrastructure' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox