[tarantool-patches] Re: [PATCH 00/11] vinyl: prepare for transaction throttling

[Konstantin Osipov]

* Vladimir Davydov <vdavydov.dev at gmail.com> [18/09/20 12:34]:
> This patch set contains preparatory patches necessary for implementing
> transaction throttling. It does some refactoring, introduces basic
> throttling based on dump bandwidth, and splits the rate limit in soft
> and hard, which is necessary for compaction based throttling. The only
> thing left is invent such a throttling policy that guarantees that
> compaction always keeps up with dumps.

Re refactoring of vy_quota: I continue to believe that there is an 
underlying low-level quota object, encapsulating the logic of a
limit and the current consumption rate.

Before throttling, we had a single resource we consume:
vinyl.memory, there was no logical limit on disk bandwidth. 

Throttling is essentially an introduction and enforcement of a logical
limit on disk bandwidth.

I disagree that with addition of this new logical limit the quota
API should be changed. The current API is conceptually very
simple. A quota is two variables: a limit and the current
consumption rate. So there are two sets of methods: to
change/adjust the limit, and to consume/release the use rate, in
other words, use/release the associated resource.

Introduction of a new resource type changes nothing about the way
we work with the quota. We simply get a new pair of limit and use
rate, and begin working with it in addition to the pair related to
memory. 

So I believe that introduction of a new resource does not mandate
the changes in the focus of quota-api from working with the
internal state of the quota (limit and use rate) to essentially
the life cycle of the LSM tree, i.e. handling of the events such
as a dump or a compaction. We may have one set of events which
lead to consumption and release of the quota today, and another
tomorrow. 

Basically, the throttling task has two subtasks:
- introduction of disk bandwidth as a separate resource which
  usage is accounted for across the code base
  This is this patch set's concern. This should have nothing
  to do with LSM details. The main consumer of the bandwidth
  is DML, since it essentially leads to level 0 dump.

- introduction of a separate entity, which is then part
  of our LSM implementation which tracks events in LSM 
  tree life cycle, such as compaction and dump, and dynamically
  adjusts the limit based on predicted implications of these
  events on the available disk bandwidth (or actually the
  remainder of it, since we assume that compaction's use
  of disk bandwidth is not limited with the quota).

We still have no simple solution for the second part, "the
predictor". It's therefore best, I agree, to separate the two
stacks of patches, since we may want to change the predictive
model in the future without changing the rate limiting logic
itself
> 
> https://github.com/tarantool/tarantool/issues/1862
> https://github.com/tarantool/tarantool/tree/dv/gh-1862-vy-throttling
> 
> Vladimir Davydov (11):
>   vinyl: merge vy_quota_release and vy_quota_update_dump_bandwidth
>   vinyl: refactor quota use/unuse methods
>   vinyl: do not try to trigger dump if it is already in progress
>   vinyl: don't start quota timer until local recovery is complete
>   vinyl: add helper to start scheduler and enable quota on startup
>   vinyl: zap vy_env::memory, read_threads, and write_threads
>   vinyl: do not account zero dump bandwidth
>   vinyl: set quota timer period to 100 ms
>   vinyl: implement basic transaction throttling
>   vinyl: implement quota wait queue without fiber_cond
>   vinyl: split quota consumption rate limit into soft and hard
> 
>  src/box/vinyl.c              |  79 +++++-----
>  src/box/vy_quota.c           | 340 +++++++++++++++++++++++++++++++++----------
>  src/box/vy_quota.h           | 146 +++++++++++++++----
>  src/box/vy_run.c             |  12 +-
>  src/box/vy_run.h             |  17 ++-
>  test/unit/vy_point_lookup.c  |   2 +-
>  test/vinyl/suite.ini         |   2 +-
>  test/vinyl/throttle.result   | 102 +++++++++++++
>  test/vinyl/throttle.test.lua |  54 +++++++
>  9 files changed, 601 insertions(+), 153 deletions(-)
>  create mode 100644 test/vinyl/throttle.result
>  create mode 100644 test/vinyl/throttle.test.lua
> 
> -- 
> 2.11.0
> 

-- 
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov