From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH 0/7] vinyl: improve stats for throttling Date: Sun, 2 Sep 2018 23:18:53 +0300 Message-Id: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: To implement transaction throttling, we need (at least) two additional metrics: - Compaction debt, i.e. how much data have to be compacted to restore the target LSM tree shape. This is needed to make decisions about decreasing transaction rate. - Disk idle time, i.e. how much time worker threads spend doing nothing. This is needed to make decisions about loosening throttling. This patch set adds those metrics. Along the way it also fixes a bug in cache accounting (patch 1) and adds some useful global statistics (patches 2 and 3). https://github.com/tarantool/tarantool/issues/1862 https://github.com/tarantool/tarantool/commits/dv/vy-stats-for-throttling Vladimir Davydov (7): vinyl: fix accounting of secondary index cache statements vinyl: add global memory stats vinyl: add global disk stats vinyl: fix force compaction logic vinyl: update compact priority usual way on range split/coalesce vinyl: keep track of compaction queue length and debt vinyl: keep track of disk idle time src/box/vinyl.c | 89 ++++++++++--------- src/box/vy_cache.c | 13 ++- src/box/vy_cache.h | 5 +- src/box/vy_lsm.c | 85 +++++++++++++----- src/box/vy_lsm.h | 42 ++++++++- src/box/vy_range.c | 42 ++++----- src/box/vy_range.h | 17 ++-- src/box/vy_scheduler.c | 61 +++++++++++-- src/box/vy_scheduler.h | 12 +++ src/box/vy_stat.h | 39 +++++--- src/box/vy_tx.c | 19 ++++ src/box/vy_tx.h | 4 + src/errinj.h | 1 + test/box/errinj.result | 4 +- test/unit/vy_iterators_helper.c | 2 +- test/unit/vy_point_lookup.c | 2 +- test/vinyl/cache.result | 55 +++++++++++- test/vinyl/cache.test.lua | 26 +++++- test/vinyl/errinj.result | 191 ++++++++++++++++++++++++++++++++++++++++ test/vinyl/errinj.test.lua | 56 ++++++++++++ test/vinyl/info.result | 159 ++++++++++++++++++++++++++++----- test/vinyl/info.test.lua | 32 ++++++- 22 files changed, 809 insertions(+), 147 deletions(-) -- 2.11.0