[PATCH 0/7] vinyl: improve stats for throttling

[Vladimir Davydov]

To implement transaction throttling, we need (at least) two additional
metrics:

 - Compaction debt, i.e. how much data have to be compacted to restore
   the target LSM tree shape. This is needed to make decisions about
   decreasing transaction rate.

 - Disk idle time, i.e. how much time worker threads spend doing
   nothing. This is needed to make decisions about loosening throttling.

This patch set adds those metrics. Along the way it also fixes a bug
in cache accounting (patch 1) and adds some useful global statistics
(patches 2 and 3).

https://github.com/tarantool/tarantool/issues/1862
https://github.com/tarantool/tarantool/commits/dv/vy-stats-for-throttling

Vladimir Davydov (7):
  vinyl: fix accounting of secondary index cache statements
  vinyl: add global memory stats
  vinyl: add global disk stats
  vinyl: fix force compaction logic
  vinyl: update compact priority usual way on range split/coalesce
  vinyl: keep track of compaction queue length and debt
  vinyl: keep track of disk idle time

 src/box/vinyl.c                 |  89 ++++++++++---------
 src/box/vy_cache.c              |  13 ++-
 src/box/vy_cache.h              |   5 +-
 src/box/vy_lsm.c                |  85 +++++++++++++-----
 src/box/vy_lsm.h                |  42 ++++++++-
 src/box/vy_range.c              |  42 ++++-----
 src/box/vy_range.h              |  17 ++--
 src/box/vy_scheduler.c          |  61 +++++++++++--
 src/box/vy_scheduler.h          |  12 +++
 src/box/vy_stat.h               |  39 +++++---
 src/box/vy_tx.c                 |  19 ++++
 src/box/vy_tx.h                 |   4 +
 src/errinj.h                    |   1 +
 test/box/errinj.result          |   4 +-
 test/unit/vy_iterators_helper.c |   2 +-
 test/unit/vy_point_lookup.c     |   2 +-
 test/vinyl/cache.result         |  55 +++++++++++-
 test/vinyl/cache.test.lua       |  26 +++++-
 test/vinyl/errinj.result        | 191 ++++++++++++++++++++++++++++++++++++++++
 test/vinyl/errinj.test.lua      |  56 ++++++++++++
 test/vinyl/info.result          | 159 ++++++++++++++++++++++++++++-----
 test/vinyl/info.test.lua        |  32 ++++++-
 22 files changed, 809 insertions(+), 147 deletions(-)

-- 
2.11.0