Re: [PATCH 7/7] vinyl: keep track of disk idle time

[Vladimir Davydov <vdavydov.dev@gmail.com>]

On Tue, Sep 04, 2018 at 02:54:04PM +0300, Vladimir Davydov wrote:
> Please ignore this patch, because it's incorrect: since we always
> keep one worker reserved for dumps it will should that the disk is
> 1/box.cfg.vinyl_writers idle even if compaction never stops, which
> is misleading. I'll rework and resend it separately.

Here goes the new version. The primary difference is that it reports
idle time separately for dump and compaction threads. To achieve that,
I had to introduce separate thread pools for dump and compaction tasks.
There's still one knob though, box.cfg.vinyl_write_threads I assign 25%
of threads (min 1) to dump tasks while the rest goes to compaction tasks
(this is similar to what RocksDB does).

Patches 1 and 2 do some trivial cleanup.

Patches 3 to 7 introduce separate thread pools for different kinds of
background tasks.

Patch 8 adds idle ratio to global statistics.

The branch is the same:

https://github.com/tarantool/tarantool/commits/dv/vy-stats-for-throttling

Vladimir Davydov (8):
  vinyl: add helper to check whether dump is in progress
  vinyl: don't use mempool for allocating background tasks
  vinyl: factor out worker pool from scheduler struct
  vinyl: move worker allocation closer to task creation
  vinyl: use separate thread pools for dump and compaction tasks
  vinyl: zap vy_worker_pool::idle_worker_count
  vinyl: don't start scheduler fiber until local recovery is complete
  vinyl: keep track of thread pool idle ratio

 src/box/vinyl.c            |  44 +++---
 src/box/vy_scheduler.c     | 334 ++++++++++++++++++++++++++++++---------------
 src/box/vy_scheduler.h     |  60 ++++++--
 test/vinyl/errinj.result   |  82 +++++++++++
 test/vinyl/errinj.test.lua |  37 +++++
 test/vinyl/info.result     |  14 +-
 test/vinyl/info.test.lua   |   2 +
 7 files changed, 423 insertions(+), 150 deletions(-)

-- 
2.11.0