[PATCH 7/7] vinyl: keep track of disk idle time
Vladimir Davydov
vdavydov.dev at gmail.com
Tue Sep 4 20:23:43 MSK 2018
On Tue, Sep 04, 2018 at 02:54:04PM +0300, Vladimir Davydov wrote:
> Please ignore this patch, because it's incorrect: since we always
> keep one worker reserved for dumps it will should that the disk is
> 1/box.cfg.vinyl_writers idle even if compaction never stops, which
> is misleading. I'll rework and resend it separately.
Here goes the new version. The primary difference is that it reports
idle time separately for dump and compaction threads. To achieve that,
I had to introduce separate thread pools for dump and compaction tasks.
There's still one knob though, box.cfg.vinyl_write_threads I assign 25%
of threads (min 1) to dump tasks while the rest goes to compaction tasks
(this is similar to what RocksDB does).
Patches 1 and 2 do some trivial cleanup.
Patches 3 to 7 introduce separate thread pools for different kinds of
background tasks.
Patch 8 adds idle ratio to global statistics.
The branch is the same:
https://github.com/tarantool/tarantool/commits/dv/vy-stats-for-throttling
Vladimir Davydov (8):
vinyl: add helper to check whether dump is in progress
vinyl: don't use mempool for allocating background tasks
vinyl: factor out worker pool from scheduler struct
vinyl: move worker allocation closer to task creation
vinyl: use separate thread pools for dump and compaction tasks
vinyl: zap vy_worker_pool::idle_worker_count
vinyl: don't start scheduler fiber until local recovery is complete
vinyl: keep track of thread pool idle ratio
src/box/vinyl.c | 44 +++---
src/box/vy_scheduler.c | 334 ++++++++++++++++++++++++++++++---------------
src/box/vy_scheduler.h | 60 ++++++--
test/vinyl/errinj.result | 82 +++++++++++
test/vinyl/errinj.test.lua | 37 +++++
test/vinyl/info.result | 14 +-
test/vinyl/info.test.lua | 2 +
7 files changed, 423 insertions(+), 150 deletions(-)
--
2.11.0
More information about the Tarantool-patches
mailing list