From: Konstantin Osipov <kostja@tarantool.org>
To: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: tarantool-patches@freelists.org
Subject: Re: [PATCH 8/8] vinyl: keep track of thread pool idle ratio
Date: Thu, 6 Sep 2018 13:26:31 +0300 [thread overview]
Message-ID: <20180906102631.GI8205@chai> (raw)
In-Reply-To: <20180906081851.tlkeedihyfe5qoff@esperanza>
* Vladimir Davydov <vdavydov.dev@gmail.com> [18/09/06 11:19]:
> > > To understand whether the disk is fully utilized or can still handle
> > > more compaction load and make right decisions regarding transaction
> > > throttling, we need a metric that would report how much time worker
> > > threads spent being idle. So this patch adds two new metrics to global
> > > statistics, disk.dump_idle_ratio and compact_idle_ratio, which show how
> > > much time dump threads and compaction threads were idle, respectively.
> > > The metrics are updated using the following formula:
> > >
> > > idle_time
> > > idle_ratio = --------------------------
> > > dump_period * worker_count
> >
> > I don't understand the formula. There can be many workers.
> > Is idle time measured per worker or per entire pool?
> >
> > If it is measured per entire pool, how is idle time calculated if
> > some workers are busy and some not?
>
> It is measured for entire pool - note that I divide the result by
> worker_count. E.g. if there were two workers and one of them were
> busy all the time between two last dumps while another were idle,
> idle_ratio would be 0.5.
This looks imprecise. Why not measure idle time of each worker and
then even it out over the total number of workers? Besides, once
again, how do you define the window over which you measure?
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
next prev parent reply other threads:[~2018-09-06 10:26 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-02 20:18 [PATCH 0/7] vinyl: improve stats for throttling Vladimir Davydov
2018-09-02 20:18 ` [PATCH 1/7] vinyl: fix accounting of secondary index cache statements Vladimir Davydov
2018-09-02 22:26 ` [tarantool-patches] " Konstantin Osipov
2018-09-02 20:18 ` [PATCH 2/7] vinyl: add global memory stats Vladimir Davydov
2018-09-02 22:27 ` [tarantool-patches] " Konstantin Osipov
2018-09-02 22:27 ` Konstantin Osipov
2018-09-03 8:10 ` Vladimir Davydov
2018-09-02 20:18 ` [PATCH 3/7] vinyl: add global disk stats Vladimir Davydov
2018-09-02 22:30 ` [tarantool-patches] " Konstantin Osipov
2018-09-02 20:18 ` [PATCH 4/7] vinyl: fix force compaction logic Vladimir Davydov
2018-09-02 20:18 ` [PATCH 5/7] vinyl: update compact priority usual way on range split/coalesce Vladimir Davydov
2018-09-02 20:18 ` [PATCH 6/7] vinyl: keep track of compaction queue length and debt Vladimir Davydov
2018-09-02 20:19 ` [PATCH 7/7] vinyl: keep track of disk idle time Vladimir Davydov
2018-09-04 11:54 ` Vladimir Davydov
2018-09-04 17:23 ` Vladimir Davydov
2018-09-04 17:23 ` [PATCH 1/8] vinyl: add helper to check whether dump is in progress Vladimir Davydov
2018-09-06 7:33 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 2/8] vinyl: don't use mempool for allocating background tasks Vladimir Davydov
2018-09-06 7:33 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 3/8] vinyl: factor out worker pool from scheduler struct Vladimir Davydov
2018-09-06 7:34 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 4/8] vinyl: move worker allocation closer to task creation Vladimir Davydov
2018-09-06 7:35 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 5/8] vinyl: use separate thread pools for dump and compaction tasks Vladimir Davydov
2018-09-06 7:37 ` Konstantin Osipov
2018-09-06 9:48 ` Vladimir Davydov
2018-09-06 10:32 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 6/8] vinyl: zap vy_worker_pool::idle_worker_count Vladimir Davydov
2018-09-06 7:38 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 7/8] vinyl: don't start scheduler fiber until local recovery is complete Vladimir Davydov
2018-09-06 7:39 ` Konstantin Osipov
2018-09-04 17:23 ` [PATCH 8/8] vinyl: keep track of thread pool idle ratio Vladimir Davydov
2018-09-06 7:49 ` Konstantin Osipov
2018-09-06 8:18 ` Vladimir Davydov
2018-09-06 10:26 ` Konstantin Osipov [this message]
2018-09-06 10:52 ` Vladimir Davydov
2018-09-06 10:57 ` Konstantin Osipov
2018-09-06 11:59 ` Vladimir Davydov
2018-09-09 11:41 ` [PATCH 0/7] vinyl: improve stats for throttling Vladimir Davydov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180906102631.GI8205@chai \
--to=kostja@tarantool.org \
--cc=tarantool-patches@freelists.org \
--cc=vdavydov.dev@gmail.com \
--subject='Re: [PATCH 8/8] vinyl: keep track of thread pool idle ratio' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox