From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH 12/18] histogram: add function for computing lower bound percentile estimate Date: Thu, 16 Aug 2018 19:12:06 +0300 [thread overview] Message-ID: <f33ba0ae6d64dbccec3b86ad7ac9ff36c018fca2.1534432819.git.vdavydov.dev@gmail.com> (raw) In-Reply-To: <cover.1534432819.git.vdavydov.dev@gmail.com> In-Reply-To: <cover.1534432819.git.vdavydov.dev@gmail.com> The value returned by histogram_percentile() is an upper bound estimate. This is fine for measuring latency, because we are interested in the worst, i.e. highest, observations, but doesn't suit particularly well if we want to keep track of the lowest observations, as it is the case with bandwidth. So this patch introduces histogram_percentile_lower(), a function that is similar to histogram_percentile(), but returns a lower bound estimate of the requested percentile. --- src/histogram.c | 13 +++++++++++++ src/histogram.h | 7 +++++++ test/unit/histogram.c | 4 ++++ 3 files changed, 24 insertions(+) diff --git a/src/histogram.c b/src/histogram.c index c934ee9f..8c6cd6c0 100644 --- a/src/histogram.c +++ b/src/histogram.c @@ -145,6 +145,19 @@ histogram_percentile(struct histogram *hist, int pct) return hist->max; } +int64_t +histogram_percentile_lower(struct histogram *hist, int pct) +{ + size_t count = 0; + + for (size_t i = 0; i < hist->n_buckets; i++) { + count += hist->buckets[i].count; + if (count * 100 > hist->total * pct) + return hist->buckets[i > 0 ? i - 1 : 0].max; + } + return hist->max; +} + int histogram_snprint(char *buf, int size, struct histogram *hist) { diff --git a/src/histogram.h b/src/histogram.h index 95d47d40..2c3df5d1 100644 --- a/src/histogram.h +++ b/src/histogram.h @@ -90,6 +90,13 @@ int64_t histogram_percentile(struct histogram *hist, int pct); /** + * Same as histogram_percentile(), but return a lower bound + * estimate of the percentile. + */ +int64_t +histogram_percentile_lower(struct histogram *hist, int pct); + +/** * Print string representation of a histogram. */ int diff --git a/test/unit/histogram.c b/test/unit/histogram.c index 4c980d24..a026be26 100644 --- a/test/unit/histogram.c +++ b/test/unit/histogram.c @@ -154,14 +154,18 @@ test_percentile(void) for (int pct = 5; pct < 100; pct += 5) { int64_t val = data[data_len * pct / 100]; int64_t expected = max; + int64_t expected_lo = max; for (size_t b = 0; b < n_buckets; b++) { if (buckets[b] >= val) { expected = buckets[b]; + expected_lo = buckets[b > 0 ? b - 1 : 0]; break; } } int64_t result = histogram_percentile(hist, pct); fail_if(result != expected); + int64_t result_lo = histogram_percentile_lower(hist, pct); + fail_if(result_lo != expected_lo); } histogram_delete(hist); -- 2.11.0
next prev parent reply other threads:[~2018-08-16 16:12 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-16 16:11 [PATCH 00/18] Implement write throttling for vinyl Vladimir Davydov 2018-08-16 16:11 ` [PATCH 01/18] vinyl: rework internal quota API Vladimir Davydov 2018-08-20 11:07 ` Konstantin Osipov 2018-08-24 8:32 ` Vladimir Davydov 2018-08-27 18:29 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 02/18] vinyl: move quota methods implementation to vy_quota.c Vladimir Davydov 2018-08-20 11:07 ` Konstantin Osipov 2018-08-27 18:30 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 03/18] vinyl: move quota related methods and variables from vy_env to vy_quota Vladimir Davydov 2018-08-20 11:08 ` Konstantin Osipov 2018-08-27 18:33 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 04/18] vinyl: implement vy_quota_wait using vy_quota_try_use Vladimir Davydov 2018-08-20 11:09 ` Konstantin Osipov 2018-08-27 18:36 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 05/18] vinyl: wake up fibers waiting for quota one by one Vladimir Davydov 2018-08-20 11:11 ` Konstantin Osipov 2018-08-24 8:33 ` Vladimir Davydov 2018-08-28 13:19 ` Vladimir Davydov 2018-08-28 14:04 ` Konstantin Osipov 2018-08-28 14:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 06/18] vinyl: do not wake up fibers waiting for quota if quota is unavailable Vladimir Davydov 2018-08-20 11:13 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 07/18] vinyl: tune dump bandwidth histogram buckets Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 15:37 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 08/18] vinyl: rename vy_quota::dump_bw to dump_bw_hist Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 16:04 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 09/18] vinyl: cache dump bandwidth for timer invocation Vladimir Davydov 2018-08-20 11:21 ` Konstantin Osipov 2018-08-28 16:10 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 10/18] vinyl: do not add initial guess to dump bandwidth histogram Vladimir Davydov 2018-08-20 11:23 ` Konstantin Osipov 2018-08-23 20:15 ` Konstantin Osipov 2018-08-28 16:15 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 11/18] vinyl: use snap_io_rate_limit for initial dump bandwidth estimate Vladimir Davydov 2018-08-20 11:24 ` Konstantin Osipov 2018-08-24 8:31 ` Vladimir Davydov 2018-08-28 16:18 ` Vladimir Davydov 2018-08-16 16:12 ` Vladimir Davydov [this message] 2018-08-20 11:29 ` [tarantool-patches] Re: [PATCH 12/18] histogram: add function for computing lower bound percentile estimate Konstantin Osipov 2018-08-24 8:30 ` Vladimir Davydov 2018-08-28 16:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 13/18] vinyl: use lower bound percentile estimate for dump bandwidth Vladimir Davydov 2018-08-28 16:51 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 14/18] vinyl: do not try to trigger dump if it is already in progress Vladimir Davydov 2018-08-16 16:12 ` [PATCH 15/18] vinyl: improve dump start/stop logging Vladimir Davydov 2018-08-23 20:18 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 16/18] vinyl: confine quota watermark within sane value range Vladimir Davydov 2018-08-16 16:12 ` [PATCH 17/18] vinyl: set quota timer period to 100 ms Vladimir Davydov 2018-08-23 20:49 ` Konstantin Osipov 2018-08-24 8:18 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 18/18] vinyl: throttle tx rate if dump does not catch up Vladimir Davydov 2018-08-23 20:54 ` Konstantin Osipov 2018-08-23 20:58 ` [tarantool-patches] " Konstantin Osipov 2018-08-24 8:21 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=f33ba0ae6d64dbccec3b86ad7ac9ff36c018fca2.1534432819.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH 12/18] histogram: add function for computing lower bound percentile estimate' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox