From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: Re: [PATCH 05/18] vinyl: wake up fibers waiting for quota one by one Date: Tue, 28 Aug 2018 16:19:14 +0300 [thread overview] Message-ID: <20180828131914.zuxe76xi3c64y5y6@esperanza> (raw) In-Reply-To: <4938e95febbd96d464ebe8f8fdbb4c1785013e1f.1534432819.git.vdavydov.dev@gmail.com> Discussed verbally. Agreed to wake up the next fiber on vy_quota_use() only if we waited for quota (otherwise we don't need to do that). Here's the updated patch: From 7c10049ebf9c96adb197a719d4ed7c16f61a4f33 Mon Sep 17 00:00:00 2001 From: Vladimir Davydov <vdavydov.dev@gmail.com> Date: Tue, 28 Aug 2018 16:15:33 +0300 Subject: [PATCH] vinyl: wake up fibers waiting for quota one by one Currently, we wake up all fibers whenever we free some memory. This is inefficient, because it might occur that all available quota gets consumed by a few fibers while the rest will have to go back to sleep. This is also kinda unfair, because waking up all fibers breaks the order in which the fibers were put to sleep. This works now, because we free memory and wake up fibers infrequently (on dump) and there normally shouldn't be any fibers waiting for quota (if there were, the latency would rocket sky high because of absence of any kind of throttling). However, once throttling is introduced, fibers waiting for quota will become the norm. So let's wake up fibers one by one: whenever we free memory we wake up the first fiber in the line, which will wake up the next fiber on success and so forth. diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c index 1e7c754f..b3f073c3 100644 --- a/src/box/vy_quota.c +++ b/src/box/vy_quota.c @@ -56,6 +56,16 @@ enum { VY_QUOTA_RATE_AVG_PERIOD = 5, }; +/** + * Returns true if the quota limit is exceeded and so consumers + * have to wait. + */ +static inline bool +vy_quota_is_exceeded(struct vy_quota *q) +{ + return q->used > q->limit; +} + static void vy_quota_timer_cb(ev_loop *loop, ev_timer *timer, int events) { @@ -152,7 +162,7 @@ vy_quota_set_limit(struct vy_quota *q, size_t limit) q->limit = q->watermark = limit; if (q->used >= limit) q->quota_exceeded_cb(q); - fiber_cond_broadcast(&q->cond); + fiber_cond_signal(&q->cond); } size_t @@ -184,28 +194,40 @@ vy_quota_release(struct vy_quota *q, size_t size) { assert(q->used >= size); q->used -= size; - fiber_cond_broadcast(&q->cond); + fiber_cond_signal(&q->cond); } int vy_quota_use(struct vy_quota *q, size_t size, double timeout) { - double start_time = ev_monotonic_now(loop()); - double deadline = start_time + timeout; - while (q->used + size > q->limit && timeout > 0) { - q->quota_exceeded_cb(q); - if (fiber_cond_wait_deadline(&q->cond, deadline) != 0) - break; /* timed out */ - } - double wait_time = ev_monotonic_now(loop()) - start_time; - if (wait_time > q->too_long_threshold) { - say_warn("waited for %zu bytes of vinyl memory quota " - "for too long: %.3f sec", size, wait_time); - } - if (q->used + size > q->limit) - return -1; q->used += size; q->use_curr += size; + if (vy_quota_is_exceeded(q)) { + /* Wait for quota. */ + double start_time = ev_monotonic_now(loop()); + double deadline = start_time + timeout; + + do { + q->quota_exceeded_cb(q); + q->used -= size; + q->use_curr -= size; + if (fiber_cond_wait_deadline(&q->cond, deadline) != 0) + return -1; /* timed out */ + q->used += size; + q->use_curr += size; + } while (vy_quota_is_exceeded(q)); + + double wait_time = ev_monotonic_now(loop()) - start_time; + if (wait_time > q->too_long_threshold) { + say_warn("waited for %zu bytes of vinyl memory quota " + "for too long: %.3f sec", size, wait_time); + } + /* + * Wake up the next fiber in the line waiting + * for quota. + */ + fiber_cond_signal(&q->cond); + } if (q->used >= q->watermark) q->quota_exceeded_cb(q); return 0; @@ -222,7 +244,7 @@ vy_quota_adjust(struct vy_quota *q, size_t reserved, size_t used) q->use_curr -= excess; else /* was reset by timeout */ q->use_curr = 0; - fiber_cond_broadcast(&q->cond); + fiber_cond_signal(&q->cond); } if (reserved < used) vy_quota_force_use(q, used - reserved);
next prev parent reply other threads:[~2018-08-28 13:19 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-16 16:11 [PATCH 00/18] Implement write throttling for vinyl Vladimir Davydov 2018-08-16 16:11 ` [PATCH 01/18] vinyl: rework internal quota API Vladimir Davydov 2018-08-20 11:07 ` Konstantin Osipov 2018-08-24 8:32 ` Vladimir Davydov 2018-08-27 18:29 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 02/18] vinyl: move quota methods implementation to vy_quota.c Vladimir Davydov 2018-08-20 11:07 ` Konstantin Osipov 2018-08-27 18:30 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 03/18] vinyl: move quota related methods and variables from vy_env to vy_quota Vladimir Davydov 2018-08-20 11:08 ` Konstantin Osipov 2018-08-27 18:33 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 04/18] vinyl: implement vy_quota_wait using vy_quota_try_use Vladimir Davydov 2018-08-20 11:09 ` Konstantin Osipov 2018-08-27 18:36 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 05/18] vinyl: wake up fibers waiting for quota one by one Vladimir Davydov 2018-08-20 11:11 ` Konstantin Osipov 2018-08-24 8:33 ` Vladimir Davydov 2018-08-28 13:19 ` Vladimir Davydov [this message] 2018-08-28 14:04 ` Konstantin Osipov 2018-08-28 14:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 06/18] vinyl: do not wake up fibers waiting for quota if quota is unavailable Vladimir Davydov 2018-08-20 11:13 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 07/18] vinyl: tune dump bandwidth histogram buckets Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 15:37 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 08/18] vinyl: rename vy_quota::dump_bw to dump_bw_hist Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 16:04 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 09/18] vinyl: cache dump bandwidth for timer invocation Vladimir Davydov 2018-08-20 11:21 ` Konstantin Osipov 2018-08-28 16:10 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 10/18] vinyl: do not add initial guess to dump bandwidth histogram Vladimir Davydov 2018-08-20 11:23 ` Konstantin Osipov 2018-08-23 20:15 ` Konstantin Osipov 2018-08-28 16:15 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 11/18] vinyl: use snap_io_rate_limit for initial dump bandwidth estimate Vladimir Davydov 2018-08-20 11:24 ` Konstantin Osipov 2018-08-24 8:31 ` Vladimir Davydov 2018-08-28 16:18 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 12/18] histogram: add function for computing lower bound percentile estimate Vladimir Davydov 2018-08-20 11:29 ` [tarantool-patches] " Konstantin Osipov 2018-08-24 8:30 ` Vladimir Davydov 2018-08-28 16:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 13/18] vinyl: use lower bound percentile estimate for dump bandwidth Vladimir Davydov 2018-08-28 16:51 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 14/18] vinyl: do not try to trigger dump if it is already in progress Vladimir Davydov 2018-08-16 16:12 ` [PATCH 15/18] vinyl: improve dump start/stop logging Vladimir Davydov 2018-08-23 20:18 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 16/18] vinyl: confine quota watermark within sane value range Vladimir Davydov 2018-08-16 16:12 ` [PATCH 17/18] vinyl: set quota timer period to 100 ms Vladimir Davydov 2018-08-23 20:49 ` Konstantin Osipov 2018-08-24 8:18 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 18/18] vinyl: throttle tx rate if dump does not catch up Vladimir Davydov 2018-08-23 20:54 ` Konstantin Osipov 2018-08-23 20:58 ` [tarantool-patches] " Konstantin Osipov 2018-08-24 8:21 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180828131914.zuxe76xi3c64y5y6@esperanza \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH 05/18] vinyl: wake up fibers waiting for quota one by one' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox