From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH 02/18] vinyl: move quota methods implementation to vy_quota.c Date: Thu, 16 Aug 2018 19:11:56 +0300 [thread overview] Message-ID: <6f86d2082c35c5b9b01b3e46a009326b8b49f636.1534432819.git.vdavydov.dev@gmail.com> (raw) In-Reply-To: <cover.1534432819.git.vdavydov.dev@gmail.com> In-Reply-To: <cover.1534432819.git.vdavydov.dev@gmail.com> None of vy_quota methods is called from a hot path - even the most frequently called ones, vy_quota_try_use and vy_quota_commit_use, are only invoked once per a transactions. So there's no need to clog the header with the methods implementation. --- src/box/CMakeLists.txt | 1 + src/box/vy_quota.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++ src/box/vy_quota.h | 110 +++++++--------------------------------- 3 files changed, 153 insertions(+), 91 deletions(-) create mode 100644 src/box/vy_quota.c diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt index ad544270..cab6a227 100644 --- a/src/box/CMakeLists.txt +++ b/src/box/CMakeLists.txt @@ -86,6 +86,7 @@ add_library(box STATIC vy_history.c vy_read_set.c vy_scheduler.c + vy_quota.c request.c space.c space_def.c diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c new file mode 100644 index 00000000..6e93d652 --- /dev/null +++ b/src/box/vy_quota.c @@ -0,0 +1,133 @@ +/* + * Copyright 2010-2018, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * AUTHORS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#include "vy_quota.h" + +#include <assert.h> +#include <stddef.h> +#include <tarantool_ev.h> + +#include "fiber.h" +#include "fiber_cond.h" +#include "say.h" + +void +vy_quota_create(struct vy_quota *q, vy_quota_exceeded_f quota_exceeded_cb) +{ + q->limit = SIZE_MAX; + q->watermark = SIZE_MAX; + q->used = 0; + q->too_long_threshold = TIMEOUT_INFINITY; + q->quota_exceeded_cb = quota_exceeded_cb; + fiber_cond_create(&q->cond); +} + +void +vy_quota_destroy(struct vy_quota *q) +{ + fiber_cond_broadcast(&q->cond); + fiber_cond_destroy(&q->cond); +} + +void +vy_quota_set_limit(struct vy_quota *q, size_t limit) +{ + q->limit = q->watermark = limit; + if (q->used >= limit) + q->quota_exceeded_cb(q); + fiber_cond_broadcast(&q->cond); +} + +void +vy_quota_set_watermark(struct vy_quota *q, size_t watermark) +{ + q->watermark = watermark; + if (q->used >= watermark) + q->quota_exceeded_cb(q); +} + +void +vy_quota_force_use(struct vy_quota *q, size_t size) +{ + q->used += size; + if (q->used >= q->watermark) + q->quota_exceeded_cb(q); +} + +void +vy_quota_dump(struct vy_quota *q, size_t size) +{ + assert(q->used >= size); + q->used -= size; + fiber_cond_broadcast(&q->cond); +} + +int +vy_quota_try_use(struct vy_quota *q, size_t size, double timeout) +{ + double start_time = ev_monotonic_now(loop()); + double deadline = start_time + timeout; + while (q->used + size > q->limit && timeout > 0) { + q->quota_exceeded_cb(q); + if (fiber_cond_wait_deadline(&q->cond, deadline) != 0) + break; /* timed out */ + } + double wait_time = ev_monotonic_now(loop()) - start_time; + if (wait_time > q->too_long_threshold) { + say_warn("waited for %zu bytes of vinyl memory quota " + "for too long: %.3f sec", size, wait_time); + } + if (q->used + size > q->limit) + return -1; + q->used += size; + if (q->used >= q->watermark) + q->quota_exceeded_cb(q); + return 0; +} + +void +vy_quota_commit_use(struct vy_quota *q, size_t reserved, size_t used) +{ + if (reserved > used) { + size_t excess = reserved - used; + assert(q->used >= excess); + q->used -= excess; + fiber_cond_broadcast(&q->cond); + } + if (reserved < used) + vy_quota_force_use(q, used - reserved); +} + +void +vy_quota_wait(struct vy_quota *q) +{ + while (q->used > q->limit) + fiber_cond_wait(&q->cond); +} diff --git a/src/box/vy_quota.h b/src/box/vy_quota.h index fd1004da..cf70b1ab 100644 --- a/src/box/vy_quota.h +++ b/src/box/vy_quota.h @@ -31,13 +31,8 @@ * SUCH DAMAGE. */ -#include <stdbool.h> #include <stddef.h> -#include <tarantool_ev.h> - -#include "fiber.h" #include "fiber_cond.h" -#include "say.h" #if defined(__cplusplus) extern "C" { @@ -83,72 +78,39 @@ struct vy_quota { vy_quota_exceeded_f quota_exceeded_cb; }; -static inline void -vy_quota_create(struct vy_quota *q, vy_quota_exceeded_f quota_exceeded_cb) -{ - q->limit = SIZE_MAX; - q->watermark = SIZE_MAX; - q->used = 0; - q->too_long_threshold = TIMEOUT_INFINITY; - q->quota_exceeded_cb = quota_exceeded_cb; - fiber_cond_create(&q->cond); -} - -static inline void -vy_quota_destroy(struct vy_quota *q) -{ - fiber_cond_broadcast(&q->cond); - fiber_cond_destroy(&q->cond); -} +void +vy_quota_create(struct vy_quota *q, vy_quota_exceeded_f quota_exceeded_cb); + +void +vy_quota_destroy(struct vy_quota *q); /** * Set memory limit. If current memory usage exceeds * the new limit, invoke the callback. */ -static inline void -vy_quota_set_limit(struct vy_quota *q, size_t limit) -{ - q->limit = q->watermark = limit; - if (q->used >= limit) - q->quota_exceeded_cb(q); - fiber_cond_broadcast(&q->cond); -} +void +vy_quota_set_limit(struct vy_quota *q, size_t limit); /** * Set memory watermark. If current memory usage exceeds * the new watermark, invoke the callback. */ -static inline void -vy_quota_set_watermark(struct vy_quota *q, size_t watermark) -{ - q->watermark = watermark; - if (q->used >= watermark) - q->quota_exceeded_cb(q); -} +void +vy_quota_set_watermark(struct vy_quota *q, size_t watermark); /** * Consume @size bytes of memory. In contrast to vy_quota_try_use() * this function does not throttle the caller. */ -static inline void -vy_quota_force_use(struct vy_quota *q, size_t size) -{ - q->used += size; - if (q->used >= q->watermark) - q->quota_exceeded_cb(q); -} +void +vy_quota_force_use(struct vy_quota *q, size_t size); /** * Function called on dump completion to release quota after * freeing memory. */ -static inline void -vy_quota_dump(struct vy_quota *q, size_t size) -{ - assert(q->used >= size); - q->used -= size; - fiber_cond_broadcast(&q->cond); -} +void +vy_quota_dump(struct vy_quota *q, size_t size); /** * Try to consume @size bytes of memory, throttle the caller @@ -184,28 +146,8 @@ vy_quota_dump(struct vy_quota *q, size_t size) * are stored in the common memory level, which isn't taken into * account while estimating the size of a memory allocation. */ -static inline int -vy_quota_try_use(struct vy_quota *q, size_t size, double timeout) -{ - double start_time = ev_monotonic_now(loop()); - double deadline = start_time + timeout; - while (q->used + size > q->limit && timeout > 0) { - q->quota_exceeded_cb(q); - if (fiber_cond_wait_deadline(&q->cond, deadline) != 0) - break; /* timed out */ - } - double wait_time = ev_monotonic_now(loop()) - start_time; - if (wait_time > q->too_long_threshold) { - say_warn("waited for %zu bytes of vinyl memory quota " - "for too long: %.3f sec", size, wait_time); - } - if (q->used + size > q->limit) - return -1; - q->used += size; - if (q->used >= q->watermark) - q->quota_exceeded_cb(q); - return 0; -} +int +vy_quota_try_use(struct vy_quota *q, size_t size, double timeout); /** * Adjust quota after allocating memory. @@ -215,28 +157,14 @@ vy_quota_try_use(struct vy_quota *q, size_t size, double timeout) * * See also vy_quota_try_use(). */ -static inline void -vy_quota_commit_use(struct vy_quota *q, size_t reserved, size_t used) -{ - if (reserved > used) { - size_t excess = reserved - used; - assert(q->used >= excess); - q->used -= excess; - fiber_cond_broadcast(&q->cond); - } - if (reserved < used) - vy_quota_force_use(q, used - reserved); -} +void +vy_quota_commit_use(struct vy_quota *q, size_t reserved, size_t used); /** * Block the caller until the quota is not exceeded. */ -static inline void -vy_quota_wait(struct vy_quota *q) -{ - while (q->used > q->limit) - fiber_cond_wait(&q->cond); -} +void +vy_quota_wait(struct vy_quota *q); #if defined(__cplusplus) } /* extern "C" */ -- 2.11.0
next prev parent reply other threads:[~2018-08-16 16:11 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-16 16:11 [PATCH 00/18] Implement write throttling for vinyl Vladimir Davydov 2018-08-16 16:11 ` [PATCH 01/18] vinyl: rework internal quota API Vladimir Davydov 2018-08-20 11:07 ` Konstantin Osipov 2018-08-24 8:32 ` Vladimir Davydov 2018-08-27 18:29 ` Vladimir Davydov 2018-08-16 16:11 ` Vladimir Davydov [this message] 2018-08-20 11:07 ` [PATCH 02/18] vinyl: move quota methods implementation to vy_quota.c Konstantin Osipov 2018-08-27 18:30 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 03/18] vinyl: move quota related methods and variables from vy_env to vy_quota Vladimir Davydov 2018-08-20 11:08 ` Konstantin Osipov 2018-08-27 18:33 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 04/18] vinyl: implement vy_quota_wait using vy_quota_try_use Vladimir Davydov 2018-08-20 11:09 ` Konstantin Osipov 2018-08-27 18:36 ` Vladimir Davydov 2018-08-16 16:11 ` [PATCH 05/18] vinyl: wake up fibers waiting for quota one by one Vladimir Davydov 2018-08-20 11:11 ` Konstantin Osipov 2018-08-24 8:33 ` Vladimir Davydov 2018-08-28 13:19 ` Vladimir Davydov 2018-08-28 14:04 ` Konstantin Osipov 2018-08-28 14:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 06/18] vinyl: do not wake up fibers waiting for quota if quota is unavailable Vladimir Davydov 2018-08-20 11:13 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 07/18] vinyl: tune dump bandwidth histogram buckets Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 15:37 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 08/18] vinyl: rename vy_quota::dump_bw to dump_bw_hist Vladimir Davydov 2018-08-20 11:15 ` Konstantin Osipov 2018-08-28 16:04 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 09/18] vinyl: cache dump bandwidth for timer invocation Vladimir Davydov 2018-08-20 11:21 ` Konstantin Osipov 2018-08-28 16:10 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 10/18] vinyl: do not add initial guess to dump bandwidth histogram Vladimir Davydov 2018-08-20 11:23 ` Konstantin Osipov 2018-08-23 20:15 ` Konstantin Osipov 2018-08-28 16:15 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 11/18] vinyl: use snap_io_rate_limit for initial dump bandwidth estimate Vladimir Davydov 2018-08-20 11:24 ` Konstantin Osipov 2018-08-24 8:31 ` Vladimir Davydov 2018-08-28 16:18 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 12/18] histogram: add function for computing lower bound percentile estimate Vladimir Davydov 2018-08-20 11:29 ` [tarantool-patches] " Konstantin Osipov 2018-08-24 8:30 ` Vladimir Davydov 2018-08-28 16:39 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 13/18] vinyl: use lower bound percentile estimate for dump bandwidth Vladimir Davydov 2018-08-28 16:51 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 14/18] vinyl: do not try to trigger dump if it is already in progress Vladimir Davydov 2018-08-16 16:12 ` [PATCH 15/18] vinyl: improve dump start/stop logging Vladimir Davydov 2018-08-23 20:18 ` Konstantin Osipov 2018-08-16 16:12 ` [PATCH 16/18] vinyl: confine quota watermark within sane value range Vladimir Davydov 2018-08-16 16:12 ` [PATCH 17/18] vinyl: set quota timer period to 100 ms Vladimir Davydov 2018-08-23 20:49 ` Konstantin Osipov 2018-08-24 8:18 ` Vladimir Davydov 2018-08-16 16:12 ` [PATCH 18/18] vinyl: throttle tx rate if dump does not catch up Vladimir Davydov 2018-08-23 20:54 ` Konstantin Osipov 2018-08-23 20:58 ` [tarantool-patches] " Konstantin Osipov 2018-08-24 8:21 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=6f86d2082c35c5b9b01b3e46a009326b8b49f636.1534432819.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH 02/18] vinyl: move quota methods implementation to vy_quota.c' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox