From: Vladimir Davydov <vdavydov.dev@gmail.com> To: kostja@tarantool.org Cc: tarantool-patches@freelists.org Subject: [PATCH v2 08/11] vinyl: do not try to trigger dump in regulator if already in progress Date: Fri, 28 Sep 2018 20:40:06 +0300 [thread overview] Message-ID: <1a139cdc45c7898eebf8b4846abeae7e0bd8b054.1538155645.git.vdavydov.dev@gmail.com> (raw) In-Reply-To: <cover.1538155645.git.vdavydov.dev@gmail.com> In-Reply-To: <cover.1538155645.git.vdavydov.dev@gmail.com> This is pointless since trigger_dump_cb callback will return right away in such a case. Let's wrap trigger_dump_cb in vy_regulator_trigger_dump method, which will actulally invoke the callback only if the previous dump has already completed (i.e. vy_regulator_dump_complete was called). This also gives us a definite place in code where we can adjust the rate limit so as to guarantee that a triggered memory dump will finish before we hit the hard memory limit (this will be done later). Needed for #1862 --- src/box/vinyl.c | 5 +++-- src/box/vy_regulator.c | 20 ++++++++++++++++++-- src/box/vy_regulator.h | 11 +++++++++-- 3 files changed, 30 insertions(+), 6 deletions(-) diff --git a/src/box/vinyl.c b/src/box/vinyl.c index 25ba92fe..c3d95777 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -2427,7 +2427,7 @@ vy_env_quota_exceeded_cb(struct vy_quota *quota) vy_regulator_no_memory(&env->regulator); } -static void +static int vy_env_trigger_dump_cb(struct vy_regulator *regulator) { struct vy_env *env = container_of(regulator, struct vy_env, regulator); @@ -2439,9 +2439,10 @@ vy_env_trigger_dump_cb(struct vy_regulator *regulator) * quota has been consumed by pending transactions. * There's nothing we can do about that. */ - return; + return -1; } vy_scheduler_trigger_dump(&env->scheduler); + return 0; } static void diff --git a/src/box/vy_regulator.c b/src/box/vy_regulator.c index 3bd4fee4..5ec5629f 100644 --- a/src/box/vy_regulator.c +++ b/src/box/vy_regulator.c @@ -31,6 +31,7 @@ #include "vy_regulator.h" #include <math.h> +#include <stdbool.h> #include <stddef.h> #include <stdint.h> #include <tarantool_ev.h> @@ -66,6 +67,18 @@ static const int VY_DUMP_BW_PCT = 10; static const size_t VY_DUMP_BW_DEFAULT = 10 * 1024 * 1024; static void +vy_regulator_trigger_dump(struct vy_regulator *regulator) +{ + if (regulator->dump_in_progress) + return; + + if (regulator->trigger_dump_cb(regulator) != 0) + return; + + regulator->dump_in_progress = true; +} + +static void vy_regulator_update_write_rate(struct vy_regulator *regulator) { size_t used_curr = regulator->quota->used; @@ -152,6 +165,7 @@ vy_regulator_create(struct vy_regulator *regulator, struct vy_quota *quota, regulator->watermark = SIZE_MAX; regulator->write_rate = 0; regulator->dump_bw = VY_DUMP_BW_DEFAULT; + regulator->dump_in_progress = false; } void @@ -170,20 +184,22 @@ vy_regulator_destroy(struct vy_regulator *regulator) void vy_regulator_no_memory(struct vy_regulator *regulator) { - regulator->trigger_dump_cb(regulator); + vy_regulator_trigger_dump(regulator); } void vy_regulator_check_watermark(struct vy_regulator *regulator) { if (regulator->quota->used >= regulator->watermark) - regulator->trigger_dump_cb(regulator); + vy_regulator_trigger_dump(regulator); } void vy_regulator_dump_complete(struct vy_regulator *regulator, size_t mem_dumped, double dump_duration) { + regulator->dump_in_progress = false; + if (dump_duration > 0) { histogram_collect(regulator->dump_bw_hist, mem_dumped / dump_duration); diff --git a/src/box/vy_regulator.h b/src/box/vy_regulator.h index 9336adc9..0ea708d6 100644 --- a/src/box/vy_regulator.h +++ b/src/box/vy_regulator.h @@ -31,6 +31,7 @@ * SUCH DAMAGE. */ +#include <stdbool.h> #include <stddef.h> #include <tarantool_ev.h> @@ -42,7 +43,7 @@ struct histogram; struct vy_quota; struct vy_regulator; -typedef void +typedef int (*vy_trigger_dump_f)(struct vy_regulator *regulator); /** @@ -59,7 +60,7 @@ struct vy_regulator { /** * Called when the regulator detects that memory usage * exceeds the computed watermark. Supposed to trigger - * memory dump. + * memory dump and return 0 on success, -1 on failure. */ vy_trigger_dump_f trigger_dump_cb; /** @@ -100,6 +101,12 @@ struct vy_regulator { * best result among 10% worst measurements. */ struct histogram *dump_bw_hist; + /** + * Set if the last triggered memory dump hasn't completed + * yet, i.e. trigger_dump_cb() was successfully invoked, + * but vy_regulator_dump_complete() hasn't been called yet. + */ + bool dump_in_progress; }; void -- 2.11.0
next prev parent reply other threads:[~2018-09-28 17:40 UTC|newest] Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-09-28 17:39 [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov 2018-09-28 17:39 ` [PATCH v2 01/11] vinyl: add helper to start scheduler and enable quota on startup Vladimir Davydov 2018-09-29 4:37 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 02/11] vinyl: factor load regulator out of quota Vladimir Davydov 2018-09-29 5:00 ` [tarantool-patches] " Konstantin Osipov 2018-09-29 11:36 ` Vladimir Davydov [not found] ` <20180929114308.GA19162@chai> 2018-10-01 10:27 ` Vladimir Davydov 2018-10-01 10:31 ` Vladimir Davydov 2018-10-02 18:16 ` [tarantool-patches] " Konstantin Osipov 2018-10-03 8:49 ` Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 03/11] vinyl: minor refactoring of quota methods Vladimir Davydov 2018-09-29 5:01 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 04/11] vinyl: move transaction size sanity check to quota Vladimir Davydov 2018-09-29 5:02 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 05/11] vinyl: implement quota wait queue without fiber_cond Vladimir Davydov 2018-09-29 5:05 ` [tarantool-patches] " Konstantin Osipov 2018-09-29 11:44 ` Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 06/11] vinyl: enable quota upon recovery completion explicitly Vladimir Davydov 2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 07/11] vinyl: zap vy_env::memory, read_threads, and write_threads Vladimir Davydov 2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` Vladimir Davydov [this message] 2018-09-28 17:40 ` [PATCH v2 09/11] vinyl: do not account zero dump bandwidth Vladimir Davydov 2018-10-12 13:27 ` Vladimir Davydov 2018-10-16 18:25 ` [tarantool-patches] " Konstantin Osipov 2018-10-17 8:44 ` Vladimir Davydov 2018-10-23 7:02 ` Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 10/11] vinyl: implement basic transaction throttling Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 11/11] vinyl: introduce quota consumer priorities Vladimir Davydov 2018-10-06 13:24 ` Konstantin Osipov 2018-10-08 11:10 ` Vladimir Davydov 2018-10-09 13:25 ` Vladimir Davydov 2018-10-11 7:02 ` Konstantin Osipov 2018-10-11 8:29 ` Vladimir Davydov 2018-10-03 9:06 ` [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1a139cdc45c7898eebf8b4846abeae7e0bd8b054.1538155645.git.vdavydov.dev@gmail.com \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH v2 08/11] vinyl: do not try to trigger dump in regulator if already in progress' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox