From: Vladimir Davydov <vdavydov.dev@gmail.com> To: Konstantin Osipov <kostja@tarantool.org> Cc: tarantool-patches@freelists.org Subject: Re: [PATCH v2 7/8] vinyl: introduce quota consumer types Date: Tue, 12 Feb 2019 18:48:16 +0300 [thread overview] Message-ID: <20190212154756.mx3kvqwndvjpewd2@esperanza> (raw) In-Reply-To: <0e5b13fdd4bee9f2e91c462f7a7a4ae91238d8ea.1548349067.git.vdavydov.dev@gmail.com> On Thu, Jan 24, 2019 at 08:12:43PM +0300, Vladimir Davydov wrote: > diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c > index 07cd5856..20d322de 100644 > --- a/src/box/vy_quota.c > +++ b/src/box/vy_quota.c > @@ -52,6 +52,43 @@ > static const double VY_QUOTA_TIMER_PERIOD = 0.1; > > /** > + * Bit mask of resources used by a particular consumer type. > + */ > +static unsigned > +vy_quota_consumer_resource_map[] = { > + /** > + * Transaction throttling pursues two goals. First, it is > + * capping memory consumption rate so that the hard memory > + * limit will not be hit before memory dump has completed > + * (memory-based throttling). Second, we must make sure > + * that compaction jobs keep up with dumps to keep read and > + * space amplification within bounds (disk-based throttling). > + * Transactions ought to respect them both. > + */ > + [VY_QUOTA_CONSUMER_TX] = (1 << VY_QUOTA_RESOURCE_DISK) | > + (1 << VY_QUOTA_RESOURCE_MEMORY), > + /** > + * Compaction jobs may need some quota too, because they > + * may generate deferred DELETEs for secondary indexes. > + * Apparently, we must not impose the rate limit that is > + * supposed to speed up compaction on them (disk-based), > + * however they still have to respect memory-based throttling > + * to avoid long stalls. > + */ > + [VY_QUOTA_CONSUMER_COMPACTION] = (1 << VY_QUOTA_RESOURCE_MEMORY), > +}; > + > +/** > + * Iterate over rate limit states that are enforced for a consumer > + * of the given type. > + */ > +#define vy_quota_consumer_for_each_rate_limit(quota, type, rl) \ > + for (struct vy_rate_limit *rl = (quota)->rate_limit; \ > + rl - (quota)->rate_limit < vy_quota_resource_type_MAX; rl++) \ > + if (vy_quota_consumer_resource_map[type] & \ > + (1 << (rl - (quota)->rate_limit))) > + Kostja: This macro looks confusing, let's inline it. I: OK, here we go: diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c index 20d322de..dc452adb 100644 --- a/src/box/vy_quota.c +++ b/src/box/vy_quota.c @@ -79,14 +79,16 @@ vy_quota_consumer_resource_map[] = { }; /** - * Iterate over rate limit states that are enforced for a consumer - * of the given type. + * Check if the rate limit corresponding to resource @resource_type + * should be applied to a consumer of type @consumer_type. */ -#define vy_quota_consumer_for_each_rate_limit(quota, type, rl) \ - for (struct vy_rate_limit *rl = (quota)->rate_limit; \ - rl - (quota)->rate_limit < vy_quota_resource_type_MAX; rl++) \ - if (vy_quota_consumer_resource_map[type] & \ - (1 << (rl - (quota)->rate_limit))) +static inline bool +vy_rate_limit_is_applicable(enum vy_quota_consumer_type consumer_type, + enum vy_quota_resource_type resource_type) +{ + return (vy_quota_consumer_resource_map[consumer_type] & + (1 << resource_type)) != 0; +} /** * Return true if the requested amount of memory may be consumed @@ -106,8 +108,10 @@ vy_quota_may_use(struct vy_quota *q, enum vy_quota_consumer_type type, q->quota_exceeded_cb(q); return false; } - vy_quota_consumer_for_each_rate_limit(q, type, rl) { - if (!vy_rate_limit_may_use(rl)) + for (int i = 0; i < vy_quota_resource_type_MAX; i++) { + struct vy_rate_limit *rl = &q->rate_limit[i]; + if (vy_rate_limit_is_applicable(type, i) && + !vy_rate_limit_may_use(rl)) return false; } return true; @@ -121,8 +125,11 @@ vy_quota_do_use(struct vy_quota *q, enum vy_quota_consumer_type type, size_t size) { q->used += size; - vy_quota_consumer_for_each_rate_limit(q, type, rl) - vy_rate_limit_use(rl, size); + for (int i = 0; i < vy_quota_resource_type_MAX; i++) { + struct vy_rate_limit *rl = &q->rate_limit[i]; + if (vy_rate_limit_is_applicable(type, i)) + vy_rate_limit_use(rl, size); + } } /** @@ -135,8 +142,11 @@ vy_quota_do_unuse(struct vy_quota *q, enum vy_quota_consumer_type type, { assert(q->used >= size); q->used -= size; - vy_quota_consumer_for_each_rate_limit(q, type, rl) - vy_rate_limit_unuse(rl, size); + for (int i = 0; i < vy_quota_resource_type_MAX; i++) { + struct vy_rate_limit *rl = &q->rate_limit[i]; + if (vy_rate_limit_is_applicable(type, i)) + vy_rate_limit_unuse(rl, size); + } } /**
next prev parent reply other threads:[~2019-02-12 15:48 UTC|newest] Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-01-24 17:12 [PATCH v2 0/8] vinyl: compaction randomization and throttling Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 1/8] vinyl: use uncompressed run size for range split/coalesce/compaction Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 2/8] vinyl: fix compaction priority calculation Vladimir Davydov 2019-02-08 17:18 ` [tarantool-patches] " Konstantin Osipov 2019-02-11 14:42 ` Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 3/8] vinyl: rename lsm->range_heap to max_compaction_priority Vladimir Davydov 2019-02-08 17:19 ` [tarantool-patches] " Konstantin Osipov 2019-02-08 17:26 ` Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 4/8] vinyl: keep track of dumps per compaction for each LSM tree Vladimir Davydov 2019-02-08 17:42 ` Vladimir Davydov 2019-02-11 18:17 ` [tarantool-patches] " Konstantin Osipov 2019-02-11 18:17 ` Konstantin Osipov 2019-02-12 10:15 ` Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 5/8] vinyl: set range size automatically Vladimir Davydov 2019-02-11 18:21 ` [tarantool-patches] " Konstantin Osipov 2019-02-12 10:16 ` Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 6/8] vinyl: randomize range compaction to avoid IO load spikes Vladimir Davydov 2019-02-11 18:21 ` [tarantool-patches] " Konstantin Osipov 2019-02-12 10:16 ` Vladimir Davydov 2019-01-24 17:12 ` [PATCH v2 7/8] vinyl: introduce quota consumer types Vladimir Davydov 2019-02-12 15:48 ` Vladimir Davydov [this message] 2019-01-24 17:12 ` [PATCH v2 8/8] vinyl: throttle tx to ensure compaction keeps up with dumps Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190212154756.mx3kvqwndvjpewd2@esperanza \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH v2 7/8] vinyl: introduce quota consumer types' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox