From: Vladimir Davydov <vdavydov.dev@gmail.com> To: Konstantin Osipov <kostja@tarantool.org> Cc: tarantool-patches@freelists.org Subject: Re: [PATCH v2 11/11] vinyl: introduce quota consumer priorities Date: Thu, 11 Oct 2018 11:29:37 +0300 [thread overview] Message-ID: <20181011082937.c5xpcpe5fime46cw@esperanza> (raw) In-Reply-To: <20181011070250.GB3110@chai> On Thu, Oct 11, 2018 at 10:02:50AM +0300, Konstantin Osipov wrote: > * Vladimir Davydov <vdavydov.dev@gmail.com> [18/10/08 14:12]: > > > > Actually, there are resource types and there are consumer types. > > I admit the fact that I mixed them may look confusing at the first > > glance. We may introduce a seprate enum for resource types with a > > mapping between them. > > > > enum vy_quota_consumer_type { > > VY_QUOTA_CONSUMER_TX = 0, > > VY_QUOTA_CONSUMER_COMPACTION = 1, > > vy_quota_consumer_max, > > }; > > > > enum vy_quota_resource_type { > > VY_QUOTA_RESOURCE_DISK = 0, > > VY_QUOTA_RESOURCE_MEMORY = 1, > > vy_quota_resource_max, > > }; > > > > static unsigned vy_quota_consumer_mask[] = { > > [VY_QUOTA_CONSUMER_TX] = (1 << VY_QUOTA_RESOURCE_DISK) | > > (1 << VY_QUOTA_RESOURCE_MEMORY), > > [VY_QUOTA_CONSUMER_COMPACTION] = (1 << VY_QUOTA_RESOURCE_MEMORY), > > }; > > > > struct vy_quota { > > ... > > struct rlist wait_queue[vy_quota_consumer_max]; > > struct vy_rate_limit rate_limit[vy_quota_resource_max]; > > }; > > > > > > This would make the code more bulky though. Do we really want to > > complicate? > > I like it. Then please see the patch: https://www.freelists.org/post/tarantool-patches/PATCH-v2-1111-vinyl-introduce-quota-consumer-priorities,3 > > > > Also, quite frankly I don't quite dig the concept of 'resources' and > > the corresponding constant names, because 'memory' rate limit may be > > confused with memory usage, which is what vy_quota is about in the first > > place. > > Yes, vy_quota is originally about usage. I don't know why you > insist on having rate limits in it ;) I'm ok if we put rate limits > into a separate object, now that we have vy_regulator it can > manage all of them. Because I want to reuse the wait queue which was initially introduced for putting consumers to sleep when they hit the memory limit. Besides, rate limit value is consumed only along with quota so I'm convinced they should be together. > > > > Quota refill interval should be varying - please schedule this > > > work in a separate patch. > > > > > > > > > > static void > > > > vy_quota_signal(struct vy_quota *q) > > > > { > > > > - if (!rlist_empty(&q->wait_queue)) { > > > > + /* > > > > + * Wake up a consumer that has waited most no matter > > > > + * whether it's high or low priority. This assures that > > > > + * high priority consumers don't uncontrollably throttle > > > > + * low priority ones. > > > > + */ > > > > + struct vy_quota_wait_node *oldest = NULL; > > > > + for (int i = 0; i < vy_quota_consumer_prio_MAX; i++) { > > > > + struct rlist *wq = &q->wait_queue[i]; > > > > + if (rlist_empty(wq)) > > > > + continue; > > > > > > I still do not understand why you need a second queue and > > > timestapms. If you need to ensure total fairness, a single queue > > > should be enough. > > > > I need to maintain one queue per consumer type, because it's possible > > that we may wake up consumers of one type, but not of the other. > > If you do this, your fairness is strictly speaking broken, unless, > then again, a type implies some kind of priority. By waking up a > consumer out of (strict) order you give away a resource which may > be needed for another consumer which (in strict single-queue > order) is ahead of it. I only wake up consumers out of (strict) order in case some consumers can't proceed in the *current* interval, i.e. I only give the *surplus* of the resource that wouldn't be consumed anyway in *this* interval. Note, the resource is replenished on a timely basis so by dispensing the surplus I don't deprive throttled consumers of any resource. > > Frankly I still don't see why you did it this way except for the > personal preference. > > > If we used a single queue, it could occur that consumers that > > could be woken up landed in the middle of the queue so that > > without scanning the queue we wouldn't be able to find them. > > Scanning a queue looks ugly. > > Don't scan the queue. If you have no resources for the first > consumer in the queue, everyone else has to wait. Make it simple > and stupid. Then compaction threads could occasionally be throttled by transactions that stalled due to the disk-based rate limit. > > > Think of the multi-queue design like this: there's one queue per each > > consumer type. When a resource is replenished, we check only those > > queues that store consumers that may proceed and choose the one that has > > waited most. > > How is that possible that some consumer can not proceed? The Transactions waiting for disk-based quota since the last interval may consume all quota that was added for the current interval and get throttled. Then we wouldn't be able to wake up compaction threads that were queued after them. Even though you might argue that this is unlikely and this should never happen once the load is stabilized, ignoring such a possibility makes me feel uncomfortable. Using one wait queue per consumer with ticketing for fairness is a simple and clear solution that assures this can't happen. It's clear both to me and Kirill. I fail to see why you find it difficult for understanding. You haven't given a single technical argument against it. You just seem to not like it for some reason, that's all... > replenishment is instantaneous for all kinds of resources. So if > there is at all a chance that the first (strictly first) consumer > has insufficient resources to proceed, it means it requires more > than is at all possible within this rate limit interval. It's > better to prohibit this altogether.
next prev parent reply other threads:[~2018-10-11 8:29 UTC|newest] Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-09-28 17:39 [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov 2018-09-28 17:39 ` [PATCH v2 01/11] vinyl: add helper to start scheduler and enable quota on startup Vladimir Davydov 2018-09-29 4:37 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 02/11] vinyl: factor load regulator out of quota Vladimir Davydov 2018-09-29 5:00 ` [tarantool-patches] " Konstantin Osipov 2018-09-29 11:36 ` Vladimir Davydov [not found] ` <20180929114308.GA19162@chai> 2018-10-01 10:27 ` Vladimir Davydov 2018-10-01 10:31 ` Vladimir Davydov 2018-10-02 18:16 ` [tarantool-patches] " Konstantin Osipov 2018-10-03 8:49 ` Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 03/11] vinyl: minor refactoring of quota methods Vladimir Davydov 2018-09-29 5:01 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 04/11] vinyl: move transaction size sanity check to quota Vladimir Davydov 2018-09-29 5:02 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 05/11] vinyl: implement quota wait queue without fiber_cond Vladimir Davydov 2018-09-29 5:05 ` [tarantool-patches] " Konstantin Osipov 2018-09-29 11:44 ` Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 06/11] vinyl: enable quota upon recovery completion explicitly Vladimir Davydov 2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 07/11] vinyl: zap vy_env::memory, read_threads, and write_threads Vladimir Davydov 2018-09-29 5:06 ` [tarantool-patches] " Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 08/11] vinyl: do not try to trigger dump in regulator if already in progress Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 09/11] vinyl: do not account zero dump bandwidth Vladimir Davydov 2018-10-12 13:27 ` Vladimir Davydov 2018-10-16 18:25 ` [tarantool-patches] " Konstantin Osipov 2018-10-17 8:44 ` Vladimir Davydov 2018-10-23 7:02 ` Konstantin Osipov 2018-09-28 17:40 ` [PATCH v2 10/11] vinyl: implement basic transaction throttling Vladimir Davydov 2018-09-28 17:40 ` [PATCH v2 11/11] vinyl: introduce quota consumer priorities Vladimir Davydov 2018-10-06 13:24 ` Konstantin Osipov 2018-10-08 11:10 ` Vladimir Davydov 2018-10-09 13:25 ` Vladimir Davydov 2018-10-11 7:02 ` Konstantin Osipov 2018-10-11 8:29 ` Vladimir Davydov [this message] 2018-10-03 9:06 ` [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181011082937.c5xpcpe5fime46cw@esperanza \ --to=vdavydov.dev@gmail.com \ --cc=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [PATCH v2 11/11] vinyl: introduce quota consumer priorities' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox