Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: Konstantin Osipov <kostja@tarantool.org>
Cc: tarantool-patches@freelists.org
Subject: Re: [PATCH v2 11/11] vinyl: introduce quota consumer priorities
Date: Thu, 11 Oct 2018 11:29:37 +0300	[thread overview]
Message-ID: <20181011082937.c5xpcpe5fime46cw@esperanza> (raw)
In-Reply-To: <20181011070250.GB3110@chai>

On Thu, Oct 11, 2018 at 10:02:50AM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [18/10/08 14:12]:
> > 
> > Actually, there are resource types and there are consumer types.
> > I admit the fact that I mixed them may look confusing at the first
> > glance. We may introduce a seprate enum for resource types with a
> > mapping between them.
> > 
> > enum vy_quota_consumer_type {
> > 	VY_QUOTA_CONSUMER_TX = 0,
> > 	VY_QUOTA_CONSUMER_COMPACTION = 1,
> > 	vy_quota_consumer_max,
> > };
> > 
> > enum vy_quota_resource_type {
> > 	VY_QUOTA_RESOURCE_DISK = 0,
> > 	VY_QUOTA_RESOURCE_MEMORY = 1,
> > 	vy_quota_resource_max,
> > };
> > 
> > static unsigned vy_quota_consumer_mask[] = {
> > 	[VY_QUOTA_CONSUMER_TX] = (1 << VY_QUOTA_RESOURCE_DISK) |
> > 				 (1 << VY_QUOTA_RESOURCE_MEMORY),
> > 	[VY_QUOTA_CONSUMER_COMPACTION] = (1 << VY_QUOTA_RESOURCE_MEMORY),
> > };
> > 
> > struct vy_quota {
> > 	...
> > 	struct rlist wait_queue[vy_quota_consumer_max];
> > 	struct vy_rate_limit rate_limit[vy_quota_resource_max];
> > };
> 
> 
> > 
> > This would make the code more bulky though. Do we really want to
> > complicate?
> 
> I like it.

Then please see the patch:

https://www.freelists.org/post/tarantool-patches/PATCH-v2-1111-vinyl-introduce-quota-consumer-priorities,3

> > 
> > Also, quite frankly I don't quite dig the concept of 'resources' and
> > the corresponding constant names, because 'memory' rate limit may be
> > confused with memory usage, which is what vy_quota is about in the first
> > place.
> 
> Yes, vy_quota is originally about usage. I don't know why you
> insist on having rate limits in it ;) I'm ok if we put rate limits
> into a separate object, now that we have vy_regulator it can
> manage all of them.

Because I want to reuse the wait queue which was initially introduced
for putting consumers to sleep when they hit the memory limit.

Besides, rate limit value is consumed only along with quota so I'm
convinced they should be together.

> 
> > > Quota refill interval should be varying - please schedule this
> > > work in a separate patch.
> > > 
> > > 
> > > >  static void
> > > >  vy_quota_signal(struct vy_quota *q)
> > > >  {
> > > > -	if (!rlist_empty(&q->wait_queue)) {
> > > > +	/*
> > > > +	 * Wake up a consumer that has waited most no matter
> > > > +	 * whether it's high or low priority. This assures that
> > > > +	 * high priority consumers don't uncontrollably throttle
> > > > +	 * low priority ones.
> > > > +	 */
> > > > +	struct vy_quota_wait_node *oldest = NULL;
> > > > +	for (int i = 0; i < vy_quota_consumer_prio_MAX; i++) {
> > > > +		struct rlist *wq = &q->wait_queue[i];
> > > > +		if (rlist_empty(wq))
> > > > +			continue;
> > > 
> > > I still do not understand why you need a second queue and
> > > timestapms. If you need to ensure total fairness, a single queue
> > > should be enough. 
> > 
> > I need to maintain one queue per consumer type, because it's possible
> > that we may wake up consumers of one type, but not of the other.
> 
> If you do this, your fairness is strictly speaking broken, unless,
> then again, a type implies some kind of priority. By waking up a
> consumer out of (strict) order you give away a resource which may
> be needed for another consumer which (in strict single-queue
> order) is ahead of it.

I only wake up consumers out of (strict) order in case some consumers
can't proceed in the *current* interval, i.e. I only give the *surplus*
of the resource that wouldn't be consumed anyway in *this* interval.
Note, the resource is replenished on a timely basis so by dispensing the
surplus I don't deprive throttled consumers of any resource.

> 
> Frankly I still don't see why you did it this way except for the
> personal preference.
> 
> > If we used a single queue, it could occur that consumers that
> > could be woken up landed in the middle of the queue so that
> > without scanning the queue we wouldn't be able to find them.
> > Scanning a queue looks ugly.
> 
> Don't scan the queue. If you have no resources for the first
> consumer in the queue, everyone else has to wait. Make it simple
> and stupid.

Then compaction threads could occasionally be throttled by transactions
that stalled due to the disk-based rate limit.

> 
> > Think of the multi-queue design like this: there's one queue per each
> > consumer type. When a resource is replenished, we check only those
> > queues that store consumers that may proceed and choose the one that has
> > waited most.
> 
> How is that possible that some consumer can not proceed? The

Transactions waiting for disk-based quota since the last interval may
consume all quota that was added for the current interval and get
throttled. Then we wouldn't be able to wake up compaction threads that
were queued after them.

Even though you might argue that this is unlikely and this should never
happen once the load is stabilized, ignoring such a possibility makes me
feel uncomfortable. Using one wait queue per consumer with ticketing for
fairness is a simple and clear solution that assures this can't happen.
It's clear both to me and Kirill. I fail to see why you find it
difficult for understanding. You haven't given a single technical
argument against it. You just seem to not like it for some reason,
that's all...

> replenishment is instantaneous for all kinds of resources. So if
> there is at all a chance that the first (strictly first) consumer
> has insufficient resources to proceed, it means it requires more
> than is at all possible within this rate limit interval. It's
> better to prohibit this altogether.

  reply	other threads:[~2018-10-11  8:29 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-28 17:39 [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov
2018-09-28 17:39 ` [PATCH v2 01/11] vinyl: add helper to start scheduler and enable quota on startup Vladimir Davydov
2018-09-29  4:37   ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 02/11] vinyl: factor load regulator out of quota Vladimir Davydov
2018-09-29  5:00   ` [tarantool-patches] " Konstantin Osipov
2018-09-29 11:36     ` Vladimir Davydov
     [not found]       ` <20180929114308.GA19162@chai>
2018-10-01 10:27         ` Vladimir Davydov
2018-10-01 10:31   ` Vladimir Davydov
2018-10-02 18:16   ` [tarantool-patches] " Konstantin Osipov
2018-10-03  8:49     ` Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 03/11] vinyl: minor refactoring of quota methods Vladimir Davydov
2018-09-29  5:01   ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 04/11] vinyl: move transaction size sanity check to quota Vladimir Davydov
2018-09-29  5:02   ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 05/11] vinyl: implement quota wait queue without fiber_cond Vladimir Davydov
2018-09-29  5:05   ` [tarantool-patches] " Konstantin Osipov
2018-09-29 11:44     ` Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 06/11] vinyl: enable quota upon recovery completion explicitly Vladimir Davydov
2018-09-29  5:06   ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 07/11] vinyl: zap vy_env::memory, read_threads, and write_threads Vladimir Davydov
2018-09-29  5:06   ` [tarantool-patches] " Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 08/11] vinyl: do not try to trigger dump in regulator if already in progress Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 09/11] vinyl: do not account zero dump bandwidth Vladimir Davydov
2018-10-12 13:27   ` Vladimir Davydov
2018-10-16 18:25     ` [tarantool-patches] " Konstantin Osipov
2018-10-17  8:44       ` Vladimir Davydov
2018-10-23  7:02         ` Konstantin Osipov
2018-09-28 17:40 ` [PATCH v2 10/11] vinyl: implement basic transaction throttling Vladimir Davydov
2018-09-28 17:40 ` [PATCH v2 11/11] vinyl: introduce quota consumer priorities Vladimir Davydov
2018-10-06 13:24   ` Konstantin Osipov
2018-10-08 11:10     ` Vladimir Davydov
2018-10-09 13:25       ` Vladimir Davydov
2018-10-11  7:02       ` Konstantin Osipov
2018-10-11  8:29         ` Vladimir Davydov [this message]
2018-10-03  9:06 ` [PATCH v2 00/11] vinyl: transaction throttling infrastructure Vladimir Davydov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181011082937.c5xpcpe5fime46cw@esperanza \
    --to=vdavydov.dev@gmail.com \
    --cc=kostja@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH v2 11/11] vinyl: introduce quota consumer priorities' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox