Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: tarantool-patches@freelists.org
Subject: [PATCH] vinyl: don't throttle DDL
Date: Fri, 31 May 2019 13:00:59 +0300	[thread overview]
Message-ID: <6a44ac6225255600cdc0ab8c082c04c9e98f89f8.1559296816.git.vdavydov.dev@gmail.com> (raw)

Since DDL is triggered by the admin, it can be deliberately initiated
when the workload is known to be low. Throttling it along with DML
requests would only cause exasperation in this case. So we don't apply
disk-based rate limit to DDL. This should be fine, because the
disk-based limit is set rather strictly to let the workload some space
to grow, see vy_regulator_update_rate_limit(), and in contrast to the
memory-based limit, exceeding the disk-based limit doesn't result in
abrupt stalls - it may only lead to a gradual accumulation of disk space
usage and read latency.

Closes #4238
---
https://github.com/tarantool/tarantool/issues/4238
https://github.com/tarantool/tarantool/commits/dv/gh-4238-vy-dont-throttle-ddl

 src/box/vinyl.c    |  6 +++---
 src/box/vy_quota.c | 13 +++++++++++++
 src/box/vy_quota.h |  2 ++
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index 8286fed7..9b4bc53b 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -4191,10 +4191,10 @@ vy_build_insert_tuple(struct vy_env *env, struct vy_lsm *lsm,
 	/* Consume memory quota. Throttle if it is exceeded. */
 	size_t mem_used_after = lsregion_used(&env->mem_env.allocator);
 	assert(mem_used_after >= mem_used_before);
-	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_TX,
+	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_DDL,
 			   mem_used_after - mem_used_before);
 	vy_regulator_check_dump_watermark(&env->regulator);
-	vy_quota_wait(&env->quota, VY_QUOTA_CONSUMER_TX);
+	vy_quota_wait(&env->quota, VY_QUOTA_CONSUMER_DDL);
 	return rc;
 }
 
@@ -4321,7 +4321,7 @@ vy_build_recover(struct vy_env *env, struct vy_lsm *lsm, struct vy_lsm *pk)
 
 	mem_used_after = lsregion_used(&env->mem_env.allocator);
 	assert(mem_used_after >= mem_used_before);
-	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_TX,
+	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_DDL,
 			   mem_used_after - mem_used_before);
 	return rc;
 }
diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c
index 035040a3..f1ac8dd9 100644
--- a/src/box/vy_quota.c
+++ b/src/box/vy_quota.c
@@ -76,6 +76,19 @@ vy_quota_consumer_resource_map[] = {
 	 * to avoid long stalls.
 	 */
 	[VY_QUOTA_CONSUMER_COMPACTION] = (1 << VY_QUOTA_RESOURCE_MEMORY),
+	/**
+	 * Since DDL is triggered by the admin, it can be deliberately
+	 * initiated when the workload is known to be low. Throttling
+	 * it along with DML requests would only cause exasperation in
+	 * this case. So we don't apply disk-based rate limit to DDL.
+	 * This should be fine, because the disk-based limit is set
+	 * rather strictly to let the workload some space to grow, see
+	 * vy_regulator_update_rate_limit(), and in contrast to the
+	 * memory-based limit, exceeding the disk-based limit doesn't
+	 * result in abrupt stalls - it may only lead to a gradual
+	 * accumulation of disk space usage and read latency.
+	 */
+	[VY_QUOTA_CONSUMER_DDL] = (1 << VY_QUOTA_RESOURCE_MEMORY),
 };
 
 /**
diff --git a/src/box/vy_quota.h b/src/box/vy_quota.h
index 7ff98cc1..63d4f6a8 100644
--- a/src/box/vy_quota.h
+++ b/src/box/vy_quota.h
@@ -150,6 +150,8 @@ enum vy_quota_consumer_type {
 	VY_QUOTA_CONSUMER_TX = 0,
 	/** Compaction job. */
 	VY_QUOTA_CONSUMER_COMPACTION = 1,
+	/** Request to build a new index. */
+	VY_QUOTA_CONSUMER_DDL = 2,
 
 	vy_quota_consumer_type_MAX,
 };
-- 
2.11.0

             reply	other threads:[~2019-05-31 10:00 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-31 10:00 Vladimir Davydov [this message]
2019-05-31 10:11 ` [tarantool-patches] " Konstantin Osipov
2019-05-31 10:35 ` Vladimir Davydov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6a44ac6225255600cdc0ab8c082c04c9e98f89f8.1559296816.git.vdavydov.dev@gmail.com \
    --to=vdavydov.dev@gmail.com \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH] vinyl: don'\''t throttle DDL' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox