[PATCH] vinyl: don't throttle DDL

[PATCH] vinyl: don't throttle DDL

[Vladimir Davydov]

Since DDL is triggered by the admin, it can be deliberately initiated
when the workload is known to be low. Throttling it along with DML
requests would only cause exasperation in this case. So we don't apply
disk-based rate limit to DDL. This should be fine, because the
disk-based limit is set rather strictly to let the workload some space
to grow, see vy_regulator_update_rate_limit(), and in contrast to the
memory-based limit, exceeding the disk-based limit doesn't result in
abrupt stalls - it may only lead to a gradual accumulation of disk space
usage and read latency.

Closes #4238
---
https://github.com/tarantool/tarantool/issues/4238
https://github.com/tarantool/tarantool/commits/dv/gh-4238-vy-dont-throttle-ddl

 src/box/vinyl.c    |  6 +++---
 src/box/vy_quota.c | 13 +++++++++++++
 src/box/vy_quota.h |  2 ++
 3 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index 8286fed7..9b4bc53b 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -4191,10 +4191,10 @@ vy_build_insert_tuple(struct vy_env *env, struct vy_lsm *lsm,
 	/* Consume memory quota. Throttle if it is exceeded. */
 	size_t mem_used_after = lsregion_used(&env->mem_env.allocator);
 	assert(mem_used_after >= mem_used_before);
-	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_TX,
+	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_DDL,
 			   mem_used_after - mem_used_before);
 	vy_regulator_check_dump_watermark(&env->regulator);
-	vy_quota_wait(&env->quota, VY_QUOTA_CONSUMER_TX);
+	vy_quota_wait(&env->quota, VY_QUOTA_CONSUMER_DDL);
 	return rc;
 }
 
@@ -4321,7 +4321,7 @@ vy_build_recover(struct vy_env *env, struct vy_lsm *lsm, struct vy_lsm *pk)
 
 	mem_used_after = lsregion_used(&env->mem_env.allocator);
 	assert(mem_used_after >= mem_used_before);
-	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_TX,
+	vy_quota_force_use(&env->quota, VY_QUOTA_CONSUMER_DDL,
 			   mem_used_after - mem_used_before);
 	return rc;
 }
diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c
index 035040a3..f1ac8dd9 100644
--- a/src/box/vy_quota.c
+++ b/src/box/vy_quota.c
@@ -76,6 +76,19 @@ vy_quota_consumer_resource_map[] = {
 	 * to avoid long stalls.
 	 */
 	[VY_QUOTA_CONSUMER_COMPACTION] = (1 << VY_QUOTA_RESOURCE_MEMORY),
+	/**
+	 * Since DDL is triggered by the admin, it can be deliberately
+	 * initiated when the workload is known to be low. Throttling
+	 * it along with DML requests would only cause exasperation in
+	 * this case. So we don't apply disk-based rate limit to DDL.
+	 * This should be fine, because the disk-based limit is set
+	 * rather strictly to let the workload some space to grow, see
+	 * vy_regulator_update_rate_limit(), and in contrast to the
+	 * memory-based limit, exceeding the disk-based limit doesn't
+	 * result in abrupt stalls - it may only lead to a gradual
+	 * accumulation of disk space usage and read latency.
+	 */
+	[VY_QUOTA_CONSUMER_DDL] = (1 << VY_QUOTA_RESOURCE_MEMORY),
 };
 
 /**
diff --git a/src/box/vy_quota.h b/src/box/vy_quota.h
index 7ff98cc1..63d4f6a8 100644
--- a/src/box/vy_quota.h
+++ b/src/box/vy_quota.h
@@ -150,6 +150,8 @@ enum vy_quota_consumer_type {
 	VY_QUOTA_CONSUMER_TX = 0,
 	/** Compaction job. */
 	VY_QUOTA_CONSUMER_COMPACTION = 1,
+	/** Request to build a new index. */
+	VY_QUOTA_CONSUMER_DDL = 2,
 
 	vy_quota_consumer_type_MAX,
 };
-- 
2.11.0

[tarantool-patches] Re: [PATCH] vinyl: don't throttle DDL

[Konstantin Osipov]

* Vladimir Davydov <vdavydov.dev@gmail.com> [19/05/31 13:01]:
> Since DDL is triggered by the admin, it can be deliberately initiated
> when the workload is known to be low. Throttling it along with DML
> requests would only cause exasperation in this case. So we don't apply
> disk-based rate limit to DDL. This should be fine, because the
> disk-based limit is set rather strictly to let the workload some space
> to grow, see vy_regulator_update_rate_limit(), and in contrast to the
> memory-based limit, exceeding the disk-based limit doesn't result in
> abrupt stalls - it may only lead to a gradual accumulation of disk space
> usage and read latency.
> 
> Closes #4238

Nice that you provisioned for this in advance. LGTM.


-- 
Konstantin Osipov, Moscow, Russia

Re: [PATCH] vinyl: don't throttle DDL

[Vladimir Davydov]

Pushed to master, 2.1, 1.10.