Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH 07/11] vinyl: do not account zero dump bandwidth
Date: Thu, 20 Sep 2018 12:34:12 +0300	[thread overview]
Message-ID: <0f5578b940031e14c8e08234648c84103e0d78b2.1537435404.git.vdavydov.dev@gmail.com> (raw)
In-Reply-To: <cover.1537435404.git.vdavydov.dev@gmail.com>
In-Reply-To: <cover.1537435404.git.vdavydov.dev@gmail.com>

Since we free memory in 16 MB blocks (see SLAB_SIZE), it may occur that
we dump almost all data stored in a block but still have to leave it be,
because it contains data of a newer generation. If the memory limit is
small (as it is typically in tests), this may result in dumping 0 bytes.
In order not to disrupt statistics and throttling transactions in vain,
let's simply ignore such results. Normally, the memory limit should be
large enough for such granularity not to affect the measurements
(hundreds of megabytes) so this problem isn't worth putting more efforts
into.
---
 src/box/vy_quota.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/src/box/vy_quota.c b/src/box/vy_quota.c
index 7cd64474..5f8c0618 100644
--- a/src/box/vy_quota.c
+++ b/src/box/vy_quota.c
@@ -205,8 +205,20 @@ vy_quota_dump(struct vy_quota *q, size_t size, double duration)
 	q->used -= size;
 	fiber_cond_signal(&q->cond);
 
-	/* Update dump bandwidth. */
-	if (duration > 0) {
+	/*
+	 * Update dump bandwidth.
+	 *
+	 * Note, since we free memory in 16 MB blocks (see SLAB_SIZE),
+	 * it may occur that we dump almost all data stored in a block
+	 * but still have to leave it be, because it contains data of
+	 * a newer generation. If the memory limit is small, this may
+	 * result in dumping 0 bytes. In order not to disrupt statistics
+	 * let's simply ignore such results. Normally, the memory limit
+	 * should be large enough for such granularity not to affect the
+	 * measurements (hundreds of megabytes) so this problem isn't
+	 * worth putting more efforts into.
+	 */
+	if (size > 0 && duration > 0) {
 		histogram_collect(q->dump_bw_hist, size / duration);
 		/*
 		 * To avoid unpredictably long stalls, we need to
-- 
2.11.0

  parent reply	other threads:[~2018-09-20  9:34 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-20  9:34 [PATCH 00/11] vinyl: prepare for transaction throttling Vladimir Davydov
2018-09-20  9:34 ` [PATCH 01/11] vinyl: merge vy_quota_release and vy_quota_update_dump_bandwidth Vladimir Davydov
2018-09-20  9:34 ` [PATCH 02/11] vinyl: refactor quota use/unuse methods Vladimir Davydov
2018-09-20  9:34 ` [PATCH 03/11] vinyl: do not try to trigger dump if it is already in progress Vladimir Davydov
2018-09-20  9:34 ` [PATCH 04/11] vinyl: don't start quota timer until local recovery is complete Vladimir Davydov
2018-09-20  9:34 ` [PATCH 05/11] vinyl: add helper to start scheduler and enable quota on startup Vladimir Davydov
2018-09-25 23:22   ` [tarantool-patches] " Konstantin Osipov
2018-09-20  9:34 ` [PATCH 06/11] vinyl: zap vy_env::memory, read_threads, and write_threads Vladimir Davydov
2018-09-25 23:23   ` [tarantool-patches] " Konstantin Osipov
2018-09-20  9:34 ` Vladimir Davydov [this message]
2018-09-25 23:24   ` [tarantool-patches] Re: [PATCH 07/11] vinyl: do not account zero dump bandwidth Konstantin Osipov
2018-09-20  9:34 ` [PATCH 08/11] vinyl: set quota timer period to 100 ms Vladimir Davydov
2018-09-20  9:34 ` [PATCH 09/11] vinyl: implement basic transaction throttling Vladimir Davydov
2018-09-20  9:34 ` [PATCH 10/11] vinyl: implement quota wait queue without fiber_cond Vladimir Davydov
2018-09-20  9:34 ` [PATCH 11/11] vinyl: split quota consumption rate limit into soft and hard Vladimir Davydov
2018-09-25 23:19 ` [tarantool-patches] Re: [PATCH 00/11] vinyl: prepare for transaction throttling Konstantin Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0f5578b940031e14c8e08234648c84103e0d78b2.1537435404.git.vdavydov.dev@gmail.com \
    --to=vdavydov.dev@gmail.com \
    --cc=kostja@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH 07/11] vinyl: do not account zero dump bandwidth' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox