Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH 8/8] memtx: run garbage collection on demand
Date: Tue, 22 May 2018 14:46:16 +0300	[thread overview]
Message-ID: <fad9a9fcc1e8eb97da03d61bebd6f0b14384460d.1526987033.git.vdavydov.dev@gmail.com> (raw)
In-Reply-To: <cover.1526987033.git.vdavydov.dev@gmail.com>
In-Reply-To: <cover.1526987033.git.vdavydov.dev@gmail.com>

When a memtx space is dropped or truncated, we delegate freeing tuples
stored in it to a background fiber so as not to block the caller (and tx
thread) for too long. Turns out it doesn't work out well for ephemeral
spaces, which share the destruction code with normal spaces: the problem
is the user might issue a lot of complex SQL SELECT statements that
create a lot of ephemeral spaces and do not yield and hence don't give
the garbage collection fiber a chance to clean up. There's a test that
emulates this, 2.0:test/sql-tap/gh-3083-ephemeral-unref-tuples.test.lua.
For this test to pass, let's run garbage collection procedure on demand,
i.e. when any of memtx allocation functions fails to allocate memory.

Follow-up #3408
---
 src/box/memtx_engine.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c
index 3b21bcaa..afb453f9 100644
--- a/src/box/memtx_engine.c
+++ b/src/box/memtx_engine.c
@@ -1069,7 +1069,11 @@ memtx_tuple_new(struct tuple_format *format, const char *data, const char *end)
 		return NULL;
 	}
 
-	struct memtx_tuple *memtx_tuple = smalloc(&memtx->alloc, total);
+	struct memtx_tuple *memtx_tuple;
+	while ((memtx_tuple = smalloc(&memtx->alloc, total)) == NULL) {
+		if (!memtx_engine_run_gc(memtx))
+			break;
+	}
 	if (memtx_tuple == NULL) {
 		diag_set(OutOfMemory, total, "slab allocator", "memtx_tuple");
 		return NULL;
@@ -1150,7 +1154,11 @@ memtx_index_extent_alloc(void *ctx)
 			 "mempool", "new slab");
 		return NULL;
 	});
-	void *ret = mempool_alloc(&memtx->index_extent_pool);
+	void *ret;
+	while ((ret = mempool_alloc(&memtx->index_extent_pool)) == NULL) {
+		if (!memtx_engine_run_gc(memtx))
+			break;
+	}
 	if (ret == NULL)
 		diag_set(OutOfMemory, MEMTX_EXTENT_SIZE,
 			 "mempool", "new slab");
@@ -1183,6 +1191,8 @@ memtx_index_extent_reserve(struct memtx_engine *memtx, int num)
 	while (memtx->num_reserved_extents < num) {
 		void *ext = mempool_alloc(&memtx->index_extent_pool);
 		if (ext == NULL) {
+			if (memtx_engine_run_gc(memtx))
+				continue;
 			diag_set(OutOfMemory, MEMTX_EXTENT_SIZE,
 				 "mempool", "new slab");
 			return -1;
-- 
2.11.0

  parent reply	other threads:[~2018-05-22 11:46 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-22 11:46 [PATCH 0/8] Follow-up on async memtx index cleanup Vladimir Davydov
2018-05-22 11:46 ` [PATCH 1/8] memtx: init index extent allocator in engine constructor Vladimir Davydov
2018-05-22 13:43   ` Konstantin Osipov
2018-05-22 11:46 ` [PATCH 2/8] memtx: fold memtx_tuple.cc into memtx_engine.c Vladimir Davydov
2018-05-22 13:45   ` Konstantin Osipov
2018-05-22 11:46 ` [PATCH 3/8] memtx: pass engine to memory allocation functions Vladimir Davydov
2018-05-22 13:47   ` Konstantin Osipov
2018-05-22 14:39     ` Vladimir Davydov
2018-05-22 11:46 ` [PATCH 4/8] memtx: move all global variables to engine Vladimir Davydov
2018-05-22 13:48   ` Konstantin Osipov
2018-05-22 11:46 ` [PATCH 5/8] memtx: destroy slab arena on engine shutdown Vladimir Davydov
2018-05-22 13:50   ` Konstantin Osipov
2018-05-22 16:26     ` Vladimir Davydov
2018-05-22 11:46 ` [PATCH 6/8] memtx: embed light hash into memtx_hash_index Vladimir Davydov
2018-05-22 13:51   ` Konstantin Osipov
2018-05-22 11:46 ` [PATCH 7/8] memtx: rework background garbage collection procedure Vladimir Davydov
2018-05-22 13:56   ` Konstantin Osipov
2018-05-22 14:49     ` Vladimir Davydov
2018-05-22 16:42       ` Konstantin Osipov
2018-05-22 11:46 ` Vladimir Davydov [this message]
2018-05-22 14:00   ` [PATCH 8/8] memtx: run garbage collection on demand Konstantin Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fad9a9fcc1e8eb97da03d61bebd6f0b14384460d.1526987033.git.vdavydov.dev@gmail.com \
    --to=vdavydov.dev@gmail.com \
    --cc=kostja@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH 8/8] memtx: run garbage collection on demand' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox