From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH 8/8] memtx: run garbage collection on demand Date: Tue, 22 May 2018 14:46:16 +0300 Message-Id: In-Reply-To: References: In-Reply-To: References: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: When a memtx space is dropped or truncated, we delegate freeing tuples stored in it to a background fiber so as not to block the caller (and tx thread) for too long. Turns out it doesn't work out well for ephemeral spaces, which share the destruction code with normal spaces: the problem is the user might issue a lot of complex SQL SELECT statements that create a lot of ephemeral spaces and do not yield and hence don't give the garbage collection fiber a chance to clean up. There's a test that emulates this, 2.0:test/sql-tap/gh-3083-ephemeral-unref-tuples.test.lua. For this test to pass, let's run garbage collection procedure on demand, i.e. when any of memtx allocation functions fails to allocate memory. Follow-up #3408 --- src/box/memtx_engine.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c index 3b21bcaa..afb453f9 100644 --- a/src/box/memtx_engine.c +++ b/src/box/memtx_engine.c @@ -1069,7 +1069,11 @@ memtx_tuple_new(struct tuple_format *format, const char *data, const char *end) return NULL; } - struct memtx_tuple *memtx_tuple = smalloc(&memtx->alloc, total); + struct memtx_tuple *memtx_tuple; + while ((memtx_tuple = smalloc(&memtx->alloc, total)) == NULL) { + if (!memtx_engine_run_gc(memtx)) + break; + } if (memtx_tuple == NULL) { diag_set(OutOfMemory, total, "slab allocator", "memtx_tuple"); return NULL; @@ -1150,7 +1154,11 @@ memtx_index_extent_alloc(void *ctx) "mempool", "new slab"); return NULL; }); - void *ret = mempool_alloc(&memtx->index_extent_pool); + void *ret; + while ((ret = mempool_alloc(&memtx->index_extent_pool)) == NULL) { + if (!memtx_engine_run_gc(memtx)) + break; + } if (ret == NULL) diag_set(OutOfMemory, MEMTX_EXTENT_SIZE, "mempool", "new slab"); @@ -1183,6 +1191,8 @@ memtx_index_extent_reserve(struct memtx_engine *memtx, int num) while (memtx->num_reserved_extents < num) { void *ext = mempool_alloc(&memtx->index_extent_pool); if (ext == NULL) { + if (memtx_engine_run_gc(memtx)) + continue; diag_set(OutOfMemory, MEMTX_EXTENT_SIZE, "mempool", "new slab"); return -1; -- 2.11.0