From: Ilya Kosarev <i.kosarev@tarantool.org> To: tarantool-patches@dev.tarantool.org Cc: v.shpilevoy@tarantool.org Subject: [Tarantool-patches] [PATCH v2 2/2] memtx: increase the memory quota if needed to truncate or delete Date: Fri, 10 Jan 2020 03:36:53 +0300 [thread overview] Message-ID: <055fe8425fe1d7a529601fc6d04da1b8bc9fd520.1578615748.git.i.kosarev@tarantool.org> (raw) In-Reply-To: <cover.1578615748.git.i.kosarev@tarantool.org> In-Reply-To: <cover.1578615748.git.i.kosarev@tarantool.org> Trying to perform space:truncate() and space:delete() while reaching memtx_memory limit we could experience slab allocator failure. This behavior seems to be quite surprising for users. Now we are increasing memtx quota if needed for truncation or deletion. After performing it quota is being set back to the previous value if possible, while it should be so for almost any case, since we are meant to free some space during deletion or truncation. Closes #3807 --- src/box/box.cc | 34 +++++++++++++++++++++++++++++++++- src/box/memtx_engine.c | 18 ++++++++++++++++++ src/box/memtx_engine.h | 5 +++++ 3 files changed, 56 insertions(+), 1 deletion(-) diff --git a/src/box/box.cc b/src/box/box.cc index 1b2b27d61..03e2f3cc9 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -1250,7 +1250,26 @@ box_delete(uint32_t space_id, uint32_t index_id, const char *key, request.index_id = index_id; request.key = key; request.key_end = key_end; - return box_process1(&request, result); + + struct space *space = space_cache_find(space_id); + if (space == NULL) + return -1; + struct memtx_engine *memtx = + (struct memtx_engine *)space->engine; + size_t total; + bool extended; + memtx_engine_guarantee_memory(memtx, memtx->arena.slab_size, + &total, &extended); + + int rc = box_process1(&request, result); + + if (extended) { + size_t new_total = quota_set(&memtx->quota, total); + if (new_total > total) + quota_set(&memtx->quota, quota_used(&memtx->quota)); + } + + return rc; } int @@ -1321,9 +1340,22 @@ space_truncate(struct space *space) ops_buf_end = mp_encode_uint(ops_buf_end, 1); assert(ops_buf_end < buf + buf_size); + struct memtx_engine *memtx = + (struct memtx_engine *)space->engine; + size_t total; + bool extended; + memtx_engine_guarantee_memory(memtx, memtx->arena.slab_size, + &total, &extended); + if (box_upsert(BOX_TRUNCATE_ID, 0, tuple_buf, tuple_buf_end, ops_buf, ops_buf_end, 0, NULL) != 0) diag_raise(); + + if (extended) { + size_t new_total = quota_set(&memtx->quota, total); + if (new_total > total) + quota_set(&memtx->quota, quota_used(&memtx->quota)); + } } int diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c index 23ccc4703..cbac015e3 100644 --- a/src/box/memtx_engine.c +++ b/src/box/memtx_engine.c @@ -1090,6 +1090,24 @@ memtx_engine_set_memory(struct memtx_engine *memtx, size_t size) return 0; } +void +memtx_engine_guarantee_memory(struct memtx_engine *memtx, + size_t request, size_t *old_total, + bool *extended) +{ + struct quota *memtx_quota = &memtx->quota; + size_t total, used; + quota_get_total_and_used(memtx_quota, &total, &used); + *old_total = total; + if (total - used < request) { + quota_set(memtx_quota, + total + request - (total - used)); + *extended = true; + return; + } + *extended = false; +} + void memtx_engine_set_max_tuple_size(struct memtx_engine *memtx, size_t max_size) { diff --git a/src/box/memtx_engine.h b/src/box/memtx_engine.h index f562c66df..f74f65e96 100644 --- a/src/box/memtx_engine.h +++ b/src/box/memtx_engine.h @@ -213,6 +213,11 @@ memtx_engine_set_snap_io_rate_limit(struct memtx_engine *memtx, double limit); int memtx_engine_set_memory(struct memtx_engine *memtx, size_t size); +void +memtx_engine_guarantee_memory(struct memtx_engine *memtx, + size_t request, size_t *old_total, + bool *extended); + void memtx_engine_set_max_tuple_size(struct memtx_engine *memtx, size_t max_size); -- 2.17.1
next prev parent reply other threads:[~2020-01-10 0:36 UTC|newest] Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-10 0:36 [Tarantool-patches] [PATCH v2 0/2] Safe truncation and deletion Ilya Kosarev 2020-01-10 0:36 ` [Tarantool-patches] [PATCH v2 1/2] b-tree: return NULL on matras_alloc fail Ilya Kosarev 2020-01-10 0:36 ` Ilya Kosarev [this message] 2020-01-13 21:31 [Tarantool-patches] [PATCH v2 0/2] Safe truncation and deletion Ilya Kosarev 2020-01-13 21:31 ` [Tarantool-patches] [PATCH v2 2/2] memtx: increase the memory quota if needed to truncate or delete Ilya Kosarev 2020-01-14 21:00 ` Vladislav Shpilevoy 2020-01-20 18:13 ` Ilya Kosarev 2020-01-24 11:21 ` Konstantin Osipov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=055fe8425fe1d7a529601fc6d04da1b8bc9fd520.1578615748.git.i.kosarev@tarantool.org \ --to=i.kosarev@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v2 2/2] memtx: increase the memory quota if needed to truncate or delete' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox