[Tarantool-patches] [PATCH 0/4] Safe truncation and deletion

[Vladislav Shpilevoy]

Here is what Kostja said, but somewhy without CCing the
mailing list, and my answers to it inlined:

> This is the first approximation of what I have proposed at best. 
> First of all, the mechanism should not be truncation specific. It should be usable by all subsystems that require emergency memory.
> 
> Second, there is no reason to patch small. All the patch needs to do is something along these lines:
> 
> 
> 1) in memtx_init, reserve the emergency slab.
> 
> 2) in memtx_tuple_alloc, refuse with error if there is no emergency slab
> 
> 3) in memtx_tuple_free, try to reserve emergency slab (replenish it) if it is empty

All of this adds code and conditions to a hot path. The way with quota
enabling/disabling works only in a rare case, when delete() fails due
to OOM, or when truncate is called, which is not often.

> 4) at start of truncate, or wherever we need emergency memory, release emergency slab. Simply return it to arena.

Once you returned it, all the other operations will be able to take
and fill it. Such as insertions. In the first truncate or delete
didn't free anything, or freed not enough to fit a new truncate
tuple here, there is no more a reserved slab for a next delete/truncate.
So your proposal does not seem to change anything.

> No need for special members or code for this outside small.
> 
> We could generalize this idea by moving all system spaces to its own arena.
> 
> On Sat, Feb 15, 2020, 01:48 Vladislav Shpilevoy <v.shpilevoy at tarantool.org> wrote:
> 
>     Thanks for the patchset!
> 
>     On 14/02/2020 20:40, Ilya Kosarev wrote:
>     > This patchset is an experiment on reserve tech used for safe truncation
>     > and the demonstration of the possible space:delete() fail on reserve.
>     > The idea of the 3rd patch (memtx: use reserved slab for truncation) was
>     > to prealloc a slab on arena to be used only for truncation tuples.
>     > This approach didn't really solve the problem. After we are getting the
>     > reserved slab with slab_map, we have a choice to put or not to put it
>     > on slab lists. In case we do put it, the problem is that any next
>     > operation, using mempool_alloc, will be able to use our slab, not only
>     > truncation. This will lead to it's fast exhaustion. On the other hand,
>     > if we skip all this lists (as it is done), we will fail at the garbage
>     > collection when trying to free those tuples, as far as their slab won't
>     > be found. See backtrace at
>     > https://gitlab.com/tarantool/tarantool/-/jobs/438082894. It doesn't
>     > look like there is any acceptable way to tune garbage collector
>     > behavior.
>     > In the 4th patch the stress_delete test with errinj is introduced to
>     > show how it may fail on reserve. It uses errinj to simulate the case
>     > where the num of reserved extents for memtx_index_extent_reserve won't
>     > be enough, setting target amount as current amount + 1.
>     >
>     > Not aimed to be pushed to master!
>     >
>     > Branch: https://github.com/tarantool/tarantool/tree/i.kosarev/gh-3807-safe-alloc-on-truncation
>     > Issue: https://github.com/tarantool/tarantool/issues/3807
> 
>     This is not a correct link. You pushed this patchset into
>     https://github.com/tarantool/tarantool/tree/i.kosarev/gh-3807-reserve-on-truncation
> 
>     Overall the patchset looks ... ghm, I would say not really good. It
>     consists of crutches more than completely. I don't think that approach
>     works, if this is what Kostja proposed.
>