[Tarantool-patches] [PATCH 0/4] Safe truncation and deletion

[Konstantin Osipov]

* Vladislav Shpilevoy <v.shpilevoy at tarantool.org> [20/02/15 19:24]:
> Here is what Kostja said, but somewhy without CCing the
> mailing list, and my answers to it inlined:
> 
> > This is the first approximation of what I have proposed at best. 
> > First of all, the mechanism should not be truncation specific. It should be usable by all subsystems that require emergency memory.
> > 
> > Second, there is no reason to patch small. All the patch needs to do is something along these lines:
> > 
> > 
> > 1) in memtx_init, reserve the emergency slab.
> > 
> > 2) in memtx_tuple_alloc, refuse with error if there is no emergency slab
> > 
> > 3) in memtx_tuple_free, try to reserve emergency slab (replenish it) if it is empty
> 
> All of this adds code and conditions to a hot path. The way with quota
> enabling/disabling works only in a rare case, when delete() fails due
> to OOM, or when truncate is called, which is not often.

First of all, the way with quota simply doesn't work. You expect
the quota to shrink back, but it never does. So you simply run out
of all existing memory. 

If it worked, the fix would to reduce the amount of available
quota by 1 slab at start, and change quota before truncate, 
and then return the quota back in its place. 

My main point: under no circumstances tarantool should go beyond
the amount of memory set by quota. If we need to reserve
memory/quota for emergency, it's fine, let's do it at start,
but going beyond is not acceptable.

Second, yes, it does add a branch to the hot path. Same as in
memtx_index_extent_reserve().  This will have no impact on performance profile -
feel free to check. I think, however, given the 10% performance
regression in tuple_format, your time optimizing performance will
be better spent elsewhere.

> > 4) at start of truncate, or wherever we need emergency memory, release emergency slab. Simply return it to arena.
> 
> Once you returned it, all the other operations will be able to take
> and fill it. Such as insertions. In the first truncate or delete
> didn't free anything, or freed not enough to fit a new truncate
> tuple here, there is no more a reserved slab for a next delete/truncate.
> So your proposal does not seem to change anything.

Ehm, I did not fully explain the proposal. It also assumes there is 
"emergency mode" flag set at start of truncate and cleared at end,
and if there is no reserve slab and there is an emergence flag
set, nothing can allocate memory.

-- 
Konstantin Osipov, Moscow, Russia