Re: [PATCH v3 3/4] box: introduce tuple_extra infrastructure

[Konstantin Osipov <kostja@tarantool.org>]

* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> Introduced a new object tuple_extra: a memory allocation is
> associated with given tuple and chunk_id. Extended tuple_format's
> vtab with few new methods to manage tuple_extras lifecycle.
> Supported tuple_extra for memtx engine: a memory
> chunks are allocated with memtx's smalloc allocator.

tuple_extra is a very general name. 

Are all extra chunks/segments of equal size, or variable size?

Why not change tuple format right away to consist of segments, and
add segments on the side? 

I've been warning about a mounting technical debt, and this is
basically beyond any reasonable limit:
- we have huge problems that large tuples lead to memory
  fragmentation
- we have an ugly realloc when building a tuple field map for
  multikey indexes
- now we have a yet another ugly thing on the side for functional 
  indexes.

I understand that coming up with a new log structured tuple allocator 
may be considered beyond the scope of this patch. But even without
a new allocator this patch is already half way through segmented
tuple structure.

So please deal with the debt first and transform tuple format to
segmented one.

The first segment should be up to a few kilobytes in size and
should contain the old-good-tuple as it is now. All subsequent
segments could be allocated using the same principles as in stl
deque - an array of chunks.


-- 
Konstantin Osipov, Moscow, Russia