From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 18 Jul 2019 12:47:57 +0300 From: Konstantin Osipov Subject: Re: [PATCH v3 3/4] box: introduce tuple_extra infrastructure Message-ID: <20190718094757.GB21191@atlas> References: <3474f2b5e217e1dcd5ab405b72f574b283b132f0.1563326037.git.kshcherbatov@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3474f2b5e217e1dcd5ab405b72f574b283b132f0.1563326037.git.kshcherbatov@tarantool.org> To: Kirill Shcherbatov Cc: tarantool-patches@freelists.org, vdavydov.dev@gmail.com List-ID: * Kirill Shcherbatov [19/07/17 09:39]: > Introduced a new object tuple_extra: a memory allocation is > associated with given tuple and chunk_id. Extended tuple_format's > vtab with few new methods to manage tuple_extras lifecycle. > Supported tuple_extra for memtx engine: a memory > chunks are allocated with memtx's smalloc allocator. tuple_extra is a very general name. Are all extra chunks/segments of equal size, or variable size? Why not change tuple format right away to consist of segments, and add segments on the side? I've been warning about a mounting technical debt, and this is basically beyond any reasonable limit: - we have huge problems that large tuples lead to memory fragmentation - we have an ugly realloc when building a tuple field map for multikey indexes - now we have a yet another ugly thing on the side for functional indexes. I understand that coming up with a new log structured tuple allocator may be considered beyond the scope of this patch. But even without a new allocator this patch is already half way through segmented tuple structure. So please deal with the debt first and transform tuple format to segmented one. The first segment should be up to a few kilobytes in size and should contain the old-good-tuple as it is now. All subsequent segments could be allocated using the same principles as in stl deque - an array of chunks. -- Konstantin Osipov, Moscow, Russia