From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 18 Apr 2019 21:30:22 +0300 From: Alexander Turenko Subject: Re: [tarantool-patches] [PATCH v3 4/7] lua: add non-recursive msgpack decoding functions Message-ID: <20190418183019.nolq3dprawgeubxw@tkn_work_nb> References: <20190418173528.GL13022@chai> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20190418173528.GL13022@chai> To: Konstantin Osipov Cc: tarantool-patches@freelists.org, Vladimir Davydov List-ID: In short: * The behaviour is what I need (I cannot skip data). * The names are confusing, but I don't see better variants. These ones at least consistent with others. I posted below relevant discussions, old variants, possible variants, usage examples and so on. WBR, Alexander Turenko. On Thu, Apr 18, 2019 at 08:35:28PM +0300, Konstantin Osipov wrote: > * Alexander Turenko [19/04/10 18:23]: > > Contracts: > > > > ``` > > msgpack.decode_array(buf.rpos, buf:size()) -> arr_len, new_rpos > > msgpack.decode_map(buf.rpos, buf:size()) -> map_len, new_rpos > > ``` > > From what they do, they actually neither decode nor skip an > array/map. They decode array/map header only. I know that msgpuck > uses this naming, but it is actually confusing. Yep, names are most hard part of the patch. They were check_array/map(), then were renamed to decode_array/map(). The relevant parts from the previous discussion: Vladimir: > Not sure about the name either, because it doesn't just checks the > msgpack - it decodes it, but can't come up with anything substantially > better. May be, msgpack.decode_array? > > https://www.freelists.org/post/tarantool-patches/PATCH-v2-56-netbox-add-helpers-to-decode-msgpack-headers,1 Me: > Re name: now I understood: decode_unchecked() is like mp_decode(), > decode() is like mp_check() + mp_decode(). So it worth to rename it to > decode_array(). Done. > > https://www.freelists.org/post/tarantool-patches/PATCH-v2-56-netbox-add-helpers-to-decode-msgpack-headers,2 All the words 'check', 'skip' and 'decode' are assumed intuitively as something related to the whole array/map. The only thing we able to do here is to add '_header' postfix (it would look a bit weird, IMHO). Should I do? > Why not just encode array/map header, but skip array/map data in > these functions and and call them msgpack.skip_array(), > msgpack.skip_map() respectively? I need to skip an array/map header (and save length), but feed bare data to a source constructor. Initially I had a parameter in merger (now it would be in a source constructor) that say how much array headers it need to skip before actually read tuples. AFAIR, the option values were BUFFER_RAW, BUFFER_SELECT, BUFFER_CALL, BUFFER_CHUNKED. Then we decided to factor it our from merger and then splitted to msgpack helpers and a net.box option. > > + > > + uint32_t len = mp_decode_array(&data); > > This doesn't actually skip array members, does it. This is exactly what I need. Don't know how to describe it in short. I receive [[tuple, tuple, ...], [tuple, tuple, ...]] in a buffer and need to feed each [tuple, tuple, ...] part to merger. The following sections of merger's examples should shed light on that I hope: https://github.com/Totktonada/tarantool-merger-examples/tree/695fc9511685033f4b4b22c0df537a12ddf2eaf6#preparing-buffers https://github.com/Totktonada/tarantool-merger-examples/tree/695fc9511685033f4b4b22c0df537a12ddf2eaf6#multiplexing-requests The usage example itself is here: https://github.com/Totktonada/tarantool-merger-examples/blob/695fc9511685033f4b4b22c0df537a12ddf2eaf6/multiplexed_example/frontend.lua#L38-L46 WBR, Alexander Turenko.