Re: [Tarantool-patches] [PATCH 2.X 6/7] module api: luaL_checkibuf & luaL_checkconstchar

[Alexander Turenko <alexander.turenko@tarantool.org>]

On Wed, Sep 30, 2020 at 01:25:15AM +0200, Vladislav Shpilevoy wrote:
> On 29.09.2020 07:53, Alexander Turenko wrote:
> >>> +/**
> >>> + * Check if a value on @a L stack by index @a idx is an ibuf
> >>> + * object. Both 'struct ibuf' and 'struct ibuf *' are accepted.
> >>> + * Returns NULL, if can't convert - not an ibuf object.
> >>> + */
> >>> +struct ibuf *
> >>> +luaL_checkibuf(struct lua_State *L, int idx);
> >>
> >> 1. IMO 'check' is almost always worse than 'is'. Because you leave
> >> a user no choice but to use lua_cpcall if he does not want an
> >> exception. With 'is' he would be able to decide whether he wants to
> >> throw. The same for the method below.
> > 
> > I personally prefer *is* Lua functions. It is good to have *is* variants
> > in the public API aside of (or even instead of) *check* ones. I don't
> > insist, however.
> 
> This is what I said. *is* is better than *check*. More freedom to decide
> what to do, when it returns false.

I just agreed explicitly.

> 
> >> Also what exactly is the reason you need the ibuf checking for?
> >> Ibuf is not exposed as a part of tarantool binary. It is a part of
> >> small library. When we want to export parts of small from inside
> >> of the executable, we need to obfuscate the types and wrap the
> >> functions, because user's small library may be different from the
> >> executable's small.
> > 
> > It seems, we really should expose non-opacue <box_ibuf_t>: rpos and wpos
> > are used in merger (it would be too expensive to wrap them into calls).
> > The module accepts ibuf created from Lua (tarantool's <struct ibuf>), so
> > linking with external small is not an option (unless we'll really care
> > about ABI).
> 
> It seems you care about ABI a lot. So why do you stop now? If we don't care
> about ABI that much, then why do we worry about box_key_part_def_t?

'unless we'll really care about ABI' was regarding the small ABI. Sorry,
now I see that it is not obvious from my wording.

I'll repeat options as a list just in case:

1. Expose <box_ibuf_t> and <box_ibuf_*>() from the module API.
2. Ensure that we can hold small's ibuf API / ABI (from several sides:
   ability to extend it in future, static asserts, testing, no implicit
   dependency on a particular toolchain behaviour) and use it directly.

Both ways are okay for me. The second one is maybe better, but it is
more dangerous considering our current deadlines.

I would highlight that just 'let's use small directly, unlikely it will
be changed' does not work in the long terms. So the second way is not
cheap, IMHO. Many things should be considered before we'll able to state
that, say, ibuf API / ABI is perfectly stable.

> 
> I am ok with box_ibuf_t, of any kind. Then we could eventually legally expose
> tarantool_lua_ibuf, as at least for single alloc+free pairs it is better than
> region (a tiny tiny better).
> 
> > Maybe we can alias <box_ibuf_t> with <struct ibuf> if we'll add static
> > asserts around the structure size and offset of fields? And expose
> > box_ibuf_*(), of course.

I meant non-opaque structure and it seems I was not right. We cannot
alias <box_ibuf_t> to <struct ibuf> now and, when we'll change
<struct ibuf>, provide another <box_ibuf_t>, which will have the same
layout as the old <struct ibuf>. The reason is that 'real' <struct ibuf>
is already available via Lua ('buffer' module and via net.box). So if
we'll going this way (expose non-opaque <box_ibuf_t>), we should typedef
it to another structure right now and return
cdata<struct this_new_structure> from Lua. It seems, it would be good to
avoid this extra complexity of code and APIs.

So let's consider opaque <box_ibuf_t> approach. It'll add indirect calls
on hot paths (for each tuple reading from a buffer or writing to a
buffer), but I don't see any good way to overcome this. Ever increasing
of tuple's reference counter will be indirect call for an external
module.

So lot calls to do memcpy() and advance a pointer...

> 
> Also looks fine. However returning to the topic, it seems not related to the
> Lua type ids. You don't need ibuf definition to get its type ID in C. All
> is done in luaL_ctypeid function taking a string anyway.
> 
> 	luaL_ctypeid(L, "struct ibuf *");
> 	luaL_ctypeid(L, "struct ibuf");
> 
> So why does Timur need that method exported? luaL_checkibuf is a trival
> function, which can be implemented by luaL_iscdata() + getting ibuf type
> id using luaL_ctypeid() and comparing result with the cdata type id.

Timur does not need this method, he can implement it on the module side.

But since our built-in module 'buffer' provides cdata<struct ibuf> and
cdata<struct ibuf *>, it would be good to have Lua/C accessors in the
module API. It is simple, so there are no much questions for design, and
it looks worthful to expose it while we're here.

I would say the same for cdata<struct key_def &>, but since there is
another similar cdata type offered by the external key_def module, it
has no practical reason: we should handle both anyway.

Maybe uuid and decimal too, but I'm not sure. It is hard to estimate how
many times those functions will be used in modules (that are written by
us or by users). I would think about this like so: since we provide
ability to write modules using Lua/C and have cdata<struct foo> objects
in Lua API, it would be good to have luaT_isfoo() and luaT_pushfoo()
functions in the module API.