Re: [PATCH v2 2/6] Add functions to ease using Lua iterators from C

[Vladimir Davydov <vdavydov.dev@gmail.com>]

On Wed, Jan 16, 2019 at 02:40:12PM +0300, Alexander Turenko wrote:
> > > > > +
> > > > > +/**
> > > > > + * Create a Lua iterator from {gen, param, state}.
> > > > 
> > > > May be, we could pass idx == 0 to create an iterator from
> > > > gen, param, state (without a table)? Would it be worthwhile?
> > > > 
> > > 
> > > I think it is good idea, because cases could be different. My thought
> > > was that we'll add another function for this case (if we'll need), but
> > > using idx == 0 is better. I created the similar API before for
> > > luaT_newtuple().
> > > 
> > > And I think the new merger API will require it.
> > 
> > The patch looks good to me now, but I'm still not sure the new merger
> > implementation will need to deal with Lua iterators in C at all, as one
> > can easily write a wrapper function in Lua turning an iterator to a
> > 'fetch' closure, which then can be passed to a source constructor.
> 
> This is not the thread about merger, but the idea looks weird for me.
> 'next' has one meaning (get one tuple), 'fetch' has another meaning (get
> next tuples batch). 'next' is written in C and predefined, 'fetch' is
> user-defined Lua function. When you'll try to express one over another
> you'll find rough edges, e. g. need of extra wrapping table.

Yep, why not wrap an iterator/table/whatever so that it works as fetch()
closure taken by one of available source constructors.

> 
> But my main objection is that it is hard to think what is going on
> when 'next' and 'fetch' are mixed.

I never mixed those.

'next' would be a C function returning C tuples. This would be a method
of a source. It would be used by a merger to get the next tuple to
merge. It would also be used by source:pairs() Lua function.

'fetch' would be a Lua function returning either Lua tuples or tuple
batches (represented by Lua tables or raw msgpack). What exactly it
returns depends on the source type.