[Tarantool-discussions] [dev] Introduction of DOUBLE field type in Tarantool

[Mergen Imeev]

On Sat, Nov 16, 2019 at 08:17:16PM +0300, Konstantin Osipov wrote:
> * Mergen Imeev <imeevma at tarantool.org> [19/11/16 20:08]:
> > After some research, I found two implementation options for
> > DOUBLE, so this will only affect Lua:
> > 1) Encode the Lua table using the hints from tuple_format. This
> > method does not look very good, because if we got a tuple, we
> > still have to re-encode it.
> 
> I think it's good enough. Whoever supplies a tuple, have encoded
> it - and could have used a format for that.
> 
> 
> > 2) After we have copied or created a tuple, re-encode it again if
> > necessary.
> > 
> > The second way looks better to me. However, there is a problem:
> > what if we get a tuple from any other interface, where instead of
> > DOUBLE we get INTEGER/UNSIGNED? For example, in netbox, we can do
> > this using something like this:
> > 
> > require('net.box').connect(3301).space.s:insert{1}
> > 
> > This is definitely possible because, as I know, we cannot expect
> > that the client have DOUBLE field type. In general case, client
> > and server could have different sets of field types.
> > 
> > And here we can again take a look at the solution that encodes the
> > tuples in space_execute_dml(), since in this case we will
> > re-encode any received tuple where INTEGER/UNSIGNED was inserted
> > into the DOUBLE field. The disadvantage of the solution is quite
> > obvious - we must check any received tuple.
> 
> Please keep in mind to not have well-behaving users pay for the
> sloppy users who you want to support. A user can always use
> ffi.cast() to ffi double to work around Lua type mess, before
> encoding. You have given yourself a goal which is simply not worth
> solving IMO - and means should solving it should not slow down the
> case when nothing needs to be done.
> 
> -- 
> Konstantin Osipov, Moscow, Russia

After discussing with @kyukhin, @Korablev77, @Totktonada, @GeorgyK
and @gerold103, we decided that we would try to create two or
three implementations and look at them and their performance.
After that we will choose. The options I'm going to implement are:
1) Convert Lua tables to msgpack using hints from the space
format.
2) Re-encode the tuple into space_execute_dml().
3) Re-encode in the place where the tuple is decoded. Somewhere
deeper in BOX than space_execute_dml().

It might also be a good idea to take a look at the NUMBER
implementation in Lua. Or just create cdata that has the same
behavior as NUMBER but always converts as DOUBLE using a
serializer.