[Tarantool-patches] [PATCH 00/14] RFC: module api: extend for external key_def Lua module

[Alexander Turenko]

A lot of code to review... but nobody will do it better than you, Vlad!
Please, look when your time will permit.

This patchset extends the module API with a set of functions (mainly
key_def and tuple related) in order to implement an external key_def Lua
module on top of them.

Timur will do the same for functions he need for the external merger
module (outside of this patchset, but on top of it).

I don't wrote necessary tests ATM, so the patchset is not ready to push
and marked as RFC (request for comments). However I did my best for all
other aspects and decided to send the patchset for review to collect a
feedback earlier. I'll work on tests while waiting for review. (I
promise I'll not ask to push it until all tests will be written.)

There is backport of this patchset to 1.10 (see the links below). It
adds two extra commits and reworks some of those fourteen ones. Please,
consider it as part of this work and take glance too (I can send it too
if there will be demand).

Regarding those two extra commits:

- d7c8a7888 collation: allow to find a collation by a name

  The alternative would be a function like [1], but direct call to the
  database from key_def_api.c looks strange for me. I think it is better
  to backport the hashmap by collation names: the patch is simple and
  proven by the time.

  A long time ago we agreed with Vladimir Davydov to backport this patch
  to 1.10 if we'll backport merger to 1.10. It didn't occur, but the
  situation is similar.

- 078239e37 refactoring: adjust contract of luaT_tuple_new()

  Here I'm in doubts.

  The alternative would be duplicate the code: rename old
  luaT_tuple_new() to somewhat like luaT_tuple_new_110() and add the new
  luaT_tuple_new() to expose it into the module API. I don't know, to
  be honest. The code is proven by time and the risk is minimal. I would
  prefer to avoid code duplication, but I'm not strict on this position
  here: I'll implement the alternative if there will be reasoned
  complains against the current variant.

What also worries me: I found that actual behaviour of luaT_tuple_new()
is different from what I expected in the past: it may raise an error
when serialization fails. I didn't expect this, so I'll look how much it
may affect key_def and merger modules (built-in and external ones). For
now I just described the actual contract in the API comments for
luaT_tuple_new() and luaT_tuple_encode().

Issues:

https://github.com/tarantool/tarantool/issues/5313 ('module api: module
API requires C11')
https://github.com/tarantool/tarantool/issues/5273 ('module api: expose
everything that is needed for external key_def module')

Branches:

https://github.com/tarantool/tarantool/tree/Totktonada/gh-5273-expand-module-api
(top 14 commits)
https://github.com/tarantool/tarantool/tree/Totktonada/gh-5273-expand-module-api-1.10
(top 16 commits)

The module based on this API:

https://github.com/Totktonada/key_def

The module repository will be moved to the tarantool organization when
it'll be ready (surely not before this patchset). Please, consider the
module as PoC (everything work, though; even on Mac OS).

References:

[1]: https://github.com/tarantool/tarantool/blob/b7c787152d8984523d0b8b4d5b08f79173c57115/src/lua/merger.c#L1686-L1708

Alexander Turenko (14):
  module api: get rid of typedef redefinitions
  WIP: module api: expose box region
  WIP: module api/lua: add luaL_iscdata() function
  WIP: module api/lua: expose luaT_tuple_new()
  WIP: module api/lua: add luaT_tuple_encode()
  WIP: refactoring: add API_EXPORT to lua/tuple functions
  WIP: refactoring: add API_EXPORT to key_def functions
  WIP: refactoring: extract key_def module API functions
  WIP: module api: add box_key_def_new_ex()
  WIP: module api: add box_key_def_dump_parts()
  WIP: module api: expose box_tuple_validate_key_parts()
  WIP: module api: expose box_key_def_merge()
  WIP: module api: expose box_tuple_extract_key_ex()
  WIP: module api: add box_key_def_validate_key()

 src/CMakeLists.txt              |   7 +-
 src/box/CMakeLists.txt          |   1 +
 src/box/field_map.h             |   4 +
 src/box/index.h                 |   5 +-
 src/box/key_def.c               |  62 ++----
 src/box/key_def.h               |  85 +++------
 src/box/key_def_api.c           | 323 ++++++++++++++++++++++++++++++++
 src/box/key_def_api.h           | 295 +++++++++++++++++++++++++++++
 src/box/lua/tuple.c             |  21 ++-
 src/box/lua/tuple.h             |  56 +++++-
 src/exports.h                   |  14 ++
 src/lib/core/fiber.c            |  24 +++
 src/lib/core/fiber.h            | 101 ++++++++++
 src/lua/utils.c                 |   6 +
 src/lua/utils.h                 |  20 ++
 test/app-tap/CMakeLists.txt     |   6 +
 test/app-tap/module_api.c       |   2 +
 test/unit/vy_iterators_helper.c |   1 +
 test/unit/vy_mem.c              |   1 +
 test/unit/vy_point_lookup.c     |   1 +
 test/unit/vy_write_iterator.c   |   1 +
 21 files changed, 915 insertions(+), 121 deletions(-)
 create mode 100644 src/box/key_def_api.c
 create mode 100644 src/box/key_def_api.h

-- 
2.25.0