[Tarantool-patches] [PATCH v3 00/16] module api: extend for external key_def Lua module

[Alexander Turenko]

Updated.

Changes from v3 to the final version (I hope it is final):

- Fixed sorting of the box region functions in src/exports.h.
- Added forgotten API_EXPORT for box_region_truncate().
- Added a comment for luaT_tuple_encode_table() reference in the Lua registry.
- Added the <size_t *key_size_ptr> parameter to box_key_def_validate_key() and
  box_key_def_validate_full_key().
- Fixed API comments for key_def functions with mention of the diagnostics
  area.
- Set a memory error to the diagnostics area for box_region_*alloc() functions.

On Tue, Oct 13, 2020 at 02:23:07AM +0300, Alexander Turenko wrote:
> Vlad,
> 
> Thank you for the past thorough reviews and answering to a lot of my
> questions!
> 
> Can you look over the updated patchset once again?
> 
> Overview
> --------
> 
> The patchset exposes and adds new functions into the module API, which
> are necessary to implement the external tuple.keydef module. Aside of
> this, a couple of relevant bugs were fixed.
> 
> The extra commits for tarantool-1.10 are the following:
> 
>  - lua: adjust contract of luaT_tuple_new()
>  - collation: allow to find a collation by a name
> 
> I'll not flood the mailing list with the 1.10 version of the patchset
> (if someone will not ask me do to that). It mostly the same or similar
> as this patchset (however there are some code differences).
> 
> Previous versions:
> 
> [v1]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-September/019583.html
> [v2]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-October/020019.html
> 
> Changelog from v2 to v3
> -----------------------
> 
> In brief: the last three commits from v2 gains test cases, the new
> function box_key_def_validate_full_key() is added.
> 
> - Clarified how default values of <box_key_part_def_t> may be changed in a
>   future tarantool versions.
> - Added a test case for 'none' collation for box_key_def_dump_parts().
> - Added -g to build module API test to obtain backtraces from GDB.
> - Added box_key_def_merge() test.
> - Added basic box_key_def_extract_key() test.
> - Added basic box_key_def_validate_key() test.
> - Clarified box_key_def_validate_key() API comment.
> - Fixed some typos, made suggested style changes.
> - Added region_truncate() in case of encoding or OOM error inside
>   luaT_tuple_encode().
> - Added box_key_def_validate_full_key().
> - Clafiried the Lua stack manipulations in luaT_tuple_encode_on_lua_ibuf().
> 
> @ChangeLog
> 
> - Module API: Get rid of typedef redefinitions for compatibility with
>   C99 (gh-5313).
> - Lua: Fixed unhandled Lua error that may lead to memory leaks and
>   inconsistencies in `<space_object>:frommap()`,
>   `<key_def_object>:compare()`, `<merge_source>:select()` (gh-5382).
> - Module API: Exposed the box region, key_def and several other
>   functions in order to implement an external tuple.keydef module on top
>   of them (gh-5273).
> 
> Issues
> ------
> 
> https://github.com/tarantool/tarantool/issues/5313 ('module api: module
> API requires C11')
> https://github.com/tarantool/tarantool/issues/5382 ('luaT_tuple_new()
> raises a Lua error that leads to various problems')
> https://github.com/tarantool/tarantool/issues/5273 ('module api: expose
> everything that is needed for external key_def module')
> 
> Branches
> --------
> 
> https://github.com/tarantool/tarantool/tree/Totktonada/gh-5273-expand-module-api
> (top 16 commits)
> https://github.com/tarantool/tarantool/tree/Totktonada/gh-5273-expand-module-api-1.10
> (top 18 commits)
> 
> The module based on this API
> ----------------------------
> 
> https://github.com/Totktonada/key_def
> 
> The module repository will be moved to the tarantool organization soon,
> when the module API will come into the server. It'll be in
> https://github.com/tarantool/tuple-keydef
> 
> Alexander Turenko (16):
>   module api: get rid of typedef redefinitions
>   module api: expose box region
>   module api/lua: add luaL_iscdata() function
>   lua: factor out tuple encoding from luaT_tuple_new
>   lua: don't raise a Lua error from luaT_tuple_new()
>   module api/lua: add luaT_tuple_encode()
>   module api/lua: expose luaT_tuple_new()
>   module api/lua: add API_EXPORT to tuple functions
>   module api: add API_EXPORT to key_def functions
>   module api: add box_key_def_new_v2()
>   module api: add box_key_def_dump_parts()
>   module api: expose box_key_def_validate_tuple()
>   module api: expose box_key_def_merge()
>   module api: expose box_key_def_extract_key()
>   module api: add box_key_def_validate_key()
>   module api: add box_key_def_validate_full_key()
> 
>  src/CMakeLists.txt               |    2 +-
>  src/box/index.h                  |    5 +-
>  src/box/key_def.c                |  264 +++++
>  src/box/key_def.h                |  268 ++++-
>  src/box/lua/tuple.c              |  210 +++-
>  src/box/lua/tuple.h              |   46 +-
>  src/exports.h                    |   15 +
>  src/lib/core/fiber.c             |   24 +
>  src/lib/core/fiber.h             |   76 ++
>  src/lua/utils.c                  |    6 +
>  src/lua/utils.h                  |   20 +
>  test/app-tap/CMakeLists.txt      |    6 +
>  test/app-tap/module_api.c        | 1711 ++++++++++++++++++++++++++++++
>  test/app-tap/module_api.test.lua |   63 +-
>  test/unit/luaT_tuple_new.c       |   42 +-
>  test/unit/luaT_tuple_new.result  |    6 +-
>  16 files changed, 2705 insertions(+), 59 deletions(-)
> 
> -- 
> 2.25.0
>