From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Alexander Turenko Subject: [PATCH 0/3] Merger Date: Sun, 16 Dec 2018 23:17:23 +0300 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit To: Vladimir Davydov Cc: Alexander Turenko , tarantool-patches@freelists.org List-ID: https://github.com/tarantool/tarantool/issues/3276 On top of 2.1: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger On top of 1.10: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger-1.10 The emails below consist content for the 2.1 branch. This is adaptation of [shard's merger][1] to be the built-in tarantool module. The main reason of have this module built-in is to create key_defs with nullability and collation support (which is not availiable in the external API). The second reason is vague plan to have some more general map-reduce support built-in in the future. Most of changes made to the original merger are splitted into the following categories: * Support buffer/table/iterator input/output to allow merger's output being input for an another merger. * Change API to net.box-like one (:select(), :pairs()) and change index parts format to be the same as for local spaces and as for net.box spaces. * Support 'chain' format for input / output buffers to handle the case when tuples are part of the wrapping array, see comment in merger.c re chaining mergers and the [real usage][2]. * Split merger and merger_iterator to allow yield during merge process and use the same merger instance in an another merge. * Document, test and comment the code. Consider usage information at top of merger.c file to get the idea what the module is. [1]: https://github.com/tarantool/shard/blob/180948e99148973e89f75f8e4784315e183e3fa2/shard/driver.c [2]: https://github.com/tarantool/graphql/pull/258 Alexander Turenko (3): Add luaT_iscallable with support of cdata metatype Add module to ease using Lua iterators from C Add merger for tuple streams extra/exports | 1 + src/CMakeLists.txt | 2 + src/lua/init.c | 5 + src/lua/lua_iterator.h | 117 +++ src/lua/merger.c | 1643 ++++++++++++++++++++++++++++++ src/lua/merger.h | 39 + src/lua/merger.lua | 19 + src/lua/utils.c | 43 + src/lua/utils.h | 10 + test/app-tap/merger.test.lua | 693 +++++++++++++ test/app-tap/module_api.c | 10 + test/app-tap/module_api.test.lua | 85 +- test/app-tap/suite.ini | 1 + 13 files changed, 2666 insertions(+), 2 deletions(-) create mode 100644 src/lua/lua_iterator.h create mode 100644 src/lua/merger.c create mode 100644 src/lua/merger.h create mode 100644 src/lua/merger.lua create mode 100755 test/app-tap/merger.test.lua -- 2.19.2