[PATCH 0/3] Merger

Alexander Turenko alexander.turenko at tarantool.org
Sun Dec 16 23:17:23 MSK 2018


https://github.com/tarantool/tarantool/issues/3276
On top of 2.1: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger
On top of 1.10: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger-1.10

The emails below consist content for the 2.1 branch.

This is adaptation of [shard's merger][1] to be the built-in tarantool
module. The main reason of have this module built-in is to create
key_defs with nullability and collation support (which is not availiable
in the external API). The second reason is vague plan to have some more
general map-reduce support built-in in the future.

Most of changes made to the original merger are splitted into the
following categories:

* Support buffer/table/iterator input/output to allow merger's output
  being input for an another merger.
* Change API to net.box-like one (:select(), :pairs()) and change index
  parts format to be the same as for local spaces and as for net.box
  spaces.
* Support 'chain' format for input / output buffers to handle the case
  when tuples are part of the wrapping array, see comment in merger.c re
  chaining mergers and the [real usage][2].
* Split merger and merger_iterator to allow yield during merge process
  and use the same merger instance in an another merge.
* Document, test and comment the code.

Consider usage information at top of merger.c file to get the idea what
the module is.

[1]: https://github.com/tarantool/shard/blob/180948e99148973e89f75f8e4784315e183e3fa2/shard/driver.c
[2]: https://github.com/tarantool/graphql/pull/258

Alexander Turenko (3):
  Add luaT_iscallable with support of cdata metatype
  Add module to ease using Lua iterators from C
  Add merger for tuple streams

 extra/exports                    |    1 +
 src/CMakeLists.txt               |    2 +
 src/lua/init.c                   |    5 +
 src/lua/lua_iterator.h           |  117 +++
 src/lua/merger.c                 | 1643 ++++++++++++++++++++++++++++++
 src/lua/merger.h                 |   39 +
 src/lua/merger.lua               |   19 +
 src/lua/utils.c                  |   43 +
 src/lua/utils.h                  |   10 +
 test/app-tap/merger.test.lua     |  693 +++++++++++++
 test/app-tap/module_api.c        |   10 +
 test/app-tap/module_api.test.lua |   85 +-
 test/app-tap/suite.ini           |    1 +
 13 files changed, 2666 insertions(+), 2 deletions(-)
 create mode 100644 src/lua/lua_iterator.h
 create mode 100644 src/lua/merger.c
 create mode 100644 src/lua/merger.h
 create mode 100644 src/lua/merger.lua
 create mode 100755 test/app-tap/merger.test.lua

-- 
2.19.2




More information about the Tarantool-patches mailing list