From: Alexander Turenko <alexander.turenko@tarantool.org> To: Vladimir Davydov <vdavydov.dev@gmail.com> Cc: Alexander Turenko <alexander.turenko@tarantool.org>, tarantool-patches@freelists.org Subject: [PATCH 0/3] Merger Date: Sun, 16 Dec 2018 23:17:23 +0300 [thread overview] Message-ID: <cover.1544989900.git.alexander.turenko@tarantool.org> (raw) https://github.com/tarantool/tarantool/issues/3276 On top of 2.1: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger On top of 1.10: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger-1.10 The emails below consist content for the 2.1 branch. This is adaptation of [shard's merger][1] to be the built-in tarantool module. The main reason of have this module built-in is to create key_defs with nullability and collation support (which is not availiable in the external API). The second reason is vague plan to have some more general map-reduce support built-in in the future. Most of changes made to the original merger are splitted into the following categories: * Support buffer/table/iterator input/output to allow merger's output being input for an another merger. * Change API to net.box-like one (:select(), :pairs()) and change index parts format to be the same as for local spaces and as for net.box spaces. * Support 'chain' format for input / output buffers to handle the case when tuples are part of the wrapping array, see comment in merger.c re chaining mergers and the [real usage][2]. * Split merger and merger_iterator to allow yield during merge process and use the same merger instance in an another merge. * Document, test and comment the code. Consider usage information at top of merger.c file to get the idea what the module is. [1]: https://github.com/tarantool/shard/blob/180948e99148973e89f75f8e4784315e183e3fa2/shard/driver.c [2]: https://github.com/tarantool/graphql/pull/258 Alexander Turenko (3): Add luaT_iscallable with support of cdata metatype Add module to ease using Lua iterators from C Add merger for tuple streams extra/exports | 1 + src/CMakeLists.txt | 2 + src/lua/init.c | 5 + src/lua/lua_iterator.h | 117 +++ src/lua/merger.c | 1643 ++++++++++++++++++++++++++++++ src/lua/merger.h | 39 + src/lua/merger.lua | 19 + src/lua/utils.c | 43 + src/lua/utils.h | 10 + test/app-tap/merger.test.lua | 693 +++++++++++++ test/app-tap/module_api.c | 10 + test/app-tap/module_api.test.lua | 85 +- test/app-tap/suite.ini | 1 + 13 files changed, 2666 insertions(+), 2 deletions(-) create mode 100644 src/lua/lua_iterator.h create mode 100644 src/lua/merger.c create mode 100644 src/lua/merger.h create mode 100644 src/lua/merger.lua create mode 100755 test/app-tap/merger.test.lua -- 2.19.2
next reply other threads:[~2018-12-16 20:17 UTC|newest] Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-12-16 20:17 Alexander Turenko [this message] 2018-12-16 20:17 ` [PATCH 1/3] Add luaT_iscallable with support of cdata metatype Alexander Turenko 2018-12-26 18:35 ` Vladimir Davydov 2018-12-28 1:46 ` Alexander Turenko 2018-12-28 8:00 ` Vladimir Davydov 2018-12-16 20:17 ` [PATCH 2/3] Add module to ease using Lua iterators from C Alexander Turenko 2018-12-26 18:45 ` Vladimir Davydov 2018-12-31 6:43 ` Alexander Turenko 2018-12-16 20:17 ` [PATCH 3/3] Add merger for tuple streams Alexander Turenko 2018-12-26 20:11 ` Vladimir Davydov 2019-01-09 21:36 ` Alexander Turenko 2018-12-18 12:16 ` [PATCH 0/3] Merger Alexander Turenko 2019-03-22 14:24 ` [tarantool-patches] [PATCH 0/3] lua: add key_def lua module Kirill Shcherbatov 2019-03-22 16:20 ` Alexander Turenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cover.1544989900.git.alexander.turenko@tarantool.org \ --to=alexander.turenko@tarantool.org \ --cc=tarantool-patches@freelists.org \ --cc=vdavydov.dev@gmail.com \ --subject='Re: [PATCH 0/3] Merger' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox