Tarantool development patches archive
 help / color / mirror / Atom feed
From: Alexander Turenko <alexander.turenko@tarantool.org>
To: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Alexander Turenko <alexander.turenko@tarantool.org>,
	tarantool-patches@freelists.org
Subject: [PATCH 0/3] Merger
Date: Sun, 16 Dec 2018 23:17:23 +0300	[thread overview]
Message-ID: <cover.1544989900.git.alexander.turenko@tarantool.org> (raw)

https://github.com/tarantool/tarantool/issues/3276
On top of 2.1: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger
On top of 1.10: https://github.com/tarantool/tarantool/tree/Totktonada/gh-3276-on-board-merger-1.10

The emails below consist content for the 2.1 branch.

This is adaptation of [shard's merger][1] to be the built-in tarantool
module. The main reason of have this module built-in is to create
key_defs with nullability and collation support (which is not availiable
in the external API). The second reason is vague plan to have some more
general map-reduce support built-in in the future.

Most of changes made to the original merger are splitted into the
following categories:

* Support buffer/table/iterator input/output to allow merger's output
  being input for an another merger.
* Change API to net.box-like one (:select(), :pairs()) and change index
  parts format to be the same as for local spaces and as for net.box
  spaces.
* Support 'chain' format for input / output buffers to handle the case
  when tuples are part of the wrapping array, see comment in merger.c re
  chaining mergers and the [real usage][2].
* Split merger and merger_iterator to allow yield during merge process
  and use the same merger instance in an another merge.
* Document, test and comment the code.

Consider usage information at top of merger.c file to get the idea what
the module is.

[1]: https://github.com/tarantool/shard/blob/180948e99148973e89f75f8e4784315e183e3fa2/shard/driver.c
[2]: https://github.com/tarantool/graphql/pull/258

Alexander Turenko (3):
  Add luaT_iscallable with support of cdata metatype
  Add module to ease using Lua iterators from C
  Add merger for tuple streams

 extra/exports                    |    1 +
 src/CMakeLists.txt               |    2 +
 src/lua/init.c                   |    5 +
 src/lua/lua_iterator.h           |  117 +++
 src/lua/merger.c                 | 1643 ++++++++++++++++++++++++++++++
 src/lua/merger.h                 |   39 +
 src/lua/merger.lua               |   19 +
 src/lua/utils.c                  |   43 +
 src/lua/utils.h                  |   10 +
 test/app-tap/merger.test.lua     |  693 +++++++++++++
 test/app-tap/module_api.c        |   10 +
 test/app-tap/module_api.test.lua |   85 +-
 test/app-tap/suite.ini           |    1 +
 13 files changed, 2666 insertions(+), 2 deletions(-)
 create mode 100644 src/lua/lua_iterator.h
 create mode 100644 src/lua/merger.c
 create mode 100644 src/lua/merger.h
 create mode 100644 src/lua/merger.lua
 create mode 100755 test/app-tap/merger.test.lua

-- 
2.19.2

             reply	other threads:[~2018-12-16 20:17 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-16 20:17 Alexander Turenko [this message]
2018-12-16 20:17 ` [PATCH 1/3] Add luaT_iscallable with support of cdata metatype Alexander Turenko
2018-12-26 18:35   ` Vladimir Davydov
2018-12-28  1:46     ` Alexander Turenko
2018-12-28  8:00       ` Vladimir Davydov
2018-12-16 20:17 ` [PATCH 2/3] Add module to ease using Lua iterators from C Alexander Turenko
2018-12-26 18:45   ` Vladimir Davydov
2018-12-31  6:43     ` Alexander Turenko
2018-12-16 20:17 ` [PATCH 3/3] Add merger for tuple streams Alexander Turenko
2018-12-26 20:11   ` Vladimir Davydov
2019-01-09 21:36     ` Alexander Turenko
2018-12-18 12:16 ` [PATCH 0/3] Merger Alexander Turenko
2019-03-22 14:24 ` [tarantool-patches] [PATCH 0/3] lua: add key_def lua module Kirill Shcherbatov
2019-03-22 16:20   ` Alexander Turenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1544989900.git.alexander.turenko@tarantool.org \
    --to=alexander.turenko@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --cc=vdavydov.dev@gmail.com \
    --subject='Re: [PATCH 0/3] Merger' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox