Tarantool development patches archive
 help / color / mirror / Atom feed
* [PATCH v5 0/3] box: functional indexes
@ 2019-07-26  9:43 Kirill Shcherbatov
  2019-07-26  9:43 ` [PATCH v5 1/3] box: introduce opts.is_multikey function option Kirill Shcherbatov
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Kirill Shcherbatov @ 2019-07-26  9:43 UTC (permalink / raw)
  To: tarantool-patches, vdavydov.dev, kostja; +Cc: Kirill Shcherbatov

This patchset introduces functional indexes in memtx.
Functional index is an index that use user-defined function to extract
a key by tuple.

In current implementation only persistent deterministic sandboxed Lua
function previously created with box.schema.func.create may be used
in functional index.
This provides a potential ability to support new languages to
transparently extend supported extractors in future (e.g. SQL extractor,
C extractor).

Because an _index space is loaded before _func space, functional index
initially created in "disabled state". Then, on loading from a new space
_func_index, it completely rebuilds.

http://github.com/tarantool/tarantool/tree/kshch/gh-1260-functional-index-new
https://github.com/tarantool/tarantool/issues/1260

Kirill Shcherbatov (3):
  box: introduce opts.is_multikey function option
  box: introduce tuple_chunk infrastructure
  box: introduce func_index

 src/box/alter.h                    |   1 +
 src/box/errcode.h                  |   3 +
 src/box/func_def.h                 |  22 +
 src/box/index.h                    |   9 +
 src/box/index_def.h                |  23 +
 src/box/key_def.h                  |  15 +-
 src/box/key_list.h                 | 105 +++++
 src/box/schema_def.h               |   9 +
 src/box/tuple.h                    |  34 ++
 src/box/tuple_format.h             |  15 +
 src/box/func_def.c                 |  19 +-
 src/box/index_def.c                |   7 +
 src/box/key_def.c                  |  47 +-
 src/box/key_list.c                 | 177 +++++++
 src/box/lua/call.c                 |   3 +
 src/box/lua/key_def.c              |   2 +-
 src/box/memtx_engine.c             |  32 ++
 src/box/memtx_space.c              |  18 +
 src/box/memtx_tree.c               | 331 ++++++++++++-
 src/box/sql.c                      |   2 +-
 src/box/sql/build.c                |   2 +-
 src/box/sql/select.c               |   2 +-
 src/box/sql/where.c                |   2 +-
 src/box/tuple.c                    |   2 +
 src/box/tuple_format.c             |   4 +
 src/box/vinyl.c                    |   9 +-
 src/box/vy_stmt.c                  |   2 +
 test/unit/luaT_tuple_new.c         |   2 +-
 test/unit/merger.test.c            |   4 +-
 src/box/CMakeLists.txt             |   1 +
 src/box/alter.cc                   |  97 +++-
 src/box/bootstrap.snap             | Bin 5863 -> 5914 bytes
 src/box/index.cc                   |  28 ++
 src/box/lua/schema.lua             |  36 +-
 src/box/lua/space.cc               |  20 +
 src/box/lua/upgrade.lua            |  18 +
 src/box/schema.cc                  |  12 +-
 src/box/tuple_compare.cc           | 117 ++++-
 src/box/tuple_extract_key.cc       |  29 +-
 src/box/tuple_hash.cc              |   1 +
 test/app-tap/tarantoolctl.test.lua |   4 +-
 test/box-py/bootstrap.result       |   5 +
 test/box/access.result             |   3 +
 test/box/access.test.lua           |   1 +
 test/box/access_misc.result        | 132 +++---
 test/box/access_sysview.result     |   6 +-
 test/box/alter.result              |   7 +-
 test/box/bitset.result             |  24 +
 test/box/bitset.test.lua           |   9 +
 test/box/function1.result          |  14 +
 test/box/function1.test.lua        |   5 +
 test/box/hash.result               |  24 +
 test/box/hash.test.lua             |   9 +
 test/box/misc.result               |   3 +
 test/box/rtree_misc.result         |  24 +
 test/box/rtree_misc.test.lua       |   9 +
 test/engine/engine.cfg             |   5 +-
 test/engine/func_index.result      | 734 +++++++++++++++++++++++++++++
 test/engine/func_index.test.lua    | 250 ++++++++++
 test/vinyl/misc.result             |  23 +
 test/vinyl/misc.test.lua           |   9 +
 test/wal_off/alter.result          |   2 +-
 62 files changed, 2453 insertions(+), 111 deletions(-)
 create mode 100644 src/box/key_list.h
 create mode 100644 src/box/key_list.c
 create mode 100644 test/engine/func_index.result
 create mode 100644 test/engine/func_index.test.lua

-- 
2.22.0

^ permalink raw reply	[flat|nested] 12+ messages in thread
* [PATCH v5 0/3] box: functional indexes
@ 2019-07-25 18:39 Kirill Shcherbatov
  2019-07-25 18:39 ` [PATCH v5 3/3] box: introduce func_index Kirill Shcherbatov
  0 siblings, 1 reply; 12+ messages in thread
From: Kirill Shcherbatov @ 2019-07-25 18:39 UTC (permalink / raw)
  To: tarantool-patches, kostja; +Cc: vdavydov.dev, v.shpilevoy, Kirill Shcherbatov

This patchset introduces functional indexes in memtx.
Functional index is an index that use user-defined function to extract
a key by tuple.

In current implementation only persistent deterministic sandboxed Lua
function previously created with box.schema.func.create may be used
in functional index.
This provides a potential ability to support new languages to
transparently extend supported extractors in future (e.g. SQL extractor,
C extractor).

Because an _index space is loaded before _func space, functional index
initially created in "disabled state". Then, on loading from a new space
_func_index, it completely rebuilds.

Comment:
  - I'll send a follow-up patch to specify function by name a bit later.

Changes in version 5:
  - better iterators, codestyle fixes

http://github.com/tarantool/tarantool/tree/kshch/gh-1260-functional-index-new
https://github.com/tarantool/tarantool/issues/1260

Kirill Shcherbatov (3):
  box: introduce opts.is_multikey function option
  box: introduce tuple_chunk infrastructure
  box: introduce func_index

 src/box/alter.h                    |   1 +
 src/box/errcode.h                  |   2 +
 src/box/func_def.h                 |  22 +
 src/box/index.h                    |   9 +
 src/box/index_def.h                |  16 +
 src/box/key_def.h                  |  15 +-
 src/box/key_list.h                 | 104 +++++
 src/box/schema_def.h               |   9 +
 src/box/tuple.h                    |  28 ++
 src/box/tuple_format.h             |   9 +
 src/box/func_def.c                 |  19 +-
 src/box/index_def.c                |   7 +
 src/box/key_def.c                  |  43 +-
 src/box/key_list.c                 | 171 +++++++
 src/box/lua/call.c                 |   3 +
 src/box/lua/key_def.c              |   2 +-
 src/box/memtx_engine.c             |  29 ++
 src/box/memtx_space.c              |  18 +
 src/box/memtx_tree.c               | 332 ++++++++++++-
 src/box/sql.c                      |   2 +-
 src/box/sql/build.c                |   2 +-
 src/box/sql/select.c               |   2 +-
 src/box/sql/where.c                |   2 +-
 src/box/tuple.c                    |   8 +
 src/box/tuple_format.c             |   4 +
 src/box/vinyl.c                    |   8 +-
 src/box/vy_stmt.c                  |   2 +
 test/unit/luaT_tuple_new.c         |   2 +-
 test/unit/merger.test.c            |   4 +-
 src/box/CMakeLists.txt             |   1 +
 src/box/alter.cc                   |  96 +++-
 src/box/bootstrap.snap             | Bin 5863 -> 5914 bytes
 src/box/index.cc                   |  28 ++
 src/box/lua/schema.lua             |  36 +-
 src/box/lua/space.cc               |  20 +
 src/box/lua/upgrade.lua            |  18 +
 src/box/schema.cc                  |  12 +-
 src/box/tuple_compare.cc           | 117 ++++-
 src/box/tuple_extract_key.cc       |  29 +-
 src/box/tuple_hash.cc              |   1 +
 test/app-tap/tarantoolctl.test.lua |   4 +-
 test/box-py/bootstrap.result       |   5 +
 test/box/access.result             |   3 +
 test/box/access.test.lua           |   1 +
 test/box/access_misc.result        | 132 +++---
 test/box/access_sysview.result     |   6 +-
 test/box/alter.result              |   7 +-
 test/box/bitset.result             |  24 +
 test/box/bitset.test.lua           |   9 +
 test/box/function1.result          |  14 +
 test/box/function1.test.lua        |   5 +
 test/box/hash.result               |  24 +
 test/box/hash.test.lua             |   9 +
 test/box/misc.result               |   2 +
 test/box/rtree_misc.result         |  24 +
 test/box/rtree_misc.test.lua       |   9 +
 test/engine/engine.cfg             |   5 +-
 test/engine/func_index.result      | 728 +++++++++++++++++++++++++++++
 test/engine/func_index.test.lua    | 250 ++++++++++
 test/vinyl/misc.result             |  23 +
 test/vinyl/misc.test.lua           |   9 +
 test/wal_off/alter.result          |   2 +-
 62 files changed, 2417 insertions(+), 111 deletions(-)
 create mode 100644 src/box/key_list.h
 create mode 100644 src/box/key_list.c
 create mode 100644 test/engine/func_index.result
 create mode 100644 test/engine/func_index.test.lua

-- 
2.22.0

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-07-26 19:00 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-26  9:43 [PATCH v5 0/3] box: functional indexes Kirill Shcherbatov
2019-07-26  9:43 ` [PATCH v5 1/3] box: introduce opts.is_multikey function option Kirill Shcherbatov
2019-07-26  9:43 ` [PATCH v5 2/3] box: introduce tuple_chunk infrastructure Kirill Shcherbatov
2019-07-26  9:43 ` [PATCH v5 3/3] box: introduce func_index Kirill Shcherbatov
2019-07-26 12:15   ` Vladimir Davydov
2019-07-26 13:58     ` [tarantool-patches] " Kirill Shcherbatov
2019-07-26 17:42 ` [tarantool-patches] [PATCH v5 0/3] box: functional indexes Kirill Yukhin
2019-07-26 19:00   ` [tarantool-patches] " Vladislav Shpilevoy
  -- strict thread matches above, loose matches on Subject: below --
2019-07-25 18:39 Kirill Shcherbatov
2019-07-25 18:39 ` [PATCH v5 3/3] box: introduce func_index Kirill Shcherbatov
2019-07-26  9:49   ` Vladimir Davydov
2019-07-26  9:57     ` Konstantin Osipov
2019-07-26 10:10       ` Vladimir Davydov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox