* [PATCH v3 0/4] box: functional indexes
@ 2019-07-17 1:20 Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
` (3 more replies)
0 siblings, 4 replies; 15+ messages in thread
From: Kirill Shcherbatov @ 2019-07-17 1:20 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev; +Cc: kostja, Kirill Shcherbatov
This patchset introduces functional indexes in memtx.
Functional index is an index that use user-defined function to extract
key by processed tuple.
In current implementation only persistent deterministic sandboxed Lua function
previously created with box.schema.func.create may be used in functional index.
This provides a potential ability to support new languages to transparently
extend supported extractors in future (e.g. SQL extractor, C extractor).
Because an _index space is loaded before _func space, functional index initially
created in "disabled state". Then, on loading from a new space _func_index, it
completely rebuilds.
Changes in version 2:
- new space _func_index
- new, new enhanced object model for functional indexes infrastructure
Branch: http://github.com/tarantool/tarantool/tree/kshch/gh-1260-functional-index-new
Issue: https://github.com/tarantool/tarantool/issues/1260
Kirill Shcherbatov (4):
box: introduce key_def->is_multikey flag
box: refactor key_validate_parts to return key_end
box: introduce tuple_extra infrastructure
box: introduce functional indexes in memtx
src/box/CMakeLists.txt | 4 +-
src/box/alter.cc | 134 +++++-
src/box/alter.h | 1 +
src/box/bootstrap.snap | Bin 5863 -> 5914 bytes
src/box/box.cc | 4 +-
src/box/call.c | 8 +-
src/box/errcode.h | 1 +
src/box/functional_extractor.c | 118 +++++
src/box/functional_extractor.h | 101 ++++
src/box/functional_key.c | 228 ++++++++++
src/box/functional_key.h | 94 ++++
src/box/index.cc | 34 +-
src/box/index.h | 38 +-
src/box/index_def.c | 11 +-
src/box/index_def.h | 9 +-
src/box/key_def.c | 39 +-
src/box/key_def.h | 42 +-
src/box/lua/key_def.c | 7 +-
src/box/lua/misc.cc | 2 +-
src/box/lua/schema.lua | 20 +
src/box/lua/space.cc | 15 +
src/box/lua/upgrade.lua | 18 +
src/box/memtx_bitset.c | 2 +-
src/box/memtx_engine.c | 36 +-
src/box/memtx_rtree.c | 2 +-
src/box/memtx_space.c | 24 +-
src/box/memtx_tree.c | 43 +-
src/box/port.h | 4 +-
src/box/schema.cc | 13 +-
src/box/schema_def.h | 9 +
src/box/sql.c | 3 +-
src/box/sql/build.c | 2 +-
src/box/sql/select.c | 3 +-
src/box/sql/where.c | 2 +-
src/box/tuple.c | 35 +-
src/box/tuple.h | 103 ++++-
src/box/tuple_bloom.c | 4 +-
src/box/tuple_compare.cc | 228 +++++++---
src/box/tuple_extract_key.cc | 21 +-
src/box/tuple_format.c | 58 ++-
src/box/tuple_format.h | 32 ++
src/box/tuple_hash.cc | 7 +-
src/box/vinyl.c | 20 +-
src/box/vy_stmt.c | 2 +
src/box/vy_stmt.h | 6 +-
test/app-tap/tarantoolctl.test.lua | 4 +-
test/box-py/bootstrap.result | 5 +
test/box/access.result | 3 +
test/box/access.test.lua | 1 +
test/box/access_misc.result | 132 +++---
test/box/access_sysview.result | 6 +-
test/box/alter.result | 7 +-
test/box/bitset.result | 24 +
test/box/bitset.test.lua | 9 +
test/box/hash.result | 24 +
test/box/hash.test.lua | 9 +
test/box/misc.result | 1 +
test/box/rtree_misc.result | 24 +
test/box/rtree_misc.test.lua | 9 +
test/engine/engine.cfg | 5 +-
test/engine/functional.result | 708 +++++++++++++++++++++++++++++
test/engine/functional.test.lua | 240 ++++++++++
test/unit/luaT_tuple_new.c | 2 +-
test/unit/merger.test.c | 8 +-
test/unit/tuple_bigref.c | 2 +-
test/unit/vy_iterators_helper.c | 2 +-
test/vinyl/misc.result | 23 +
test/vinyl/misc.test.lua | 9 +
test/wal_off/alter.result | 2 +-
69 files changed, 2619 insertions(+), 227 deletions(-)
create mode 100644 src/box/functional_extractor.c
create mode 100644 src/box/functional_extractor.h
create mode 100644 src/box/functional_key.c
create mode 100644 src/box/functional_key.h
create mode 100644 test/engine/functional.result
create mode 100644 test/engine/functional.test.lua
--
2.22.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 1/4] box: introduce key_def->is_multikey flag
2019-07-17 1:20 [PATCH v3 0/4] box: functional indexes Kirill Shcherbatov
@ 2019-07-17 1:20 ` Kirill Shcherbatov
2019-07-17 14:34 ` Konstantin Osipov
2019-07-18 11:20 ` [tarantool-patches] " Kirill Yukhin
2019-07-17 1:20 ` [PATCH v3 2/4] box: refactor key_validate_parts to return key_end Kirill Shcherbatov
` (2 subsequent siblings)
3 siblings, 2 replies; 15+ messages in thread
From: Kirill Shcherbatov @ 2019-07-17 1:20 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev; +Cc: kostja, Kirill Shcherbatov
Previously only key definitions that have JSON paths were able
to define multikey index. We used to check multikey_path != NULL
test to determine whether given key definition is multikey.
In further patches with functional indexes this rule becomes
outdated. Functional index extracted key definition may be
multikey, but has no JSON paths.
So an explicit is_multikey flag was introduced.
Needed for #1260
---
src/box/index_def.c | 2 +-
src/box/key_def.c | 2 ++
src/box/key_def.h | 12 ++++--------
src/box/memtx_bitset.c | 2 +-
src/box/memtx_engine.c | 3 +--
src/box/memtx_rtree.c | 2 +-
src/box/memtx_space.c | 6 +++---
src/box/memtx_tree.c | 4 ++--
src/box/tuple.c | 2 +-
src/box/tuple_bloom.c | 4 ++--
src/box/tuple_compare.cc | 22 +++++++++++-----------
| 12 ++++++------
src/box/tuple_hash.cc | 6 +++---
src/box/vinyl.c | 5 ++---
src/box/vy_stmt.h | 6 +++---
15 files changed, 43 insertions(+), 47 deletions(-)
diff --git a/src/box/index_def.c b/src/box/index_def.c
index 28de89274..eb309a30c 100644
--- a/src/box/index_def.c
+++ b/src/box/index_def.c
@@ -291,7 +291,7 @@ index_def_is_valid(struct index_def *index_def, const char *space_name)
space_name, "too many key parts");
return false;
}
- if (index_def->iid == 0 && key_def_is_multikey(index_def->key_def)) {
+ if (index_def->iid == 0 && index_def->key_def->is_multikey) {
diag_set(ClientError, ER_MODIFY_INDEX, index_def->name,
space_name, "primary key cannot be multikey");
return false;
diff --git a/src/box/key_def.c b/src/box/key_def.c
index eebfb7fe4..2aa095091 100644
--- a/src/box/key_def.c
+++ b/src/box/key_def.c
@@ -181,6 +181,7 @@ key_def_set_part_path(struct key_def *def, uint32_t part_no, const char *path,
def->multikey_path = part->path;
def->multikey_fieldno = part->fieldno;
def->multikey_path_len = (uint32_t) multikey_path_len;
+ def->is_multikey = true;
} else if (def->multikey_fieldno != part->fieldno ||
json_path_cmp(path, multikey_path_len, def->multikey_path,
def->multikey_path_len,
@@ -752,6 +753,7 @@ key_def_merge(const struct key_def *first, const struct key_def *second)
new_def->is_nullable = first->is_nullable || second->is_nullable;
new_def->has_optional_parts = first->has_optional_parts ||
second->has_optional_parts;
+ new_def->is_multikey = first->is_multikey || second->is_multikey;
/* JSON paths data in the new key_def. */
char *path_pool = (char *)new_def + key_def_sizeof(new_part_count, 0);
diff --git a/src/box/key_def.h b/src/box/key_def.h
index f4a1a8fd1..ab4b7c087 100644
--- a/src/box/key_def.h
+++ b/src/box/key_def.h
@@ -196,6 +196,8 @@ struct key_def {
bool is_nullable;
/** True if some key part has JSON path. */
bool has_json_paths;
+ /** True if it is multikey key definition. */
+ bool is_multikey;
/**
* True, if some key parts can be absent in a tuple. These
* fields assumed to be MP_NIL.
@@ -220,14 +222,14 @@ struct key_def {
const char *multikey_path;
/**
* The length of the key_def::multikey_path.
- * Valid when key_def_is_multikey(key_def) is true,
+ * Valid when key_def->is_multikey is true,
* undefined otherwise.
*/
uint32_t multikey_path_len;
/**
* The index of the root field of the multikey JSON
* path index key_def::multikey_path.
- * Valid when key_def_is_multikey(key_def) is true,
+ * Valid when key_def->is_multikey is true,
* undefined otherwise.
*/
uint32_t multikey_fieldno;
@@ -469,12 +471,6 @@ key_def_is_sequential(const struct key_def *key_def)
return true;
}
-static inline bool
-key_def_is_multikey(const struct key_def *key_def)
-{
- return key_def->multikey_path != NULL;
-}
-
/**
* Return true if @a key_def defines has fields that requires
* special collation comparison.
diff --git a/src/box/memtx_bitset.c b/src/box/memtx_bitset.c
index 8c626684c..59bc96427 100644
--- a/src/box/memtx_bitset.c
+++ b/src/box/memtx_bitset.c
@@ -276,7 +276,7 @@ memtx_bitset_index_replace(struct index *base, struct tuple *old_tuple,
struct memtx_bitset_index *index = (struct memtx_bitset_index *)base;
assert(!base->def->opts.is_unique);
- assert(!key_def_is_multikey(base->def->key_def));
+ assert(!base->def->key_def->is_multikey);
assert(old_tuple != NULL || new_tuple != NULL);
(void) mode;
diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c
index 869cd343a..428491c2d 100644
--- a/src/box/memtx_engine.c
+++ b/src/box/memtx_engine.c
@@ -1251,7 +1251,6 @@ memtx_index_def_change_requires_rebuild(struct index *index,
TUPLE_INDEX_BASE) != 0)
return true;
}
- assert(key_def_is_multikey(old_cmp_def) ==
- key_def_is_multikey(new_cmp_def));
+ assert(old_cmp_def->is_multikey == new_cmp_def->is_multikey);
return false;
}
diff --git a/src/box/memtx_rtree.c b/src/box/memtx_rtree.c
index b91a405d7..8badad797 100644
--- a/src/box/memtx_rtree.c
+++ b/src/box/memtx_rtree.c
@@ -120,7 +120,7 @@ extract_rectangle(struct rtree_rect *rect, struct tuple *tuple,
struct index_def *index_def)
{
assert(index_def->key_def->part_count == 1);
- assert(!key_def_is_multikey(index_def->key_def));
+ assert(!index_def->key_def->is_multikey);
const char *elems = tuple_field_by_part(tuple,
index_def->key_def->parts, MULTIKEY_NONE);
unsigned dimension = index_def->opts.dimension;
diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c
index 09277b029..a8e7b8080 100644
--- a/src/box/memtx_space.c
+++ b/src/box/memtx_space.c
@@ -653,7 +653,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"HASH index must be unique");
return -1;
}
- if (key_def_is_multikey(index_def->key_def)) {
+ if (index_def->key_def->is_multikey) {
diag_set(ClientError, ER_MODIFY_INDEX,
index_def->name, space_name(space),
"HASH index cannot be multikey");
@@ -682,7 +682,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"RTREE index field type must be ARRAY");
return -1;
}
- if (key_def_is_multikey(index_def->key_def)) {
+ if (index_def->key_def->is_multikey) {
diag_set(ClientError, ER_MODIFY_INDEX,
index_def->name, space_name(space),
"RTREE index cannot be multikey");
@@ -710,7 +710,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"BITSET index field type must be NUM or STR");
return -1;
}
- if (key_def_is_multikey(index_def->key_def)) {
+ if (index_def->key_def->is_multikey) {
diag_set(ClientError, ER_MODIFY_INDEX,
index_def->name, space_name(space),
"BITSET index cannot be multikey");
diff --git a/src/box/memtx_tree.c b/src/box/memtx_tree.c
index 268ce4050..3edf94044 100644
--- a/src/box/memtx_tree.c
+++ b/src/box/memtx_tree.c
@@ -882,7 +882,7 @@ memtx_tree_index_end_build(struct index *base)
struct key_def *cmp_def = memtx_tree_cmp_def(&index->tree);
qsort_arg(index->build_array, index->build_array_size,
sizeof(index->build_array[0]), memtx_tree_qcompare, cmp_def);
- if (key_def_is_multikey(cmp_def)) {
+ if (cmp_def->is_multikey) {
/*
* Multikey index may have equal(in terms of
* cmp_def) keys inserted by different multikey
@@ -1027,7 +1027,7 @@ memtx_tree_index_new(struct memtx_engine *memtx, struct index_def *def)
"malloc", "struct memtx_tree_index");
return NULL;
}
- const struct index_vtab *vtab = key_def_is_multikey(def->key_def) ?
+ const struct index_vtab *vtab = def->key_def->is_multikey ?
&memtx_tree_index_multikey_vtab :
&memtx_tree_index_vtab;
if (index_create(&index->base, (struct engine *)memtx,
diff --git a/src/box/tuple.c b/src/box/tuple.c
index a7ef332b2..c0e94d55b 100644
--- a/src/box/tuple.c
+++ b/src/box/tuple.c
@@ -553,7 +553,7 @@ tuple_raw_multikey_count(struct tuple_format *format, const char *data,
const uint32_t *field_map,
struct key_def *key_def)
{
- assert(key_def_is_multikey(key_def));
+ assert(key_def->is_multikey);
const char *array_raw =
tuple_field_raw_by_path(format, data, field_map,
key_def->multikey_fieldno,
diff --git a/src/box/tuple_bloom.c b/src/box/tuple_bloom.c
index 8b74da22c..420a7c60e 100644
--- a/src/box/tuple_bloom.c
+++ b/src/box/tuple_bloom.c
@@ -109,7 +109,7 @@ tuple_bloom_builder_add(struct tuple_bloom_builder *builder,
int multikey_idx)
{
assert(builder->part_count == key_def->part_count);
- assert(!key_def_is_multikey(key_def) || multikey_idx != MULTIKEY_NONE);
+ assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
uint32_t h = HASH_SEED;
uint32_t carry = 0;
@@ -204,7 +204,7 @@ bool
tuple_bloom_maybe_has(const struct tuple_bloom *bloom, struct tuple *tuple,
struct key_def *key_def, int multikey_idx)
{
- assert(!key_def_is_multikey(key_def) || multikey_idx != MULTIKEY_NONE);
+ assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
if (bloom->is_legacy) {
return bloom_maybe_has(&bloom->parts[0],
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index 95a0f58c9..c03b584d0 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -460,9 +460,9 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
assert(!has_optional_parts || is_nullable);
assert(is_nullable == key_def->is_nullable);
assert(has_optional_parts == key_def->has_optional_parts);
- assert(key_def_is_multikey(key_def) == is_multikey);
- assert(!is_multikey || (is_multikey == has_json_paths &&
- tuple_a_hint != HINT_NONE && tuple_b_hint != HINT_NONE));
+ assert(key_def->is_multikey == is_multikey);
+ assert(!is_multikey || (tuple_a_hint != HINT_NONE &&
+ tuple_b_hint != HINT_NONE));
int rc = 0;
if (!is_multikey && (rc = hint_cmp(tuple_a_hint, tuple_b_hint)) != 0)
return rc;
@@ -619,9 +619,9 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
assert(has_optional_parts == key_def->has_optional_parts);
assert(key != NULL || part_count == 0);
assert(part_count <= key_def->part_count);
- assert(key_def_is_multikey(key_def) == is_multikey);
- assert(!is_multikey || (is_multikey == has_json_paths &&
- tuple_hint != HINT_NONE && key_hint == HINT_NONE));
+ assert(key_def->is_multikey == is_multikey);
+ assert(!is_multikey || (tuple_hint != HINT_NONE &&
+ key_hint == HINT_NONE));
int rc = 0;
if (!is_multikey && (rc = hint_cmp(tuple_hint, key_hint)) != 0)
return rc;
@@ -1573,7 +1573,7 @@ template <enum field_type type, bool is_nullable>
static hint_t
key_hint(const char *key, uint32_t part_count, struct key_def *key_def)
{
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
if (part_count == 0)
return HINT_NONE;
return field_hint<type, is_nullable>(key, key_def->parts->coll);
@@ -1583,7 +1583,7 @@ template <enum field_type type, bool is_nullable>
static hint_t
tuple_hint(struct tuple *tuple, struct key_def *key_def)
{
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
const char *field = tuple_field_by_part(tuple, key_def->parts,
MULTIKEY_NONE);
if (is_nullable && field == NULL)
@@ -1607,7 +1607,7 @@ key_hint_multikey(const char *key, uint32_t part_count, struct key_def *key_def)
* do nothing on key hint calculation an it is valid
* because it is never used(unlike tuple hint).
*/
- assert(key_def_is_multikey(key_def));
+ assert(key_def->is_multikey);
return HINT_NONE;
}
@@ -1641,7 +1641,7 @@ key_def_set_hint_func(struct key_def *def)
static void
key_def_set_hint_func(struct key_def *def)
{
- if (key_def_is_multikey(def)) {
+ if (def->is_multikey) {
def->key_hint = key_hint_multikey;
def->tuple_hint = tuple_hint_multikey;
return;
@@ -1756,7 +1756,7 @@ static void
key_def_set_compare_func_json(struct key_def *def)
{
assert(def->has_json_paths);
- if (key_def_is_multikey(def)) {
+ if (def->is_multikey) {
def->tuple_compare = tuple_compare_slowpath
<is_nullable, has_optional_parts, true, true>;
def->tuple_compare_with_key = tuple_compare_with_key_slowpath
--git a/src/box/tuple_extract_key.cc b/src/box/tuple_extract_key.cc
index 3bd3cde70..471c7df80 100644
--- a/src/box/tuple_extract_key.cc
+++ b/src/box/tuple_extract_key.cc
@@ -118,8 +118,8 @@ tuple_extract_key_slowpath(struct tuple *tuple, struct key_def *key_def,
assert(has_optional_parts == key_def->has_optional_parts);
assert(contains_sequential_parts ==
key_def_contains_sequential_parts(key_def));
- assert(is_multikey == key_def_is_multikey(key_def));
- assert(!key_def_is_multikey(key_def) || multikey_idx != MULTIKEY_NONE);
+ assert(is_multikey == key_def->is_multikey);
+ assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
assert(mp_sizeof_nil() == 1);
const char *data = tuple_data(tuple);
uint32_t part_count = key_def->part_count;
@@ -250,7 +250,7 @@ tuple_extract_key_slowpath_raw(const char *data, const char *data_end,
assert(has_json_paths == key_def->has_json_paths);
assert(!has_optional_parts || key_def->is_nullable);
assert(has_optional_parts == key_def->has_optional_parts);
- assert(!key_def_is_multikey(key_def) || multikey_idx != MULTIKEY_NONE);
+ assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
assert(mp_sizeof_nil() == 1);
/* allocate buffer with maximal possible size */
char *key = (char *) region_alloc(&fiber()->gc, data_end - data);
@@ -366,7 +366,7 @@ static void
key_def_set_extract_func_plain(struct key_def *def)
{
assert(!def->has_json_paths);
- assert(!key_def_is_multikey(def));
+ assert(!def->is_multikey);
if (key_def_is_sequential(def)) {
assert(contains_sequential_parts || def->part_count == 1);
def->tuple_extract_key = tuple_extract_key_sequential
@@ -387,7 +387,7 @@ static void
key_def_set_extract_func_json(struct key_def *def)
{
assert(def->has_json_paths);
- if (key_def_is_multikey(def)) {
+ if (def->is_multikey) {
def->tuple_extract_key = tuple_extract_key_slowpath
<contains_sequential_parts,
has_optional_parts, true, true>;
@@ -452,7 +452,7 @@ tuple_key_contains_null(struct tuple *tuple, struct key_def *def,
int
tuple_validate_key_parts(struct key_def *key_def, struct tuple *tuple)
{
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
for (uint32_t idx = 0; idx < key_def->part_count; idx++) {
struct key_part *part = &key_def->parts[idx];
const char *field = tuple_field_by_part(tuple, part,
diff --git a/src/box/tuple_hash.cc b/src/box/tuple_hash.cc
index 223c5f6f4..780e3d053 100644
--- a/src/box/tuple_hash.cc
+++ b/src/box/tuple_hash.cc
@@ -155,7 +155,7 @@ struct TupleHash
{
static uint32_t hash(struct tuple *tuple, struct key_def *key_def)
{
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
uint32_t h = HASH_SEED;
uint32_t carry = 0;
uint32_t total_size = 0;
@@ -172,7 +172,7 @@ template <>
struct TupleHash<FIELD_TYPE_UNSIGNED> {
static uint32_t hash(struct tuple *tuple, struct key_def *key_def)
{
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
const char *field = tuple_field_by_part(tuple,
key_def->parts,
MULTIKEY_NONE);
@@ -364,7 +364,7 @@ tuple_hash_slowpath(struct tuple *tuple, struct key_def *key_def)
{
assert(has_json_paths == key_def->has_json_paths);
assert(has_optional_parts == key_def->has_optional_parts);
- assert(!key_def_is_multikey(key_def));
+ assert(!key_def->is_multikey);
uint32_t h = HASH_SEED;
uint32_t carry = 0;
uint32_t total_size = 0;
diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index 5c0114ac6..f68958ba1 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -1019,8 +1019,7 @@ vinyl_index_def_change_requires_rebuild(struct index *index,
TUPLE_INDEX_BASE) != 0)
return true;
}
- assert(key_def_is_multikey(old_cmp_def) ==
- key_def_is_multikey(new_cmp_def));
+ assert(old_cmp_def->is_multikey == new_cmp_def->is_multikey);
return false;
}
@@ -1643,7 +1642,7 @@ vy_check_is_unique_secondary(struct vy_tx *tx, const struct vy_read_view **rv,
{
if (!lsm->check_is_unique)
return 0;
- if (!key_def_is_multikey(lsm->cmp_def)) {
+ if (!lsm->cmp_def->is_multikey) {
return vy_check_is_unique_secondary_one(tx, rv,
space_name, index_name, lsm, stmt,
MULTIKEY_NONE);
diff --git a/src/box/vy_stmt.h b/src/box/vy_stmt.h
index a80456add..25219230d 100644
--- a/src/box/vy_stmt.h
+++ b/src/box/vy_stmt.h
@@ -696,7 +696,7 @@ vy_stmt_str(struct tuple *stmt);
static inline int
vy_entry_multikey_idx(struct vy_entry entry, struct key_def *key_def)
{
- if (!key_def_is_multikey(key_def) || vy_stmt_is_key(entry.stmt))
+ if (!key_def->is_multikey || vy_stmt_is_key(entry.stmt))
return MULTIKEY_NONE;
assert(entry.hint != HINT_NONE);
return (int)entry.hint;
@@ -766,11 +766,11 @@ vy_entry_compare_with_raw_key(struct vy_entry entry,
*/
#define vy_stmt_foreach_entry(entry, src_stmt, key_def) \
for (uint32_t multikey_idx = 0, \
- multikey_count = !key_def_is_multikey((key_def)) ? 1 : \
+ multikey_count = !(key_def)->is_multikey ? 1 : \
tuple_multikey_count((src_stmt), (key_def)); \
multikey_idx < multikey_count && \
(((entry).stmt = (src_stmt)), \
- ((entry).hint = !key_def_is_multikey((key_def)) ? \
+ ((entry).hint = !(key_def)->is_multikey ? \
vy_stmt_hint((src_stmt), (key_def)) : \
multikey_idx), true); \
++multikey_idx)
--
2.22.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 2/4] box: refactor key_validate_parts to return key_end
2019-07-17 1:20 [PATCH v3 0/4] box: functional indexes Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
@ 2019-07-17 1:20 ` Kirill Shcherbatov
2019-07-18 9:41 ` Konstantin Osipov
2019-07-18 11:21 ` [tarantool-patches] " Kirill Yukhin
2019-07-17 1:20 ` [PATCH v3 3/4] box: introduce tuple_extra infrastructure Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
3 siblings, 2 replies; 15+ messages in thread
From: Kirill Shcherbatov @ 2019-07-17 1:20 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev; +Cc: kostja, Kirill Shcherbatov
The key_validate_parts helper is refactored to return a pointer
to the end of a given key argument in case of success.
This is required to effectively validate a sequence of keys in
scope of functional multikey indexes.
Needed for #1260
---
src/box/index.cc | 6 ++++--
src/box/key_def.c | 4 +++-
src/box/key_def.h | 4 +++-
src/box/lua/key_def.c | 5 +++--
src/box/vinyl.c | 4 +++-
5 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/src/box/index.cc b/src/box/index.cc
index 7f26c9bc2..843d0e73d 100644
--- a/src/box/index.cc
+++ b/src/box/index.cc
@@ -126,8 +126,9 @@ key_validate(const struct index_def *index_def, enum iterator_type type,
part_count);
return -1;
}
+ const char *key_end;
if (key_validate_parts(index_def->key_def, key,
- part_count, true) != 0)
+ part_count, true, &key_end) != 0)
return -1;
}
return 0;
@@ -143,7 +144,8 @@ exact_key_validate(struct key_def *key_def, const char *key,
part_count);
return -1;
}
- return key_validate_parts(key_def, key, part_count, false);
+ const char *key_end;
+ return key_validate_parts(key_def, key, part_count, false, &key_end);
}
char *
diff --git a/src/box/key_def.c b/src/box/key_def.c
index 2aa095091..ee758eefa 100644
--- a/src/box/key_def.c
+++ b/src/box/key_def.c
@@ -834,7 +834,8 @@ out:
int
key_validate_parts(const struct key_def *key_def, const char *key,
- uint32_t part_count, bool allow_nullable)
+ uint32_t part_count, bool allow_nullable,
+ const char **key_end)
{
for (uint32_t i = 0; i < part_count; i++) {
const struct key_part *part = &key_def->parts[i];
@@ -844,5 +845,6 @@ key_validate_parts(const struct key_def *key_def, const char *key,
return -1;
mp_next(&key);
}
+ *key_end = key;
return 0;
}
diff --git a/src/box/key_def.h b/src/box/key_def.h
index ab4b7c087..df83d055c 100644
--- a/src/box/key_def.h
+++ b/src/box/key_def.h
@@ -443,13 +443,15 @@ key_def_find_pk_in_cmp_def(const struct key_def *cmp_def,
* @param key MessagePack'ed data for matching.
* @param part_count Field count in the key.
* @param allow_nullable True if nullable parts are allowed.
+ * @param key_end[out] The end of the validated key.
*
* @retval 0 The key is valid.
* @retval -1 The key is invalid.
*/
int
key_validate_parts(const struct key_def *key_def, const char *key,
- uint32_t part_count, bool allow_nullable);
+ uint32_t part_count, bool allow_nullable,
+ const char **key_end);
/**
* Return true if @a index_def defines a sequential key without
diff --git a/src/box/lua/key_def.c b/src/box/lua/key_def.c
index 052a1c85d..041b5ec98 100644
--- a/src/box/lua/key_def.c
+++ b/src/box/lua/key_def.c
@@ -338,9 +338,10 @@ lbox_key_def_compare_with_key(struct lua_State *L)
struct region *region = &fiber()->gc;
size_t region_svp = region_used(region);
size_t key_len;
- const char *key = lbox_encode_tuple_on_gc(L, 3, &key_len);
+ const char *key_end, *key = lbox_encode_tuple_on_gc(L, 3, &key_len);
uint32_t part_count = mp_decode_array(&key);
- if (key_validate_parts(key_def, key, part_count, true) != 0) {
+ if (key_validate_parts(key_def, key, part_count, true,
+ &key_end) != 0) {
region_truncate(region, region_svp);
tuple_unref(tuple);
return luaT_error(L);
diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index f68958ba1..cf7af26b7 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -1751,7 +1751,9 @@ vy_unique_key_validate(struct vy_lsm *lsm, const char *key,
original_part_count, part_count);
return -1;
}
- return key_validate_parts(lsm->cmp_def, key, part_count, false);
+ const char *key_end;
+ return key_validate_parts(lsm->cmp_def, key, part_count, false,
+ &key_end);
}
/**
--
2.22.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 3/4] box: introduce tuple_extra infrastructure
2019-07-17 1:20 [PATCH v3 0/4] box: functional indexes Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 2/4] box: refactor key_validate_parts to return key_end Kirill Shcherbatov
@ 2019-07-17 1:20 ` Kirill Shcherbatov
2019-07-18 9:47 ` Konstantin Osipov
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
3 siblings, 1 reply; 15+ messages in thread
From: Kirill Shcherbatov @ 2019-07-17 1:20 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev; +Cc: kostja, Kirill Shcherbatov
Introduced a new object tuple_extra: a memory allocation is
associated with given tuple and chunk_id. Extended tuple_format's
vtab with few new methods to manage tuple_extras lifecycle.
Supported tuple_extra for memtx engine: a memory
chunks are allocated with memtx's smalloc allocator.
Needed for #1260
---
src/box/memtx_engine.c | 29 +++++++++++++++++++++++++++++
src/box/tuple.c | 8 ++++++++
src/box/tuple.h | 39 +++++++++++++++++++++++++++++++++++++++
src/box/tuple_format.h | 15 +++++++++++++++
src/box/vy_stmt.c | 2 ++
5 files changed, 93 insertions(+)
diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c
index 428491c2d..49efcc192 100644
--- a/src/box/memtx_engine.c
+++ b/src/box/memtx_engine.c
@@ -1126,9 +1126,38 @@ memtx_tuple_delete(struct tuple_format *format, struct tuple *tuple)
smfree_delayed(&memtx->alloc, memtx_tuple, total);
}
+void
+metmx_tuple_extra_delete(struct tuple_format *format,
+ struct tuple_extra *tuple_extra)
+{
+ struct memtx_engine *memtx = (struct memtx_engine *)format->engine;
+ uint32_t sz = tuple_extra_sz(tuple_extra->data_sz);
+ smfree(&memtx->alloc, tuple_extra, sz);
+}
+
+struct tuple_extra *
+memtx_tuple_extra_new(struct tuple_format *format, struct tuple *tuple,
+ uint32_t chunk_id, uint32_t data_sz)
+{
+ struct memtx_engine *memtx = (struct memtx_engine *)format->engine;
+ uint32_t sz = tuple_extra_sz(data_sz);
+ struct tuple_extra *tuple_extra =
+ (struct tuple_extra *) smalloc(&memtx->alloc, sz);
+ if (tuple == NULL) {
+ diag_set(OutOfMemory, sz, "smalloc", "tuple");
+ return NULL;
+ }
+ tuple_extra->tuple = tuple;
+ tuple_extra->chunk_id = chunk_id;
+ tuple_extra->data_sz = data_sz;
+ return tuple_extra;
+}
+
struct tuple_format_vtab memtx_tuple_format_vtab = {
memtx_tuple_delete,
memtx_tuple_new,
+ metmx_tuple_extra_delete,
+ memtx_tuple_extra_new,
};
/**
diff --git a/src/box/tuple.c b/src/box/tuple.c
index c0e94d55b..ca0e5230a 100644
--- a/src/box/tuple.c
+++ b/src/box/tuple.c
@@ -67,6 +67,8 @@ runtime_tuple_new(struct tuple_format *format, const char *data, const char *end
static struct tuple_format_vtab tuple_format_runtime_vtab = {
runtime_tuple_delete,
runtime_tuple_new,
+ NULL,
+ NULL,
};
static struct tuple *
@@ -785,3 +787,9 @@ mp_str(const char *data)
return "<failed to format message pack>";
return buf;
}
+
+uint32_t
+tuple_extra_sz(uint32_t data_sz)
+{
+ return sizeof(struct tuple_extra) + data_sz;
+}
diff --git a/src/box/tuple.h b/src/box/tuple.h
index 99dfeb82d..70119292e 100644
--- a/src/box/tuple.h
+++ b/src/box/tuple.h
@@ -447,6 +447,45 @@ tuple_delete(struct tuple *tuple)
format->vtab.tuple_delete(format, tuple);
}
+/** Tuple metadata hashtable entry. */
+struct tuple_extra {
+ /** Tuple pointer is used as a hash key. */
+ struct tuple *tuple;
+ /** An extention identifier. */
+ uint32_t chunk_id;
+ /** The payload size. Needed to perform memory release.*/
+ uint32_t data_sz;
+ /** Metadata object payload. */
+ char data[0];
+};
+
+/** Calculate the size of tuple_extra object by given data_sz. */
+uint32_t
+tuple_extra_sz(uint32_t data_sz);
+
+/**
+ * Allocate a new extra allocation for given tuple and
+ * unique chunk_id identifier.
+ */
+static inline struct tuple_extra *
+tuple_extra_new(struct tuple *tuple, uint32_t chunk_id, uint32_t data_sz)
+{
+ struct tuple_format *format = tuple_format(tuple);
+ return format->vtab.tuple_extra_new(format, tuple, chunk_id, data_sz);
+}
+
+/**
+ * Free allocated tuple extra for given tuple and unique
+ * chunk_id identifier.
+ */
+static inline void
+tuple_extra_delete(struct tuple_extra *tuple_extra)
+{
+ struct tuple_format *format =
+ tuple_format(tuple_extra->tuple);
+ format->vtab.tuple_extra_delete(format, tuple_extra);
+}
+
/**
* Check tuple data correspondence to space format.
* Actually, checks everything that is checked by
diff --git a/src/box/tuple_format.h b/src/box/tuple_format.h
index e4f5f0018..ac4f8a702 100644
--- a/src/box/tuple_format.h
+++ b/src/box/tuple_format.h
@@ -64,6 +64,7 @@ enum { TUPLE_INDEX_BASE = 1 };
enum { TUPLE_OFFSET_SLOT_NIL = INT32_MAX };
struct tuple;
+struct tuple_extra;
struct tuple_format;
struct coll;
@@ -82,6 +83,20 @@ struct tuple_format_vtab {
struct tuple*
(*tuple_new)(struct tuple_format *format, const char *data,
const char *end);
+ /**
+ * Free allocated tuple extra for given tuple and unique
+ * chunk_id identifier.
+ */
+ void
+ (*tuple_extra_delete)(struct tuple_format *format,
+ struct tuple_extra *tuple_extra);
+ /**
+ * Allocate a new extra allocation for given tuple and
+ * unique chunk_id identifier.
+ */
+ struct tuple_extra *
+ (*tuple_extra_new)(struct tuple_format *format, struct tuple *tuple,
+ uint32_t chunk_id, uint32_t data_sz);
};
/** Tuple field meta information for tuple_format. */
diff --git a/src/box/vy_stmt.c b/src/box/vy_stmt.c
index f936cd61f..6b09e5e53 100644
--- a/src/box/vy_stmt.c
+++ b/src/box/vy_stmt.c
@@ -118,6 +118,8 @@ vy_stmt_env_create(struct vy_stmt_env *env)
{
env->tuple_format_vtab.tuple_new = vy_tuple_new;
env->tuple_format_vtab.tuple_delete = vy_tuple_delete;
+ env->tuple_format_vtab.tuple_extra_new = NULL;
+ env->tuple_format_vtab.tuple_extra_delete = NULL;
env->max_tuple_size = 1024 * 1024;
env->key_format = vy_stmt_format_new(env, NULL, 0, NULL, 0, 0, NULL);
if (env->key_format == NULL)
--
2.22.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 [PATCH v3 0/4] box: functional indexes Kirill Shcherbatov
` (2 preceding siblings ...)
2019-07-17 1:20 ` [PATCH v3 3/4] box: introduce tuple_extra infrastructure Kirill Shcherbatov
@ 2019-07-17 1:20 ` Kirill Shcherbatov
2019-07-18 9:50 ` [tarantool-patches] " Konstantin Osipov
` (4 more replies)
3 siblings, 5 replies; 15+ messages in thread
From: Kirill Shcherbatov @ 2019-07-17 1:20 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev; +Cc: kostja, Kirill Shcherbatov
Closes #1260
@TarantoolBot document
Title: introduce functional indexes in memtx
Now you can define a functional index using registered persistent
function as key extractor.
There is some restrictions for function and key definition for
functional index:
- referenced extractor function must be persistent, deterministic
and return scalar type or array.
- key parts describe the referenced routine's returned key
(when the routine returned value type is array in meaning of
multikey index key): all parts must be sequential and the
first part's fieldno must be 1.
- key parts can't use JSON paths.
- you are not allowed to access tuple fields by names in functional
index extractor
Functional index can't be primary.
You are not allowed to change functional index extractor function
while there are some functional indexes depends of it.
Example:
s = box.schema.space.create('withdata')
s:format({{name = 'name', type = 'string'},
{name = 'address', type = 'string'}})
pk = s:create_index('name', {parts = {1, 'string'}})
lua_code = [[function(tuple)
local address = string.split(tuple[2])
local ret = {}
for _, v in pairs(address) do
table.insert(ret, {utf8.upper(v)})
end
return unpack(ret)
end]]
box.schema.func.create('addr_extractor', {body = lua_code,
is_deterministic = true, is_sandboxed = true})
idx = s:create_index('addr', {unique = false, functional_fid =
box.func.addr_extractor.id, is_multikey = true,
parts = {{1, 'string'}}})
s:insert({"James", "SIS Building Lambeth London UK"})
s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
idx:select('UK')
---
- - ['James', 'SIS Building Lambeth London UK']
- ['Sherlock', '221B Baker St Marylebone London NW1 6XE UK']
...
---
src/box/CMakeLists.txt | 4 +-
src/box/alter.cc | 134 +++++-
src/box/alter.h | 1 +
src/box/bootstrap.snap | Bin 5863 -> 5914 bytes
src/box/box.cc | 4 +-
src/box/call.c | 8 +-
src/box/errcode.h | 1 +
| 118 +++++
| 101 ++++
src/box/functional_key.c | 228 ++++++++++
src/box/functional_key.h | 94 ++++
src/box/index.cc | 28 ++
src/box/index.h | 38 +-
src/box/index_def.c | 9 +
src/box/index_def.h | 9 +-
src/box/key_def.c | 33 +-
src/box/key_def.h | 26 +-
src/box/lua/key_def.c | 2 +-
src/box/lua/misc.cc | 2 +-
src/box/lua/schema.lua | 20 +
src/box/lua/space.cc | 15 +
src/box/lua/upgrade.lua | 18 +
src/box/memtx_engine.c | 4 +
src/box/memtx_space.c | 18 +
src/box/memtx_tree.c | 41 +-
src/box/port.h | 4 +-
src/box/schema.cc | 13 +-
src/box/schema_def.h | 9 +
src/box/sql.c | 3 +-
src/box/sql/build.c | 2 +-
src/box/sql/select.c | 3 +-
src/box/sql/where.c | 2 +-
src/box/tuple.c | 25 +-
src/box/tuple.h | 64 ++-
src/box/tuple_compare.cc | 206 ++++++---
| 9 +-
src/box/tuple_format.c | 58 ++-
src/box/tuple_format.h | 17 +
src/box/tuple_hash.cc | 1 +
src/box/vinyl.c | 11 +-
test/app-tap/tarantoolctl.test.lua | 4 +-
test/box-py/bootstrap.result | 5 +
test/box/access.result | 3 +
test/box/access.test.lua | 1 +
test/box/access_misc.result | 132 +++---
test/box/access_sysview.result | 6 +-
test/box/alter.result | 7 +-
test/box/bitset.result | 24 +
test/box/bitset.test.lua | 9 +
test/box/hash.result | 24 +
test/box/hash.test.lua | 9 +
test/box/misc.result | 1 +
test/box/rtree_misc.result | 24 +
test/box/rtree_misc.test.lua | 9 +
test/engine/engine.cfg | 5 +-
test/engine/functional.result | 708 +++++++++++++++++++++++++++++
test/engine/functional.test.lua | 240 ++++++++++
test/unit/luaT_tuple_new.c | 2 +-
test/unit/merger.test.c | 8 +-
test/unit/tuple_bigref.c | 2 +-
test/unit/vy_iterators_helper.c | 2 +-
test/vinyl/misc.result | 23 +
test/vinyl/misc.test.lua | 9 +
test/wal_off/alter.result | 2 +-
64 files changed, 2468 insertions(+), 174 deletions(-)
create mode 100644 src/box/functional_extractor.c
create mode 100644 src/box/functional_extractor.h
create mode 100644 src/box/functional_key.c
create mode 100644 src/box/functional_key.h
create mode 100644 test/engine/functional.result
create mode 100644 test/engine/functional.test.lua
diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 481842a39..ddc8440a6 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -51,8 +51,9 @@ add_library(tuple STATIC
coll_id_cache.c
field_def.c
opt_def.c
+ functional_extractor.c
)
-target_link_libraries(tuple json box_error core ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES} misc bit)
+target_link_libraries(tuple json box_error core coll ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES} misc bit)
add_library(xlog STATIC xlog.c)
target_link_libraries(xlog core box_error crc32 ${ZSTD_LIBRARIES})
@@ -125,6 +126,7 @@ add_library(box STATIC
wal.c
call.c
merger.c
+ functional_key.c
${lua_sources}
lua/init.c
lua/call.c
diff --git a/src/box/alter.cc b/src/box/alter.cc
index 1dbfe6b26..fdf38598e 100644
--- a/src/box/alter.cc
+++ b/src/box/alter.cc
@@ -30,6 +30,7 @@
*/
#include "alter.h"
#include "assoc.h"
+#include "box.h"
#include "ck_constraint.h"
#include "column_mask.h"
#include "schema.h"
@@ -38,6 +39,7 @@
#include "index.h"
#include "fk_constraint.h"
#include "func.h"
+#include "functional_extractor.h"
#include "coll_id_cache.h"
#include "coll_id_def.h"
#include "txn.h"
@@ -235,6 +237,21 @@ index_opts_decode(struct index_opts *opts, const char *map,
"bloom_fpr must be greater than 0 and "
"less than or equal to 1");
}
+ /**
+ * Can't verify functional index extractor routine
+ * reference on load because the function object
+ * had not been registered in Tarantool yet.
+ */
+ if (strcmp(box_status(), "loading") != 0 && opts->functional_fid > 0) {
+ struct func *func = func_cache_find(opts->functional_fid);
+ if (func->def->language != FUNC_LANGUAGE_LUA ||
+ func->def->body == NULL || !func->def->is_deterministic ||
+ !func->def->is_sandboxed) {
+ tnt_raise(ClientError, ER_WRONG_INDEX_OPTIONS, 0,
+ "referenced function doesn't satisfy "
+ "functional index constraints");
+ }
+ }
}
/**
@@ -291,7 +308,8 @@ index_def_new_from_tuple(struct tuple *tuple, struct space *space)
space->def->fields,
space->def->field_count, &fiber()->gc) != 0)
diag_raise();
- key_def = key_def_new(part_def, part_count);
+ key_def = key_def_new(part_def, part_count, opts.functional_fid > 0,
+ opts.is_multikey);
if (key_def == NULL)
diag_raise();
struct index_def *index_def =
@@ -1376,6 +1394,60 @@ RebuildIndex::~RebuildIndex()
index_def_delete(new_index_def);
}
+/**
+ * RebuildFunctionalIndex - prepare functional index definition,
+ * drop the old index data and rebuild index from by reading the
+ * primary key.
+ */
+class RebuildFunctionalIndex: public RebuildIndex
+{
+ struct index_def *
+ functional_index_def_new(struct index_def *index_def,
+ struct func *func)
+ {
+ struct index_def *new_index_def = index_def_dup_xc(index_def);
+ old_extractor = index_def->key_def->functional_extractor;
+ if (func != NULL) {
+ new_extractor = functional_extractor_new(func);
+ if (new_extractor == NULL)
+ diag_raise();
+ } else {
+ new_extractor = NULL;
+ }
+ new_index_def->key_def->functional_extractor = new_extractor;
+ new_index_def->cmp_def->functional_extractor = new_extractor;
+ return new_index_def;
+ }
+ /** New functional extractor. */
+ struct functional_extractor *new_extractor;
+ /** Old functional extractor. */
+ struct functional_extractor *old_extractor;
+public:
+ RebuildFunctionalIndex(struct alter_space *alter,
+ struct index_def *old_index_def_arg,
+ struct func *func) :
+ RebuildIndex(alter,
+ functional_index_def_new(old_index_def_arg, func),
+ old_index_def_arg) {}
+ virtual void commit(struct alter_space *alter, int64_t signature);
+ virtual ~RebuildFunctionalIndex();
+};
+
+void
+RebuildFunctionalIndex::commit(struct alter_space *alter, int64_t signature)
+{
+ new_extractor = NULL;
+ if (old_extractor != NULL)
+ functional_extractor_delete(old_extractor);
+ RebuildIndex::commit(alter, signature);
+}
+
+RebuildFunctionalIndex::~RebuildFunctionalIndex()
+{
+ if (new_extractor != NULL)
+ functional_extractor_delete(new_extractor);
+}
+
/** TruncateIndex - truncate an index. */
class TruncateIndex: public AlterSpaceOp
{
@@ -2847,6 +2919,12 @@ on_replace_dd_func(struct trigger * /* trigger */, void *event)
(unsigned) old_func->def->uid,
"function has grants");
}
+ if (old_func != NULL &&
+ space_has_data(BOX_FUNC_INDEX_ID, 1, old_func->def->fid)) {
+ tnt_raise(ClientError, ER_DROP_FUNCTION,
+ (unsigned) old_func->def->uid,
+ "function has references");
+ }
struct trigger *on_commit =
txn_alter_trigger_new(on_drop_func_commit, old_func);
struct trigger *on_rollback =
@@ -4695,6 +4773,56 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event)
trigger_run_xc(&on_alter_space, space);
}
+/** A trigger invoked on replace in the _func_index space. */
+static void
+on_replace_dd_func_index(struct trigger *trigger, void *event)
+{
+ (void) trigger;
+ struct txn *txn = (struct txn *) event;
+ struct txn_stmt *stmt = txn_current_stmt(txn);
+ struct tuple *old_tuple = stmt->old_tuple;
+ struct tuple *new_tuple = stmt->new_tuple;
+
+ struct alter_space *alter = NULL;
+ struct func *func = NULL;
+ struct index *index;
+ struct space *space;
+ if (old_tuple == NULL && new_tuple != NULL) {
+ uint32_t space_id = tuple_field_u32_xc(new_tuple,
+ BOX_FUNC_INDEX_FIELD_SPACE_ID);
+ uint32_t index_id = tuple_field_u32_xc(new_tuple,
+ BOX_FUNC_INDEX_FIELD_INDEX_ID);
+ uint32_t fid = tuple_field_u32_xc(new_tuple,
+ BOX_FUNC_INDEX_FUNCTION_ID);
+ space = space_cache_find_xc(space_id);
+ index = index_find_xc(space, index_id);
+ func = func_cache_find(fid);
+ } else if (old_tuple != NULL && new_tuple == NULL) {
+ uint32_t space_id = tuple_field_u32_xc(old_tuple,
+ BOX_FUNC_INDEX_FIELD_SPACE_ID);
+ uint32_t index_id = tuple_field_u32_xc(old_tuple,
+ BOX_FUNC_INDEX_FIELD_INDEX_ID);
+ space = space_cache_find_xc(space_id);
+ index = index_find_xc(space, index_id);
+ func = NULL;
+ } else {
+ assert(old_tuple != NULL && new_tuple != NULL);
+ tnt_raise(ClientError, ER_UNSUPPORTED, "func_index", "alter");
+ }
+
+ alter = alter_space_new(space);
+ auto scoped_guard = make_scoped_guard([=] {alter_space_delete(alter);});
+ alter_space_move_indexes(alter, 0, index->def->iid);
+ (void) new RebuildFunctionalIndex(alter, index->def, func);
+ alter_space_move_indexes(alter, index->def->iid + 1,
+ space->index_id_max + 1);
+ (void) new MoveCkConstraints(alter);
+ (void) new UpdateSchemaVersion(alter);
+ alter_space_do(txn, alter);
+
+ scoped_guard.is_active = false;
+}
+
struct trigger alter_space_on_replace_space = {
RLIST_LINK_INITIALIZER, on_replace_dd_space, NULL, NULL
};
@@ -4755,4 +4883,8 @@ struct trigger on_replace_ck_constraint = {
RLIST_LINK_INITIALIZER, on_replace_dd_ck_constraint, NULL, NULL
};
+struct trigger on_replace_func_index = {
+ RLIST_LINK_INITIALIZER, on_replace_dd_func_index, NULL, NULL
+};
+
/* vim: set foldmethod=marker */
diff --git a/src/box/alter.h b/src/box/alter.h
index c339ccea6..1bc837359 100644
--- a/src/box/alter.h
+++ b/src/box/alter.h
@@ -47,5 +47,6 @@ extern struct trigger on_replace_space_sequence;
extern struct trigger on_replace_trigger;
extern struct trigger on_replace_fk_constraint;
extern struct trigger on_replace_ck_constraint;
+extern struct trigger on_replace_func_index;
#endif /* INCLUDES_TARANTOOL_BOX_ALTER_H */
diff --git a/src/box/bootstrap.snap b/src/box/bootstrap.snap
index ae698731e480e0d8cc250659222032291922e549..a52256b65c113a2055f0a8f65f9b90398d3816d5 100644
diff --git a/src/box/box.cc b/src/box/box.cc
index 80249919e..9e6f2da62 100644
--- a/src/box/box.cc
+++ b/src/box/box.cc
@@ -74,6 +74,7 @@
#include "call.h"
#include "func.h"
#include "sequence.h"
+#include "functional_key.h"
static char status[64] = "unknown";
@@ -2068,7 +2069,8 @@ box_init(void)
if (module_init() != 0)
diag_raise();
- if (tuple_init(lua_hash) != 0)
+ if (tuple_init(lua_hash, functional_key_get,
+ functional_key_get_raw) != 0)
diag_raise();
sequence_init();
diff --git a/src/box/call.c b/src/box/call.c
index ac2bf3004..d2a89ced3 100644
--- a/src/box/call.c
+++ b/src/box/call.c
@@ -46,13 +46,15 @@
static const struct port_vtab port_msgpack_vtab;
void
-port_msgpack_create(struct port *base, const char *data, uint32_t data_sz)
+port_msgpack_create(struct port *base, const char *data, uint32_t data_sz,
+ bool is_single_tuple)
{
struct port_msgpack *port_msgpack = (struct port_msgpack *) base;
memset(port_msgpack, 0, sizeof(*port_msgpack));
port_msgpack->vtab = &port_msgpack_vtab;
port_msgpack->data = data;
port_msgpack->data_sz = data_sz;
+ port_msgpack->is_single_tuple = is_single_tuple;
}
static const char *
@@ -113,7 +115,7 @@ box_process_call(struct call_request *request, struct port *port)
int rc;
struct port args;
port_msgpack_create(&args, request->args,
- request->args_end - request->args);
+ request->args_end - request->args, false);
struct func *func = func_by_name(name, name_len);
if (func != NULL) {
rc = func_call(func, &args, port);
@@ -149,7 +151,7 @@ box_process_eval(struct call_request *request, struct port *port)
return -1;
struct port args;
port_msgpack_create(&args, request->args,
- request->args_end - request->args);
+ request->args_end - request->args, false);
const char *expr = request->expr;
uint32_t expr_len = mp_decode_strl(&expr);
struct txn *txn;
diff --git a/src/box/errcode.h b/src/box/errcode.h
index 361ad3a45..87001f80d 100644
--- a/src/box/errcode.h
+++ b/src/box/errcode.h
@@ -250,6 +250,7 @@ struct errcode_record {
/*195 */_(ER_CREATE_CK_CONSTRAINT, "Failed to create check constraint '%s': %s") \
/*196 */_(ER_CK_CONSTRAINT_FAILED, "Check constraint failed '%s': %s") \
/*197 */_(ER_SQL_COLUMN_COUNT, "Unequal number of entries in row expression: left side has %u, but right side - %u") \
+ /*198 */_(ER_FUNCTIONAL_EXTRACTOR, "Functional index extractor '%s' error: %s") \
/*
* !IMPORTANT! Please follow instructions at start of the file
--git a/src/box/functional_extractor.c b/src/box/functional_extractor.c
new file mode 100644
index 000000000..ff69aa063
--- /dev/null
+++ b/src/box/functional_extractor.c
@@ -0,0 +1,118 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "functional_extractor.h"
+#include "func.h"
+#include "func_def.h"
+#include "tuple.h"
+
+/** Map: (struct tuple *) => (struct tuple_extra *). */
+#define MH_SOURCE 1
+#define mh_name _tuple_extra
+#define mh_key_t struct tuple *
+#define mh_node_t struct tuple_extra *
+#define mh_arg_t uint32_t
+#define mh_hash(a, arg) ((uintptr_t)(*(a))->tuple)
+#define mh_hash_key(a, arg) ((uintptr_t) a)
+#define mh_cmp(a, b, arg) ((*(a))->tuple != (*(b))->tuple || \
+ (*(a))->chunk_id != (*(b))->chunk_id)
+#define mh_cmp_key(a, b, arg) ((a) != (*(b))->tuple || (arg) != (*(b))->chunk_id)
+#include "salad/mhash.h"
+
+int
+functional_extractor_hash_register(struct functional_extractor *extractor,
+ struct tuple_extra *tuple_extra)
+{
+ mh_int_t id = mh_tuple_extra_put(extractor->hash,
+ (const struct tuple_extra **)&tuple_extra, NULL, 0);
+ if (id == mh_end(extractor->hash))
+ return -1;
+ return 0;
+}
+
+void
+functional_extractor_hash_unregister(struct functional_extractor *extractor,
+ struct tuple_extra *tuple_extra)
+{
+ mh_int_t id = mh_tuple_extra_find(extractor->hash, tuple_extra->tuple,
+ tuple_extra->chunk_id);
+ assert(id != mh_end(extractor->hash));
+ mh_tuple_extra_del(extractor->hash, id, tuple_extra->chunk_id);
+}
+
+struct tuple_extra *
+functional_extractor_hash_lookup(struct functional_extractor *extractor,
+ struct tuple *tuple)
+{
+ mh_int_t id = mh_tuple_extra_find(extractor->hash, tuple,
+ extractor->func->def->fid);
+ if (id == mh_end(extractor->hash))
+ return NULL;
+ struct tuple_extra **tuple_extra_ptr =
+ mh_tuple_extra_node(extractor->hash, id);
+ assert(tuple_extra_ptr != NULL && *tuple_extra_ptr != NULL);
+ return *tuple_extra_ptr;
+}
+
+struct functional_extractor *
+functional_extractor_new(struct func *func)
+{
+ struct functional_extractor *extractor = malloc(sizeof(*extractor));
+ if (extractor == NULL) {
+ diag_set(OutOfMemory, sizeof(*extractor), "malloc",
+ "extractor");
+ return NULL;
+ }
+ extractor->func = func;
+ extractor->hash = mh_tuple_extra_new();
+ if (extractor->hash == NULL) {
+ diag_set(OutOfMemory, sizeof(*extractor->hash),
+ "malloc", "hash");
+ free(extractor);
+ return NULL;
+ }
+ return extractor;
+}
+
+void
+functional_extractor_delete(struct functional_extractor *extractor)
+{
+ while (mh_size(extractor->hash) > 0) {
+ mh_int_t i = mh_first(extractor->hash);
+ struct tuple_extra **tuple_extra =
+ mh_tuple_extra_node(extractor->hash, i);
+ assert(tuple_extra != NULL && *tuple_extra != NULL);
+ mh_tuple_extra_del(extractor->hash, i, 0);
+ tuple_extra_delete(*tuple_extra);
+ }
+ mh_tuple_extra_delete(extractor->hash);
+ TRASH(extractor);
+ free(extractor);
+}
--git a/src/box/functional_extractor.h b/src/box/functional_extractor.h
new file mode 100644
index 000000000..1496ef864
--- /dev/null
+++ b/src/box/functional_extractor.h
@@ -0,0 +1,101 @@
+#ifndef TARANTOOL_BOX_FUNCTIONAL_EXTRACTOR_H_INCLUDED
+#define TARANTOOL_BOX_FUNCTIONAL_EXTRACTOR_H_INCLUDED
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "inttypes.h"
+#include <stdio.h>
+
+struct func;
+struct tuple;
+struct tuple_extra;
+struct mh_tuple_extra_t;
+
+/**
+ * Functional extractor is a proxy object to tune functional index
+ * extractor calls.
+ */
+struct functional_extractor {
+ /** A pointer to functional index extractor routine. */
+ struct func *func;
+ /**
+ * Hash of extracted and registered functional index
+ * keys: <tuple *, functional_fid> -> tuple_extra *
+ */
+ struct mh_tuple_extra_t *hash;
+};
+
+/**
+ * Register a new functional index key represented as
+ * engine-dependent tuple_extra memory chunk.
+ */
+int
+functional_extractor_hash_register(struct functional_extractor *extractor,
+ struct tuple_extra *tuple_extra);
+
+/**
+ * Unregister a given functional index key represented as
+ * engine-dependent tuple_extra.
+ */
+void
+functional_extractor_hash_unregister(struct functional_extractor *extractor,
+ struct tuple_extra *tuple_extra);
+
+/**
+ * Lookup for a registered functional index key represented as a
+ * engine-dependent tuple_extra.
+ */
+struct tuple_extra *
+functional_extractor_hash_lookup(struct functional_extractor *extractor,
+ struct tuple *tuple);
+
+/**
+ * Create a new functional index extractor object for
+ * given function.
+ */
+struct functional_extractor *
+functional_extractor_new(struct func *func);
+
+/**
+ * Release a given functional index extractor object and free
+ * associated memory.
+ */
+void
+functional_extractor_delete(struct functional_extractor *extractor);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* TARANTOOL_BOX_FUNCTIONAL_EXTRACTOR_H_INCLUDED */
diff --git a/src/box/functional_key.c b/src/box/functional_key.c
new file mode 100644
index 000000000..23ea5dd5f
--- /dev/null
+++ b/src/box/functional_key.c
@@ -0,0 +1,228 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "functional_key.h"
+#include "functional_extractor.h"
+#include "fiber.h"
+#include "func.h"
+#include "port.h"
+#include "stdbool.h"
+#include "tuple.h"
+
+/** Calculate the size of the functional key map. */
+static inline uint32_t
+functional_key_map_sz(uint32_t key_count)
+{
+ return sizeof(uint32_t) * key_count;
+}
+
+/**
+ * Execute a given functional index extractor function and
+ * return an extracted key_data and key_data_sz.
+ */
+static const char *
+functional_key_extract(struct func *func, struct port *in_port,
+ uint32_t *key_data_sz)
+{
+ struct port out_port;
+ int rc = func_call(func, in_port, &out_port);
+ if (rc != 0)
+ goto error;
+ const char *key_data = port_get_msgpack(&out_port, key_data_sz);
+ port_destroy(&out_port);
+ if (key_data == NULL)
+ goto error;
+
+ return key_data;
+error:
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR, func->def->name,
+ diag_last_error(diag_get())->errmsg);
+ return NULL;
+}
+
+/**
+ * Process a given data and initialize a key_map allocation.
+ * Perform key validation if validate == true.
+ */
+static int
+functional_key_map_create(struct func *func, struct key_def *key_def,
+ const char *data, uint32_t key_count,
+ uint32_t *key_map)
+{
+ const char *key = data;
+ for (uint32_t key_idx = 0; key_idx < key_count; key_idx++) {
+ if (key_map != NULL)
+ key_map[key_idx] = key - data;
+ if (mp_typeof(*key) != MP_ARRAY) {
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR,
+ func->def->name,
+ "returned key type is invalid");
+ return -1;
+ }
+ const char *key_end;
+ uint32_t part_count = mp_decode_array(&key);
+ if (key_def->functional_part_count != part_count) {
+ const char *error_msg =
+ tt_sprintf(tnt_errcode_desc(ER_EXACT_MATCH),
+ key_def->functional_part_count,
+ part_count);
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR,
+ func->def->name, error_msg);
+ return -1;
+ }
+ if (key_validate_parts(key_def, key, part_count, true,
+ &key_end) != 0) {
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR,
+ func->def->name,
+ diag_last_error(diag_get())->errmsg);
+ return -1;
+ }
+ key = key_end;
+ }
+ if (key_map != NULL)
+ key_map[0] = key_count;
+ return 0;
+}
+
+const char *
+functional_key_get_raw(const char *tuple, const char *tuple_end,
+ struct key_def *key_def, uint32_t *key_count)
+{
+ struct functional_extractor *extractor = key_def->functional_extractor;
+ assert(extractor != NULL);
+ struct func *func = extractor->func;
+ /*
+ * Evaluate functional index extractor function with
+ * given tuple data.
+ */
+ struct port in_port;
+ port_msgpack_create(&in_port, tuple, tuple_end - tuple, true);
+ uint32_t key_data_sz;
+ const char *key_data =
+ functional_key_extract(func, &in_port, &key_data_sz);
+ if (key_data == NULL)
+ return NULL;
+
+ *key_count = mp_decode_array(&key_data);
+ if (!key_def->is_multikey && *key_count > 1) {
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR,
+ func->def->name, "to many keys were returned");
+ return NULL;
+ }
+ if (functional_key_map_create(func, key_def, key_data, *key_count,
+ NULL) != 0)
+ return NULL;
+ return key_data;
+}
+
+struct tuple_extra *
+functional_key_new(struct tuple *tuple, struct key_def *key_def)
+{
+ struct functional_extractor *extractor = key_def->functional_extractor;
+ assert(extractor != NULL);
+ struct func *func = extractor->func;
+
+ struct port in_port;
+ port_tuple_create(&in_port);
+ port_tuple_add(&in_port, tuple);
+ uint32_t key_data_sz;
+ const char *key_data =
+ functional_key_extract(func, &in_port, &key_data_sz);
+ port_destroy(&in_port);
+ if (key_data == NULL)
+ return NULL;
+
+ /*
+ * Test whether extractor function returned only one
+ * value in case on non-multikey index.
+ */
+ uint32_t key_count = mp_decode_array(&key_data);
+ if (!key_def->is_multikey && key_count > 1) {
+ diag_set(ClientError, ER_FUNCTIONAL_EXTRACTOR,
+ func->def->name, "to many keys were returned");
+ return NULL;
+ }
+
+ uint32_t key_map_sz = functional_key_map_sz(key_count);
+ key_data_sz -= mp_sizeof_array(key_count);
+ struct tuple_extra *tuple_extra =
+ tuple_extra_new(tuple, func->def->fid, key_data_sz + key_map_sz);
+ if (tuple_extra == NULL)
+ return NULL;
+
+ memcpy(tuple_extra->data + key_map_sz, key_data, key_data_sz);
+ uint32_t *key_map = (uint32_t *) tuple_extra->data;
+ key_data = tuple_extra->data + key_map_sz;
+
+ /** Prepare functional multikey key map and validate. */
+ if (functional_key_map_create(func, key_def, key_data, key_count,
+ key_map) != 0)
+ goto error;
+ if (functional_extractor_hash_register(extractor, tuple_extra) != 0)
+ goto error;
+
+ return tuple_extra;
+error:
+ tuple_extra_delete(tuple_extra);
+ return NULL;
+}
+
+void
+functional_key_delete(struct tuple *tuple, struct key_def *key_def)
+{
+ struct functional_extractor *extractor = key_def->functional_extractor;
+ assert(extractor != NULL);
+
+ struct tuple_extra *tuple_extra =
+ functional_extractor_hash_lookup(extractor, tuple);
+ if (tuple_extra == NULL)
+ return;
+ functional_extractor_hash_unregister(extractor, tuple_extra);
+ tuple_extra_delete(tuple_extra);
+}
+
+const char *
+functional_key_get(struct tuple *tuple, struct key_def *key_def,
+ uint32_t *key_count, uint32_t **key_map)
+{
+ struct functional_extractor *extractor = key_def->functional_extractor;
+ assert(extractor != NULL);
+
+ struct tuple_extra *tuple_extra =
+ functional_extractor_hash_lookup(extractor, tuple);
+ if (unlikely(tuple_extra == NULL)) {
+ tuple_extra = functional_key_new(tuple, key_def);
+ if (tuple_extra == NULL)
+ return NULL;
+ }
+ *key_map = (uint32_t *) tuple_extra->data;
+ *key_count = (*key_map)[0];
+ return tuple_extra->data + *key_count * sizeof(uint32_t);
+}
diff --git a/src/box/functional_key.h b/src/box/functional_key.h
new file mode 100644
index 000000000..5d8a78812
--- /dev/null
+++ b/src/box/functional_key.h
@@ -0,0 +1,94 @@
+#ifndef TARANTOOL_BOX_FUNCTIONAL_KEY_H_INCLUDED
+#define TARANTOOL_BOX_FUNCTIONAL_KEY_H_INCLUDED
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "inttypes.h"
+#include "stdbool.h"
+
+struct tuple;
+struct key_def;
+struct tuple_extra;
+
+/**
+ * Evaluate functional index extractor with give tuple, validate
+ * result, alloc and register engine-dependent tuple_extra
+ * extention representing functional key;
+ *
+ * 4b 4b 4b
+ * +-----------+-----------+-----------+------+-----+------+
+ * | key_count | ... |keyN_offset| key1 | ... | keyN |
+ * +-----------+-----------+-----------+------+-----+------+
+ * |________________________^
+ * <----------------------------------->
+ * (sizeof(uint32_t) * key_count) bytes
+ *
+ * Returns not NULL tuple_extra pointer in case of success and
+ * NULL otherwise. The diag message is set.
+ */
+struct tuple_extra *
+functional_key_new(struct tuple *tuple, struct key_def *key_def);
+
+/**
+ * Unregister and destroy a functional key associated with given
+ * tuple and key definition.
+ */
+void
+functional_key_delete(struct tuple *tuple, struct key_def *key_def);
+
+/**
+ * Get functional index key by given tuple and functional
+ * key defintion. Function is expected a functional key to be
+ * registered in functional extractor cache; if not so, does it.
+ *
+ * Returns not NULL functional index key pointer or NULL.
+ */
+const char *
+functional_key_get(struct tuple *tuple, struct key_def *key_def,
+ uint32_t *key_count, uint32_t **key_map);
+
+/**
+ * Get functional index key by given tuple data and functional
+ * key defintion.
+ * Returns not NULL functional index key pointer or NULL.
+ */
+const char *
+functional_key_get_raw(const char *tuple, const char *tuple_end,
+ struct key_def *key_def, uint32_t *key_count);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* TARANTOOL_BOX_FUNCTIONAL_KEY_H_INCLUDED */
diff --git a/src/box/index.cc b/src/box/index.cc
index 843d0e73d..00a1b502e 100644
--- a/src/box/index.cc
+++ b/src/box/index.cc
@@ -679,6 +679,16 @@ generic_index_replace(struct index *index, struct tuple *old_tuple,
return -1;
}
+struct iterator *
+generic_index_create_iterator(struct index *base, enum iterator_type type,
+ const char *key, uint32_t part_count)
+{
+ (void) type; (void) key; (void) part_count;
+ diag_set(UnsupportedIndexFeature, base->def, "read view");
+ return NULL;
+}
+
+
struct snapshot_iterator *
generic_index_create_snapshot_iterator(struct index *index)
{
@@ -729,4 +739,22 @@ generic_index_end_build(struct index *)
{
}
+int
+disabled_index_build_next(struct index *index, struct tuple *tuple)
+{
+ (void) index; (void) tuple;
+ return 0;
+}
+
+int
+disabled_index_replace(struct index *index, struct tuple *old_tuple,
+ struct tuple *new_tuple, enum dup_replace_mode mode,
+ struct tuple **result)
+{
+ (void) old_tuple; (void) new_tuple; (void) mode;
+ (void) index;
+ *result = NULL;
+ return 0;
+}
+
/* }}} */
diff --git a/src/box/index.h b/src/box/index.h
index 97d600c96..75cf1eda6 100644
--- a/src/box/index.h
+++ b/src/box/index.h
@@ -33,6 +33,9 @@
#include <stdbool.h>
#include "trivia/util.h"
#include "iterator_type.h"
+#include "tuple.h"
+#include "functional_extractor.h"
+#include "functional_key.h"
#include "index_def.h"
#if defined(__cplusplus)
@@ -595,12 +598,35 @@ index_get(struct index *index, const char *key,
return index->vtab->get(index, key, part_count, result);
}
+static inline int
+index_functional_replace(struct index *index, struct tuple *old_tuple,
+ struct tuple *new_tuple, enum dup_replace_mode mode,
+ struct tuple **result)
+{
+ struct key_def *key_def = index->def->key_def;
+ if (new_tuple != NULL && functional_key_new(new_tuple, key_def) == NULL)
+ return -1;
+ int rc = index->vtab->replace(index, old_tuple, new_tuple, mode,
+ result);
+ if (rc == 0 && old_tuple != NULL)
+ functional_key_delete(old_tuple, key_def);
+ else if (rc != 0 && new_tuple != NULL)
+ functional_key_delete(new_tuple, key_def);
+ return rc;
+}
+
static inline int
index_replace(struct index *index, struct tuple *old_tuple,
struct tuple *new_tuple, enum dup_replace_mode mode,
struct tuple **result)
{
- return index->vtab->replace(index, old_tuple, new_tuple, mode, result);
+ if (index->def->key_def->functional_extractor != NULL) {
+ return index_functional_replace(index, old_tuple, new_tuple,
+ mode, result);
+ } else {
+ return index->vtab->replace(index, old_tuple, new_tuple, mode,
+ result);
+ }
}
static inline struct iterator *
@@ -685,9 +711,19 @@ void generic_index_compact(struct index *);
void generic_index_reset_stat(struct index *);
void generic_index_begin_build(struct index *);
int generic_index_reserve(struct index *, uint32_t);
+struct iterator *
+generic_index_create_iterator(struct index *base, enum iterator_type type,
+ const char *key, uint32_t part_count);
int generic_index_build_next(struct index *, struct tuple *);
void generic_index_end_build(struct index *);
+int
+disabled_index_build_next(struct index *index, struct tuple *tuple);
+int
+disabled_index_replace(struct index *index, struct tuple *old_tuple,
+ struct tuple *new_tuple, enum dup_replace_mode mode,
+ struct tuple **result);
+
#if defined(__cplusplus)
} /* extern "C" */
diff --git a/src/box/index_def.c b/src/box/index_def.c
index eb309a30c..82479d2cb 100644
--- a/src/box/index_def.c
+++ b/src/box/index_def.c
@@ -50,6 +50,8 @@ const struct index_opts index_opts_default = {
/* .bloom_fpr = */ 0.05,
/* .lsn = */ 0,
/* .stat = */ NULL,
+ /* .functional_fid = */ 0,
+ /* .is_multikey = */ false,
};
const struct opt_def index_opts_reg[] = {
@@ -63,6 +65,8 @@ const struct opt_def index_opts_reg[] = {
OPT_DEF("run_size_ratio", OPT_FLOAT, struct index_opts, run_size_ratio),
OPT_DEF("bloom_fpr", OPT_FLOAT, struct index_opts, bloom_fpr),
OPT_DEF("lsn", OPT_INT64, struct index_opts, lsn),
+ OPT_DEF("functional_fid", OPT_UINT32, struct index_opts, functional_fid),
+ OPT_DEF("is_multikey", OPT_BOOL, struct index_opts, is_multikey),
OPT_DEF_LEGACY("sql"),
OPT_END,
};
@@ -296,6 +300,11 @@ index_def_is_valid(struct index_def *index_def, const char *space_name)
space_name, "primary key cannot be multikey");
return false;
}
+ if (index_def->iid == 0 && key_def_is_functional(index_def->key_def)) {
+ diag_set(ClientError, ER_MODIFY_INDEX, index_def->name,
+ space_name, "primary key cannot be functional");
+ return false;
+ }
for (uint32_t i = 0; i < index_def->key_def->part_count; i++) {
assert(index_def->key_def->parts[i].type < field_type_MAX);
if (index_def->key_def->parts[i].fieldno > BOX_INDEX_FIELD_MAX) {
diff --git a/src/box/index_def.h b/src/box/index_def.h
index 6dac28377..ab7429c50 100644
--- a/src/box/index_def.h
+++ b/src/box/index_def.h
@@ -163,6 +163,10 @@ struct index_opts {
* filled after running ANALYZE command.
*/
struct index_stat *stat;
+ /** Identifier of the functional index extractor. */
+ uint32_t functional_fid;
+ /** Whether functional index extractor is multikey. */
+ bool is_multikey;
};
extern const struct index_opts index_opts_default;
@@ -207,6 +211,10 @@ index_opts_cmp(const struct index_opts *o1, const struct index_opts *o2)
return o1->run_size_ratio < o2->run_size_ratio ? -1 : 1;
if (o1->bloom_fpr != o2->bloom_fpr)
return o1->bloom_fpr < o2->bloom_fpr ? -1 : 1;
+ if (o1->functional_fid != o2->functional_fid)
+ return o1->functional_fid - o2->functional_fid;
+ if (o1->is_multikey != o2->is_multikey)
+ return o1->is_multikey - o2->is_multikey;
return 0;
}
@@ -297,7 +305,6 @@ index_def_update_optionality(struct index_def *def, uint32_t min_field_count)
key_def_update_optionality(def->key_def, min_field_count);
key_def_update_optionality(def->cmp_def, min_field_count);
}
-
/**
* Add an index definition to a list, preserving the
* first position of the primary key.
diff --git a/src/box/key_def.c b/src/box/key_def.c
index ee758eefa..bfbca6b30 100644
--- a/src/box/key_def.c
+++ b/src/box/key_def.c
@@ -241,7 +241,8 @@ key_def_set_part(struct key_def *def, uint32_t part_no, uint32_t fieldno,
}
struct key_def *
-key_def_new(const struct key_part_def *parts, uint32_t part_count)
+key_def_new(const struct key_part_def *parts, uint32_t part_count,
+ bool is_functional, bool is_multikey)
{
size_t sz = 0;
for (uint32_t i = 0; i < part_count; i++)
@@ -255,7 +256,6 @@ key_def_new(const struct key_part_def *parts, uint32_t part_count)
def->part_count = part_count;
def->unique_part_count = part_count;
-
/* A pointer to the JSON paths data in the new key_def. */
char *path_pool = (char *)def + key_def_sizeof(part_count, 0);
for (uint32_t i = 0; i < part_count; i++) {
@@ -266,8 +266,7 @@ key_def_new(const struct key_part_def *parts, uint32_t part_count)
if (coll_id == NULL) {
diag_set(ClientError, ER_WRONG_INDEX_OPTIONS,
i + 1, "collation was not found by ID");
- key_def_delete(def);
- return NULL;
+ goto error;
}
coll = coll_id->coll;
}
@@ -276,14 +275,25 @@ key_def_new(const struct key_part_def *parts, uint32_t part_count)
part->nullable_action, coll, part->coll_id,
part->sort_order, part->path, path_len,
&path_pool, TUPLE_OFFSET_SLOT_NIL,
- 0) != 0) {
- key_def_delete(def);
- return NULL;
+ 0) != 0)
+ goto error;
+ }
+ if (is_functional) {
+ def->functional_part_count = part_count;
+ def->is_multikey = is_multikey;
+ if (!key_def_is_sequential(def) || parts->fieldno != 0 ||
+ def->has_json_paths) {
+ diag_set(ClientError, ER_WRONG_INDEX_OPTIONS, 0,
+ "invalid functional key definition");
+ goto error;
}
}
assert(path_pool == (char *)def + sz);
key_def_set_func(def);
return def;
+error:
+ key_def_delete(def);
+ return NULL;
}
int
@@ -677,6 +687,7 @@ key_def_find_by_fieldno(const struct key_def *key_def, uint32_t fieldno)
const struct key_part *
key_def_find(const struct key_def *key_def, const struct key_part *to_find)
{
+ assert(!key_def_is_functional(key_def));
const struct key_part *part = key_def->parts;
const struct key_part *end = part + key_def->part_count;
for (; part != end; part++) {
@@ -708,6 +719,9 @@ static bool
key_def_can_merge(const struct key_def *key_def,
const struct key_part *to_merge)
{
+ if (key_def_is_functional(key_def))
+ return true;
+
const struct key_part *part = key_def_find(key_def, to_merge);
if (part == NULL)
return true;
@@ -722,6 +736,7 @@ key_def_can_merge(const struct key_def *key_def,
struct key_def *
key_def_merge(const struct key_def *first, const struct key_def *second)
{
+ assert(!key_def_is_functional(second));
uint32_t new_part_count = first->part_count + second->part_count;
/*
* Find and remove part duplicates, i.e. parts counted
@@ -754,6 +769,8 @@ key_def_merge(const struct key_def *first, const struct key_def *second)
new_def->has_optional_parts = first->has_optional_parts ||
second->has_optional_parts;
new_def->is_multikey = first->is_multikey || second->is_multikey;
+ new_def->functional_part_count = first->functional_part_count;
+ new_def->functional_extractor = first->functional_extractor;
/* JSON paths data in the new key_def. */
char *path_pool = (char *)new_def + key_def_sizeof(new_part_count, 0);
@@ -826,7 +843,7 @@ key_def_find_pk_in_cmp_def(const struct key_def *cmp_def,
}
/* Finally, allocate the new key definition. */
- extracted_def = key_def_new(parts, pk_def->part_count);
+ extracted_def = key_def_new(parts, pk_def->part_count, false, false);
out:
region_truncate(region, region_svp);
return extracted_def;
diff --git a/src/box/key_def.h b/src/box/key_def.h
index df83d055c..d163d1380 100644
--- a/src/box/key_def.h
+++ b/src/box/key_def.h
@@ -192,6 +192,13 @@ struct key_def {
* unique_part_count == part count of a merged key_def.
*/
uint32_t unique_part_count;
+ /**
+ * Count of parts in functional index defintion.
+ * All functional_part_count key_part(s) of an
+ * initialized key def instance have func != NULL pointer.
+ * != 0 iff it is functional index definition.
+ */
+ uint32_t functional_part_count;
/** True, if at least one part can store NULL. */
bool is_nullable;
/** True if some key part has JSON path. */
@@ -205,6 +212,12 @@ struct key_def {
bool has_optional_parts;
/** Key fields mask. @sa column_mask.h for details. */
uint64_t column_mask;
+ /**
+ * The pointer to a functional index extractor.
+ * != NULL for initialized functional index key definition.
+ * Key definition object doesn't manage this reference.
+ */
+ struct functional_extractor *functional_extractor;
/**
* In case of the multikey index, a pointer to the
* JSON path string, the path to the root node of
@@ -330,7 +343,8 @@ key_def_sizeof(uint32_t part_count, uint32_t path_pool_size)
* and initialize its parts.
*/
struct key_def *
-key_def_new(const struct key_part_def *parts, uint32_t part_count);
+key_def_new(const struct key_part_def *parts, uint32_t part_count,
+ bool is_functional, bool is_multikey);
/**
* Dump part definitions of the given key def.
@@ -341,6 +355,16 @@ int
key_def_dump_parts(const struct key_def *def, struct key_part_def *parts,
struct region *region);
+/**
+ * Return true if a given key definition defines functional index
+ * key.
+ */
+static inline bool
+key_def_is_functional(const struct key_def *key_def)
+{
+ return key_def->functional_part_count > 0;
+}
+
/**
* Update 'has_optional_parts' of @a key_def with correspondence
* to @a min_field_count.
diff --git a/src/box/lua/key_def.c b/src/box/lua/key_def.c
index 041b5ec98..e572fec1b 100644
--- a/src/box/lua/key_def.c
+++ b/src/box/lua/key_def.c
@@ -445,7 +445,7 @@ lbox_key_def_new(struct lua_State *L)
lua_pop(L, 1);
}
- struct key_def *key_def = key_def_new(parts, part_count);
+ struct key_def *key_def = key_def_new(parts, part_count, false, false);
region_truncate(region, region_svp);
if (key_def == NULL)
return luaT_error(L);
diff --git a/src/box/lua/misc.cc b/src/box/lua/misc.cc
index 7b8b9dcad..74cff84fd 100644
--- a/src/box/lua/misc.cc
+++ b/src/box/lua/misc.cc
@@ -86,7 +86,7 @@ port_msgpack_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
struct port_msgpack *port = (struct port_msgpack *) base;
const char *args = port->data;
- uint32_t arg_count = mp_decode_array(&args);
+ uint32_t arg_count = port->is_single_tuple ? 1 : mp_decode_array(&args);
for (uint32_t i = 0; i < arg_count; i++)
luamp_decode(L, luaL_msgpack_default, &args);
}
diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua
index aadcd3fa9..20c96b6e5 100644
--- a/src/box/lua/schema.lua
+++ b/src/box/lua/schema.lua
@@ -515,6 +515,7 @@ box.schema.space.drop = function(space_id, space_name, opts)
local _space_sequence = box.space[box.schema.SPACE_SEQUENCE_ID]
local _fk_constraint = box.space[box.schema.FK_CONSTRAINT_ID]
local _ck_constraint = box.space[box.schema.CK_CONSTRAINT_ID]
+ local _func_index = box.space[box.schema.FUNC_INDEX_ID]
local sequence_tuple = _space_sequence:delete{space_id}
if sequence_tuple ~= nil and sequence_tuple.is_generated == true then
-- Delete automatically generated sequence.
@@ -529,6 +530,9 @@ box.schema.space.drop = function(space_id, space_name, opts)
for _, t in _ck_constraint.index.primary:pairs({space_id}) do
_ck_constraint:delete({space_id, t.name})
end
+ for _, t in _func_index.index.primary:pairs({space_id}) do
+ _func_index:delete({space_id, t.index_id, t.func_id})
+ end
local keys = _vindex:select(space_id)
for i = #keys, 1, -1 do
local v = keys[i]
@@ -956,6 +960,8 @@ local index_options = {
range_size = 'number',
page_size = 'number',
bloom_fpr = 'number',
+ functional_fid = 'number',
+ is_multikey = 'boolean',
}
--
@@ -1061,6 +1067,8 @@ box.schema.index.create = function(space_id, name, options)
run_count_per_level = options.run_count_per_level,
run_size_ratio = options.run_size_ratio,
bloom_fpr = options.bloom_fpr,
+ functional_fid = options.functional_fid,
+ is_multikey = options.is_multikey,
}
local field_type_aliases = {
num = 'unsigned'; -- Deprecated since 1.7.2
@@ -1086,6 +1094,10 @@ box.schema.index.create = function(space_id, name, options)
space.name, name)
_index:insert{space_id, iid, name, options.type, index_opts, parts}
space_sequence_alter_commit(sequence_proxy)
+ if (options.functional_fid ~= nil and options.functional_fid > 0) then
+ local _func_index = box.space[box.schema.FUNC_INDEX_ID]
+ _func_index:insert{space_id, iid, options.functional_fid}
+ end
return space.index[name]
end
@@ -1101,6 +1113,10 @@ box.schema.index.drop = function(space_id, index_id)
end
end
local _index = box.space[box.schema.INDEX_ID]
+ local _func_index = box.space[box.schema.FUNC_INDEX_ID]
+ for _, v in box.space._func_index:pairs{space_id, index_id} do
+ _func_index:delete(v)
+ end
_index:delete{space_id, index_id}
end
@@ -1202,6 +1218,10 @@ box.schema.index.alter = function(space_id, index_id, options)
space.name, options.name)
_index:replace{space_id, index_id, options.name, options.type,
index_opts, parts}
+ if (options.functional_fid ~= nil and options.functional_fid > 0) then
+ local _func_index = box.space[box.schema.FUNC_INDEX_ID]
+ _func_index:insert{space_id, iid, options.functional_fid}
+ end
space_sequence_alter_commit(sequence_proxy)
end
diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc
index 18039fd6a..c13ffe2a4 100644
--- a/src/box/lua/space.cc
+++ b/src/box/lua/space.cc
@@ -335,6 +335,19 @@ lbox_fillspace(struct lua_State *L, struct space *space, int i)
lua_setfield(L, -2, "dimension");
}
+ if (index_opts->functional_fid > 0) {
+ lua_pushstring(L, "functional");
+ lua_newtable(L);
+
+ lua_pushnumber(L, index_opts->functional_fid);
+ lua_setfield(L, -2, "fid");
+
+ lua_pushboolean(L, index_opts->is_multikey);
+ lua_setfield(L, -2, "is_multikey");
+
+ lua_settable(L, -3);
+ }
+
lua_pushstring(L, index_type_strs[index_def->type]);
lua_setfield(L, -2, "type");
@@ -629,6 +642,8 @@ box_lua_space_init(struct lua_State *L)
lua_setfield(L, -2, "VSEQUENCE_ID");
lua_pushnumber(L, BOX_SPACE_SEQUENCE_ID);
lua_setfield(L, -2, "SPACE_SEQUENCE_ID");
+ lua_pushnumber(L, BOX_FUNC_INDEX_ID);
+ lua_setfield(L, -2, "FUNC_INDEX_ID");
lua_pushnumber(L, BOX_SYSTEM_ID_MIN);
lua_setfield(L, -2, "SYSTEM_ID_MIN");
lua_pushnumber(L, BOX_SYSTEM_ID_MAX);
diff --git a/src/box/lua/upgrade.lua b/src/box/lua/upgrade.lua
index 02c1cb0ff..f570a1c08 100644
--- a/src/box/lua/upgrade.lua
+++ b/src/box/lua/upgrade.lua
@@ -885,11 +885,29 @@ local function upgrade_func_to_2_2_1()
collation = 'unicode_ci'}}})
end
+local function create_func_index()
+ log.info("Create _func_index space")
+ local _func_index = box.space[box.schema.FUNC_INDEX_ID]
+ local _space = box.space._space
+ local _index = box.space._index
+ local format = {{name='space_id', type='unsigned'},
+ {name='index_id', type='unsigned'},
+ {name='func_id', type='unsigned'}}
+ _space:insert{_func_index.id, ADMIN, '_func_index', 'memtx', 0,
+ setmap({}), format}
+ _index:insert{_func_index.id, 0, 'primary', 'tree', {unique = true},
+ {{0, 'unsigned'}, {1, 'unsigned'}, {2, 'unsigned'}}}
+ _index:insert{_func_index.id, 1, 'fid', 'tree', {unique = false},
+ {{2, 'unsigned'}}}
+
+end
+
local function upgrade_to_2_2_1()
upgrade_sequence_to_2_2_1()
upgrade_ck_constraint_to_2_2_1()
create_vcollation_space()
upgrade_func_to_2_2_1()
+ create_func_index()
end
--------------------------------------------------------------------------------
diff --git a/src/box/memtx_engine.c b/src/box/memtx_engine.c
index 49efcc192..883c79ff7 100644
--- a/src/box/memtx_engine.c
+++ b/src/box/memtx_engine.c
@@ -1250,6 +1250,10 @@ memtx_index_def_change_requires_rebuild(struct index *index,
return true;
if (!old_def->opts.is_unique && new_def->opts.is_unique)
return true;
+ if (old_def->opts.is_multikey != new_def->opts.is_multikey)
+ return true;
+ if (old_def->opts.functional_fid != new_def->opts.functional_fid)
+ return true;
const struct key_def *old_cmp_def, *new_cmp_def;
if (index_depends_on_pk(index)) {
diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c
index a8e7b8080..216ef3912 100644
--- a/src/box/memtx_space.c
+++ b/src/box/memtx_space.c
@@ -659,6 +659,12 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"HASH index cannot be multikey");
return -1;
}
+ if (key_def_is_functional(index_def->key_def)) {
+ diag_set(ClientError, ER_MODIFY_INDEX,
+ index_def->name, space_name(space),
+ "HASH index cannot be functional");
+ return -1;
+ }
break;
case TREE:
/* TREE index has no limitations. */
@@ -688,6 +694,12 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"RTREE index cannot be multikey");
return -1;
}
+ if (key_def_is_functional(index_def->key_def)) {
+ diag_set(ClientError, ER_MODIFY_INDEX,
+ index_def->name, space_name(space),
+ "RTREE index cannot be functional");
+ return -1;
+ }
/* no furter checks of parts needed */
return 0;
case BITSET:
@@ -716,6 +728,12 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
"BITSET index cannot be multikey");
return -1;
}
+ if (key_def_is_functional(index_def->key_def)) {
+ diag_set(ClientError, ER_MODIFY_INDEX,
+ index_def->name, space_name(space),
+ "BITSET index cannot be functional");
+ return -1;
+ }
/* no furter checks of parts needed */
return 0;
default:
diff --git a/src/box/memtx_tree.c b/src/box/memtx_tree.c
index 3edf94044..5606bdda9 100644
--- a/src/box/memtx_tree.c
+++ b/src/box/memtx_tree.c
@@ -1017,6 +1017,36 @@ static const struct index_vtab memtx_tree_index_multikey_vtab = {
/* .end_build = */ memtx_tree_index_end_build,
};
+static const struct index_vtab memtx_tree_index_disabled_vtab = {
+ /* .destroy = */ memtx_tree_index_destroy,
+ /* .commit_create = */ generic_index_commit_create,
+ /* .abort_create = */ generic_index_abort_create,
+ /* .commit_modify = */ generic_index_commit_modify,
+ /* .commit_drop = */ generic_index_commit_drop,
+ /* .update_def = */ generic_index_update_def,
+ /* .depends_on_pk = */ generic_index_depends_on_pk,
+ /* .def_change_requires_rebuild = */
+ generic_index_def_change_requires_rebuild,
+ /* .size = */ generic_index_size,
+ /* .bsize = */ generic_index_bsize,
+ /* .min = */ generic_index_min,
+ /* .max = */ generic_index_max,
+ /* .random = */ generic_index_random,
+ /* .count = */ generic_index_count,
+ /* .get = */ generic_index_get,
+ /* .replace = */ disabled_index_replace,
+ /* .create_iterator = */ generic_index_create_iterator,
+ /* .create_snapshot_iterator = */
+ generic_index_create_snapshot_iterator,
+ /* .stat = */ generic_index_stat,
+ /* .compact = */ generic_index_compact,
+ /* .reset_stat = */ generic_index_reset_stat,
+ /* .begin_build = */ generic_index_begin_build,
+ /* .reserve = */ generic_index_reserve,
+ /* .build_next = */ disabled_index_build_next,
+ /* .end_build = */ generic_index_end_build,
+};
+
struct index *
memtx_tree_index_new(struct memtx_engine *memtx, struct index_def *def)
{
@@ -1027,9 +1057,14 @@ memtx_tree_index_new(struct memtx_engine *memtx, struct index_def *def)
"malloc", "struct memtx_tree_index");
return NULL;
}
- const struct index_vtab *vtab = def->key_def->is_multikey ?
- &memtx_tree_index_multikey_vtab :
- &memtx_tree_index_vtab;
+ const struct index_vtab *vtab;
+ if (key_def_is_functional(def->key_def) &&
+ def->key_def->functional_extractor == NULL)
+ vtab = &memtx_tree_index_disabled_vtab;
+ else if (def->key_def->is_multikey)
+ vtab = &memtx_tree_index_multikey_vtab;
+ else
+ vtab = &memtx_tree_index_vtab;
if (index_create(&index->base, (struct engine *)memtx,
vtab, def) != 0) {
free(index);
diff --git a/src/box/port.h b/src/box/port.h
index a7f5d81bd..2136f0ec2 100644
--- a/src/box/port.h
+++ b/src/box/port.h
@@ -86,6 +86,7 @@ struct port_msgpack {
const struct port_vtab *vtab;
const char *data;
uint32_t data_sz;
+ bool is_single_tuple;
};
static_assert(sizeof(struct port_msgpack) <= sizeof(struct port),
@@ -93,7 +94,8 @@ static_assert(sizeof(struct port_msgpack) <= sizeof(struct port),
/** Initialize a port to dump raw data. */
void
-port_msgpack_create(struct port *port, const char *data, uint32_t data_sz);
+port_msgpack_create(struct port *base, const char *data, uint32_t data_sz,
+ bool is_single_tuple);
/** Port for storing the result of a Lua CALL/EVAL. */
struct port_lua {
diff --git a/src/box/schema.cc b/src/box/schema.cc
index 20666386d..d5ccdd1b6 100644
--- a/src/box/schema.cc
+++ b/src/box/schema.cc
@@ -266,7 +266,8 @@ sc_space_new(uint32_t id, const char *name,
uint32_t key_part_count,
struct trigger *replace_trigger)
{
- struct key_def *key_def = key_def_new(key_parts, key_part_count);
+ struct key_def *key_def = key_def_new(key_parts, key_part_count,
+ false, false);
if (key_def == NULL)
diag_raise();
auto key_def_guard =
@@ -462,6 +463,16 @@ schema_init()
sc_space_new(BOX_CK_CONSTRAINT_ID, "_ck_constraint", key_parts, 2,
&on_replace_ck_constraint);
+ /* _func_index - check constraints. */
+ key_parts[0].fieldno = 0; /* space id */
+ key_parts[0].type = FIELD_TYPE_UNSIGNED;
+ key_parts[1].fieldno = 1; /* index id */
+ key_parts[1].type = FIELD_TYPE_UNSIGNED;
+ key_parts[2].fieldno = 2; /* function id */
+ key_parts[2].type = FIELD_TYPE_UNSIGNED;
+ sc_space_new(BOX_FUNC_INDEX_ID, "_func_index", key_parts, 3,
+ &on_replace_func_index);
+
/*
* _vinyl_deferred_delete - blackhole that is needed
* for writing deferred DELETE statements generated by
diff --git a/src/box/schema_def.h b/src/box/schema_def.h
index a97b6d531..85f652d52 100644
--- a/src/box/schema_def.h
+++ b/src/box/schema_def.h
@@ -112,6 +112,8 @@ enum {
BOX_FK_CONSTRAINT_ID = 356,
/** Space id of _ck_contraint. */
BOX_CK_CONSTRAINT_ID = 364,
+ /** Space id of _func_index. */
+ BOX_FUNC_INDEX_ID = 372,
/** End of the reserved range of system spaces. */
BOX_SYSTEM_ID_MAX = 511,
BOX_ID_NIL = 2147483647
@@ -267,6 +269,13 @@ enum {
BOX_CK_CONSTRAINT_FIELD_CODE = 4,
};
+/** _func_index fields. */
+enum {
+ BOX_FUNC_INDEX_FIELD_SPACE_ID = 0,
+ BOX_FUNC_INDEX_FIELD_INDEX_ID = 1,
+ BOX_FUNC_INDEX_FUNCTION_ID = 2,
+};
+
/*
* Different objects which can be subject to access
* control.
diff --git a/src/box/sql.c b/src/box/sql.c
index 4c9a4c15b..ba5bcbf10 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -347,7 +347,8 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
}
}
struct key_def *ephemer_key_def = key_def_new(ephemer_key_parts,
- field_count);
+ field_count, false,
+ false);
if (ephemer_key_def == NULL)
return NULL;
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 396de63fd..9fd9f1552 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -2307,7 +2307,7 @@ index_fill_def(struct Parse *parse, struct index *index,
part->coll_id = coll_id;
part->path = NULL;
}
- key_def = key_def_new(key_parts, expr_list->nExpr);
+ key_def = key_def_new(key_parts, expr_list->nExpr, false, false);
if (key_def == NULL)
goto tnt_error;
/*
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 7c8da251e..40d3394ea 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1438,7 +1438,8 @@ sql_key_info_to_key_def(struct sql_key_info *key_info)
{
if (key_info->key_def == NULL) {
key_info->key_def = key_def_new(key_info->parts,
- key_info->part_count);
+ key_info->part_count,
+ false, false);
}
return key_info->key_def;
}
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 5458c6a75..b85243a88 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -2775,7 +2775,7 @@ whereLoopAddBtree(WhereLoopBuilder * pBuilder, /* WHERE clause information */
part.coll_id = COLL_NONE;
part.path = NULL;
- struct key_def *key_def = key_def_new(&part, 1);
+ struct key_def *key_def = key_def_new(&part, 1, false, false);
if (key_def == NULL) {
tnt_error:
pWInfo->pParse->is_aborted = true;
diff --git a/src/box/tuple.c b/src/box/tuple.c
index ca0e5230a..80312da71 100644
--- a/src/box/tuple.c
+++ b/src/box/tuple.c
@@ -57,6 +57,9 @@ struct tuple *box_tuple_last;
struct tuple_format *tuple_format_runtime;
+functional_key_extractor box_functional_key_get = NULL;
+functional_key_extractor_raw box_functional_key_get_raw = NULL;
+
static void
runtime_tuple_delete(struct tuple_format *format, struct tuple *tuple);
@@ -122,7 +125,8 @@ runtime_tuple_delete(struct tuple_format *format, struct tuple *tuple)
int
tuple_validate_raw(struct tuple_format *format, const char *tuple)
{
- if (tuple_format_field_count(format) == 0)
+ if (tuple_format_field_count(format) == 0 &&
+ rlist_empty(&format->functional_handle))
return 0; /* Nothing to check */
struct region *region = &fiber()->gc;
@@ -130,7 +134,18 @@ tuple_validate_raw(struct tuple_format *format, const char *tuple)
struct field_map_builder builder;
int rc = tuple_field_map_create(format, tuple, true, &builder);
region_truncate(region, region_svp);
- return rc;
+ if (rc != 0 || rlist_empty(&format->functional_handle))
+ return rc;
+ struct functional_handle *handle;
+ const char *tuple_end = tuple;
+ mp_next(&tuple_end);
+ rlist_foreach_entry(handle, &format->functional_handle, link) {
+ uint32_t key_count;
+ if (box_functional_key_get_raw(tuple, tuple_end, handle->key_def,
+ &key_count) == NULL)
+ return -1;
+ }
+ return 0;
}
/**
@@ -288,7 +303,8 @@ tuple_unref_slow(struct tuple *tuple)
/* }}} Bigref */
int
-tuple_init(field_name_hash_f hash)
+tuple_init(field_name_hash_f hash, functional_key_extractor functional_key_get,
+ functional_key_extractor_raw functional_key_get_raw)
{
if (tuple_format_init() != 0)
return -1;
@@ -319,6 +335,9 @@ tuple_init(field_name_hash_f hash)
if (coll_id_cache_init() != 0)
return -1;
+ box_functional_key_get = functional_key_get;
+ box_functional_key_get_raw = functional_key_get_raw;
+
return 0;
}
diff --git a/src/box/tuple.h b/src/box/tuple.h
index 70119292e..907d561b7 100644
--- a/src/box/tuple.h
+++ b/src/box/tuple.h
@@ -45,6 +45,18 @@ extern "C" {
struct slab_arena;
struct quota;
struct key_part;
+struct tuple;
+
+typedef const char *
+(*functional_key_extractor)(struct tuple *tuple, struct key_def *key_def,
+ uint32_t *key_count, uint32_t **key_map);
+
+typedef const char *
+(*functional_key_extractor_raw)(const char *tuple, const char *tuple_end,
+ struct key_def *key_def, uint32_t *key_count);
+
+extern functional_key_extractor box_functional_key_get;
+extern functional_key_extractor_raw box_functional_key_get_raw;
/**
* A format for standalone tuples allocated on runtime arena.
@@ -54,7 +66,8 @@ extern struct tuple_format *tuple_format_runtime;
/** Initialize tuple library */
int
-tuple_init(field_name_hash_f hash);
+tuple_init(field_name_hash_f hash, functional_key_extractor functional_key_get,
+ functional_key_extractor_raw functional_key_get_raw);
/** Cleanup tuple library */
void
@@ -727,6 +740,40 @@ tuple_field_by_part(struct tuple *tuple, struct key_part *part,
multikey_idx);
}
+/**
+ * Get a field refereed by index @part of functional key
+ * definition in tuple.
+ * @param tuple Tuple to get the field from.
+ * @param part Index part to use.
+ * @param multikey_idx A multikey index hint.
+ * @retval Field data if the field exists or NULL.
+ */
+static inline const char *
+tuple_field_by_functional_def(struct tuple *tuple, struct key_part *part,
+ struct key_def *key_def, int multikey_idx)
+{
+ assert(key_def_is_functional(key_def));
+ uint32_t key_count, *key_map;
+ const char *key = box_functional_key_get(tuple, key_def, &key_count,
+ &key_map);
+ if (unlikely(key == NULL)) {
+ panic_syserror("Functional index runtime exception: %s",
+ diag_last_error(diag_get())->errmsg);
+ }
+ if (multikey_idx != MULTIKEY_NONE && multikey_idx > 0) {
+ if (unlikely((uint32_t) multikey_idx > key_count))
+ return NULL;
+ key += key_map[multikey_idx];
+ }
+ assert(mp_typeof(*key) == MP_ARRAY);
+ uint32_t part_count = mp_decode_array(&key);
+ if (unlikely(part->fieldno >= part_count))
+ return NULL;
+ for (uint32_t k = 0; k < part->fieldno; k++)
+ mp_next(&key);
+ return key;
+}
+
/**
* Get count of multikey index keys in tuple by given multikey
* index definition.
@@ -750,8 +797,21 @@ tuple_raw_multikey_count(struct tuple_format *format, const char *data,
static inline uint32_t
tuple_multikey_count(struct tuple *tuple, struct key_def *key_def)
{
- return tuple_raw_multikey_count(tuple_format(tuple), tuple_data(tuple),
+ if (key_def_is_functional(key_def)) {
+ assert(key_def->is_multikey);
+ uint32_t key_count, *key_map;
+ const char *key = box_functional_key_get(tuple, key_def,
+ &key_count, &key_map);
+ if (unlikely(key == NULL)) {
+ panic_syserror("Functional index runtime exception: %s",
+ diag_last_error(diag_get())->errmsg);
+ }
+ return key_count;
+ } else {
+ return tuple_raw_multikey_count(tuple_format(tuple),
+ tuple_data(tuple),
tuple_field_map(tuple), key_def);
+ }
}
/**
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index c03b584d0..8ec8a2fd4 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -450,13 +450,15 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
}
template<bool is_nullable, bool has_optional_parts, bool has_json_paths,
- bool is_multikey>
+ bool is_multikey, bool is_functional>
static inline int
tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
struct tuple *tuple_b, hint_t tuple_b_hint,
struct key_def *key_def)
{
assert(has_json_paths == key_def->has_json_paths);
+ assert(is_functional == key_def_is_functional(key_def));
+ assert(!key_def_is_functional(key_def) || !has_optional_parts);
assert(!has_optional_parts || is_nullable);
assert(is_nullable == key_def->is_nullable);
assert(has_optional_parts == key_def->has_optional_parts);
@@ -470,7 +472,7 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
const char *tuple_a_raw = tuple_data(tuple_a);
const char *tuple_b_raw = tuple_data(tuple_b);
if (key_def->part_count == 1 && part->fieldno == 0 &&
- (!has_json_paths || part->path == NULL)) {
+ (!has_json_paths || part->path == NULL) && !is_functional) {
/*
* First field can not be optional - empty tuples
* can not exist.
@@ -498,7 +500,7 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
struct tuple_format *format_b = tuple_format(tuple_b);
const uint32_t *field_map_a = tuple_field_map(tuple_a);
const uint32_t *field_map_b = tuple_field_map(tuple_b);
- struct key_part *end;
+ struct key_part *end, *functional_end;
const char *field_a, *field_b;
enum mp_type a_type, b_type;
if (is_nullable)
@@ -506,21 +508,48 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
else
end = part + key_def->part_count;
+ functional_end = part + key_def->functional_part_count;
+ for (; is_functional && part < functional_end; part++) {
+ field_a = tuple_field_by_functional_def(tuple_a, part, key_def,
+ is_multikey ? (int)tuple_a_hint : MULTIKEY_NONE);
+ field_b = tuple_field_by_functional_def(tuple_b, part, key_def,
+ is_multikey ? (int)tuple_b_hint : MULTIKEY_NONE);
+ if (! is_nullable) {
+ rc = tuple_compare_field(field_a, field_b, part->type,
+ part->coll);
+ if (rc != 0)
+ return rc;
+ else
+ continue;
+ }
+ a_type = mp_typeof(*field_a);
+ b_type = mp_typeof(*field_b);
+ if (a_type == MP_NIL) {
+ if (b_type != MP_NIL)
+ return -1;
+ was_null_met = true;
+ } else if (b_type == MP_NIL) {
+ return 1;
+ } else {
+ rc = tuple_compare_field_with_type(field_a, a_type,
+ field_b, b_type,
+ part->type,
+ part->coll);
+ if (rc != 0)
+ return rc;
+ }
+ }
+
for (; part < end; part++) {
- if (is_multikey) {
+ if (has_json_paths) {
field_a = tuple_field_raw_by_part(format_a, tuple_a_raw,
- field_map_a, part,
- (int)tuple_a_hint);
+ field_map_a, part,
+ is_multikey ?
+ (int)tuple_a_hint : MULTIKEY_NONE);
field_b = tuple_field_raw_by_part(format_b, tuple_b_raw,
- field_map_b, part,
- (int)tuple_b_hint);
- } else if (has_json_paths) {
- field_a = tuple_field_raw_by_part(format_a, tuple_a_raw,
- field_map_a, part,
- MULTIKEY_NONE);
- field_b = tuple_field_raw_by_part(format_b, tuple_b_raw,
- field_map_b, part,
- MULTIKEY_NONE);
+ field_map_b, part,
+ is_multikey ?
+ (int)tuple_b_hint : MULTIKEY_NONE);
} else {
field_a = tuple_field_raw(format_a, tuple_a_raw,
field_map_a, part->fieldno);
@@ -573,20 +602,15 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
*/
end = key_def->parts + key_def->part_count;
for (; part < end; ++part) {
- if (is_multikey) {
- field_a = tuple_field_raw_by_part(format_a, tuple_a_raw,
- field_map_a, part,
- (int)tuple_a_hint);
- field_b = tuple_field_raw_by_part(format_b, tuple_b_raw,
- field_map_b, part,
- (int)tuple_b_hint);
- } else if (has_json_paths) {
+ if (has_json_paths) {
field_a = tuple_field_raw_by_part(format_a, tuple_a_raw,
- field_map_a, part,
- MULTIKEY_NONE);
+ field_map_a, part,
+ is_multikey ?
+ (int)tuple_a_hint : MULTIKEY_NONE);
field_b = tuple_field_raw_by_part(format_b, tuple_b_raw,
- field_map_b, part,
- MULTIKEY_NONE);
+ field_map_b, part,
+ is_multikey ?
+ (int)tuple_b_hint : MULTIKEY_NONE);
} else {
field_a = tuple_field_raw(format_a, tuple_a_raw,
field_map_a, part->fieldno);
@@ -607,13 +631,14 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
}
template<bool is_nullable, bool has_optional_parts, bool has_json_paths,
- bool is_multikey>
+ bool is_multikey, bool is_functional>
static inline int
tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
const char *key, uint32_t part_count,
hint_t key_hint, struct key_def *key_def)
{
assert(has_json_paths == key_def->has_json_paths);
+ assert(is_functional == key_def_is_functional(key_def));
assert(!has_optional_parts || is_nullable);
assert(is_nullable == key_def->is_nullable);
assert(has_optional_parts == key_def->has_optional_parts);
@@ -632,14 +657,15 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
enum mp_type a_type, b_type;
if (likely(part_count == 1)) {
const char *field;
- if (is_multikey) {
- field = tuple_field_raw_by_part(format, tuple_raw,
- field_map, part,
- (int)tuple_hint);
+ if (is_functional) {
+ field = tuple_field_by_functional_def(tuple, part,
+ key_def, is_multikey ?
+ (int)tuple_hint : MULTIKEY_NONE);
} else if (has_json_paths) {
field = tuple_field_raw_by_part(format, tuple_raw,
- field_map, part,
- MULTIKEY_NONE);
+ field_map, part,
+ is_multikey ?
+ (int)tuple_hint : MULTIKEY_NONE);
} else {
field = tuple_field_raw(format, tuple_raw, field_map,
part->fieldno);
@@ -665,16 +691,42 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
}
struct key_part *end = part + part_count;
+ struct key_part *functional_end =
+ part + MIN(part_count, key_def->functional_part_count);
+ for (; is_functional && part < functional_end; part++) {
+ const char *field = tuple_field_by_functional_def(tuple, part,
+ key_def, is_multikey ?
+ (int)tuple_hint : MULTIKEY_NONE);
+ if (! is_nullable) {
+ rc = tuple_compare_field(field, key, part->type,
+ part->coll);
+ if (rc != 0)
+ return rc;
+ else
+ continue;
+ }
+ a_type = mp_typeof(*field);
+ b_type = mp_typeof(*key);
+ if (a_type == MP_NIL) {
+ if (b_type != MP_NIL)
+ return -1;
+ } else if (b_type == MP_NIL) {
+ return 1;
+ } else {
+ rc = tuple_compare_field_with_type(field, a_type, key,
+ b_type, part->type,
+ part->coll);
+ if (rc != 0)
+ return rc;
+ }
+ }
for (; part < end; ++part, mp_next(&key)) {
const char *field;
- if (is_multikey) {
- field = tuple_field_raw_by_part(format, tuple_raw,
- field_map, part,
- (int)tuple_hint);
- } else if (has_json_paths) {
+ if (has_json_paths) {
field = tuple_field_raw_by_part(format, tuple_raw,
- field_map, part,
- MULTIKEY_NONE);
+ field_map, part,
+ is_multikey ?
+ (int)tuple_hint : MULTIKEY_NONE);
} else {
field = tuple_field_raw(format, tuple_raw, field_map,
part->fieldno);
@@ -1579,13 +1631,19 @@ key_hint(const char *key, uint32_t part_count, struct key_def *key_def)
return field_hint<type, is_nullable>(key, key_def->parts->coll);
}
-template <enum field_type type, bool is_nullable>
+template <enum field_type type, bool is_nullable, bool is_functional>
static hint_t
tuple_hint(struct tuple *tuple, struct key_def *key_def)
{
assert(!key_def->is_multikey);
- const char *field = tuple_field_by_part(tuple, key_def->parts,
- MULTIKEY_NONE);
+ const char *field;
+ if (is_functional) {
+ field = tuple_field_by_functional_def(tuple, key_def->parts,
+ key_def, MULTIKEY_NONE);
+ } else {
+ field = tuple_field_by_part(tuple, key_def->parts,
+ MULTIKEY_NONE);
+ }
if (is_nullable && field == NULL)
return hint_nil();
return field_hint<type, is_nullable>(field, key_def->parts->coll);
@@ -1625,7 +1683,10 @@ static void
key_def_set_hint_func(struct key_def *def)
{
def->key_hint = key_hint<type, is_nullable>;
- def->tuple_hint = tuple_hint<type, is_nullable>;
+ if (key_def_is_functional(def))
+ def->tuple_hint = tuple_hint<type, is_nullable, true>;
+ else
+ def->tuple_hint = tuple_hint<type, is_nullable, false>;
}
template<enum field_type type>
@@ -1720,13 +1781,14 @@ key_def_set_compare_func_fast(struct key_def *def)
if (cmp == NULL) {
cmp = is_sequential ?
tuple_compare_sequential<false, false> :
- tuple_compare_slowpath<false, false, false, false>;
+ tuple_compare_slowpath<false, false, false,
+ false, false>;
}
if (cmp_wk == NULL) {
cmp_wk = is_sequential ?
tuple_compare_with_key_sequential<false, false> :
tuple_compare_with_key_slowpath<false, false,
- false, false>;
+ false, false, false>;
}
def->tuple_compare = cmp;
@@ -1745,9 +1807,11 @@ key_def_set_compare_func_plain(struct key_def *def)
<is_nullable, has_optional_parts>;
} else {
def->tuple_compare = tuple_compare_slowpath
- <is_nullable, has_optional_parts, false, false>;
+ <is_nullable, has_optional_parts, false,
+ false, false>;
def->tuple_compare_with_key = tuple_compare_with_key_slowpath
- <is_nullable, has_optional_parts, false, false>;
+ <is_nullable, has_optional_parts, false,
+ false, false>;
}
}
@@ -1758,14 +1822,36 @@ key_def_set_compare_func_json(struct key_def *def)
assert(def->has_json_paths);
if (def->is_multikey) {
def->tuple_compare = tuple_compare_slowpath
- <is_nullable, has_optional_parts, true, true>;
+ <is_nullable, has_optional_parts, true,
+ true, false>;
+ def->tuple_compare_with_key = tuple_compare_with_key_slowpath
+ <is_nullable, has_optional_parts, true,
+ true, false>;
+ } else {
+ def->tuple_compare = tuple_compare_slowpath
+ <is_nullable, has_optional_parts, true,
+ false, false>;
+ def->tuple_compare_with_key = tuple_compare_with_key_slowpath
+ <is_nullable, has_optional_parts, true,
+ false, false>;
+ }
+}
+
+template<bool is_nullable>
+static void
+key_def_set_compare_func_functional(struct key_def *def)
+{
+ assert(key_def_is_functional(def));
+ if (def->is_multikey) {
+ def->tuple_compare = tuple_compare_slowpath
+ <is_nullable, false, false, true, true>;
def->tuple_compare_with_key = tuple_compare_with_key_slowpath
- <is_nullable, has_optional_parts, true, true>;
+ <is_nullable, false, false, true, true>;
} else {
def->tuple_compare = tuple_compare_slowpath
- <is_nullable, has_optional_parts, true, false>;
+ <is_nullable, false, false, false, true>;
def->tuple_compare_with_key = tuple_compare_with_key_slowpath
- <is_nullable, has_optional_parts, true, false>;
+ <is_nullable, false, false, false, true>;
}
}
@@ -1773,9 +1859,10 @@ void
key_def_set_compare_func(struct key_def *def)
{
if (!key_def_has_collation(def) &&
- !def->is_nullable && !def->has_json_paths) {
+ !def->is_nullable && !def->has_json_paths &&
+ !key_def_is_functional(def)) {
key_def_set_compare_func_fast(def);
- } else if (!def->has_json_paths) {
+ } else if (!def->has_json_paths && !key_def_is_functional(def)) {
if (def->is_nullable && def->has_optional_parts) {
key_def_set_compare_func_plain<true, true>(def);
} else if (def->is_nullable && !def->has_optional_parts) {
@@ -1784,7 +1871,16 @@ key_def_set_compare_func(struct key_def *def)
assert(!def->is_nullable && !def->has_optional_parts);
key_def_set_compare_func_plain<false, false>(def);
}
+ } else if (key_def_is_functional(def)) {
+ assert(!def->has_json_paths);
+ assert(!def->has_optional_parts);
+ if (def->is_nullable) {
+ key_def_set_compare_func_functional<true>(def);
+ } else {
+ key_def_set_compare_func_functional<false>(def);
+ }
} else {
+ assert(def->has_json_paths);
if (def->is_nullable && def->has_optional_parts) {
key_def_set_compare_func_json<true, true>(def);
} else if (def->is_nullable && !def->has_optional_parts) {
--git a/src/box/tuple_extract_key.cc b/src/box/tuple_extract_key.cc
index 471c7df80..21e04fd8c 100644
--- a/src/box/tuple_extract_key.cc
+++ b/src/box/tuple_extract_key.cc
@@ -120,6 +120,7 @@ tuple_extract_key_slowpath(struct tuple *tuple, struct key_def *key_def,
key_def_contains_sequential_parts(key_def));
assert(is_multikey == key_def->is_multikey);
assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
+ assert(!key_def_is_functional(key_def));
assert(mp_sizeof_nil() == 1);
const char *data = tuple_data(tuple);
uint32_t part_count = key_def->part_count;
@@ -251,6 +252,7 @@ tuple_extract_key_slowpath_raw(const char *data, const char *data_end,
assert(!has_optional_parts || key_def->is_nullable);
assert(has_optional_parts == key_def->has_optional_parts);
assert(!key_def->is_multikey || multikey_idx != MULTIKEY_NONE);
+ assert(!key_def_is_functional(key_def));
assert(mp_sizeof_nil() == 1);
/* allocate buffer with maximal possible size */
char *key = (char *) region_alloc(&fiber()->gc, data_end - data);
@@ -367,6 +369,7 @@ key_def_set_extract_func_plain(struct key_def *def)
{
assert(!def->has_json_paths);
assert(!def->is_multikey);
+ assert(!key_def_is_functional(def));
if (key_def_is_sequential(def)) {
assert(contains_sequential_parts || def->part_count == 1);
def->tuple_extract_key = tuple_extract_key_sequential
@@ -387,6 +390,7 @@ static void
key_def_set_extract_func_json(struct key_def *def)
{
assert(def->has_json_paths);
+ assert(!key_def_is_functional(def));
if (def->is_multikey) {
def->tuple_extract_key = tuple_extract_key_slowpath
<contains_sequential_parts,
@@ -406,7 +410,10 @@ key_def_set_extract_func(struct key_def *key_def)
bool contains_sequential_parts =
key_def_contains_sequential_parts(key_def);
bool has_optional_parts = key_def->has_optional_parts;
- if (!key_def->has_json_paths) {
+ if (key_def_is_functional(key_def)) {
+ key_def->tuple_extract_key = NULL;
+ key_def->tuple_extract_key_raw = NULL;
+ } else if (!key_def->has_json_paths) {
if (!contains_sequential_parts && !has_optional_parts) {
key_def_set_extract_func_plain<false, false>(key_def);
} else if (!contains_sequential_parts && has_optional_parts) {
diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c
index 02fadf1cf..1ebafd83e 100644
--- a/src/box/tuple_format.c
+++ b/src/box/tuple_format.c
@@ -35,6 +35,7 @@
#include "coll_id_cache.h"
#include "tt_static.h"
+
#include "third_party/PMurHash.h"
/** Global table of tuple formats */
@@ -462,6 +463,12 @@ tuple_format_create(struct tuple_format *format, struct key_def * const *keys,
/* extract field type info */
for (uint16_t key_no = 0; key_no < key_count; ++key_no) {
const struct key_def *key_def = keys[key_no];
+ /*
+ * Functional key definitions are not the part
+ * of the space format.
+ */
+ if (key_def_is_functional(key_def))
+ continue;
bool is_sequential = key_def_is_sequential(key_def);
const struct key_part *part = key_def->parts;
const struct key_part *parts_end = part + key_def->part_count;
@@ -605,16 +612,52 @@ tuple_format_destroy_fields(struct tuple_format *format)
json_tree_destroy(&format->fields);
}
+static struct functional_handle *
+functional_handle_new(const struct key_def *key_def)
+{
+ struct functional_handle *handle = malloc(sizeof(*handle));
+ if (handle == NULL) {
+ diag_set(OutOfMemory, sizeof(*handle), "malloc", "handle");
+ return NULL;
+ }
+ handle->key_def = key_def_dup(key_def);
+ if (handle->key_def == NULL) {
+ free(handle);
+ return NULL;
+ }
+ rlist_create(&handle->link);
+ return handle;
+}
+
+static void
+functional_handle_delete(struct functional_handle *handle)
+{
+ key_def_delete(handle->key_def);
+ free(handle);
+}
+
static struct tuple_format *
tuple_format_alloc(struct key_def * const *keys, uint16_t key_count,
uint32_t space_field_count, struct tuple_dictionary *dict)
{
+ struct tuple_format *format = NULL;
+ struct rlist functional_handle;
+ rlist_create(&functional_handle);
/* Size of area to store JSON paths data. */
uint32_t path_pool_size = 0;
uint32_t index_field_count = 0;
/* find max max field no */
for (uint16_t key_no = 0; key_no < key_count; ++key_no) {
const struct key_def *key_def = keys[key_no];
+ if (key_def_is_functional(key_def) &&
+ key_def->functional_extractor != NULL) {
+ struct functional_handle *handle =
+ functional_handle_new(key_def);
+ if (handle == NULL)
+ goto error;
+ rlist_add(&functional_handle, &handle->link);
+ continue;
+ }
const struct key_part *part = key_def->parts;
const struct key_part *pend = part + key_def->part_count;
for (; part < pend; part++) {
@@ -626,7 +669,7 @@ tuple_format_alloc(struct key_def * const *keys, uint16_t key_count,
uint32_t field_count = MAX(space_field_count, index_field_count);
uint32_t allocation_size = sizeof(struct tuple_format) + path_pool_size;
- struct tuple_format *format = malloc(allocation_size);
+ format = malloc(allocation_size);
if (format == NULL) {
diag_set(OutOfMemory, allocation_size, "malloc",
"tuple format");
@@ -635,8 +678,7 @@ tuple_format_alloc(struct key_def * const *keys, uint16_t key_count,
if (json_tree_create(&format->fields) != 0) {
diag_set(OutOfMemory, 0, "json_lexer_create",
"tuple field tree");
- free(format);
- return NULL;
+ goto error;
}
for (uint32_t fieldno = 0; fieldno < field_count; fieldno++) {
struct tuple_field *field = tuple_field_new();
@@ -662,6 +704,8 @@ tuple_format_alloc(struct key_def * const *keys, uint16_t key_count,
format->dict = dict;
tuple_dictionary_ref(dict);
}
+ rlist_create(&format->functional_handle);
+ rlist_swap(&format->functional_handle, &functional_handle);
format->total_field_count = field_count;
format->required_fields = NULL;
format->fields_depth = 1;
@@ -672,7 +716,10 @@ tuple_format_alloc(struct key_def * const *keys, uint16_t key_count,
format->min_field_count = 0;
format->epoch = 0;
return format;
-error:
+error:;
+ struct functional_handle *handle, *tmp;
+ rlist_foreach_entry_safe(handle, &functional_handle, link, tmp)
+ functional_handle_delete(handle);
tuple_format_destroy_fields(format);
free(format);
return NULL;
@@ -682,6 +729,9 @@ error:
static inline void
tuple_format_destroy(struct tuple_format *format)
{
+ struct functional_handle *handle, *tmp;
+ rlist_foreach_entry_safe(handle, &format->functional_handle, link, tmp)
+ functional_handle_delete(handle);
free(format->required_fields);
tuple_format_destroy_fields(format);
tuple_dictionary_unref(format->dict);
diff --git a/src/box/tuple_format.h b/src/box/tuple_format.h
index ac4f8a702..99ae5f5d5 100644
--- a/src/box/tuple_format.h
+++ b/src/box/tuple_format.h
@@ -68,6 +68,21 @@ struct tuple_extra;
struct tuple_format;
struct coll;
+/**
+ * A functional handle is a tuple_format extention used when
+ * functional index key is defined. The functional handle
+ * represents a single functional index reference and allows
+ * to perform a validation of the functional index extracted key.
+ */
+struct functional_handle {
+ struct rlist link;
+ /**
+ * The key definition that describe keys produced by
+ * functional index extractor.
+ */
+ struct key_def *key_def;
+};
+
/** Engine-specific tuple format methods. */
struct tuple_format_vtab {
/**
@@ -233,6 +248,8 @@ struct tuple_format {
* A maximum depth of format::fields subtree.
*/
uint32_t fields_depth;
+ /** A functional index handle list. */
+ struct rlist functional_handle;
/**
* Fields comprising the format, organized in a tree.
* First level nodes correspond to tuple fields.
diff --git a/src/box/tuple_hash.cc b/src/box/tuple_hash.cc
index 780e3d053..dad87871a 100644
--- a/src/box/tuple_hash.cc
+++ b/src/box/tuple_hash.cc
@@ -365,6 +365,7 @@ tuple_hash_slowpath(struct tuple *tuple, struct key_def *key_def)
assert(has_json_paths == key_def->has_json_paths);
assert(has_optional_parts == key_def->has_optional_parts);
assert(!key_def->is_multikey);
+ assert(!key_def_is_functional(key_def));
uint32_t h = HASH_SEED;
uint32_t carry = 0;
uint32_t total_size = 0;
diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index cf7af26b7..9d4912230 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -701,6 +701,11 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def)
return -1;
}
}
+ if (key_def_is_functional(index_def->key_def)) {
+ diag_set(ClientError, ER_UNSUPPORTED, "Vinyl",
+ "functional indexes");
+ return -1;
+ }
return 0;
}
@@ -986,6 +991,10 @@ vinyl_index_def_change_requires_rebuild(struct index *index,
if (!old_def->opts.is_unique && new_def->opts.is_unique)
return true;
+ if (old_def->opts.is_multikey != new_def->opts.is_multikey)
+ return true;
+ if (old_def->opts.functional_fid != new_def->opts.functional_fid)
+ return true;
assert(index_depends_on_pk(index));
const struct key_def *old_cmp_def = old_def->cmp_def;
@@ -3165,7 +3174,7 @@ vy_send_lsm(struct vy_join_ctx *ctx, struct vy_lsm_recovery_info *lsm_info)
/* Create key definition and tuple format. */
ctx->key_def = key_def_new(lsm_info->key_parts,
- lsm_info->key_part_count);
+ lsm_info->key_part_count, false, false);
if (ctx->key_def == NULL)
goto out;
ctx->format = vy_stmt_format_new(&ctx->env->stmt_env, &ctx->key_def, 1,
diff --git a/test/app-tap/tarantoolctl.test.lua b/test/app-tap/tarantoolctl.test.lua
index 957b883f4..df2ee377f 100755
--- a/test/app-tap/tarantoolctl.test.lua
+++ b/test/app-tap/tarantoolctl.test.lua
@@ -405,8 +405,8 @@ do
check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1", "\n", 3)
check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1 --replica 2", "\n", 3)
check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 2", "\n", 0)
- check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 23)
- check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 50)
+ check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 24)
+ check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 52)
end)
end)
diff --git a/test/box-py/bootstrap.result b/test/box-py/bootstrap.result
index f2d1f46fb..a5d645df8 100644
--- a/test/box-py/bootstrap.result
+++ b/test/box-py/bootstrap.result
@@ -93,6 +93,8 @@ box.space._space:select{}
- [364, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'},
{'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'},
{'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]]
+ - [372, 1, '_func_index', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'},
+ {'name': 'index_id', 'type': 'unsigned'}, {'name': 'func_id', 'type': 'unsigned'}]]
...
box.space._index:select{}
---
@@ -148,6 +150,9 @@ box.space._index:select{}
- [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]]
- [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]]
- [364, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]]
+ - [372, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'unsigned'],
+ [2, 'unsigned']]]
+ - [372, 1, 'fid', 'tree', {'unique': false}, [[2, 'unsigned']]]
...
box.space._user:select{}
---
diff --git a/test/box/access.result b/test/box/access.result
index 5ee92a443..ba72b5f74 100644
--- a/test/box/access.result
+++ b/test/box/access.result
@@ -1496,6 +1496,9 @@ box.schema.user.grant('tester', 'read', 'space', '_fk_constraint')
box.schema.user.grant('tester', 'read', 'space', '_ck_constraint')
---
...
+box.schema.user.grant('tester', 'read', 'space', '_func_index')
+---
+...
box.session.su("tester")
---
...
diff --git a/test/box/access.test.lua b/test/box/access.test.lua
index 79340b0f5..219cdb04a 100644
--- a/test/box/access.test.lua
+++ b/test/box/access.test.lua
@@ -557,6 +557,7 @@ box.schema.user.grant('tester', 'read', 'space', '_space_sequence')
box.schema.user.grant('tester', 'read', 'space', '_trigger')
box.schema.user.grant('tester', 'read', 'space', '_fk_constraint')
box.schema.user.grant('tester', 'read', 'space', '_ck_constraint')
+box.schema.user.grant('tester', 'read', 'space', '_func_index')
box.session.su("tester")
-- successful create
s1 = box.schema.space.create("test_space")
diff --git a/test/box/access_misc.result b/test/box/access_misc.result
index c69cf0283..31b935914 100644
--- a/test/box/access_misc.result
+++ b/test/box/access_misc.result
@@ -833,140 +833,142 @@ box.space._space:select()
- [364, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'},
{'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'},
{'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]]
+ - [372, 1, '_func_index', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'},
+ {'name': 'index_id', 'type': 'unsigned'}, {'name': 'func_id', 'type': 'unsigned'}]]
...
box.space._func:select()
---
- - [1, 1, 'box.schema.user.info', 1, 'LUA', '', 'function', [], 'any', 'none', 'none',
- false, false, true, ['LUA'], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, ['LUA'], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [2, 1, 'TRIM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [3, 1, 'TYPEOF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [4, 1, 'PRINTF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [5, 1, 'UNICODE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [6, 1, 'CHAR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [7, 1, 'HEX', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [8, 1, 'VERSION', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [9, 1, 'QUOTE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [10, 1, 'REPLACE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [11, 1, 'SUBSTR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [12, 1, 'GROUP_CONCAT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [13, 1, 'JULIANDAY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [14, 1, 'DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [15, 1, 'TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [16, 1, 'DATETIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [17, 1, 'STRFTIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [18, 1, 'CURRENT_TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [19, 1, 'CURRENT_TIMESTAMP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none',
- 'none', false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ 'none', false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [20, 1, 'CURRENT_DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [21, 1, 'LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [22, 1, 'POSITION', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [23, 1, 'ROUND', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [24, 1, 'UPPER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [25, 1, 'LOWER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [26, 1, 'IFNULL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [27, 1, 'RANDOM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [28, 1, 'CEIL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [29, 1, 'CEILING', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [30, 1, 'CHARACTER_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none',
- 'none', false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ 'none', false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [31, 1, 'CHAR_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [32, 1, 'FLOOR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [33, 1, 'MOD', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [34, 1, 'OCTET_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [35, 1, 'ROW_COUNT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [36, 1, 'COUNT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [37, 1, 'LIKE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [38, 1, 'ABS', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [39, 1, 'EXP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [40, 1, 'LN', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [41, 1, 'POWER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [42, 1, 'SQRT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [43, 1, 'SUM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [44, 1, 'TOTAL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [45, 1, 'AVG', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [46, 1, 'RANDOMBLOB', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [47, 1, 'NULLIF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [48, 1, 'ZEROBLOB', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [49, 1, 'MIN', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [50, 1, 'MAX', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [51, 1, 'COALESCE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [52, 1, 'EVERY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [53, 1, 'EXISTS', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [54, 1, 'EXTRACT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [55, 1, 'SOME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false,
- false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [56, 1, 'GREATER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [57, 1, 'LESSER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [58, 1, 'SOUNDEX', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [59, 1, 'LIKELIHOOD', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [60, 1, 'LIKELY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [61, 1, 'UNLIKELY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none',
- false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [62, 1, '_sql_stat_get', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none',
- 'none', false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ 'none', false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [63, 1, '_sql_stat_push', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none',
- 'none', false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ 'none', false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [64, 1, '_sql_stat_init', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none',
- 'none', false, false, true, [], {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ 'none', false, false, true, [], {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
- [65, 1, 'LUA', 1, 'LUA', 'function(code) return assert(loadstring(code))() end',
'function', ['string'], 'any', 'none', 'none', false, false, true, ['LUA', 'SQL'],
- {}, '', '2019-07-11 13:57:48', '2019-07-11 13:57:48']
+ {}, '', '2019-07-15 17:54:58', '2019-07-15 17:54:58']
...
session = nil
---
diff --git a/test/box/access_sysview.result b/test/box/access_sysview.result
index d65aa37ae..a82127ebb 100644
--- a/test/box/access_sysview.result
+++ b/test/box/access_sysview.result
@@ -246,11 +246,11 @@ box.session.su('guest')
...
#box.space._vspace:select{}
---
-- 24
+- 25
...
#box.space._vindex:select{}
---
-- 51
+- 53
...
#box.space._vuser:select{}
---
@@ -282,7 +282,7 @@ box.session.su('guest')
...
#box.space._vindex:select{}
---
-- 51
+- 53
...
#box.space._vuser:select{}
---
diff --git a/test/box/alter.result b/test/box/alter.result
index a6db011ff..91a239bbc 100644
--- a/test/box/alter.result
+++ b/test/box/alter.result
@@ -92,7 +92,7 @@ space = box.space[t[1]]
...
space.id
---
-- 365
+- 373
...
space.field_count
---
@@ -137,7 +137,7 @@ space_deleted
...
space:replace{0}
---
-- error: Space '365' does not exist
+- error: Space '373' does not exist
...
_index:insert{_space.id, 0, 'primary', 'tree', {unique=true}, {{0, 'unsigned'}}}
---
@@ -218,6 +218,9 @@ _index:select{}
- [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]]
- [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]]
- [364, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]]
+ - [372, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'unsigned'],
+ [2, 'unsigned']]]
+ - [372, 1, 'fid', 'tree', {'unique': false}, [[2, 'unsigned']]]
...
-- modify indexes of a system space
_index:delete{_index.id, 0}
diff --git a/test/box/bitset.result b/test/box/bitset.result
index 78f74ec37..dbaa7e74a 100644
--- a/test/box/bitset.result
+++ b/test/box/bitset.result
@@ -1996,3 +1996,27 @@ _ = s:create_index('bitset', {type = 'bitset', parts = {{'[2][*]', 'unsigned'}}}
s:drop()
---
...
+-- Bitset index cannot be functional.
+s = box.schema.space.create('withdata')
+---
+...
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+---
+...
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+---
+...
+_ = s:create_index('pk')
+---
+...
+_ = s:create_index('idx', {type = 'bitset', functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+---
+- error: 'Can''t create or modify index ''idx'' in space ''withdata'': BITSET index
+ cannot be functional'
+...
+s:drop()
+---
+...
+box.schema.func.drop('s')
+---
+...
diff --git a/test/box/bitset.test.lua b/test/box/bitset.test.lua
index eb013a1c0..025799aef 100644
--- a/test/box/bitset.test.lua
+++ b/test/box/bitset.test.lua
@@ -153,3 +153,12 @@ s = box.schema.space.create('test')
_ = s:create_index('primary')
_ = s:create_index('bitset', {type = 'bitset', parts = {{'[2][*]', 'unsigned'}}})
s:drop()
+
+-- Bitset index cannot be functional.
+s = box.schema.space.create('withdata')
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+_ = s:create_index('pk')
+_ = s:create_index('idx', {type = 'bitset', functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+s:drop()
+box.schema.func.drop('s')
diff --git a/test/box/hash.result b/test/box/hash.result
index 9f08c49b8..b2a303a52 100644
--- a/test/box/hash.result
+++ b/test/box/hash.result
@@ -847,3 +847,27 @@ _ = s:create_index('hash', {type = 'hash', parts = {{'[2][*]', 'unsigned'}}})
s:drop()
---
...
+-- Hash index cannot be functional.
+s = box.schema.space.create('withdata')
+---
+...
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+---
+...
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+---
+...
+_ = s:create_index('pk')
+---
+...
+_ = s:create_index('idx', {type = 'hash', functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+---
+- error: 'Can''t create or modify index ''idx'' in space ''withdata'': HASH index
+ cannot be functional'
+...
+s:drop()
+---
+...
+box.schema.func.drop('s')
+---
+...
diff --git a/test/box/hash.test.lua b/test/box/hash.test.lua
index 9801873c4..b35d58623 100644
--- a/test/box/hash.test.lua
+++ b/test/box/hash.test.lua
@@ -353,3 +353,12 @@ s = box.schema.space.create('test')
_ = s:create_index('primary')
_ = s:create_index('hash', {type = 'hash', parts = {{'[2][*]', 'unsigned'}}})
s:drop()
+
+-- Hash index cannot be functional.
+s = box.schema.space.create('withdata')
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+_ = s:create_index('pk')
+_ = s:create_index('idx', {type = 'hash', functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+s:drop()
+box.schema.func.drop('s')
diff --git a/test/box/misc.result b/test/box/misc.result
index 791730935..7449fdc84 100644
--- a/test/box/misc.result
+++ b/test/box/misc.result
@@ -526,6 +526,7 @@ t;
195: box.error.CREATE_CK_CONSTRAINT
196: box.error.CK_CONSTRAINT_FAILED
197: box.error.SQL_COLUMN_COUNT
+ 198: box.error.FUNCTIONAL_EXTRACTOR
...
test_run:cmd("setopt delimiter ''");
---
diff --git a/test/box/rtree_misc.result b/test/box/rtree_misc.result
index 6e48bacc7..5d0d54a6e 100644
--- a/test/box/rtree_misc.result
+++ b/test/box/rtree_misc.result
@@ -682,3 +682,27 @@ _ = s:create_index('rtree', {type = 'rtree', parts = {{'[2][*]', 'array'}}})
s:drop()
---
...
+-- Rtree index cannot be functional.
+s = box.schema.space.create('withdata')
+---
+...
+lua_code = [[function(tuple) return {tuple[1] + tuple[2], tuple[1] - tuple[2]} end]]
+---
+...
+box.schema.func.create('fextract', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+---
+...
+_ = s:create_index('pk')
+---
+...
+_ = s:create_index('idx', {type = 'rtree', functional_fid = box.func.fextract.id, parts = {{1, 'array'}}})
+---
+- error: 'Can''t create or modify index ''idx'' in space ''withdata'': RTREE index
+ cannot be functional'
+...
+s:drop()
+---
+...
+box.schema.func.drop('fextract')
+---
+...
diff --git a/test/box/rtree_misc.test.lua b/test/box/rtree_misc.test.lua
index 000a928e8..b95e43bb4 100644
--- a/test/box/rtree_misc.test.lua
+++ b/test/box/rtree_misc.test.lua
@@ -243,3 +243,12 @@ s = box.schema.space.create('test')
_ = s:create_index('primary')
_ = s:create_index('rtree', {type = 'rtree', parts = {{'[2][*]', 'array'}}})
s:drop()
+
+-- Rtree index cannot be functional.
+s = box.schema.space.create('withdata')
+lua_code = [[function(tuple) return {tuple[1] + tuple[2], tuple[1] - tuple[2]} end]]
+box.schema.func.create('fextract', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+_ = s:create_index('pk')
+_ = s:create_index('idx', {type = 'rtree', functional_fid = box.func.fextract.id, parts = {{1, 'array'}}})
+s:drop()
+box.schema.func.drop('fextract')
diff --git a/test/engine/engine.cfg b/test/engine/engine.cfg
index 9f07629b4..826e8eb1f 100644
--- a/test/engine/engine.cfg
+++ b/test/engine/engine.cfg
@@ -2,7 +2,10 @@
"*": {
"memtx": {"engine": "memtx"},
"vinyl": {"engine": "vinyl"}
- }
+ },
+ "functional.test.lua": {
+ "memtx": {"engine": "memtx"}
+ }
}
diff --git a/test/engine/functional.result b/test/engine/functional.result
new file mode 100644
index 000000000..c06183cda
--- /dev/null
+++ b/test/engine/functional.result
@@ -0,0 +1,708 @@
+-- test-run result file version 2
+test_run = require('test_run').new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+--
+-- gh-1260: Funclional indexes.
+--
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+ | ---
+ | ...
+lua_code2 = [[function(tuple) return {tuple[1] + tuple[2], 2 * tuple[1] + tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('s_nonpersistent')
+ | ---
+ | ...
+box.schema.func.create('s_ivaliddef1', {body = lua_code, is_deterministic = false, is_sandboxed = true})
+ | ---
+ | ...
+box.schema.func.create('s_ivaliddef2', {body = lua_code, is_deterministic = true, is_sandboxed = false})
+ | ---
+ | ...
+
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+box.schema.func.create('ss', {body = lua_code2, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+
+-- Functional index can't be primary.
+_ = s:create_index('idx', {functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | - error: 'Can''t create or modify index ''idx'' in space ''withdata'': primary key
+ | cannot be functional'
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+-- Invalid fid.
+_ = s:create_index('idx', {functional_fid = 6666, parts = {{1, 'unsigned'}}})
+ | ---
+ | - error: Function '6666' does not exist
+ | ...
+-- Can't use non-persistent function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_nonpersistent.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): referenced function doesn''t satisfy functional
+ | index constraints'
+ | ...
+-- Can't use non-deterministic function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_ivaliddef1.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): referenced function doesn''t satisfy functional
+ | index constraints'
+ | ...
+-- Can't use non-sandboxed function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_ivaliddef2.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): referenced function doesn''t satisfy functional
+ | index constraints'
+ | ...
+-- Can't use non-sequential parts in returned key definition.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{1, 'unsigned'}, {3, 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): invalid functional key definition'
+ | ...
+-- Can't use parts started not by 1 field.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{2, 'unsigned'}, {3, 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): invalid functional key definition'
+ | ...
+-- Can't use JSON paths in returned key definiton.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{"[1]data", 'unsigned'}}})
+ | ---
+ | - error: 'Wrong index options (field 0): invalid functional key definition'
+ | ...
+
+-- Can't drop a function referenced by functional index.
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | ...
+box.schema.func.drop('s')
+ | ---
+ | - error: 'Can''t drop function 1: function has references'
+ | ...
+box.snapshot()
+ | ---
+ | - ok
+ | ...
+test_run:cmd("restart server default")
+ |
+box.schema.func.drop('s')
+ | ---
+ | - error: 'Can''t drop function 1: function has references'
+ | ...
+s = box.space.withdata
+ | ---
+ | ...
+idx = s.index.idx
+ | ---
+ | ...
+idx:drop()
+ | ---
+ | ...
+box.schema.func.drop('s')
+ | ---
+ | ...
+
+test_run = require('test_run').new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+-- Invalid functional index extractor routine return: the extractor must return keys.
+lua_code = [[function(tuple) return "hello" end]]
+ | ---
+ | ...
+box.schema.func.create('invalidreturn0', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn0.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''invalidreturn0'' error: returned key type is
+ | invalid'
+ | ...
+idx:drop()
+ | ---
+ | ...
+
+-- Invalid functional index extractor routine return: a stirng instead of unsigned
+lua_code = [[function(tuple) return {"hello"} end]]
+ | ---
+ | ...
+box.schema.func.create('invalidreturn1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn1.id, parts = {{1, 'unsigned'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''invalidreturn1'' error: Supplied key type of
+ | part 0 does not match index part type: expected unsigned'
+ | ...
+idx:drop()
+ | ---
+ | ...
+
+-- Invalid functional index extractor routine return: undefined multikey return.
+lua_code = [[function(tuple) return {"hello", "world"}, {"my", "hart"} end]]
+ | ---
+ | ...
+box.schema.func.create('invalidreturn2', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn2.id, parts = {{1, 'string'}, {2, 'string'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''invalidreturn2'' error: to many keys were returned'
+ | ...
+idx:drop()
+ | ---
+ | ...
+
+-- Invalid functional index extractor routine return: the second returned key invalid.
+lua_code = [[function(tuple) return {"hello", "world"}, {1, 2} end]]
+ | ---
+ | ...
+box.schema.func.create('invalidreturn3', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn3.id, is_multikey = true, parts = {{1, 'unsigned'}, {2, 'unsigned'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''invalidreturn3'' error: Supplied key type of
+ | part 0 does not match index part type: expected unsigned'
+ | ...
+idx:drop()
+ | ---
+ | ...
+
+-- Invalid functional extractor: runtime extractor error
+test_run:cmd("setopt delimiter ';'")
+ | ---
+ | - true
+ | ...
+lua_code = [[function(tuple)
+ local json = require('json')
+ return json.encode(tuple)
+ end]]
+test_run:cmd("setopt delimiter ''");
+ | ---
+ | ...
+box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {functional_fid = box.func.runtimeerror.id, parts = {{1, 'string'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''runtimeerror'' error: [string "return function(tuple) local
+ | ..."]:1: attempt to call global ''require'' (a nil value)'
+ | ...
+idx:drop()
+ | ---
+ | ...
+
+-- Remove old persistent functions
+for _, v in pairs(box.func) do if v.is_persistent then box.schema.func.drop(v.name) end end
+ | ---
+ | ...
+s:drop()
+ | ---
+ | ...
+
+-- Functional test cases.
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+s:insert({1, 2})
+ | ---
+ | - [1, 2]
+ | ...
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+s:insert({2, 1})
+ | ---
+ | - error: Duplicate key exists in unique index 'idx' in space 'withdata'
+ | ...
+idx:get(3)
+ | ---
+ | - [1, 2]
+ | ...
+idx:delete(3)
+ | ---
+ | - [1, 2]
+ | ...
+s:select()
+ | ---
+ | - []
+ | ...
+s:insert({2, 1})
+ | ---
+ | - [2, 1]
+ | ...
+idx:get(3)
+ | ---
+ | - [2, 1]
+ | ...
+s:drop()
+ | ---
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+
+-- Multikey functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]}, {tuple[1] + tuple[2]}, {tuple[1]} end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+s:insert({1, 2})
+ | ---
+ | - [1, 2]
+ | ...
+s:insert({3, 5})
+ | ---
+ | - error: Duplicate key exists in unique index 'idx' in space 'withdata'
+ | ...
+s:insert({5, 3})
+ | ---
+ | - [5, 3]
+ | ...
+idx:select()
+ | ---
+ | - - [1, 2]
+ | - [1, 2]
+ | - [5, 3]
+ | - [5, 3]
+ | ...
+idx:get(8)
+ | ---
+ | - [5, 3]
+ | ...
+idx:get(3)
+ | ---
+ | - [1, 2]
+ | ...
+idx:get(1)
+ | ---
+ | - [1, 2]
+ | ...
+idx:get(5)
+ | ---
+ | - [5, 3]
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+
+-- Multikey multipart functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {600 + tuple[1], 600 + tuple[2]}, {500 + tuple[1], 500 + tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}, {2, 'integer'}}})
+ | ---
+ | ...
+s:insert({1, 2})
+ | ---
+ | - [1, 2]
+ | ...
+s:insert({2, 1})
+ | ---
+ | - [2, 1]
+ | ...
+s:insert({3, 3})
+ | ---
+ | - [3, 3]
+ | ...
+idx:select({600}, {iterator = "GE"})
+ | ---
+ | - - [1, 2]
+ | - [2, 1]
+ | - [3, 3]
+ | ...
+idx:get({603, 603})
+ | ---
+ | - [3, 3]
+ | ...
+idx:select({503}, {iterator = "LE"})
+ | ---
+ | - - [3, 3]
+ | - [2, 1]
+ | - [1, 2]
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+
+-- Multikey non-unique functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {500 + tuple[1]}, {500 + tuple[2]}, {500 + tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+idx = s:create_index('idx', {unique = false, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+s:insert({1, 2})
+ | ---
+ | - [1, 2]
+ | ...
+s:insert({2, 1})
+ | ---
+ | - [2, 1]
+ | ...
+idx:select({501})
+ | ---
+ | - - [1, 2]
+ | - [2, 1]
+ | ...
+idx:select({502})
+ | ---
+ | - - [1, 2]
+ | - [2, 1]
+ | ...
+s:replace({1, 3})
+ | ---
+ | - [1, 3]
+ | ...
+idx:select({501})
+ | ---
+ | - - [1, 3]
+ | - [2, 1]
+ | ...
+idx:select({502})
+ | ---
+ | - - [2, 1]
+ | ...
+idx:select({503})
+ | ---
+ | - - [1, 3]
+ | ...
+box.snapshot()
+ | ---
+ | - ok
+ | ...
+test_run:cmd("restart server default")
+ |
+s = box.space.withdata
+ | ---
+ | ...
+idx = s.index.idx
+ | ---
+ | ...
+idx:select({501})
+ | ---
+ | - - [1, 3]
+ | - [2, 1]
+ | ...
+idx:select({502})
+ | ---
+ | - - [2, 1]
+ | ...
+idx:select({503})
+ | ---
+ | - - [1, 3]
+ | ...
+s:replace({1, 2})
+ | ---
+ | - [1, 2]
+ | ...
+idx:select({501})
+ | ---
+ | - - [1, 2]
+ | - [2, 1]
+ | ...
+idx:select({502})
+ | ---
+ | - - [1, 2]
+ | - [2, 1]
+ | ...
+idx:select({503})
+ | ---
+ | - []
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+
+-- Multikey UTF-8 address extractor
+test_run = require('test_run').new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+pk = s:create_index('name', {parts = {1, 'string'}})
+ | ---
+ | ...
+s:insert({"James", "SIS Building Lambeth London UK"})
+ | ---
+ | - ['James', 'SIS Building Lambeth London UK']
+ | ...
+s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
+ | ---
+ | - ['Sherlock', '221B Baker St Marylebone London NW1 6XE UK']
+ | ...
+-- Create functional index on space with data
+test_run:cmd("setopt delimiter ';'")
+ | ---
+ | - true
+ | ...
+lua_code = [[function(tuple)
+ local address = string.split(tuple[2])
+ local ret = {}
+ for _, v in pairs(address) do table.insert(ret, {utf8.upper(v)}) end
+ return unpack(ret)
+ end]]
+test_run:cmd("setopt delimiter ''");
+ | ---
+ | ...
+box.schema.func.create('addr_extractor', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('addr', {unique = false, functional_fid = box.func.addr_extractor.id, is_multikey = true, parts = {{1, 'string', collation = 'unicode_ci'}}})
+ | ---
+ | ...
+idx:select('uk')
+ | ---
+ | - - ['James', 'SIS Building Lambeth London UK']
+ | - ['Sherlock', '221B Baker St Marylebone London NW1 6XE UK']
+ | ...
+idx:select('Sis')
+ | ---
+ | - - ['James', 'SIS Building Lambeth London UK']
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('addr_extractor')
+ | ---
+ | ...
+
+-- Partial index with functional index extractor
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+lua_code = [[function(tuple) if tuple[1] % 2 == 1 then return {tuple[1]} end end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - [1]
+ | ...
+s:insert({2})
+ | ---
+ | - [2]
+ | ...
+s:insert({3})
+ | ---
+ | - [3]
+ | ...
+s:insert({4})
+ | ---
+ | - [4]
+ | ...
+idx:select()
+ | ---
+ | - - [1]
+ | - [3]
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+
+-- Return nil from functional index extractor.
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+lua_code = [[function(tuple) return {nil} end]]
+ | ---
+ | ...
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+idx = s:create_index('idx', {unique = false, functional_fid = box.func.extr.id, parts = {{1, 'integer', is_nullable = true}}})
+ | ---
+ | ...
+s:insert({1})
+ | ---
+ | - error: 'Functional index extractor ''extr'' error: Invalid key part count in an
+ | exact match (expected 1, got 0)'
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('extr')
+ | ---
+ | ...
+
+-- Multiple functional indexes
+s = box.schema.space.create('withdata', {engine = engine})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+lua_code = [[function(tuple) return {tuple[1] - tuple[2]} end]]
+ | ---
+ | ...
+box.schema.func.create('sub', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+idx1 = s:create_index('s_idx', {unique = true, functional_fid = box.func.s.id, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+idx2 = s:create_index('sub_idx', {unique = true, functional_fid = box.func.sub.id, parts = {{1, 'integer'}}})
+ | ---
+ | ...
+s:insert({4, 1})
+ | ---
+ | - [4, 1]
+ | ...
+idx1:get(5)
+ | ---
+ | - [4, 1]
+ | ...
+idx2:get(3)
+ | ---
+ | - [4, 1]
+ | ...
+idx1:drop()
+ | ---
+ | ...
+idx2:get(3)
+ | ---
+ | - [4, 1]
+ | ...
+s:drop()
+ | ---
+ | ...
+collectgarbage()
+ | ---
+ | - 0
+ | ...
+box.schema.func.drop('s')
+ | ---
+ | ...
+box.schema.func.drop('sub')
+ | ---
+ | ...
diff --git a/test/engine/functional.test.lua b/test/engine/functional.test.lua
new file mode 100644
index 000000000..a84b5e38e
--- /dev/null
+++ b/test/engine/functional.test.lua
@@ -0,0 +1,240 @@
+test_run = require('test_run').new()
+engine = test_run:get_cfg('engine')
+
+--
+-- gh-1260: Funclional indexes.
+--
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+lua_code2 = [[function(tuple) return {tuple[1] + tuple[2], 2 * tuple[1] + tuple[2]} end]]
+box.schema.func.create('s_nonpersistent')
+box.schema.func.create('s_ivaliddef1', {body = lua_code, is_deterministic = false, is_sandboxed = true})
+box.schema.func.create('s_ivaliddef2', {body = lua_code, is_deterministic = true, is_sandboxed = false})
+
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+box.schema.func.create('ss', {body = lua_code2, is_deterministic = true, is_sandboxed = true})
+
+-- Functional index can't be primary.
+_ = s:create_index('idx', {functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+pk = s:create_index('pk')
+-- Invalid fid.
+_ = s:create_index('idx', {functional_fid = 6666, parts = {{1, 'unsigned'}}})
+-- Can't use non-persistent function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_nonpersistent.id, parts = {{1, 'unsigned'}}})
+-- Can't use non-deterministic function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_ivaliddef1.id, parts = {{1, 'unsigned'}}})
+-- Can't use non-sandboxed function in functional index.
+_ = s:create_index('idx', {functional_fid = box.func.s_ivaliddef2.id, parts = {{1, 'unsigned'}}})
+-- Can't use non-sequential parts in returned key definition.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{1, 'unsigned'}, {3, 'unsigned'}}})
+-- Can't use parts started not by 1 field.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{2, 'unsigned'}, {3, 'unsigned'}}})
+-- Can't use JSON paths in returned key definiton.
+_ = s:create_index('idx', {functional_fid = box.func.ss.id, parts = {{"[1]data", 'unsigned'}}})
+
+-- Can't drop a function referenced by functional index.
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+box.schema.func.drop('s')
+box.snapshot()
+test_run:cmd("restart server default")
+box.schema.func.drop('s')
+s = box.space.withdata
+idx = s.index.idx
+idx:drop()
+box.schema.func.drop('s')
+
+test_run = require('test_run').new()
+engine = test_run:get_cfg('engine')
+
+-- Invalid functional index extractor routine return: the extractor must return keys.
+lua_code = [[function(tuple) return "hello" end]]
+box.schema.func.create('invalidreturn0', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn0.id, parts = {{1, 'unsigned'}}})
+s:insert({1})
+idx:drop()
+
+-- Invalid functional index extractor routine return: a stirng instead of unsigned
+lua_code = [[function(tuple) return {"hello"} end]]
+box.schema.func.create('invalidreturn1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn1.id, parts = {{1, 'unsigned'}}})
+s:insert({1})
+idx:drop()
+
+-- Invalid functional index extractor routine return: undefined multikey return.
+lua_code = [[function(tuple) return {"hello", "world"}, {"my", "hart"} end]]
+box.schema.func.create('invalidreturn2', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn2.id, parts = {{1, 'string'}, {2, 'string'}}})
+s:insert({1})
+idx:drop()
+
+-- Invalid functional index extractor routine return: the second returned key invalid.
+lua_code = [[function(tuple) return {"hello", "world"}, {1, 2} end]]
+box.schema.func.create('invalidreturn3', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {functional_fid = box.func.invalidreturn3.id, is_multikey = true, parts = {{1, 'unsigned'}, {2, 'unsigned'}}})
+s:insert({1})
+idx:drop()
+
+-- Invalid functional extractor: runtime extractor error
+test_run:cmd("setopt delimiter ';'")
+lua_code = [[function(tuple)
+ local json = require('json')
+ return json.encode(tuple)
+ end]]
+test_run:cmd("setopt delimiter ''");
+box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {functional_fid = box.func.runtimeerror.id, parts = {{1, 'string'}}})
+s:insert({1})
+idx:drop()
+
+-- Remove old persistent functions
+for _, v in pairs(box.func) do if v.is_persistent then box.schema.func.drop(v.name) end end
+s:drop()
+
+-- Functional test cases.
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+pk = s:create_index('pk')
+s:insert({1, 2})
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, parts = {{1, 'integer'}}})
+s:insert({2, 1})
+idx:get(3)
+idx:delete(3)
+s:select()
+s:insert({2, 1})
+idx:get(3)
+s:drop()
+box.schema.func.drop('extr')
+collectgarbage()
+
+-- Multikey functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]}, {tuple[1] + tuple[2]}, {tuple[1]} end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+pk = s:create_index('pk')
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+s:insert({1, 2})
+s:insert({3, 5})
+s:insert({5, 3})
+idx:select()
+idx:get(8)
+idx:get(3)
+idx:get(1)
+idx:get(5)
+s:drop()
+collectgarbage()
+box.schema.func.drop('extr')
+
+-- Multikey multipart functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {600 + tuple[1], 600 + tuple[2]}, {500 + tuple[1], 500 + tuple[2]} end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+pk = s:create_index('pk')
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}, {2, 'integer'}}})
+s:insert({1, 2})
+s:insert({2, 1})
+s:insert({3, 3})
+idx:select({600}, {iterator = "GE"})
+idx:get({603, 603})
+idx:select({503}, {iterator = "LE"})
+s:drop()
+collectgarbage()
+box.schema.func.drop('extr')
+
+-- Multikey non-unique functional index.
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {500 + tuple[1]}, {500 + tuple[2]}, {500 + tuple[2]} end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+pk = s:create_index('pk')
+idx = s:create_index('idx', {unique = false, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+s:insert({1, 2})
+s:insert({2, 1})
+idx:select({501})
+idx:select({502})
+s:replace({1, 3})
+idx:select({501})
+idx:select({502})
+idx:select({503})
+box.snapshot()
+test_run:cmd("restart server default")
+s = box.space.withdata
+idx = s.index.idx
+idx:select({501})
+idx:select({502})
+idx:select({503})
+s:replace({1, 2})
+idx:select({501})
+idx:select({502})
+idx:select({503})
+s:drop()
+collectgarbage()
+box.schema.func.drop('extr')
+
+-- Multikey UTF-8 address extractor
+test_run = require('test_run').new()
+engine = test_run:get_cfg('engine')
+s = box.schema.space.create('withdata', {engine = engine})
+pk = s:create_index('name', {parts = {1, 'string'}})
+s:insert({"James", "SIS Building Lambeth London UK"})
+s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
+-- Create functional index on space with data
+test_run:cmd("setopt delimiter ';'")
+lua_code = [[function(tuple)
+ local address = string.split(tuple[2])
+ local ret = {}
+ for _, v in pairs(address) do table.insert(ret, {utf8.upper(v)}) end
+ return unpack(ret)
+ end]]
+test_run:cmd("setopt delimiter ''");
+box.schema.func.create('addr_extractor', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('addr', {unique = false, functional_fid = box.func.addr_extractor.id, is_multikey = true, parts = {{1, 'string', collation = 'unicode_ci'}}})
+idx:select('uk')
+idx:select('Sis')
+s:drop()
+collectgarbage()
+box.schema.func.drop('addr_extractor')
+
+-- Partial index with functional index extractor
+s = box.schema.space.create('withdata', {engine = engine})
+pk = s:create_index('pk')
+lua_code = [[function(tuple) if tuple[1] % 2 == 1 then return {tuple[1]} end end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {unique = true, functional_fid = box.func.extr.id, is_multikey = true, parts = {{1, 'integer'}}})
+s:insert({1})
+s:insert({2})
+s:insert({3})
+s:insert({4})
+idx:select()
+s:drop()
+collectgarbage()
+box.schema.func.drop('extr')
+
+-- Return nil from functional index extractor.
+s = box.schema.space.create('withdata', {engine = engine})
+pk = s:create_index('pk')
+lua_code = [[function(tuple) return {nil} end]]
+box.schema.func.create('extr', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+idx = s:create_index('idx', {unique = false, functional_fid = box.func.extr.id, parts = {{1, 'integer', is_nullable = true}}})
+s:insert({1})
+s:drop()
+collectgarbage()
+box.schema.func.drop('extr')
+
+-- Multiple functional indexes
+s = box.schema.space.create('withdata', {engine = engine})
+lua_code = [[function(tuple) return {tuple[1] + tuple[2]} end]]
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+lua_code = [[function(tuple) return {tuple[1] - tuple[2]} end]]
+box.schema.func.create('sub', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+pk = s:create_index('pk')
+idx1 = s:create_index('s_idx', {unique = true, functional_fid = box.func.s.id, parts = {{1, 'integer'}}})
+idx2 = s:create_index('sub_idx', {unique = true, functional_fid = box.func.sub.id, parts = {{1, 'integer'}}})
+s:insert({4, 1})
+idx1:get(5)
+idx2:get(3)
+idx1:drop()
+idx2:get(3)
+s:drop()
+collectgarbage()
+box.schema.func.drop('s')
+box.schema.func.drop('sub')
diff --git a/test/unit/luaT_tuple_new.c b/test/unit/luaT_tuple_new.c
index 0a16fa039..d5c59c975 100644
--- a/test/unit/luaT_tuple_new.c
+++ b/test/unit/luaT_tuple_new.c
@@ -124,7 +124,7 @@ test_basic(struct lua_State *L)
part.nullable_action = ON_CONFLICT_ACTION_DEFAULT;
part.sort_order = SORT_ORDER_ASC;
part.path = NULL;
- struct key_def *key_def = key_def_new(&part, 1);
+ struct key_def *key_def = key_def_new(&part, 1, false, false);
box_tuple_format_t *another_format = box_tuple_format_new(&key_def, 1);
key_def_delete(key_def);
diff --git a/test/unit/merger.test.c b/test/unit/merger.test.c
index b4a989a20..4bf1d8bfb 100644
--- a/test/unit/merger.test.c
+++ b/test/unit/merger.test.c
@@ -214,7 +214,8 @@ test_merger(struct tuple_format *format)
merge_source_array_new(true),
};
- struct key_def *key_def = key_def_new(&key_part_unsigned, 1);
+ struct key_def *key_def = key_def_new(&key_part_unsigned, 1,
+ false, false);
struct merge_source *merger = merger_new(key_def, sources, source_count,
false);
key_def_delete(key_def);
@@ -252,7 +253,8 @@ test_basic()
plan(4);
header();
- struct key_def *key_def = key_def_new(&key_part_integer, 1);
+ struct key_def *key_def = key_def_new(&key_part_integer, 1,
+ false, false);
struct tuple_format *format = box_tuple_format_new(&key_def, 1);
assert(format != NULL);
@@ -273,7 +275,7 @@ main()
{
memory_init();
fiber_init(fiber_c_invoke);
- tuple_init(NULL);
+ tuple_init(NULL, NULL, NULL);
int rc = test_basic();
diff --git a/test/unit/tuple_bigref.c b/test/unit/tuple_bigref.c
index 20eab61f6..bc207c1d8 100644
--- a/test/unit/tuple_bigref.c
+++ b/test/unit/tuple_bigref.c
@@ -143,7 +143,7 @@ main()
memory_init();
fiber_init(fiber_c_invoke);
- tuple_init(NULL);
+ tuple_init(NULL, NULL, NULL);
tuple_end = mp_encode_array(tuple_end, 1);
tuple_end = mp_encode_uint(tuple_end, 2);
diff --git a/test/unit/vy_iterators_helper.c b/test/unit/vy_iterators_helper.c
index 0d20f19ef..9d8522698 100644
--- a/test/unit/vy_iterators_helper.c
+++ b/test/unit/vy_iterators_helper.c
@@ -18,7 +18,7 @@ vy_iterator_C_test_init(size_t cache_size)
memory_init();
fiber_init(fiber_c_invoke);
- tuple_init(NULL);
+ tuple_init(NULL, NULL, NULL);
vy_stmt_env_create(&stmt_env);
vy_cache_env_create(&cache_env, cord_slab_cache());
vy_cache_env_set_quota(&cache_env, cache_size);
diff --git a/test/vinyl/misc.result b/test/vinyl/misc.result
index b2aacdc55..af8502dba 100644
--- a/test/vinyl/misc.result
+++ b/test/vinyl/misc.result
@@ -432,3 +432,26 @@ stat.bytes_compressed < stat.bytes / 10
s:drop()
---
...
+-- Vinyl doesn't support functional indexes.
+s = box.schema.space.create('withdata', {engine = 'vinyl'})
+---
+...
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+---
+...
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+---
+...
+_ = s:create_index('pk')
+---
+...
+_ = s:create_index('idx', {functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+---
+- error: Vinyl does not support functional indexes
+...
+s:drop()
+---
+...
+box.schema.func.drop('s')
+---
+...
diff --git a/test/vinyl/misc.test.lua b/test/vinyl/misc.test.lua
index f8da578d0..2016ca45f 100644
--- a/test/vinyl/misc.test.lua
+++ b/test/vinyl/misc.test.lua
@@ -182,3 +182,12 @@ test_run:wait_cond(function() return i:stat().disk.compaction.count > 0 end)
stat = i:stat().disk
stat.bytes_compressed < stat.bytes / 10
s:drop()
+
+-- Vinyl doesn't support functional indexes.
+s = box.schema.space.create('withdata', {engine = 'vinyl'})
+lua_code = [[function(tuple) return tuple[1] + tuple[2] end]]
+box.schema.func.create('s', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+_ = s:create_index('pk')
+_ = s:create_index('idx', {functional_fid = box.func.s.id, parts = {{1, 'unsigned'}}})
+s:drop()
+box.schema.func.drop('s')
diff --git a/test/wal_off/alter.result b/test/wal_off/alter.result
index bce15711d..62cb11db7 100644
--- a/test/wal_off/alter.result
+++ b/test/wal_off/alter.result
@@ -28,7 +28,7 @@ end;
...
#spaces;
---
-- 65503
+- 65502
...
-- cleanup
for k, v in pairs(spaces) do
--
2.22.0
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/4] box: introduce key_def->is_multikey flag
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
@ 2019-07-17 14:34 ` Konstantin Osipov
2019-07-18 11:20 ` [tarantool-patches] " Kirill Yukhin
1 sibling, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-17 14:34 UTC (permalink / raw)
To: Kirill Shcherbatov; +Cc: tarantool-patches, vdavydov.dev
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> Previously only key definitions that have JSON paths were able
> to define multikey index. We used to check multikey_path != NULL
> test to determine whether given key definition is multikey.
> In further patches with functional indexes this rule becomes
> outdated. Functional index extracted key definition may be
> multikey, but has no JSON paths.
> So an explicit is_multikey flag was introduced.
The patch is lgtm, but shouldn't you also rename multikey_path to
json_path?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/4] box: refactor key_validate_parts to return key_end
2019-07-17 1:20 ` [PATCH v3 2/4] box: refactor key_validate_parts to return key_end Kirill Shcherbatov
@ 2019-07-18 9:41 ` Konstantin Osipov
2019-07-18 11:21 ` [tarantool-patches] " Kirill Yukhin
1 sibling, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 9:41 UTC (permalink / raw)
To: Kirill Shcherbatov; +Cc: tarantool-patches, vdavydov.dev
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> The key_validate_parts helper is refactored to return a pointer
> to the end of a given key argument in case of success.
> This is required to effectively validate a sequence of keys in
> scope of functional multikey indexes.
lgtm
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 3/4] box: introduce tuple_extra infrastructure
2019-07-17 1:20 ` [PATCH v3 3/4] box: introduce tuple_extra infrastructure Kirill Shcherbatov
@ 2019-07-18 9:47 ` Konstantin Osipov
0 siblings, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 9:47 UTC (permalink / raw)
To: Kirill Shcherbatov; +Cc: tarantool-patches, vdavydov.dev
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> Introduced a new object tuple_extra: a memory allocation is
> associated with given tuple and chunk_id. Extended tuple_format's
> vtab with few new methods to manage tuple_extras lifecycle.
> Supported tuple_extra for memtx engine: a memory
> chunks are allocated with memtx's smalloc allocator.
tuple_extra is a very general name.
Are all extra chunks/segments of equal size, or variable size?
Why not change tuple format right away to consist of segments, and
add segments on the side?
I've been warning about a mounting technical debt, and this is
basically beyond any reasonable limit:
- we have huge problems that large tuples lead to memory
fragmentation
- we have an ugly realloc when building a tuple field map for
multikey indexes
- now we have a yet another ugly thing on the side for functional
indexes.
I understand that coming up with a new log structured tuple allocator
may be considered beyond the scope of this patch. But even without
a new allocator this patch is already half way through segmented
tuple structure.
So please deal with the debt first and transform tuple format to
segmented one.
The first segment should be up to a few kilobytes in size and
should contain the old-good-tuple as it is now. All subsequent
segments could be allocated using the same principles as in stl
deque - an array of chunks.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
@ 2019-07-18 9:50 ` Konstantin Osipov
2019-07-18 9:54 ` Konstantin Osipov
` (3 subsequent siblings)
4 siblings, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 9:50 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, Kirill Shcherbatov
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> Closes #1260
>
> @TarantoolBot document
> Title: introduce functional indexes in memtx
> Now you can define a functional index using registered persistent
> function as key extractor.
>
> There is some restrictions for function and key definition for
> functional index:
> - referenced extractor function must be persistent, deterministic
> and return scalar type or array.
> - key parts describe the referenced routine's returned key
> (when the routine returned value type is array in meaning of
> multikey index key): all parts must be sequential and the
> first part's fieldno must be 1.
> - key parts can't use JSON paths.
> - you are not allowed to access tuple fields by names in functional
> index extractor
>
> Functional index can't be primary.
> You are not allowed to change functional index extractor function
> while there are some functional indexes depends of it.
>
> Example:
> s = box.schema.space.create('withdata')
> s:format({{name = 'name', type = 'string'},
> {name = 'address', type = 'string'}})
> pk = s:create_index('name', {parts = {1, 'string'}})
> lua_code = [[function(tuple)
> local address = string.split(tuple[2])
> local ret = {}
> for _, v in pairs(address) do
> table.insert(ret, {utf8.upper(v)})
> end
> return unpack(ret)
> end]]
> box.schema.func.create('addr_extractor', {body = lua_code,
> is_deterministic = true, is_sandboxed = true})
> idx = s:create_index('addr', {unique = false, functional_fid =
> box.func.addr_extractor.id, is_multikey = true,
> parts = {{1, 'string'}}})
> s:insert({"James", "SIS Building Lambeth London UK"})
> s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
> idx:select('UK')
> ---
> - - ['James', 'SIS Building Lambeth London UK']
> - ['Sherlock', '221B Baker St Marylebone London NW1 6XE UK']
this is a huge patch with only 3 tiny prerequisites.
Are you sure it's not possible to split it into pieces?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
2019-07-18 9:50 ` [tarantool-patches] " Konstantin Osipov
@ 2019-07-18 9:54 ` Konstantin Osipov
2019-07-18 9:55 ` Konstantin Osipov
` (2 subsequent siblings)
4 siblings, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 9:54 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, Kirill Shcherbatov
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> +++ b/src/box/CMakeLists.txt
> @@ -51,8 +51,9 @@ add_library(tuple STATIC
> coll_id_cache.c
> field_def.c
> opt_def.c
> + functional_extractor.c
since we started to use func_ and not function_ and functional_ as
an abbreviation, let's be consistent and use it everywere, so
func_extractor. extractor is not a good name either (and most
likely not a good division of responsibilities), but I'll get
to it later.
> + functional_key.c
func_key
when a patch adds new modules the commit comment should contain
an explanation about what these modules do and why they were
added.
> index 1dbfe6b26..fdf38598e 100644
> --- a/src/box/alter.cc
> +++ b/src/box/alter.cc
> @@ -30,6 +30,7 @@
> */
> #include "alter.h"
> #include "assoc.h"
> +#include "box.h"
Kirill,
please stop bloating header file dependencies, especially circular
ones. If you need to include a dependency like this, you haven't
dealt with technical debt. There should be a separate patch which
improves modules structure so that this dependency is avoided.
The dependency issue is described in the SOP, so I should not
have to point it out time and again.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
2019-07-18 9:50 ` [tarantool-patches] " Konstantin Osipov
2019-07-18 9:54 ` Konstantin Osipov
@ 2019-07-18 9:55 ` Konstantin Osipov
2019-07-18 10:02 ` Konstantin Osipov
2019-07-22 14:45 ` Vladimir Davydov
4 siblings, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 9:55 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, Kirill Shcherbatov
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> + /**
> + * Can't verify functional index extractor routine
> + * reference on load because the function object
> + * had not been registered in Tarantool yet.
> + */
Next, please set up your git order. Header file changes should
come before .c file changes, and new/added files should come
first.
Please reset with a reasonable git order so that the patch is
possible to review.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
` (2 preceding siblings ...)
2019-07-18 9:55 ` Konstantin Osipov
@ 2019-07-18 10:02 ` Konstantin Osipov
2019-07-22 14:45 ` Vladimir Davydov
4 siblings, 0 replies; 15+ messages in thread
From: Konstantin Osipov @ 2019-07-18 10:02 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, Kirill Shcherbatov
* Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/07/17 09:39]:
> Closes #1260
Re extractor - no vendor calls functional index function an
extractor. Please use an established name: simply func_index_func
is good enough to me, but we could find a good name in the server
team chat.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 1/4] box: introduce key_def->is_multikey flag
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
2019-07-17 14:34 ` Konstantin Osipov
@ 2019-07-18 11:20 ` Kirill Yukhin
1 sibling, 0 replies; 15+ messages in thread
From: Kirill Yukhin @ 2019-07-18 11:20 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, kostja, Kirill Shcherbatov
Hello,
On 17 Jul 04:20, Kirill Shcherbatov wrote:
> Previously only key definitions that have JSON paths were able
> to define multikey index. We used to check multikey_path != NULL
> test to determine whether given key definition is multikey.
> In further patches with functional indexes this rule becomes
> outdated. Functional index extracted key definition may be
> multikey, but has no JSON paths.
> So an explicit is_multikey flag was introduced.
>
> Needed for #1260
I've checked the patch into master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [tarantool-patches] [PATCH v3 2/4] box: refactor key_validate_parts to return key_end
2019-07-17 1:20 ` [PATCH v3 2/4] box: refactor key_validate_parts to return key_end Kirill Shcherbatov
2019-07-18 9:41 ` Konstantin Osipov
@ 2019-07-18 11:21 ` Kirill Yukhin
1 sibling, 0 replies; 15+ messages in thread
From: Kirill Yukhin @ 2019-07-18 11:21 UTC (permalink / raw)
To: tarantool-patches; +Cc: vdavydov.dev, kostja, Kirill Shcherbatov
Hello,
On 17 Jul 04:20, Kirill Shcherbatov wrote:
> The key_validate_parts helper is refactored to return a pointer
> to the end of a given key argument in case of success.
> This is required to effectively validate a sequence of keys in
> scope of functional multikey indexes.
>
> Needed for #1260
I've checked the patch into master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 4/4] box: introduce functional indexes in memtx
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
` (3 preceding siblings ...)
2019-07-18 10:02 ` Konstantin Osipov
@ 2019-07-22 14:45 ` Vladimir Davydov
4 siblings, 0 replies; 15+ messages in thread
From: Vladimir Davydov @ 2019-07-22 14:45 UTC (permalink / raw)
To: Kirill Shcherbatov; +Cc: tarantool-patches, kostja
On Wed, Jul 17, 2019 at 04:20:45AM +0300, Kirill Shcherbatov wrote:
> idx = s:create_index('addr', {unique = false, functional_fid =
> box.func.addr_extractor.id, is_multikey = true,
> parts = {{1, 'string'}}})
Regarding the API: please use 'func = ' instead of 'functional_fid = '
and allow to specify a function by name.
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2019-07-22 14:45 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-17 1:20 [PATCH v3 0/4] box: functional indexes Kirill Shcherbatov
2019-07-17 1:20 ` [PATCH v3 1/4] box: introduce key_def->is_multikey flag Kirill Shcherbatov
2019-07-17 14:34 ` Konstantin Osipov
2019-07-18 11:20 ` [tarantool-patches] " Kirill Yukhin
2019-07-17 1:20 ` [PATCH v3 2/4] box: refactor key_validate_parts to return key_end Kirill Shcherbatov
2019-07-18 9:41 ` Konstantin Osipov
2019-07-18 11:21 ` [tarantool-patches] " Kirill Yukhin
2019-07-17 1:20 ` [PATCH v3 3/4] box: introduce tuple_extra infrastructure Kirill Shcherbatov
2019-07-18 9:47 ` Konstantin Osipov
2019-07-17 1:20 ` [PATCH v3 4/4] box: introduce functional indexes in memtx Kirill Shcherbatov
2019-07-18 9:50 ` [tarantool-patches] " Konstantin Osipov
2019-07-18 9:54 ` Konstantin Osipov
2019-07-18 9:55 ` Konstantin Osipov
2019-07-18 10:02 ` Konstantin Osipov
2019-07-22 14:45 ` Vladimir Davydov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox