From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 8FCDC274D7 for ; Wed, 17 Apr 2019 04:21:02 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R9s9W1w3_Z7X for ; Wed, 17 Apr 2019 04:18:56 -0400 (EDT) Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id A39341FB71 for ; Wed, 17 Apr 2019 04:18:56 -0400 (EDT) From: Kirill Yukhin Subject: [tarantool-patches] [PATCH v2] sysview: set format for spaces with sysview engine Date: Wed, 17 Apr 2019 11:18:49 +0300 Message-Id: <17415db95b6b5e4d929077a05fd7549772aa56c5.1555488781.git.kyukhin@tarantool.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: v.shpilevoy@tarantool.org Cc: tarantool-patches@freelists.org, Kirill Yukhin The patch sets format for spaces with sysview engine. This is done due to following reasons: 1. Since an SQL view looks into underneath space's format, set it for spaces with sysview engine. Before the patch, spaces with sysview enginge didn't have its own tuples and hence didn't need to have a format. 2. To use sysview engine to deal with SQL views. This will allow to use sysview machinery to query SQL views from Lua land. Closes #4111 --- v2: - Fixed most review inputs to v1: identation, phrasing, test. Branch: https://github.com/tarantool/tarantool/commits/kyukhin/gh-4111-sysview-format Issue: https://github.com/tarantool/tarantool/issues/4111 src/box/index_def.c | 22 +++++++ src/box/index_def.h | 12 ++++ src/box/memtx_space.c | 10 +-- src/box/sysview.c | 25 +++++++- src/box/tuple_format.c | 5 +- test/box/access_sysview.result | 68 ++++++++++++++++++++- test/sql/gh-4111-format-in-sysview.result | 28 +++++++++ test/sql/gh-4111-format-in-sysview.test.lua | 13 ++++ test/wal_off/alter.result | 2 +- 9 files changed, 172 insertions(+), 13 deletions(-) create mode 100644 test/sql/gh-4111-format-in-sysview.result create mode 100644 test/sql/gh-4111-format-in-sysview.test.lua diff --git a/src/box/index_def.c b/src/box/index_def.c index c743d12..f509443 100644 --- a/src/box/index_def.c +++ b/src/box/index_def.c @@ -33,6 +33,7 @@ #include "identifier.h" #include "tuple_format.h" #include "json/json.h" +#include "fiber.h" const char *index_type_strs[] = { "HASH", "TREE", "BITSET", "RTREE" }; @@ -245,6 +246,27 @@ index_def_cmp(const struct index_def *key1, const struct index_def *key2) key2->key_def->parts, key2->key_def->part_count); } +struct key_def ** +key_def_array(struct rlist *index_defs, int *size) +{ + int key_count = 0; + struct index_def *index_def; + rlist_foreach_entry(index_def, index_defs, link) + key_count++; + size_t sz = sizeof(struct key_def *) * key_count; + struct key_def **keys = (struct key_def **) region_alloc(&fiber()->gc, + sz); + if (keys == NULL) { + diag_set(OutOfMemory, sz, "region_alloc", "keys"); + return NULL; + } + *size = key_count; + key_count = 0; + rlist_foreach_entry(index_def, index_defs, link) + keys[key_count++] = index_def->key_def; + return keys; +} + bool index_def_is_valid(struct index_def *index_def, const char *space_name) diff --git a/src/box/index_def.h b/src/box/index_def.h index 7717ecd..0eb9b45 100644 --- a/src/box/index_def.h +++ b/src/box/index_def.h @@ -336,6 +336,18 @@ index_def_new(uint32_t space_id, uint32_t iid, const char *name, const struct index_opts *opts, struct key_def *key_def, struct key_def *pk_def); +/** + * Create an array (on a region) of key_defs from array of index + * definitions. + * + * @param index_defs List head. + * @param size[out] Array size. + * @retval not NULL Array of pointers to key_def + * @retval NULL Memory error. + */ +struct key_def ** +key_def_array(struct rlist *index_defs, int *size); + /** * One key definition is greater than the other if it's id is * greater, it's name is greater, it's index type is greater diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index 7bb46c7..fa02806 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -976,19 +976,11 @@ memtx_space_new(struct memtx_engine *memtx, /* Create a format from key and field definitions. */ int key_count = 0; - struct index_def *index_def; - rlist_foreach_entry(index_def, key_list, link) - key_count++; - struct key_def **keys = region_alloc(&fiber()->gc, - sizeof(*keys) * key_count); + struct key_def **keys = key_def_array(key_list, &key_count); if (keys == NULL) { free(memtx_space); return NULL; } - key_count = 0; - rlist_foreach_entry(index_def, key_list, link) - keys[key_count++] = index_def->key_def; - struct tuple_format *format = tuple_format_new(&memtx_tuple_format_vtab, memtx, keys, key_count, def->fields, def->field_count, diff --git a/src/box/sysview.c b/src/box/sysview.c index f9edd2d..d65137d 100644 --- a/src/box/sysview.c +++ b/src/box/sysview.c @@ -517,8 +517,31 @@ sysview_engine_create_space(struct engine *engine, struct space_def *def, "malloc", "struct space"); return NULL; } + int key_count = 0; + /* + * Despite the fact that space with sysview engine + * actually doesn't own tuples, setup of format will be + * useful in order to unify it with SQL views and to use + * same machinery to do selects from such views from Lua + * land. + */ + struct key_def **keys = key_def_array(key_list, &key_count); + if (keys == NULL) { + free(space); + return NULL; + } + struct tuple_format *format = + tuple_format_new(NULL, NULL, keys, key_count, def->fields, + def->field_count, def->exact_field_count, + def->dict, def->opts.is_temporary, + def->opts.is_ephemeral); + if (format == NULL) { + free(space); + return NULL; + } + tuple_format_ref(format); if (space_create(space, engine, &sysview_space_vtab, - def, key_list, NULL) != 0) { + def, key_list, format) != 0) { free(space); return NULL; } diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c index 093046b..804a678 100644 --- a/src/box/tuple_format.c +++ b/src/box/tuple_format.c @@ -699,7 +699,10 @@ tuple_format_new(struct tuple_format_vtab *vtab, void *engine, tuple_format_alloc(keys, key_count, space_field_count, dict); if (format == NULL) return NULL; - format->vtab = *vtab; + if (vtab != NULL) + format->vtab = *vtab; + else + memset(&format->vtab, 0, sizeof(format->vtab)); format->engine = engine; format->is_temporary = is_temporary; format->is_ephemeral = is_ephemeral; diff --git a/test/box/access_sysview.result b/test/box/access_sysview.result index ae04266..6b51566 100644 --- a/test/box/access_sysview.result +++ b/test/box/access_sysview.result @@ -635,6 +635,7 @@ seq:drop() -- box.space._vspace.index[1]:alter({parts = { 2, 'string' }}) --- +- error: Field 2 has type 'unsigned' in space format, but type 'string' in index definition ... box.space._vspace.index[1]:select('xxx') --- @@ -642,7 +643,72 @@ box.space._vspace.index[1]:select('xxx') ... box.space._vspace.index[1]:select(1) --- -- error: 'Supplied key type of part 0 does not match index part type: expected string' +- - [257, 1, '_vinyl_deferred_delete', 'blackhole', 0, {'group_id': 1}, [{'name': 'space_id', + 'type': 'unsigned'}, {'name': 'lsn', 'type': 'unsigned'}, {'name': 'tuple', + 'type': 'array'}]] + - [272, 1, '_schema', 'memtx', 0, {}, [{'type': 'string', 'name': 'key'}, {'type': 'any', + 'name': 'value', 'is_nullable': true}]] + - [276, 1, '_collation', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, { + 'name': 'name', 'type': 'string'}, {'name': 'owner', 'type': 'unsigned'}, + {'name': 'type', 'type': 'string'}, {'name': 'locale', 'type': 'string'}, { + 'name': 'opts', 'type': 'map'}]] + - [280, 1, '_space', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'engine', + 'type': 'string'}, {'name': 'field_count', 'type': 'unsigned'}, {'name': 'flags', + 'type': 'map'}, {'name': 'format', 'type': 'array'}]] + - [281, 1, '_vspace', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'engine', + 'type': 'string'}, {'name': 'field_count', 'type': 'unsigned'}, {'name': 'flags', + 'type': 'map'}, {'name': 'format', 'type': 'array'}]] + - [284, 1, '_sequence', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'step', + 'type': 'integer'}, {'name': 'min', 'type': 'integer'}, {'name': 'max', 'type': 'integer'}, + {'name': 'start', 'type': 'integer'}, {'name': 'cache', 'type': 'integer'}, + {'name': 'cycle', 'type': 'boolean'}]] + - [285, 1, '_sequence_data', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, + {'name': 'value', 'type': 'integer'}]] + - [286, 1, '_vsequence', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, + {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, { + 'name': 'step', 'type': 'integer'}, {'name': 'min', 'type': 'integer'}, { + 'name': 'max', 'type': 'integer'}, {'name': 'start', 'type': 'integer'}, { + 'name': 'cache', 'type': 'integer'}, {'name': 'cycle', 'type': 'boolean'}]] + - [288, 1, '_index', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'iid', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'type', + 'type': 'string'}, {'name': 'opts', 'type': 'map'}, {'name': 'parts', 'type': 'array'}]] + - [289, 1, '_vindex', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'iid', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'type', + 'type': 'string'}, {'name': 'opts', 'type': 'map'}, {'name': 'parts', 'type': 'array'}]] + - [296, 1, '_func', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', + 'type': 'unsigned'}]] + - [297, 1, '_vfunc', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', + 'type': 'unsigned'}]] + - [304, 1, '_user', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'type', + 'type': 'string'}, {'name': 'auth', 'type': 'map'}]] + - [305, 1, '_vuser', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', + 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'type', + 'type': 'string'}, {'name': 'auth', 'type': 'map'}]] + - [312, 1, '_priv', 'memtx', 0, {}, [{'name': 'grantor', 'type': 'unsigned'}, { + 'name': 'grantee', 'type': 'unsigned'}, {'name': 'object_type', 'type': 'string'}, + {'name': 'object_id', 'type': 'scalar'}, {'name': 'privilege', 'type': 'unsigned'}]] + - [313, 1, '_vpriv', 'sysview', 0, {}, [{'name': 'grantor', 'type': 'unsigned'}, + {'name': 'grantee', 'type': 'unsigned'}, {'name': 'object_type', 'type': 'string'}, + {'name': 'object_id', 'type': 'scalar'}, {'name': 'privilege', 'type': 'unsigned'}]] + - [320, 1, '_cluster', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'uuid', + 'type': 'string'}]] + - [328, 1, '_trigger', 'memtx', 0, {}, [{'name': 'name', 'type': 'string'}, {'name': 'space_id', + 'type': 'unsigned'}, {'name': 'opts', 'type': 'map'}]] + - [330, 1, '_truncate', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'count', + 'type': 'unsigned'}]] + - [340, 1, '_space_sequence', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, + {'name': 'sequence_id', 'type': 'unsigned'}, {'name': 'is_generated', 'type': 'boolean'}]] + - [356, 1, '_fk_constraint', 'memtx', 0, {}, [{'name': 'name', 'type': 'string'}, + {'name': 'child_id', 'type': 'unsigned'}, {'name': 'parent_id', 'type': 'unsigned'}, + {'name': 'is_deferred', 'type': 'boolean'}, {'name': 'match', 'type': 'string'}, + {'name': 'on_delete', 'type': 'string'}, {'name': 'on_update', 'type': 'string'}, + {'name': 'child_cols', 'type': 'array'}, {'name': 'parent_cols', 'type': 'array'}]] ... box.space._vspace.index[1]:alter({parts = { 2, 'unsigned' }}) --- diff --git a/test/sql/gh-4111-format-in-sysview.result b/test/sql/gh-4111-format-in-sysview.result new file mode 100644 index 0000000..95a81c8 --- /dev/null +++ b/test/sql/gh-4111-format-in-sysview.result @@ -0,0 +1,28 @@ +-- Make sure, that it is possible to create a VIEW which +-- refers to "_v" space, i.e. to sysview engine. +-- Before gh-4111 was fixed, attempt to create such a view +-- failed due to lack of format in a space with sysview +-- engine. +test_run = require('test_run').new() +--- +... +box.space._vspace.index[1]:count(1) > 0 +--- +- true +... +box.execute([[CREATE VIEW t AS SELECT "name" FROM "_vspace" y]]) +--- +- row_count: 1 +... +box.execute([[SELECT * from t WHERE "name" = 'T']]) +--- +- metadata: + - name: name + type: string + rows: + - ['T'] +... +box.execute([[DROP VIEW t]]) +--- +- row_count: 1 +... diff --git a/test/sql/gh-4111-format-in-sysview.test.lua b/test/sql/gh-4111-format-in-sysview.test.lua new file mode 100644 index 0000000..59eee89 --- /dev/null +++ b/test/sql/gh-4111-format-in-sysview.test.lua @@ -0,0 +1,13 @@ +-- Make sure, that it is possible to create a VIEW which +-- refers to "_v" space, i.e. to sysview engine. +-- Before gh-4111 was fixed, attempt to create such a view +-- failed due to lack of format in a space with sysview +-- engine. + +test_run = require('test_run').new() + +box.space._vspace.index[1]:count(1) > 0 + +box.execute([[CREATE VIEW t AS SELECT "name" FROM "_vspace" y]]) +box.execute([[SELECT * from t WHERE "name" = 'T']]) +box.execute([[DROP VIEW t]]) diff --git a/test/wal_off/alter.result b/test/wal_off/alter.result index becdf13..9257ccf 100644 --- a/test/wal_off/alter.result +++ b/test/wal_off/alter.result @@ -28,7 +28,7 @@ end; ... #spaces; --- -- 65511 +- 65488 ... -- cleanup for k, v in pairs(spaces) do -- 2.20.1