From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 257422106F for ; Wed, 10 Jul 2019 07:01:15 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3WAoShl7OpyU for ; Wed, 10 Jul 2019 07:01:14 -0400 (EDT) Received: from smtp60.i.mail.ru (smtp60.i.mail.ru [217.69.128.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 4BDEC1FAA9 for ; Wed, 10 Jul 2019 07:01:13 -0400 (EDT) From: Kirill Shcherbatov Subject: [tarantool-patches] [PATCH v2 12/12] sql: use schema's func hash instead of FuncDef hash Date: Wed, 10 Jul 2019 14:01:00 +0300 Message-Id: <8fa5473273ccef8386d519c19ae75a57c8ed51ae.1562756438.git.kshcherbatov@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org, korablev@tarantool.org Cc: kostja@tarantool.org, Kirill Shcherbatov Part of #2200 Closes #4113 Closes #2233 @TarantoolBot document Title: Reworked SQL functions machinery Now all functions in Tarantool use uniform API and calling convention. Therefore it is possible to call C and Lua functions from SQL. To make a function available in SQL, you need to define it with list of types of it's arguments 'param_list', the type of returned value 'returns' and mention 'SQL' language in exports parameter. Deterministic functions must be defined with is_deterministic = true (the query planner use this information to build an effective vdbe code). Example: -- C function is called in SQL box.schema.func.create("function1.divide", {language = 'C', returns = 'number', param_list = {'number', 'number'}, is_deterministic = true, exports = {'LUA', 'SQL'}}) box.execute('SELECT "function1.divide"(6, 3)') --- - metadata: - name: '"function1.divide"(6, 3)' type: number rows: - [2] ... -- Persistent Lua function is called in SQL box.schema.func.create("SUMMARIZE", {language = 'LUA', returns = 'number', body = 'function (a, b) return a + b end', is_deterministic = true, param_list = {'number', 'number'}, exports = {'LUA', 'SQL'}}) box.execute('SELECT summarize(1, 2)') --- - metadata: - name: summarize(1, 2) type: number rows: - [3] ... Moreover there is a predefined Lua function "LUA" that allows to call Lua code in SQL. The argument of LUA function is a valid Lua code that returns some scalar type. Example: box.execute('SELECT lua(\'return box.cfg.memtx_memory\')') --- - metadata: - name: lua('return box.cfg.memtx_memory') type: any rows: - [268435456] ... --- src/box/CMakeLists.txt | 1 - src/box/alter.cc | 1 + src/box/func.c | 3 +- src/box/func.h | 5 + src/box/func_def.c | 19 + src/box/func_def.h | 2 + src/box/lua/call.c | 8 +- src/box/lua/lua_sql.c | 217 ---------- src/box/lua/lua_sql.h | 39 -- src/box/lua/schema.lua | 4 +- src/box/port.c | 5 +- src/box/sql.h | 13 + src/box/sql/analyze.c | 40 +- src/box/sql/callback.c | 208 ---------- src/box/sql/date.c | 28 -- src/box/sql/expr.c | 51 ++- src/box/sql/func.c | 697 +++++++++++++++++++++++++------- src/box/sql/global.c | 7 - src/box/sql/main.c | 134 ------ src/box/sql/resolve.c | 55 ++- src/box/sql/select.c | 10 +- src/box/sql/sqlInt.h | 180 +-------- src/box/sql/vdbe.c | 19 +- src/box/sql/vdbe.h | 6 +- src/box/sql/vdbeInt.h | 20 +- src/box/sql/vdbeapi.c | 11 +- src/box/sql/vdbeaux.c | 31 +- src/box/sql/vdbemem.c | 73 ++-- src/box/sql/whereexpr.c | 4 +- src/lib/coll/coll.c | 1 + test/box/function1.result | 119 ++++++ test/box/function1.test.lua | 35 ++ test/sql-tap/alias.test.lua | 11 +- test/sql-tap/check.test.lua | 13 +- test/sql-tap/func5.test.lua | 29 +- test/sql-tap/lua_sql.test.lua | 120 +++--- test/sql-tap/subquery.test.lua | 21 +- test/sql-tap/trigger9.test.lua | 6 +- test/sql-tap/where2.test.lua | 4 +- test/sql/errinj.result | 25 -- test/sql/errinj.test.lua | 10 - test/sql/func-recreate.result | 41 +- test/sql/func-recreate.test.lua | 28 +- 43 files changed, 1109 insertions(+), 1245 deletions(-) delete mode 100644 src/box/lua/lua_sql.c delete mode 100644 src/box/lua/lua_sql.h diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt index 481842a39..a0160912d 100644 --- a/src/box/CMakeLists.txt +++ b/src/box/CMakeLists.txt @@ -128,7 +128,6 @@ add_library(box STATIC ${lua_sources} lua/init.c lua/call.c - lua/lua_sql.c lua/cfg.cc lua/console.c lua/tuple.c diff --git a/src/box/alter.cc b/src/box/alter.cc index c92a1f710..cd85552a2 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -2786,6 +2786,7 @@ func_def_new_from_tuple(struct tuple *tuple) def->exports.lua = true; def->param_count = 0; } + def->sql_flags = 0; def_guard.is_active = false; return def; } diff --git a/src/box/func.c b/src/box/func.c index e36649d15..6e7d04f25 100644 --- a/src/box/func.c +++ b/src/box/func.c @@ -34,13 +34,13 @@ #include "assoc.h" #include "lua/utils.h" #include "lua/call.h" -#include "lua/lua_sql.h" #include "error.h" #include "sql.h" #include "diag.h" #include "port.h" #include "schema.h" #include "session.h" +#include "small/region.h" #include /** @@ -438,6 +438,7 @@ func_c_new(struct func_def *def) return NULL; } func->base.vtab = func_c_vtab; + rlist_create(&func->base.signature); func->func = NULL; func->module = NULL; return &func->base; diff --git a/src/box/func.h b/src/box/func.h index 7e4dd37a3..133172bf0 100644 --- a/src/box/func.h +++ b/src/box/func.h @@ -82,6 +82,11 @@ struct func { * Cached runtime access information. */ struct access access[BOX_USER_MAX]; + /** + * A list of other functions with given name. + * Is valid for SQL builtins. + */ + struct rlist signature; }; /** diff --git a/src/box/func_def.c b/src/box/func_def.c index fb9f77df8..ffd76a514 100644 --- a/src/box/func_def.c +++ b/src/box/func_def.c @@ -34,9 +34,28 @@ func_def_cmp(struct func_def *def1, struct func_def *def2) return def1->aggregate - def2->aggregate; if (def1->param_count != def2->param_count) return def1->param_count - def2->param_count; + if (def1->sql_flags != def2->sql_flags) + return def1->sql_flags - def2->sql_flags; if ((def1->comment != NULL) != (def2->comment != NULL)) return def1->comment - def2->comment; if (def1->comment != NULL && strcmp(def1->comment, def2->comment) != 0) return strcmp(def1->comment, def2->comment); return 0; } + +struct func_def * +func_def_dup(struct func_def *def) +{ + uint32_t body_offset, comment_offset; + uint32_t sz = func_def_sizeof(def->name_len, + def->body != NULL ? strlen(def->body) : 0, + def->comment != NULL ? strlen(def->comment) : 0, + &body_offset, &comment_offset); + struct func_def *new = (struct func_def *) malloc(sz); + memcpy(new, def, sz); + if (new->body != NULL) + new->body = (char *)new + body_offset; + if (new->comment != NULL) + new->comment = (char *)new + comment_offset; + return new; +} diff --git a/src/box/func_def.h b/src/box/func_def.h index 508580f78..809d74c42 100644 --- a/src/box/func_def.h +++ b/src/box/func_def.h @@ -89,6 +89,8 @@ struct func_def { * available. */ bool is_sandboxed; + /** A set of SQL_FUNCTION_* flags. */ + uint16_t sql_flags; /** The count of function's input arguments. */ int param_count; /** The type of the value returned by function. */ diff --git a/src/box/lua/call.c b/src/box/lua/call.c index 752f05745..528c41310 100644 --- a/src/box/lua/call.c +++ b/src/box/lua/call.c @@ -44,7 +44,6 @@ #include "box/port.h" #include "box/lua/tuple.h" #include "small/obuf.h" -#include "lua_sql.h" #include "trivia/util.h" #include "mpstream.h" @@ -501,13 +500,16 @@ port_lua_destroy(struct port *base) extern const char * port_lua_dump_plain(struct port *port, uint32_t *size); +extern struct sql_value * +port_lua_get_vdbemem(struct port *base, uint32_t *size); + static const struct port_vtab port_lua_vtab = { .dump_msgpack = port_lua_dump, .dump_msgpack_16 = port_lua_dump_16, .dump_lua = port_lua_dump_lua, .dump_plain = port_lua_dump_plain, .get_msgpack = port_lua_get_msgpack, - .get_vdbemem = NULL, + .get_vdbemem = port_lua_get_vdbemem, .get_context = NULL, .destroy = port_lua_destroy, }; @@ -717,6 +719,7 @@ func_lua_new(struct func_def *def) func->lua_ref = LUA_REFNIL; func->base.vtab = func_lua_vtab; } + rlist_create(&func->base.signature); return &func->base; } @@ -959,7 +962,6 @@ static struct trigger on_alter_func_in_lua = { static const struct luaL_Reg boxlib_internal[] = { {"call_loadproc", lbox_call_loadproc}, - {"sql_create_function", lbox_sql_create_function}, {"module_reload", lbox_module_reload}, {"func_call", lbox_func_call}, {NULL, NULL} diff --git a/src/box/lua/lua_sql.c b/src/box/lua/lua_sql.c deleted file mode 100644 index 0c5797fa2..000000000 --- a/src/box/lua/lua_sql.c +++ /dev/null @@ -1,217 +0,0 @@ -/* - * Copyright 2010-2017, Tarantool AUTHORS, please see AUTHORS file. - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * 1. Redistributions of source code must retain the above - * copyright notice, this list of conditions and the - * following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED - * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL - * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR - * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF - * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - */ - -#include "lua.h" -#include "lua/utils.h" - -#include "box/lua/call.h" -#include "box/sql/sqlInt.h" -#include "box/port.h" -#include "box/sql/vdbeInt.h" - -struct lua_sql_func_info { - int func_ref; -}; - -/** - * This function is callback which is called by sql engine. - * - * Purpose of this function is to call lua func from sql. - * Lua func should be previously registered in sql - * (see lbox_sql_create_function). - */ -static int -lua_sql_call(struct func *func, struct port *args, struct port *ret) -{ - (void) func; - uint32_t argc; - struct Mem *argv = (struct Mem *)port_get_vdbemem(args, &argc); - if (argv == NULL) - return -1; - struct sql_context *ctx = (struct sql_context *) port_get_context(args); - assert(ctx != NULL); - struct Mem *val = vdbemem_alloc_on_region(1); - if (val == NULL) - return -1; - port_vdbemem_create(ret, (struct sql_value *)val, 1, NULL); - - lua_State *L = lua_newthread(tarantool_L); - int coro_ref = luaL_ref(tarantool_L, LUA_REGISTRYINDEX); - struct lua_sql_func_info *func_info = sql_user_data(ctx); - - lua_rawgeti(L, LUA_REGISTRYINDEX, func_info->func_ref); - for (uint32_t i = 0; i < argc; i++) { - sql_value *param = (sql_value *)&argv[i]; - switch (sql_value_type(param)) { - case MP_INT: - luaL_pushint64(L, sql_value_int64(param)); - break; - case MP_DOUBLE: - lua_pushnumber(L, sql_value_double(param)); - break; - case MP_STR: - lua_pushstring(L, (const char *) sql_value_text(param)); - break; - case MP_BIN: - lua_pushlstring(L, sql_value_blob(param), - (size_t) sql_value_bytes(param)); - break; - case MP_NIL: - lua_rawgeti(L, LUA_REGISTRYINDEX, luaL_nil_ref); - break; - case MP_BOOL: - lua_pushboolean(L, sql_value_boolean(param)); - break; - default: - diag_set(ClientError, ER_SQL_EXECUTE, "Unsupported "\ - "type passed to Lua"); - goto error; - } - } - if (lua_pcall(L, lua_gettop(L) - 1, 1, 0) != 0){ - diag_set(ClientError, ER_SQL_EXECUTE, lua_tostring(L, -1)); - goto error; - } - switch(lua_type(L, -1)) { - case LUA_TBOOLEAN: - mem_set_bool(val, lua_toboolean(L, -1)); - break; - case LUA_TNUMBER: - sqlVdbeMemSetDouble(val, lua_tonumber(L, -1)); - break; - case LUA_TSTRING: - if (sqlVdbeMemSetStr(val, lua_tostring(L, -1), -1, - 1, SQL_TRANSIENT) != 0) - return -1; - break; - case LUA_TNIL: - sqlVdbeMemSetNull(val); - break; - default: - diag_set(ClientError, ER_SQL_EXECUTE, "Unsupported type "\ - "passed from Lua"); - goto error; - } - return 0; -error: - luaL_unref(tarantool_L, LUA_REGISTRYINDEX, coro_ref); - return -1; -} - -static void -lua_sql_destroy(void *p) -{ - struct lua_sql_func_info *func_info = p; - luaL_unref(tarantool_L, LUA_REGISTRYINDEX, func_info->func_ref); - free(func_info); - return; -} - -/** - * A helper to register lua function in SQL during runtime. - * It makes available queries like this: "SELECT lua_func(arg);" - * - * sql_create_function *p argument is used to store func ref - * to lua function (it identifies actual lua func to call if there - * are many of them). SQL function must have name and type of - * returning value. Additionally, it can feature number of - * arguments and deterministic flag. - */ -int -lbox_sql_create_function(struct lua_State *L) -{ - struct sql *db = sql_get(); - if (db == NULL) - return luaL_error(L, "Please call box.cfg{} first"); - int argc = lua_gettop(L); - /* - * Three function prototypes are possible: - * 1. sql_create_function("func_name", "type", func); - * 2. sql_create_function("func_name", "type", func, - * func_arg_num); - * 3. sql_create_function("func_name", "type", func, - * func_arg_num, is_deterministic); - */ - if (!(argc == 3 && lua_isstring(L, 1) && lua_isstring(L, 2) && - lua_isfunction(L, 3)) && - !(argc == 4 && lua_isstring(L, 1) && lua_isstring(L, 2) && - lua_isfunction(L, 3) && lua_isnumber(L, 4)) && - !(argc == 5 && lua_isstring(L, 1) && lua_isstring(L, 2) && - lua_isfunction(L, 3) && lua_isnumber(L, 4) && - lua_isboolean(L, 5))) - return luaL_error(L, "Invalid arguments"); - enum field_type type; - const char *type_arg = lua_tostring(L, 2); - if (strcmp(type_arg, "INT") == 0 || strcmp(type_arg, "INTEGER") == 0) - type = FIELD_TYPE_INTEGER; - else if (strcmp(type_arg, "TEXT") == 0) - type = FIELD_TYPE_STRING; - else if (strcmp(type_arg, "FLOAT") == 0) - type = FIELD_TYPE_NUMBER; - else if (strcmp(type_arg, "NUM") == 0) - type = FIELD_TYPE_NUMBER; - else if (strcmp(type_arg, "BLOB") == 0) - type = FIELD_TYPE_SCALAR; - else if (strcmp(type_arg, "BOOL") == 0 || - strcmp(type_arg, "BOOLEAN") == 0) - type = FIELD_TYPE_BOOLEAN; - else - return luaL_error(L, "Unknown type"); - /* -1 indicates any number of arguments. */ - int func_arg_num = -1; - bool is_deterministic = false; - if (argc == 4) { - func_arg_num = lua_tointeger(L, 4); - lua_pop(L, 1); - } else if (argc == 5) { - is_deterministic = lua_toboolean(L, 5); - func_arg_num = lua_tointeger(L, 4); - lua_pop(L, 2); - } - size_t name_len; - const char *name = lua_tolstring(L, 1, &name_len); - char *normalized_name = - sql_normalized_name_region_new(&fiber()->gc, name, name_len); - if (normalized_name == NULL) - return luaT_error(L); - struct lua_sql_func_info *func_info = - (struct lua_sql_func_info *) malloc(sizeof(*func_info)); - if (func_info == NULL) - return luaL_error(L, "out of memory"); - func_info->func_ref = luaL_ref(L, LUA_REGISTRYINDEX); - int rc = sql_create_function_v2(db, normalized_name, type, func_arg_num, - is_deterministic ? SQL_DETERMINISTIC : 0, - func_info, lua_sql_call, NULL, NULL, - lua_sql_destroy); - if (rc != 0) - return luaT_error(L); - return 0; -} diff --git a/src/box/lua/lua_sql.h b/src/box/lua/lua_sql.h deleted file mode 100644 index b81093eca..000000000 --- a/src/box/lua/lua_sql.h +++ /dev/null @@ -1,39 +0,0 @@ -/* - * Copyright 2010-2017, Tarantool AUTHORS, please see AUTHORS file. - * - * Redistribution and use in source and binary forms, with or - * without modification, are permitted provided that the following - * conditions are met: - * - * 1. Redistributions of source code must retain the above - * copyright notice, this list of conditions and the - * following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above - * copyright notice, this list of conditions and the following - * disclaimer in the documentation and/or other materials - * provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED - * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL - * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR - * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF - * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - */ - -#ifndef TARANTOOL_LUA_SQL_H -#define TARANTOOL_LUA_SQL_H - -int -lbox_sql_create_function(struct lua_State *L); - -#endif //TARANTOOL_LUA_SQL_H - diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index aadcd3fa9..5c65eb792 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -2109,7 +2109,9 @@ box.schema.func.create = function(name, opts) if_not_exists = 'boolean', language = 'string', body = 'string', is_deterministic = 'boolean', - is_sandboxed = 'boolean', comment = 'string' }) + is_sandboxed = 'boolean', comment = 'string', + returns = 'string', param_list = 'table', + exports = 'table'}) local _func = box.space[box.schema.FUNC_ID] local _vfunc = box.space[box.schema.VFUNC_ID] local func = _vfunc.index.name:get{name} diff --git a/src/box/port.c b/src/box/port.c index 9e4ab9453..02c6a2245 100644 --- a/src/box/port.c +++ b/src/box/port.c @@ -140,13 +140,16 @@ port_free(void) mempool_destroy(&port_tuple_entry_pool); } +extern struct sql_value * +port_tuple_get_vdbemem(struct port *base, uint32_t *size); + const struct port_vtab port_tuple_vtab = { .dump_msgpack = port_tuple_dump_msgpack, .dump_msgpack_16 = port_tuple_dump_msgpack_16, .dump_lua = port_tuple_dump_lua, .dump_plain = NULL, .get_msgpack = NULL, - .get_vdbemem = NULL, + .get_vdbemem = port_tuple_get_vdbemem, .get_context = NULL, .destroy = port_tuple_destroy, }; diff --git a/src/box/sql.h b/src/box/sql.h index a078bfdec..ac10ae400 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -33,6 +33,7 @@ #include #include +#include "box/func.h" #if defined(__cplusplus) extern "C" { @@ -409,6 +410,18 @@ vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref, struct func * func_sql_builtin_new(struct func_def *def); +struct func_sql_builtin { + /** Function object base class. */ + struct func base; + /** User data to pass in call method. */ + void *user_data; + /** Finilize method for aggregate function. */ + int (*finalize)(struct func *func, struct port *args, struct port *ret); +}; + +struct func * +sql_func_by_signature(const char *name, uint32_t name_len, int argc); + #if defined(__cplusplus) } /* extern "C" { */ #endif diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c index 7764b48c5..02c922314 100644 --- a/src/box/sql/analyze.c +++ b/src/box/sql/analyze.c @@ -264,7 +264,7 @@ stat4Destructor(void *pOld) * return value is BLOB, but it is really just a pointer to the Stat4Accum * object. */ -static int +MAYBE_UNUSED int sql_builtin_stat_init(struct func *func, struct port *args, struct port *ret) { (void) func; @@ -544,7 +544,7 @@ samplePushPrevious(Stat4Accum * p, int iChng) * * The R parameter is only used for STAT4 */ -static int +MAYBE_UNUSED int sql_builtin_stat_push(struct func *func, struct port *args, struct port *ret) { (void) func; @@ -623,7 +623,7 @@ sql_builtin_stat_push(struct func *func, struct port *args, struct port *ret) * The content to returned is determined by the parameter J * which is one of the STAT_GET_xxxx values defined above. */ -static int +MAYBE_UNUSED int sql_builtin_stat_get(struct func *func, struct port *args, struct port *ret) { (void) func; @@ -736,11 +736,11 @@ vdbe_emit_analyze_stat_get(struct Vdbe * v, int regStat4, int iParam, { assert(regOut != regStat4 && regOut != regStat4 + 1); sqlVdbeAddOp2(v, OP_Integer, iParam, regStat4 + 1); - struct FuncDef *func = - sqlFindFunction(sql_get(), "_sql_stat_get", 2, 0); + struct func *func = + sql_func_by_signature("_sql_stat_get", strlen("_sql_stat_get"), 2); assert(func != NULL); sqlVdbeAddOp4(v, OP_Function0, 0, regStat4, regOut, - (char *)func, P4_FUNCDEF); + (char *)func, P4_FUNC); sqlVdbeChangeP5(v, 2); } @@ -876,11 +876,12 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space) sqlVdbeAddOp2(v, OP_Count, idx_cursor, stat4_reg + 3); sqlVdbeAddOp2(v, OP_Integer, part_count, stat4_reg + 1); sqlVdbeAddOp2(v, OP_Integer, part_count, stat4_reg + 2); - struct FuncDef *init_func = - sqlFindFunction(sql_get(), "_sql_stat_init", 3, 0); + struct func *init_func = + sql_func_by_signature("_sql_stat_init", + strlen("_sql_stat_init"), 3); assert(init_func != NULL); sqlVdbeAddOp4(v, OP_Function0, 0, stat4_reg + 1, stat4_reg, - (char *)init_func, P4_FUNCDEF); + (char *)init_func, P4_FUNC); sqlVdbeChangeP5(v, 3); /* * Implementation of the following: @@ -977,11 +978,12 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space) sqlVdbeAddOp3(v, OP_MakeRecord, stat_key_reg, pk_part_count, key_reg); assert(chng_reg == (stat4_reg + 1)); - struct FuncDef *push_func = - sqlFindFunction(sql_get(), "_sql_stat_push", 3, 0); + struct func *push_func = + sql_func_by_signature("_sql_stat_push", + strlen("_sql_stat_push"), 3); assert(push_func != NULL); sqlVdbeAddOp4(v, OP_Function0, 1, stat4_reg, tmp_reg, - (char *)push_func, P4_FUNCDEF); + (char *)push_func, P4_FUNC); sqlVdbeChangeP5(v, 3); sqlVdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr); /* Add the entry to the stat1 table. */ @@ -1768,17 +1770,3 @@ fail: box_txn_rollback(); return -1; } - -void -sql_register_analyze_builtins(void) -{ - static FuncDef funcs[] = { - FUNCTION(_sql_stat_get, 2, 0, 0, sql_builtin_stat_get, - FIELD_TYPE_ANY), - FUNCTION(_sql_stat_push, 3, 0, 0, sql_builtin_stat_push, - FIELD_TYPE_ANY), - FUNCTION(_sql_stat_init, 3, 0, 0, sql_builtin_stat_init, - FIELD_TYPE_ANY), - }; - sqlInsertBuiltinFuncs(funcs, nelem(funcs)); -} diff --git a/src/box/sql/callback.c b/src/box/sql/callback.c index 42fec2c6a..290363db6 100644 --- a/src/box/sql/callback.c +++ b/src/box/sql/callback.c @@ -56,211 +56,3 @@ sql_get_coll_seq(Parse *parser, const char *name, uint32_t *coll_id) return p->coll; } } - -/* During the search for the best function definition, this procedure - * is called to test how well the function passed as the first argument - * matches the request for a function with nArg arguments in a system - * that uses encoding enc. The value returned indicates how well the - * request is matched. A higher value indicates a better match. - * - * If nArg is -1 that means to only return a match (non-zero) if p->nArg - * is also -1. In other words, we are searching for a function that - * takes a variable number of arguments. - * - * If nArg is -2 that means that we are searching for any function - * regardless of the number of arguments it uses, so return a positive - * match score for any - * - * The returned value is always between 0 and 6, as follows: - * - * 0: Not a match. - * 1: UTF8/16 conversion required and function takes any number of arguments. - * 2: UTF16 byte order change required and function takes any number of args. - * 3: encoding matches and function takes any number of arguments - * 4: UTF8/16 conversion required - argument count matches exactly - * 5: UTF16 byte order conversion required - argument count matches exactly - * 6: Perfect match: encoding and argument count match exactly. - * - * If nArg==(-2) then any function with a non-null xSFunc is - * a perfect match and any function with xSFunc NULL is - * a non-match. - */ -#define FUNC_PERFECT_MATCH 4 /* The score for a perfect match */ -static int -matchQuality(FuncDef * p, /* The function we are evaluating for match quality */ - int nArg /* Desired number of arguments. (-1)==any */ - ) -{ - int match; - - /* nArg of -2 is a special case */ - if (nArg == (-2)) - return (p->xSFunc == 0) ? 0 : FUNC_PERFECT_MATCH; - - /* Wrong number of arguments means "no match" */ - if (p->nArg != nArg && p->nArg >= 0) - return 0; - - /* Give a better score to a function with a specific number of arguments - * than to function that accepts any number of arguments. - */ - if (p->nArg == nArg) { - match = 4; - } else { - match = 1; - } - - return match; -} - -/* - * Search a FuncDefHash for a function with the given name. Return - * a pointer to the matching FuncDef if found, or 0 if there is no match. - */ -static FuncDef * -functionSearch(int h, /* Hash of the name */ - const char *zFunc /* Name of function */ - ) -{ - FuncDef *p; - for (p = sqlBuiltinFunctions.a[h]; p; p = p->u.pHash) { - if (sqlStrICmp(p->zName, zFunc) == 0) { - return p; - } - } - return 0; -} - -/* - * Insert a new FuncDef into a FuncDefHash hash table. - */ -void -sqlInsertBuiltinFuncs(FuncDef * aDef, /* List of global functions to be inserted */ - int nDef /* Length of the apDef[] list */ - ) -{ - int i; - for (i = 0; i < nDef; i++) { - FuncDef *pOther; - const char *zName = aDef[i].zName; - int nName = sqlStrlen30(zName); - int h = - (sqlUpperToLower[(u8) zName[0]] + - nName) % SQL_FUNC_HASH_SZ; - pOther = functionSearch(h, zName); - if (pOther) { - assert(pOther != &aDef[i] && pOther->pNext != &aDef[i]); - aDef[i].pNext = pOther->pNext; - pOther->pNext = &aDef[i]; - } else { - aDef[i].pNext = 0; - aDef[i].u.pHash = sqlBuiltinFunctions.a[h]; - sqlBuiltinFunctions.a[h] = &aDef[i]; - } - } -} - -/* - * Locate a user function given a name, a number of arguments and a flag - * indicating whether the function prefers UTF-16 over UTF-8. Return a - * pointer to the FuncDef structure that defines that function, or return - * NULL if the function does not exist. - * - * If the createFlag argument is true, then a new (blank) FuncDef - * structure is created and liked into the "db" structure if a - * no matching function previously existed. - * - * If nArg is -2, then the first valid function found is returned. A - * function is valid if xSFunc is non-zero. The nArg==(-2) - * case is used to see if zName is a valid function name for some number - * of arguments. If nArg is -2, then createFlag must be 0. - * - * If createFlag is false, then a function with the required name and - * number of arguments may be returned even if the eTextRep flag does not - * match that requested. - */ -FuncDef * -sqlFindFunction(sql * db, /* An open database */ - const char *zName, /* Name of the function. zero-terminated */ - int nArg, /* Number of arguments. -1 means any number */ - u8 createFlag /* Create new entry if true and does not otherwise exist */ - ) -{ - FuncDef *p; /* Iterator variable */ - FuncDef *pBest = 0; /* Best match found so far */ - int bestScore = 0; /* Score of best match */ - int h; /* Hash value */ - int nName; /* Length of the name */ - - assert(nArg >= (-2)); - assert(nArg >= (-1) || createFlag == 0); - nName = sqlStrlen30(zName); - - /* First search for a match amongst the application-defined functions. - */ - p = (FuncDef *) sqlHashFind(&db->aFunc, zName); - while (p) { - int score = matchQuality(p, nArg); - if (score > bestScore) { - pBest = p; - bestScore = score; - } - p = p->pNext; - } - - /* If no match is found, search the built-in functions. - * - * If the SQL_PreferBuiltin flag is set, then search the built-in - * functions even if a prior app-defined function was found. And give - * priority to built-in functions. - * - * Except, if createFlag is true, that means that we are trying to - * install a new function. Whatever FuncDef structure is returned it will - * have fields overwritten with new information appropriate for the - * new function. But the FuncDefs for built-in functions are read-only. - * So we must not search for built-ins when creating a new function. - */ - if (!createFlag && (pBest == NULL)) { - bestScore = 0; - h = (sqlUpperToLower[(u8) zName[0]] + - nName) % SQL_FUNC_HASH_SZ; - p = functionSearch(h, zName); - while (p) { - int score = matchQuality(p, nArg); - if (score > bestScore) { - pBest = p; - bestScore = score; - } - p = p->pNext; - } - } - - /* If the createFlag parameter is true and the search did not reveal an - * exact match for the name, number of arguments and encoding, then add a - * new entry to the hash table and return it. - */ - if (createFlag && bestScore < FUNC_PERFECT_MATCH && - (pBest = - sqlDbMallocZero(db, sizeof(*pBest) + nName + 1)) != 0) { - FuncDef *pOther; - pBest->zName = (const char *)&pBest[1]; - pBest->nArg = (u16) nArg; - pBest->funcFlags = 0; - memcpy((char *)&pBest[1], zName, nName + 1); - pOther = - (FuncDef *) sqlHashInsert(&db->aFunc, pBest->zName, - pBest); - if (pOther == pBest) { - sqlDbFree(db, pBest); - sqlOomFault(db); - return 0; - } else { - pBest->pNext = pOther; - } - } - - if (pBest && (pBest->xSFunc || createFlag)) { - return pBest; - } - return 0; -} diff --git a/src/box/sql/date.c b/src/box/sql/date.c index 2e2a71ad2..dffc23616 100644 --- a/src/box/sql/date.c +++ b/src/box/sql/date.c @@ -1290,31 +1290,3 @@ currentTimeFunc(sql_context * context, int argc, sql_value ** argv) } } #endif - -/* - * This function registered all of the above C functions as SQL - * functions. This should be the only routine in this file with - * external linkage. - */ -void -sqlRegisterDateTimeFunctions(void) -{ - static FuncDef aDateTimeFuncs[] = { -#if 0 - DFUNCTION(julianday, -1, 0, 0, juliandayFunc, FIELD_TYPE_NUMBER), - DFUNCTION(date, -1, 0, 0, dateFunc, FIELD_TYPE_STRING), - DFUNCTION(time, -1, 0, 0, timeFunc, FIELD_TYPE_STRING), - DFUNCTION(datetime, -1, 0, 0, datetimeFunc, FIELD_TYPE_STRING), - DFUNCTION(strftime, -1, 0, 0, strftimeFunc, FIELD_TYPE_STRING), - DFUNCTION(current_time, 0, 0, 0, ctimeFunc, FIELD_TYPE_STRING), - DFUNCTION(current_timestamp, 0, 0, 0, ctimestampFunc, - FIELD_TYPE_STRING), - DFUNCTION(current_date, 0, 0, 0, cdateFunc, FIELD_TYPE_STRING), - STR_FUNCTION(current_time, 0, "%H:%M:%S", 0, currentTimeFunc), - STR_FUNCTION(current_date, 0, "%Y-%m-%d", 0, currentTimeFunc), - STR_FUNCTION(current_timestamp, 0, "%Y-%m-%d %H:%M:%S", 0, - currentTimeFunc), -#endif - }; - sqlInsertBuiltinFuncs(aDateTimeFuncs, ArraySize(aDateTimeFuncs)); -} diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c index 7ee3e992e..003e49840 100644 --- a/src/box/sql/expr.c +++ b/src/box/sql/expr.c @@ -35,6 +35,8 @@ */ #include "box/coll_id_cache.h" #include "coll/coll.h" +#include "box/func.h" +#include "box/field_def.h" #include "sqlInt.h" #include "tarantoolInt.h" #include "box/schema.h" @@ -275,12 +277,13 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id, if (op == TK_FUNCTION) { uint32_t arg_count = p->x.pList == NULL ? 0 : p->x.pList->nExpr; - struct FuncDef *func = sqlFindFunction(parse->db, - p->u.zToken, - arg_count, 0); + struct func *func = + sql_func_by_signature(p->u.zToken, + strlen(p->u.zToken), arg_count); if (func == NULL) break; - if ((func->funcFlags & SQL_FUNC_DERIVEDCOLL) != 0) { + if ((func->def->sql_flags & + SQL_FUNC_DERIVEDCOLL) != 0) { /* * Now we use quite straightforward * approach assuming that resulting @@ -289,7 +292,7 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id, * built-in functions: trim, upper, * lower, replace, substr. */ - assert(func->ret_type == FIELD_TYPE_STRING); + assert(func->def->returns == FIELD_TYPE_STRING); p = p->x.pList->a->pExpr; continue; } @@ -3923,11 +3926,9 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) case TK_FUNCTION:{ ExprList *pFarg; /* List of function arguments */ int nFarg; /* Number of function arguments */ - FuncDef *pDef; /* The function definition object */ const char *zId; /* The function name */ u32 constMask = 0; /* Mask of function arguments that are constant */ int i; /* Loop counter */ - sql *db = pParse->db; /* The database connection */ struct coll *coll = NULL; assert(!ExprHasProperty(pExpr, EP_xIsSelect)); @@ -3939,8 +3940,10 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) nFarg = pFarg ? pFarg->nExpr : 0; assert(!ExprHasProperty(pExpr, EP_IntValue)); zId = pExpr->u.zToken; - pDef = sqlFindFunction(db, zId, nFarg, 0); - if (pDef == 0 || pDef->xFinalize != 0) { + struct func *func = + sql_func_by_signature(zId, strlen(zId), nFarg); + if (func == NULL || + func->def->aggregate == FUNC_AGGREGATE_GROUP) { diag_set(ClientError, ER_NO_SUCH_FUNCTION, zId); pParse->is_aborted = true; @@ -3950,7 +3953,7 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) * IFNULL() functions. This avoids unnecessary evaluation of * arguments past the first non-NULL argument. */ - if (pDef->funcFlags & SQL_FUNC_COALESCE) { + if (func->def->sql_flags & SQL_FUNC_COALESCE) { int endCoalesce = sqlVdbeMakeLabel(v); assert(nFarg >= 2); sqlExprCode(pParse, pFarg->a[0].pExpr, @@ -3987,7 +3990,7 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) * is done using ANSI rules from * collations_check_compatibility(). */ - if ((pDef->funcFlags & SQL_FUNC_NEEDCOLL) != 0) { + if ((func->def->sql_flags & SQL_FUNC_NEEDCOLL) != 0) { struct coll *unused = NULL; uint32_t curr_id = COLL_NONE; bool is_curr_forced = false; @@ -4034,9 +4037,8 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) * or OPFLAG_TYPEOFARG respectively, to avoid unnecessary data * loading. */ - if ((pDef-> - funcFlags & (SQL_FUNC_LENGTH | - SQL_FUNC_TYPEOF)) != 0) { + if ((func->def->sql_flags & (SQL_FUNC_LENGTH | + SQL_FUNC_TYPEOF)) != 0) { u8 exprOp; assert(nFarg == 1); assert(pFarg->a[0].pExpr != 0); @@ -4051,8 +4053,7 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) funcFlags & OPFLAG_LENGTHARG); pFarg->a[0].pExpr->op2 = - pDef-> - funcFlags & + func->def->sql_flags & (OPFLAG_LENGTHARG | OPFLAG_TYPEOFARG); } @@ -4066,12 +4067,12 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) } else { r1 = 0; } - if (pDef->funcFlags & SQL_FUNC_NEEDCOLL) { + if (func->def->sql_flags & SQL_FUNC_NEEDCOLL) { sqlVdbeAddOp4(v, OP_CollSeq, 0, 0, 0, (char *)coll, P4_COLLSEQ); } sqlVdbeAddOp4(v, OP_Function0, constMask, r1, - target, (char *)pDef, P4_FUNCDEF); + target, (char *)func, P4_FUNC); sqlVdbeChangeP5(v, (u8) nFarg); if (nFarg && constMask == 0) { sqlReleaseTempRange(pParse, r1, nFarg); @@ -5376,12 +5377,18 @@ analyzeAggregate(Walker * pWalker, Expr * pExpr) pItem->iMem = ++pParse->nMem; assert(!ExprHasProperty (pExpr, EP_IntValue)); - pItem->pFunc = sqlFindFunction( - pParse->db, + pItem->func = sql_func_by_signature( pExpr->u.zToken, + strlen(pExpr->u.zToken), pExpr->x.pList ? - pExpr->x.pList->nExpr : 0, - 0); + pExpr->x.pList-> + nExpr : 0); + assert(pItem->func->def-> + language == + FUNC_LANGUAGE_SQL_BUILTIN && + pItem->func->def-> + aggregate == + FUNC_AGGREGATE_GROUP); if (pExpr->flags & EP_Distinct) { pItem->iDistinct = pParse->nTab++; diff --git a/src/box/sql/func.c b/src/box/sql/func.c index f79939cb6..e35b5c2a2 100644 --- a/src/box/sql/func.c +++ b/src/box/sql/func.c @@ -39,14 +39,19 @@ #include "version.h" #include "coll/coll.h" #include "box/func.h" +#include "lua/utils.h" +#include "box/tuple.h" +#include "mpstream.h" #include "box/port.h" #include "tarantoolInt.h" #include "box/session.h" +#include "box/schema.h" #include #include #include #include #include +#include "small/rlist.h" /* * Return the collating function associated with a function. @@ -101,17 +106,222 @@ port_vdbemem_get_context(struct port *base) return port->ctx; } +void +port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat) +{ + (void) is_flat; + struct port_vdbemem *port = (struct port_vdbemem *) base; + assert(is_flat == true); + for (uint32_t i = 0; i < port->size; i++) { + sql_value *param = + (sql_value *)((struct Mem *)port->mem + i); + switch (sql_value_type(param)) { + case MP_INT: + luaL_pushint64(L, sql_value_int64(param)); + break; + case MP_DOUBLE: + lua_pushnumber(L, sql_value_double(param)); + break; + case MP_STR: + lua_pushstring(L, (const char *) sql_value_text(param)); + break; + case MP_BIN: + lua_pushlstring(L, sql_value_blob(param), + (size_t) sql_value_bytes(param)); + break; + case MP_NIL: + lua_pushnil(L); + break; + case MP_BOOL: + lua_pushboolean(L, sql_value_boolean(param)); + break; + default: + unreachable(); + } + } +} + +const char * +port_vdbemem_get_msgpack(struct port *base, uint32_t *size) +{ + struct port_vdbemem *port = (struct port_vdbemem *) base; + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + bool is_error = false; + struct mpstream stream; + mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb, + set_encode_error, &is_error); + mpstream_encode_array(&stream, port->size); + + for (uint32_t i = 0; i < port->size && !is_error; i++) { + sql_value *param = + (sql_value *)((struct Mem *)port->mem + i); + switch (sql_value_type(param)) { + case MP_INT: { + sql_int64 val = sql_value_int64(param); + if (val < 0) + mpstream_encode_int(&stream, val); + else + mpstream_encode_uint(&stream, val); + break; + } + case MP_DOUBLE: { + mpstream_encode_double(&stream, + sql_value_double(param)); + break; + } + case MP_STR: { + mpstream_encode_str(&stream, + (const char *) sql_value_text(param)); + break; + } + case MP_BIN: { + mpstream_encode_binl(&stream, sql_value_bytes(param)); + mpstream_memcpy(&stream, sql_value_blob(param), + sql_value_bytes(param)); + break; + } + case MP_NIL: { + mpstream_encode_nil(&stream); + break; + } + case MP_BOOL: { + mpstream_encode_bool(&stream, sql_value_boolean(param)); + break; + } + default: + unreachable(); + } + } + mpstream_flush(&stream); + *size = region_used(region) - region_svp; + if (is_error) + goto error; + const char *ret = (char *)region_join(region, *size); + if (ret == NULL) + goto error; + return ret; +error: + diag_set(OutOfMemory, *size, "region", "ret"); + return NULL; +} + static const struct port_vtab port_vdbemem_vtab = { .dump_msgpack = NULL, .dump_msgpack_16 = NULL, - .dump_lua = NULL, + .dump_lua = port_vdbemem_dump_lua, .dump_plain = NULL, - .get_msgpack = NULL, + .get_msgpack = port_vdbemem_get_msgpack, .get_vdbemem = port_vdbemem_get_vdbemem, .get_context = port_vdbemem_get_context, .destroy = NULL, }; + +struct sql_value * +port_lua_get_vdbemem(struct port *base, uint32_t *size) +{ + struct port_lua *port = (struct port_lua *) base; + struct lua_State *L = port->L; + int argc = lua_gettop(L); + *size = argc; + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + struct Mem *val = vdbemem_alloc_on_region(argc); + if (val == NULL) + return NULL; + for (int i = 0; i < argc; i++) { + switch(lua_type(L, -1 - i)) { + case LUA_TBOOLEAN: + mem_set_bool(&val[i], lua_toboolean(L, -1 - i)); + break; + case LUA_TNUMBER: + sqlVdbeMemSetDouble(&val[i], lua_tonumber(L, -1 - i)); + break; + case LUA_TSTRING: + if (sqlVdbeMemSetStr(&val[i], lua_tostring(L, -1 - i), -1, + 1, SQL_TRANSIENT) != 0) + goto error; + break; + case LUA_TNIL: + sqlVdbeMemSetNull(&val[i]); + break; + default: + diag_set(ClientError, ER_SQL_EXECUTE, + "Unsupported type passed from Lua"); + goto error; + } + } + return (struct sql_value *)val; +error: + for (int i = 0; i < argc; i++) + sqlVdbeMemRelease(&val[i]); + region_truncate(region, region_svp); + return NULL; +} + +struct sql_value * +port_tuple_get_vdbemem(struct port *base, uint32_t *size) +{ + struct port_tuple *port = (struct port_tuple *)base; + *size = port->size; + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + struct Mem *val = vdbemem_alloc_on_region(port->size); + if (val == NULL) + return NULL; + int i = 0; + struct port_tuple_entry *pe; + for (pe = port->first; pe != NULL; pe = pe->next) { + const char *data = tuple_data(pe->tuple); + if (mp_typeof(*data) != MP_ARRAY || + mp_decode_array(&data) != 1) { + diag_set(ClientError, ER_SQL_EXECUTE, + "Unsupported type passed from C"); + goto error; + } + uint32_t len; + const char *str; + switch (mp_typeof(*data)) { + case MP_BOOL: + mem_set_bool(&val[i], mp_decode_bool(&data)); + break; + case MP_FLOAT: + sqlVdbeMemSetDouble(&val[i], mp_decode_float(&data)); + break; + case MP_DOUBLE: + sqlVdbeMemSetDouble(&val[i], mp_decode_double(&data)); + break; + case MP_INT: + sqlVdbeMemSetInt64(val, mp_decode_int(&data)); + break; + case MP_UINT: + sqlVdbeMemSetInt64(val, mp_decode_uint(&data)); + break; + case MP_STR: + str = mp_decode_str(&data, &len); + if (sqlVdbeMemSetStr(&val[i], str, len, + 1, SQL_TRANSIENT) != 0) + goto error; + break; + case MP_NIL: + sqlVdbeMemSetNull(val); + break; + default: + diag_set(ClientError, ER_SQL_EXECUTE, + "Unsupported type passed from C"); + goto error; + } + i++; + } + return (struct sql_value *) val; +error: + for (int i = 0; i < port->size; i++) + sqlVdbeMemRelease(&val[i]); + region_truncate(region, region_svp); + return NULL; +} + /* * Implementation of the non-aggregate min() and max() functions */ @@ -714,19 +924,6 @@ sql_builtin_ICU##case_type(struct func *func, struct port *args, \ ICU_CASE_CONVERT(Lower); ICU_CASE_CONVERT(Upper); - -/* - * Some functions like COALESCE() and IFNULL() are implemented - * as VDBE code so that unused argument values do not have to be - * computed. - * However, we still need some kind of function implementation for - * this routines in the function table. The sql_builtin_noop macro - * provides this. sql_builtin_noop will never be called so it - * doesn't matter what the implementation is. We might as well - * use the "version()" function as a substitute. - */ -#define sql_builtin_noop sql_builtin_version - /* * Implementation of random(). Return a random integer. */ @@ -2028,142 +2225,302 @@ sql_builtin_group_concat_finalize(struct func *func, struct port *args, } int -sql_is_like_func(struct sql *db, struct Expr *expr, int *is_like_ci) +sql_is_like_func(struct Expr *expr, int *is_like_ci) { if (expr->op != TK_FUNCTION || !expr->x.pList || expr->x.pList->nExpr != 2) return 0; assert(!ExprHasProperty(expr, EP_xIsSelect)); - struct FuncDef *func = sqlFindFunction(db, expr->u.zToken, 2, 0); + struct func *func = + sql_func_by_signature(expr->u.zToken, strlen(expr->u.zToken), 2); assert(func != NULL); - if ((func->funcFlags & SQL_FUNC_LIKE) == 0) + if ((func->def->sql_flags & SQL_FUNC_LIKE) == 0) return 0; *is_like_ci = (current_session()->sql_flags & LIKE_CASE_SENS_FLAG) == 0; return 1; } -/* - * All of the FuncDef structures in the aBuiltinFunc[] array above - * to the global function hash table. This occurs at start-time (as - * a consequence of calling sql_initialize()). - * - * After this routine runs - */ -void -sqlRegisterBuiltinFunctions(void) +static struct func_vtab func_sql_builtin_vtab; + +struct func * +sql_func_by_signature(const char *name, uint32_t name_len, int argc) { - /* - * The following array holds FuncDef structures for all of the functions - * defined in this file. - * - * The array cannot be constant since changes are made to the - * FuncDef.pHash elements at start-time. The elements of this array - * are read-only after initialization is complete. - * - * For peak efficiency, put the most frequently used function last. - */ - static FuncDef aBuiltinFunc[] = { - FUNCTION_COLL(trim, 1, 3, 0, sql_builtin_trim_one_arg), - FUNCTION_COLL(trim, 2, 3, 0, sql_builtin_trim_two_args), - FUNCTION_COLL(trim, 3, 3, 0, sql_builtin_trim_three_args), - FUNCTION(min, -1, 0, 1, sql_builtin_minmax, FIELD_TYPE_SCALAR), - FUNCTION(min, 0, 0, 1, 0, FIELD_TYPE_SCALAR), - AGGREGATE2(min, 1, 0, 1, sql_builtin_minmax_step, - sql_builtin_minmax_finalize, SQL_FUNC_MINMAX, - FIELD_TYPE_SCALAR), - FUNCTION(max, -1, 1, 1, sql_builtin_minmax, FIELD_TYPE_SCALAR), - FUNCTION(max, 0, 1, 1, 0, FIELD_TYPE_SCALAR), - AGGREGATE2(max, 1, 1, 1, sql_builtin_minmax_step, - sql_builtin_minmax_finalize, SQL_FUNC_MINMAX, - FIELD_TYPE_SCALAR), - FUNCTION2(typeof, 1, 0, 0, sql_builtin_typeof, SQL_FUNC_TYPEOF, - FIELD_TYPE_STRING), - FUNCTION2(length, 1, 0, 0, sql_builtin_length, SQL_FUNC_LENGTH, - FIELD_TYPE_INTEGER), - FUNCTION(position, 2, 0, 1, sql_builtin_position, - FIELD_TYPE_INTEGER), - FUNCTION(printf, -1, 0, 0, sql_builtin_printf, - FIELD_TYPE_STRING), - FUNCTION(unicode, 1, 0, 0, sql_builtin_unicode, - FIELD_TYPE_STRING), - FUNCTION(char, -1, 0, 0, sql_builtin_char, FIELD_TYPE_STRING), - FUNCTION(abs, 1, 0, 0, sql_builtin_abs, FIELD_TYPE_NUMBER), - FUNCTION(round, 1, 0, 0, sql_builtin_round, FIELD_TYPE_INTEGER), - FUNCTION(round, 2, 0, 0, sql_builtin_round, FIELD_TYPE_INTEGER), - FUNCTION_COLL(upper, 1, 0, 1, sql_builtin_ICUUpper), - FUNCTION_COLL(lower, 1, 0, 1, sql_builtin_ICULower), - FUNCTION(hex, 1, 0, 0, sql_builtin_hex, FIELD_TYPE_STRING), - FUNCTION2(ifnull, 2, 0, 0, sql_builtin_noop, SQL_FUNC_COALESCE, - FIELD_TYPE_INTEGER), - VFUNCTION(random, 0, 0, 0, sql_builtin_random, - FIELD_TYPE_INTEGER), - VFUNCTION(randomblob, 1, 0, 0, sql_builtin_random_blob, - FIELD_TYPE_SCALAR), - FUNCTION(nullif, 2, 0, 1, sql_builtin_nullif, - FIELD_TYPE_SCALAR), - FUNCTION(version, 0, 0, 0, sql_builtin_version, - FIELD_TYPE_STRING), - FUNCTION(quote, 1, 0, 0, sql_builtin_quote, FIELD_TYPE_STRING), - VFUNCTION(row_count, 0, 0, 0, sql_builtin_row_count, - FIELD_TYPE_INTEGER), - FUNCTION_COLL(replace, 3, 0, 0, sql_builtin_replace), - FUNCTION(zeroblob, 1, 0, 0, sql_builtin_zeroblob, - FIELD_TYPE_SCALAR), - FUNCTION_COLL(substr, 2, 0, 0, sql_builtin_substr), - FUNCTION_COLL(substr, 3, 0, 0, sql_builtin_substr), - AGGREGATE(sum, 1, 0, 0, sql_builtin_sum_step, - sql_builtin_sum_finalize, FIELD_TYPE_NUMBER), - AGGREGATE(total, 1, 0, 0, sql_builtin_sum_step, - sql_builtin_total_finalize, FIELD_TYPE_NUMBER), - AGGREGATE(avg, 1, 0, 0, sql_builtin_sum_step, - sql_builtin_avg_finalize, FIELD_TYPE_NUMBER), - AGGREGATE2(count, 0, 0, 0, sql_builtin_count_step, - sql_builtin_count_finalize, SQL_FUNC_COUNT, - FIELD_TYPE_INTEGER), - AGGREGATE(count, 1, 0, 0, sql_builtin_count_step, - sql_builtin_count_finalize, FIELD_TYPE_INTEGER), - AGGREGATE(group_concat, 1, 0, 0, sql_builtin_group_concat_step, - sql_builtin_group_concat_finalize, FIELD_TYPE_STRING), - AGGREGATE(group_concat, 2, 0, 0, sql_builtin_group_concat_step, - sql_builtin_group_concat_finalize, FIELD_TYPE_STRING), - FUNCTION2(like, 2, 1, 0, sql_builtin_like, SQL_FUNC_LIKE, - FIELD_TYPE_INTEGER), - FUNCTION2(like, 3, 1, 0, sql_builtin_like, SQL_FUNC_LIKE, - FIELD_TYPE_INTEGER), - FUNCTION(coalesce, 1, 0, 0, 0, FIELD_TYPE_SCALAR), - FUNCTION(coalesce, 0, 0, 0, 0, FIELD_TYPE_SCALAR), - FUNCTION2(coalesce, -1, 0, 0, sql_builtin_noop, - SQL_FUNC_COALESCE, FIELD_TYPE_SCALAR), - }; - sql_register_analyze_builtins(); - sqlRegisterDateTimeFunctions(); - sqlInsertBuiltinFuncs(aBuiltinFunc, ArraySize(aBuiltinFunc)); - -#if 0 /* Enable to print out how the built-in functions are hashed */ - { - int i; - FuncDef *p; - for (i = 0; i < SQL_FUNC_HASH_SZ; i++) { - printf("FUNC-HASH %02d:", i); - for (p = sqlBuiltinFunctions.a[i]; p; - p = p->u.pHash) { - int n = sqlStrlen30(p->zName); - int h = p->zName[0] + n; - printf(" %s(%d)", p->zName, h); - } - printf("\n"); + struct func *ret = NULL; + struct func *main_func = func_by_name(name, name_len); + if (main_func == NULL || argc == -2) + return main_func; + if (main_func->def->param_count == argc) { + ret = main_func; + goto end; + } + ret = main_func->def->param_count < 0 ? main_func : NULL; + struct func *func; + rlist_foreach_entry(func, &main_func->signature, signature) { + /** Exact match. */ + if (func->def->param_count == argc) { + ret = func; + goto end; } + /** Multiparam function. */ + if (func->def->param_count < 0) + ret = func; } -#endif +end: + return ret != NULL && ret->def->exports.sql ? ret : NULL; } -struct func_sql_builtin { - /** Function object base class. */ - struct func base; +static int +sql_builtin_stub(struct func *func, struct port *args, struct port *ret) +{ + (void) func; (void) args; (void) ret; + struct Mem *val = vdbemem_alloc_on_region(1); + if (val == NULL) + return -1; + diag_set(ClientError, ER_SQL_EXECUTE, "function is not implemented"); + return -1; +} + +struct sql_builtin_entry { + int param_count; + enum field_type returns; + uint16_t sql_flags; + int (*call)(struct func *func, struct port *args, struct port *ret); + int (*finalize)(struct func *func, struct port *args, struct port *ret); + void *user_data; + enum func_aggregate aggregate; + bool is_deterministic; + bool is_available; }; -static struct func_vtab func_sql_builtin_vtab; +#define BUILTIN_FUNCTION_ENTRY(param_count, returns, flags, call,\ + user_data, is_deterministic, \ + is_available) \ + {param_count, returns, flags, call, NULL, user_data, \ + FUNC_AGGREGATE_NONE, is_deterministic, is_available} + +#define BUILTIN_AGGREGATE_ENTRY(param_count, returns, flags, call,\ + finalize, user_data, is_available)\ + {param_count, returns, flags, call, finalize, user_data, \ + FUNC_AGGREGATE_GROUP, false, is_available} + +#define BUILTIN_STUB_ENTRY() \ + {-1, FIELD_TYPE_ANY, 0, sql_builtin_stub, NULL, \ + NULL, FUNC_AGGREGATE_NONE, false, false} + +/** + * A sequence of SQL builtins definitions in + * lexicographic order. + */ +static struct { + const char *name; + int entries; + struct sql_builtin_entry entry[5]; +} sql_builtins[] = { + {"ABS", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_NUMBER, 0, + sql_builtin_abs, NULL, true, true), + }}, + {"AVG", 1, { + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_NUMBER, 0, + sql_builtin_sum_step, + sql_builtin_avg_finalize, NULL, true), + }}, + {"CEIL", 1, {BUILTIN_STUB_ENTRY()}}, + {"CEILING", 1, {BUILTIN_STUB_ENTRY()}}, + {"CHAR", 1, { + BUILTIN_FUNCTION_ENTRY(-1, FIELD_TYPE_STRING, 0, + sql_builtin_char, NULL, true, true), + }}, + {"CHARACTER_LENGTH", 1, {BUILTIN_STUB_ENTRY()}}, + {"CHAR_LENGTH", 1, {BUILTIN_STUB_ENTRY()}}, + {"COALESCE", 3, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_SCALAR, 0, + sql_builtin_stub, NULL, true, false), + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_SCALAR, 0, + sql_builtin_stub, NULL, true, false), + BUILTIN_FUNCTION_ENTRY(-1, FIELD_TYPE_SCALAR, SQL_FUNC_COALESCE, + sql_builtin_stub, NULL, true, true), + }}, + {"COUNT", 2, { + BUILTIN_AGGREGATE_ENTRY(0, FIELD_TYPE_INTEGER, + SQL_FUNC_COUNT, + sql_builtin_count_step, + sql_builtin_count_finalize, NULL, true), + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_INTEGER, 0, + sql_builtin_count_step, + sql_builtin_count_finalize, NULL, true), + }}, + {"CURRENT_DATE", 1, {BUILTIN_STUB_ENTRY()}}, + {"CURRENT_TIME", 1, {BUILTIN_STUB_ENTRY()}}, + {"CURRENT_TIMESTAMP", 1, {BUILTIN_STUB_ENTRY()}}, + {"DATE", 1, {BUILTIN_STUB_ENTRY()}}, + {"DATETIME", 1, {BUILTIN_STUB_ENTRY()}}, + {"EVERY", 1, {BUILTIN_STUB_ENTRY()}}, + {"EXISTS", 1, {BUILTIN_STUB_ENTRY()}}, + {"EXP", 1, {BUILTIN_STUB_ENTRY()}}, + {"EXTRACT", 1, {BUILTIN_STUB_ENTRY()}}, + {"FLOOR", 1, {BUILTIN_STUB_ENTRY()}}, + {"GREATER", 1, {BUILTIN_STUB_ENTRY()}}, + {"GROUP_CONCAT", 2, { + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_STRING, 0, + sql_builtin_group_concat_step, + sql_builtin_group_concat_finalize, NULL, true), + BUILTIN_AGGREGATE_ENTRY(2, FIELD_TYPE_STRING, 1, + sql_builtin_group_concat_step, + sql_builtin_group_concat_finalize, NULL, true), + }}, + {"HEX", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, 0, + sql_builtin_hex, NULL, true, true), + }}, + {"IFNULL", 1, { + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_INTEGER, SQL_FUNC_COALESCE, + sql_builtin_stub, NULL, true, true), + }}, + {"JULIANDAY", 1, {BUILTIN_STUB_ENTRY()}}, + {"LENGTH", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_INTEGER, SQL_FUNC_LENGTH, + sql_builtin_length, NULL, true, true), + }}, + {"LESSER", 1, {BUILTIN_STUB_ENTRY()}}, + {"LIKE", 2, { + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_INTEGER, SQL_FUNC_LIKE, + sql_builtin_like, SQL_INT_TO_PTR(1), + true, true), + BUILTIN_FUNCTION_ENTRY(3, FIELD_TYPE_INTEGER, SQL_FUNC_LIKE, + sql_builtin_like, SQL_INT_TO_PTR(1), + true, true), + }}, + {"LN", 1, {BUILTIN_STUB_ENTRY()}}, + {"LOWER", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, + SQL_FUNC_DERIVEDCOLL | SQL_FUNC_NEEDCOLL, + sql_builtin_ICULower, NULL, true, true), + }}, + {"MAX", 3, { + BUILTIN_FUNCTION_ENTRY(-1, FIELD_TYPE_SCALAR, SQL_FUNC_NEEDCOLL, + sql_builtin_minmax, SQL_INT_TO_PTR(1), + true, true), + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_SCALAR, 0, + sql_builtin_stub, SQL_INT_TO_PTR(1), + true, false), + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_SCALAR, + SQL_FUNC_NEEDCOLL | SQL_FUNC_MINMAX, + sql_builtin_minmax_step, + sql_builtin_minmax_finalize, + SQL_INT_TO_PTR(1), true), + }}, + {"MIN", 3, { + BUILTIN_FUNCTION_ENTRY(-1, FIELD_TYPE_SCALAR, SQL_FUNC_NEEDCOLL, + sql_builtin_minmax, SQL_INT_TO_PTR(0), + true, true), + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_SCALAR, 0, + sql_builtin_stub, SQL_INT_TO_PTR(0), + true, false), + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_SCALAR, + SQL_FUNC_NEEDCOLL | SQL_FUNC_MINMAX, + sql_builtin_minmax_step, + sql_builtin_minmax_finalize, + SQL_INT_TO_PTR(0), true), + }}, + {"MOD", 1, {BUILTIN_STUB_ENTRY()}}, + {"NULLIF", 1, { + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_SCALAR, SQL_FUNC_NEEDCOLL, + sql_builtin_nullif, NULL, true, true), + }}, + {"OCTET_LENGTH", 1, {BUILTIN_STUB_ENTRY()}}, + {"POSITION", 1, { + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_INTEGER, SQL_FUNC_NEEDCOLL, + sql_builtin_position, NULL, true, true), + }}, + {"POWER", 1, {BUILTIN_STUB_ENTRY()}}, + {"PRINTF", 1, { + BUILTIN_FUNCTION_ENTRY(-1, FIELD_TYPE_STRING, 0, + sql_builtin_printf, NULL, true, true), + }}, + {"QUOTE", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, 0, + sql_builtin_quote, NULL, true, true), + }}, + {"RANDOM", 1, { + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_INTEGER, 0, + sql_builtin_random, NULL, false, true), + }}, + {"RANDOMBLOB", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_SCALAR, 0, + sql_builtin_random_blob, NULL, false, true), + }}, + {"REPLACE", 1, { + BUILTIN_FUNCTION_ENTRY(3, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_replace, NULL, true, true), + }}, + {"ROUND", 2, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_INTEGER, 0, + sql_builtin_round, NULL, true, true), + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_INTEGER, 0, + sql_builtin_round, NULL, true, true), + }}, + {"ROW_COUNT", 1, { + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_INTEGER, 0, + sql_builtin_row_count, NULL, true, true), + }}, + {"SOME", 1, {BUILTIN_STUB_ENTRY()}}, + {"SQRT", 1, {BUILTIN_STUB_ENTRY()}}, + {"STRFTIME", 1, {BUILTIN_STUB_ENTRY()}}, + {"SUBSTR", 2, { + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_substr, NULL, true, true), + BUILTIN_FUNCTION_ENTRY(3, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_substr, NULL, true, true), + }}, + {"SUM", 1, { + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_NUMBER, 0, + sql_builtin_sum_step, + sql_builtin_sum_finalize, NULL, true), + }}, + {"TIME", 1, {BUILTIN_STUB_ENTRY()}}, + {"TOTAL", 1, { + BUILTIN_AGGREGATE_ENTRY(1, FIELD_TYPE_NUMBER, 0, + sql_builtin_sum_step, + sql_builtin_total_finalize, NULL, true), + }}, + {"TRIM", 3, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_trim_one_arg, NULL, + true, true), + BUILTIN_FUNCTION_ENTRY(2, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_trim_two_args, NULL, + true, true), + BUILTIN_FUNCTION_ENTRY(3, FIELD_TYPE_STRING, SQL_FUNC_DERIVEDCOLL, + sql_builtin_trim_three_args, NULL, + true, true), + }}, + {"TYPEOF", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, SQL_FUNC_TYPEOF, + sql_builtin_typeof, NULL, true, true), + }}, + {"UNICODE", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, 0, + sql_builtin_unicode, NULL, true, true), + }}, + {"UPPER", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_STRING, + SQL_FUNC_DERIVEDCOLL | SQL_FUNC_NEEDCOLL, + sql_builtin_ICUUpper, NULL, true, true), + }}, + {"VERSION", 1, { + BUILTIN_FUNCTION_ENTRY(0, FIELD_TYPE_STRING, 0, + sql_builtin_version, NULL, true, true), + }}, + {"ZEROBLOB", 1, { + BUILTIN_FUNCTION_ENTRY(1, FIELD_TYPE_SCALAR, 0, + sql_builtin_zeroblob, NULL, true, true), + }}, + {"_sql_stat_get", 1, {BUILTIN_STUB_ENTRY()}}, + {"_sql_stat_init", 1, {BUILTIN_STUB_ENTRY()}}, + {"_sql_stat_push", 1, {BUILTIN_STUB_ENTRY()}}, +}; struct func * func_sql_builtin_new(struct func_def *def) @@ -2175,26 +2532,88 @@ func_sql_builtin_new(struct func_def *def) "with SQL language"); return NULL; } - struct func_sql_builtin *func = - (struct func_sql_builtin *) malloc(sizeof(*func)); - if (func == NULL) { - diag_set(OutOfMemory, sizeof(*func), "malloc", "func"); + /** Binary search for corresponding builtin entry. */ + int idx = -1, left = 0, right = nelem(sql_builtins) - 1; + while (left <= right) { + uint32_t mid = (left + right) / 2; + int rc = strcmp(def->name, sql_builtins[mid].name); + if (rc == 0) { + idx = mid; + break; + } + if (rc < 0) + right = mid - 1; + else + left = mid + 1; + } + if (idx == -1) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "unknown sql builtin name"); + return NULL; + } + int func_cnt = sql_builtins[idx].entries; + struct func_sql_builtin *funcs = + (struct func_sql_builtin *) malloc(sizeof(*funcs) * func_cnt); + if (funcs == NULL) { + diag_set(OutOfMemory, sizeof(*funcs) * func_cnt, + "malloc", "func"); return NULL; } + int func_idx = 0; + for (; func_idx < func_cnt; func_idx++) { + struct func_def *curr_def; + if (func_idx == 0) { + curr_def = def; + rlist_create(&funcs[func_idx].base.signature); + } else { + curr_def = func_def_dup(def); + if (curr_def == NULL) + goto error; + rlist_add(&funcs[0].base.signature, + &funcs[func_idx].base.signature); + } + curr_def->is_deterministic = + sql_builtins[idx].entry[func_idx].is_deterministic; + curr_def->sql_flags = + sql_builtins[idx].entry[func_idx].sql_flags; + curr_def->param_count = + sql_builtins[idx].entry[func_idx].param_count; + curr_def->returns = + sql_builtins[idx].entry[func_idx].returns; + curr_def->aggregate = + sql_builtins[idx].entry[func_idx].aggregate; + curr_def->exports.sql = + sql_builtins[idx].entry[func_idx].is_available; + funcs[func_idx].base.def = curr_def; + funcs[func_idx].user_data = + sql_builtins[idx].entry[func_idx].user_data; + funcs[func_idx].finalize = + sql_builtins[idx].entry[func_idx].finalize; + funcs[func_idx].base.vtab = func_sql_builtin_vtab; + funcs[func_idx].base.vtab.call = + sql_builtins[idx].entry[func_idx].call; + } /** Don't export SQL builtins in Lua for now. */ def->exports.lua = false; - func->base.vtab = func_sql_builtin_vtab; - return &func->base; + return &funcs[0].base; +error: + for (int i = 1; i < func_idx; i++) + free(funcs[func_idx].base.def); + free(funcs); + return NULL; } static void func_sql_builtin_destroy(struct func *base) { assert(base != NULL && base->def->language == FUNC_LANGUAGE_SQL_BUILTIN); + struct func *func, *tmp; + rlist_foreach_entry_safe(func, &base->signature, signature, tmp) + free(func->def); free(base); } static struct func_vtab func_sql_builtin_vtab = { - .call = NULL, + .call = sql_builtin_stub, .destroy = func_sql_builtin_destroy, }; diff --git a/src/box/sql/global.c b/src/box/sql/global.c index 6cadef809..c25b83de1 100644 --- a/src/box/sql/global.c +++ b/src/box/sql/global.c @@ -162,13 +162,6 @@ SQL_WSD struct sqlConfig sqlConfig = { 0x7ffffffe /* iOnceResetThreshold */ }; -/* - * Hash table for global functions - functions common to all - * database connections. After initialization, this table is - * read-only. - */ -FuncDefHash sqlBuiltinFunctions; - /* * The value of the "pending" byte must be 0x40000000 (1 byte past the * 1-gibabyte boundary) in a compatible database. sql never uses diff --git a/src/box/sql/main.c b/src/box/sql/main.c index baaf7dcc9..16505ac7f 100644 --- a/src/box/sql/main.c +++ b/src/box/sql/main.c @@ -127,9 +127,6 @@ sql_initialize(void) if (sqlGlobalConfig.isInit == 0 && sqlGlobalConfig.inProgress == 0) { sqlGlobalConfig.inProgress = 1; - memset(&sqlBuiltinFunctions, 0, - sizeof(sqlBuiltinFunctions)); - sqlRegisterBuiltinFunctions(); sql_os_init(); sqlGlobalConfig.isInit = 1; sqlGlobalConfig.inProgress = 0; @@ -177,25 +174,6 @@ sqlCloseSavepoints(Vdbe * pVdbe) pVdbe->anonymous_savepoint = NULL; } -/* - * Invoke the destructor function associated with FuncDef p, if any. Except, - * if this is not the last copy of the function, do not invoke it. Multiple - * copies of a single function are created when create_function() is called - * with SQL_ANY as the encoding. - */ -static void -functionDestroy(sql * db, FuncDef * p) -{ - FuncDestructor *pDestructor = p->u.pDestructor; - if (pDestructor) { - pDestructor->nRef--; - if (pDestructor->nRef == 0) { - pDestructor->xDestroy(pDestructor->pUserData); - sqlDbFree(db, pDestructor); - } - } -} - /* * Rollback all database files. If tripCode is not 0, then * any write cursors are invalidated ("tripped" - as in "tripping a circuit @@ -214,118 +192,6 @@ sqlRollbackAll(Vdbe * pVdbe) } } -/* - * This function is exactly the same as sql_create_function(), except - * that it is designed to be called by internal code. The difference is - * that if a malloc() fails in sql_create_function(), an error code - * is returned and the mallocFailed flag cleared. - */ -int -sqlCreateFunc(struct sql * db, const char *zFunctionName, enum field_type type, - int nArg, int flags, void *pUserData, - int (*xSFunc) (struct func *func, struct port *args, - struct port *ret), - int (*xStep) (struct func *func, struct port *args, - struct port *ret), - int (*xFinal) (struct func *func, struct port *args, - struct port *ret), - struct FuncDestructor * pDestructor) -{ - FuncDef *p; - int extraFlags; - - if (zFunctionName == 0 || - (xSFunc && (xFinal || xStep)) || - (!xSFunc && (xFinal && !xStep)) || - (!xSFunc && (!xFinal && xStep)) || - (nArg < -1 || nArg > SQL_MAX_FUNCTION_ARG) || - (255 < (sqlStrlen30(zFunctionName)))) { - diag_set(ClientError, ER_CREATE_FUNCTION, zFunctionName, - "wrong function definition"); - return -1; - } - - assert(SQL_FUNC_CONSTANT == SQL_DETERMINISTIC); - extraFlags = flags & SQL_DETERMINISTIC; - - - /* Check if an existing function is being overridden or deleted. If so, - * and there are active VMs, then return an error. If a function - * is being overridden/deleted but there are no active VMs, allow the - * operation to continue but invalidate all precompiled statements. - */ - p = sqlFindFunction(db, zFunctionName, nArg, 0); - if (p && p->nArg == nArg) { - if (db->nVdbeActive) { - diag_set(ClientError, ER_CREATE_FUNCTION, zFunctionName, - "unable to create function due to active "\ - "statements"); - return -1; - } else { - sqlExpirePreparedStatements(db); - } - } - - p = sqlFindFunction(db, zFunctionName, nArg, 1); - assert(p || db->mallocFailed); - if (p == NULL) - return -1; - - /* If an older version of the function with a configured destructor is - * being replaced invoke the destructor function here. - */ - functionDestroy(db, p); - - if (pDestructor) { - pDestructor->nRef++; - } - p->u.pDestructor = pDestructor; - p->funcFlags = extraFlags; - testcase(p->funcFlags & SQL_DETERMINISTIC); - p->xSFunc = xSFunc ? xSFunc : xStep; - p->xFinalize = xFinal; - p->pUserData = pUserData; - p->nArg = (u16) nArg; - p->ret_type = type; - return 0; -} - - -int -sql_create_function_v2(struct sql * db, const char *zFunc, enum field_type type, - int nArg, int flags, void *p, - int (*xSFunc) (struct func *func, struct port *args, - struct port *ret), - int (*xStep) (struct func *func, struct port *args, - struct port *ret), - int (*xFinal) (struct func *func, struct port *args, - struct port *ret), - void (*xDestroy) (void *ctx)) -{ - FuncDestructor *pArg = 0; - - if (xDestroy) { - pArg = - (FuncDestructor *) sqlDbMallocZero(db, - sizeof - (FuncDestructor)); - if (!pArg) { - xDestroy(p); - return -1; - } - pArg->xDestroy = xDestroy; - pArg->pUserData = p; - } - int rc = sqlCreateFunc(db, zFunc, type, nArg, flags, p, xSFunc, xStep, - xFinal, pArg); - if (pArg && pArg->nRef == 0) { - assert(rc != 0); - xDestroy(p); - sqlDbFree(db, pArg); - } - return rc; -} - /* * This array defines hard upper bounds on limit values. The * initializer must be kept in sync with the SQL_LIMIT_* diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c index 7c81b1980..7c67d5d56 100644 --- a/src/box/sql/resolve.c +++ b/src/box/sql/resolve.c @@ -38,6 +38,8 @@ #include "sqlInt.h" #include #include +#include "box/func.h" +#include "box/field_def.h" /* * Walk the expression tree pExpr and increase the aggregate function @@ -576,36 +578,31 @@ resolveExprStep(Walker * pWalker, Expr * pExpr) int no_such_func = 0; /* True if no such function exists */ int wrong_num_args = 0; /* True if wrong number of arguments */ int is_agg = 0; /* True if is an aggregate function */ - int nId; /* Number of characters in function name */ - const char *zId; /* The function name. */ - FuncDef *pDef; /* Information about the function */ - assert(!ExprHasProperty(pExpr, EP_xIsSelect)); - zId = pExpr->u.zToken; - nId = sqlStrlen30(zId); - pDef = sqlFindFunction(pParse->db, zId, n, 0); - if (pDef == 0) { - pDef = - sqlFindFunction(pParse->db, zId, -2,0); - if (pDef == 0) { + const char *name = pExpr->u.zToken; + uint32_t name_len = strlen(name); + struct func *func = + sql_func_by_signature(name, name_len, n); + if (func == 0) { + func = sql_func_by_signature(name, name_len, -2); + if (func == NULL) { no_such_func = 1; } else { wrong_num_args = 1; } } else { - is_agg = pDef->xFinalize != 0; - pExpr->type = pDef->ret_type; - if (pDef-> - funcFlags & (SQL_FUNC_CONSTANT | - SQL_FUNC_SLOCHNG)) { + is_agg = func->def->language == + FUNC_LANGUAGE_SQL_BUILTIN && + func->def->aggregate == + FUNC_AGGREGATE_GROUP; + pExpr->type = func->def->returns; + if (func->def->is_deterministic) { /* For the purposes of the EP_ConstFunc flag, date and time * functions and other functions that change slowly are considered * constant because they are constant for the duration of one query */ ExprSetProperty(pExpr, EP_ConstFunc); - } - if ((pDef->funcFlags & SQL_FUNC_CONSTANT) == - 0) { + } else { /* Date/time functions that use 'now', and other functions * that might change over time cannot be used * in an index. @@ -614,22 +611,21 @@ resolveExprStep(Walker * pWalker, Expr * pExpr) } } if (is_agg && (pNC->ncFlags & NC_AllowAgg) == 0) { - const char *err = - tt_sprintf("misuse of aggregate "\ - "function %.*s()", nId, zId); - diag_set(ClientError, ER_SQL_PARSER_GENERIC, err); + diag_set(ClientError, ER_SQL_PARSER_GENERIC, + tt_sprintf("misuse of aggregate " + "function %s()", name)); pParse->is_aborted = true; pNC->nErr++; is_agg = 0; } else if (no_such_func && pParse->db->init.busy == 0) { - diag_set(ClientError, ER_NO_SUCH_FUNCTION, zId); + diag_set(ClientError, ER_NO_SUCH_FUNCTION, + name); pParse->is_aborted = true; pNC->nErr++; } else if (wrong_num_args) { - const char *err = "wrong number of arguments "\ - "to function %.*s()"; diag_set(ClientError, ER_SQL_PARSER_GENERIC, - tt_sprintf(err, nId, zId)); + tt_sprintf("wrong number of arguments " + "to function %s()", name)); pParse->is_aborted = true; pNC->nErr++; } @@ -648,7 +644,7 @@ resolveExprStep(Walker * pWalker, Expr * pExpr) pExpr->op2++; pNC2 = pNC2->pNext; } - assert(pDef != 0); + assert(func != NULL); if (pNC2) { assert(SQL_FUNC_MINMAX == NC_MinMaxAgg); @@ -656,8 +652,7 @@ resolveExprStep(Walker * pWalker, Expr * pExpr) funcFlags & SQL_FUNC_MINMAX) != 0); pNC2->ncFlags |= - NC_HasAgg | (pDef-> - funcFlags & + NC_HasAgg | (func->def->sql_flags & SQL_FUNC_MINMAX); } diff --git a/src/box/sql/select.c b/src/box/sql/select.c index 7c8da251e..70b39ff30 100644 --- a/src/box/sql/select.c +++ b/src/box/sql/select.c @@ -4374,7 +4374,7 @@ is_simple_count(struct Select *select, struct AggInfo *agg_info) return NULL; if (NEVER(agg_info->nFunc == 0)) return NULL; - if ((agg_info->aFunc[0].pFunc->funcFlags & SQL_FUNC_COUNT) == 0) + if ((agg_info->aFunc[0].func->def->sql_flags & SQL_FUNC_COUNT) == 0) return NULL; if (expr->flags & EP_Distinct) return NULL; @@ -5269,7 +5269,7 @@ finalizeAggFunctions(Parse * pParse, AggInfo * pAggInfo) assert(!ExprHasProperty(pF->pExpr, EP_xIsSelect)); sqlVdbeAddOp2(v, OP_AggFinal, pF->iMem, pList ? pList->nExpr : 0); - sqlVdbeAppendP4(v, pF->pFunc, P4_FUNCDEF); + sqlVdbeAppendP4(v, pF->func, P4_FUNC); } } @@ -5310,11 +5310,11 @@ updateAccumulator(Parse * pParse, AggInfo * pAggInfo) vdbe_insert_distinct(pParse, pF->iDistinct, pF->reg_eph, addrNext, 1, regAgg); } - if (pF->pFunc->funcFlags & SQL_FUNC_NEEDCOLL) { + if ((pF->func->def->sql_flags & SQL_FUNC_NEEDCOLL) != 0) { struct coll *coll = NULL; struct ExprList_item *pItem; int j; - assert(pList != 0); /* pList!=0 if pF->pFunc has NEEDCOLL */ + assert(pList != 0); bool unused; uint32_t id; for (j = 0, pItem = pList->a; coll == NULL && j < nArg; @@ -5329,7 +5329,7 @@ updateAccumulator(Parse * pParse, AggInfo * pAggInfo) (char *)coll, P4_COLLSEQ); } sqlVdbeAddOp3(v, OP_AggStep0, 0, regAgg, pF->iMem); - sqlVdbeAppendP4(v, pF->pFunc, P4_FUNCDEF); + sqlVdbeAppendP4(v, pF->func, P4_FUNC); sqlVdbeChangeP5(v, (u8) nArg); sql_expr_type_cache_change(pParse, regAgg, nArg); sqlReleaseTempRange(pParse, regAgg, nArg); diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index ce15500e5..dc0a06f5a 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -520,19 +520,6 @@ sql_initialize(void); #define SQL_TRACE_ROW 0x04 #define SQL_TRACE_CLOSE 0x08 -#define SQL_DETERMINISTIC 0x800 - -int -sql_create_function_v2(struct sql * db, const char *zFunc, enum field_type type, - int nArg, int flags, void *p, - int (*xSFunc) (struct func *func, struct port *args, - struct port *ret), - int (*xStep) (struct func *func, struct port *args, - struct port *ret), - int (*xFinal) (struct func *func, struct port *args, - struct port *ret), - void (*xDestroy) (void *ctx)); - #define SQL_OPEN_READONLY 0x00000001 /* Ok for sql_open_v2() */ #define SQL_OPEN_READWRITE 0x00000002 /* Ok for sql_open_v2() */ #define SQL_OPEN_CREATE 0x00000004 /* Ok for sql_open_v2() */ @@ -983,9 +970,6 @@ typedef struct Column Column; typedef struct Expr Expr; typedef struct ExprList ExprList; typedef struct ExprSpan ExprSpan; -typedef struct FuncDestructor FuncDestructor; -typedef struct FuncDef FuncDef; -typedef struct FuncDefHash FuncDefHash; typedef struct IdList IdList; typedef struct KeyClass KeyClass; typedef struct NameContext NameContext; @@ -1030,18 +1014,6 @@ typedef int VList; */ #define SQL_N_LIMIT (SQL_LIMIT_TRIGGER_DEPTH+1) -/* - * A hash table for built-in function definitions. (Application-defined - * functions use a regular table table from hash.h.) - * - * Hash each FuncDef structure into one of the FuncDefHash.a[] slots. - * Collisions are on the FuncDef.u.pHash chain. - */ -#define SQL_FUNC_HASH_SZ 23 -struct FuncDefHash { - FuncDef *a[SQL_FUNC_HASH_SZ]; /* Hash table for functions */ -}; - /* * Each database connection is an instance of the following structure. */ @@ -1152,62 +1124,14 @@ struct type_def { }; /* - * Each SQL function is defined by an instance of the following - * structure. For global built-in functions (ex: substr(), max(), count()) - * a pointer to this structure is held in the sqlBuiltinFunctions object. - * For per-connection application-defined functions, a pointer to this - * structure is held in the db->aHash hash table. - * - * The u.pHash field is used by the global built-ins. The u.pDestructor - * field is used by per-connection app-def functions. - */ -struct FuncDef { - i8 nArg; /* Number of arguments. -1 means unlimited */ - u16 funcFlags; /* Some combination of sql_FUNC_* */ - void *pUserData; /* User data parameter */ - FuncDef *pNext; /* Next function with same name */ - int (*xSFunc) (struct func *func, struct port *args, struct port *ret); - int (*xFinalize) (struct func *func, struct port *args, struct port *ret); - const char *zName; /* SQL name of the function. */ - union { - FuncDef *pHash; /* Next with a different name but the same hash */ - FuncDestructor *pDestructor; /* Reference counted destructor function */ - } u; - /* Return type. */ - enum field_type ret_type; -}; - -/* - * This structure encapsulates a user-function destructor callback (as - * configured using create_function_v2()) and a reference counter. When - * create_function_v2() is called to create a function with a destructor, - * a single object of this type is allocated. FuncDestructor.nRef is set to - * the number of FuncDef objects created (either 1 or 3, depending on whether - * or not the specified encoding is sql_ANY). The FuncDef.pDestructor - * member of each of the new FuncDef objects is set to point to the allocated - * FuncDestructor. - * - * Thereafter, when one of the FuncDef objects is deleted, the reference - * count on this object is decremented. When it reaches 0, the destructor - * is invoked and the FuncDestructor structure freed. - */ -struct FuncDestructor { - int nRef; - void (*xDestroy) (void *); - void *pUserData; -}; - -/* - * Possible values for FuncDef.flags. Note that the _LENGTH and _TYPEOF - * values must correspond to OPFLAG_LENGTHARG and OPFLAG_TYPEOFARG. And - * sql_FUNC_CONSTANT must be the same as sql_DETERMINISTIC. There - * are assert() statements in the code to verify this. + * Possible values for FuncDef.flags. Note that the _LENGTH and + * _TYPEOF values must correspond to OPFLAG_LENGTHARG and + * OPFLAG_TYPEOFARG. * * Value constraints (enforced via assert()): * SQL_FUNC_MINMAX == NC_MinMaxAgg == SF_MinMaxAgg * SQL_FUNC_LENGTH == OPFLAG_LENGTHARG * SQL_FUNC_TYPEOF == OPFLAG_TYPEOFARG - * SQL_FUNC_CONSTANT == sql_DETERMINISTIC from the API */ #define SQL_FUNC_LIKE 0x0004 /* Candidate for the LIKE optimization */ #define SQL_FUNC_EPHEM 0x0010 /* Ephemeral. Delete with VDBE */ @@ -1229,11 +1153,7 @@ struct FuncDestructor { * argument. */ #define SQL_FUNC_DERIVEDCOLL 0x0400 -#define SQL_FUNC_CONSTANT 0x0800 /* Constant inputs give a constant output */ #define SQL_FUNC_MINMAX 0x1000 /* True for min() and max() aggregates */ -#define SQL_FUNC_SLOCHNG 0x2000 /* "Slow Change". Value constant during a - * single query - might change over time - */ /* * Trim side mask components. TRIM_LEADING means to trim left side @@ -1246,72 +1166,6 @@ enum trim_side_mask { TRIM_BOTH = TRIM_LEADING | TRIM_TRAILING }; -/* - * The following three macros, FUNCTION(), LIKEFUNC() and AGGREGATE() are - * used to create the initializers for the FuncDef structures. - * - * FUNCTION(zName, nArg, iArg, bNC, xFunc) - * Used to create a scalar function definition of a function zName - * implemented by C function xFunc that accepts nArg arguments. The - * value passed as iArg is cast to a (void*) and made available - * as the user-data (sql_user_data()) for the function. If - * argument bNC is true, then the sql_FUNC_NEEDCOLL flag is set. - * - * FUNCTION_COLL - * Like FUNCTION except it assumes that function returns - * STRING which collation should be derived from first - * argument (trim, substr etc). - * - * VFUNCTION(zName, nArg, iArg, bNC, xFunc) - * Like FUNCTION except it omits the sql_FUNC_CONSTANT flag. - * - * DFUNCTION(zName, nArg, iArg, bNC, xFunc) - * Like FUNCTION except it omits the sql_FUNC_CONSTANT flag and - * adds the sql_FUNC_SLOCHNG flag. Used for date & time functions, - * but not during a single query. - * - * AGGREGATE(zName, nArg, iArg, bNC, xStep, xFinal) - * Used to create an aggregate function definition implemented by - * the C functions xStep and xFinal. The first four parameters - * are interpreted in the same way as the first 4 parameters to - * FUNCTION(). - * - * LIKEFUNC(zName, nArg, pArg, flags) - * Used to create a scalar function definition of a function zName - * that accepts nArg arguments and is implemented by a call to C - * function likeFunc. Argument pArg is cast to a (void *) and made - * available as the function user-data (sql_user_data()). The - * FuncDef.flags variable is set to the value passed as the flags - * parameter. - */ -#define FUNCTION(zName, nArg, iArg, bNC, xFunc, type) \ - {nArg, SQL_FUNC_CONSTANT|(bNC*SQL_FUNC_NEEDCOLL), \ - SQL_INT_TO_PTR(iArg), 0, xFunc, 0, #zName, {0}, type} -#define FUNCTION_COLL(zName, nArg, iArg, bNC, xFunc) \ - {nArg, SQL_FUNC_CONSTANT|SQL_FUNC_DERIVEDCOLL|(bNC*SQL_FUNC_NEEDCOLL), \ - SQL_INT_TO_PTR(iArg), 0, xFunc, 0, #zName, {0}, FIELD_TYPE_STRING} -#define VFUNCTION(zName, nArg, iArg, bNC, xFunc, type) \ - {nArg, (bNC*SQL_FUNC_NEEDCOLL), \ - SQL_INT_TO_PTR(iArg), 0, xFunc, 0, #zName, {0}, type} -#define DFUNCTION(zName, nArg, iArg, bNC, xFunc, type) \ - {nArg, SQL_FUNC_SLOCHNG|(bNC*SQL_FUNC_NEEDCOLL), \ - SQL_INT_TO_PTR(iArg), 0, xFunc, 0, #zName, {0}, type} -#define FUNCTION2(zName, nArg, iArg, bNC, xFunc, extraFlags, type) \ - {nArg,SQL_FUNC_CONSTANT|(bNC*SQL_FUNC_NEEDCOLL)|extraFlags,\ - SQL_INT_TO_PTR(iArg), 0, xFunc, 0, #zName, {0}, type} -#define STR_FUNCTION(zName, nArg, pArg, bNC, xFunc) \ - {nArg, SQL_FUNC_SLOCHNG|(bNC*SQL_FUNC_NEEDCOLL), \ - pArg, 0, xFunc, 0, #zName, {SQL_AFF_STRING, {0}}} -#define LIKEFUNC(zName, nArg, arg, flags, type) \ - {nArg, SQL_FUNC_CONSTANT|flags, \ - (void *)(SQL_INT_TO_PTR(arg)), 0, likeFunc, 0, #zName, {0}, type} -#define AGGREGATE(zName, nArg, arg, nc, xStep, xFinal, type) \ - {nArg, (nc*SQL_FUNC_NEEDCOLL), \ - SQL_INT_TO_PTR(arg), 0, xStep,xFinal,#zName, {0}, type} -#define AGGREGATE2(zName, nArg, arg, nc, xStep, xFinal, extraFlags, type) \ - {nArg, (nc*SQL_FUNC_NEEDCOLL)|extraFlags, \ - SQL_INT_TO_PTR(arg), 0, xStep,xFinal,#zName, {0}, type} - /* * All current savepoints are stored in a linked list starting at * sql.pSavepoint. The first element in the list is the most recently @@ -1504,7 +1358,8 @@ struct AggInfo { */ struct AggInfo_func { /* For each aggregate function */ Expr *pExpr; /* Expression encoding the function */ - FuncDef *pFunc; /* The aggregate function implementation */ + /** The aggregate function implementation. */ + struct func *func; int iMem; /* Memory location that acts as accumulator */ int iDistinct; /* Ephemeral table used to enforce DISTINCT */ /** @@ -1654,7 +1509,8 @@ struct Expr { #define EP_Static 0x008000 /* Held in memory not obtained from malloc() */ #define EP_MemToken 0x010000 /* Need to sqlDbFree() Expr.zToken */ #define EP_NoReduce 0x020000 /* Cannot EXPRDUP_REDUCE this Expr */ -#define EP_ConstFunc 0x080000 /* A sql_FUNC_CONSTANT or _SLOCHNG function */ +/** A deterministic function. */ +#define EP_ConstFunc 0x080000 #define EP_CanBeNull 0x100000 /* Can be null despite NOT NULL constraint */ #define EP_Subquery 0x200000 /* Tree contains a TK_SELECT operator */ #define EP_Alias 0x400000 /* Is an alias for a result set column */ @@ -3462,10 +3318,6 @@ void sqlSelectSetName(Select *, const char *); #else #define sqlSelectSetName(A,B) #endif -void sqlInsertBuiltinFuncs(FuncDef *, int); -FuncDef *sqlFindFunction(sql *, const char *, int, u8); -void sqlRegisterBuiltinFunctions(void); -void sqlRegisterDateTimeFunctions(void); /** * Evaluate a view and store its result in an ephemeral table. @@ -4037,7 +3889,6 @@ extern const unsigned char sqlUpperToLower[]; extern const unsigned char sqlCtypeMap[]; extern const Token sqlIntTokens[]; extern SQL_WSD struct sqlConfig sqlConfig; -extern FuncDefHash sqlBuiltinFunctions; extern int sqlPendingByte; /** @@ -4218,18 +4069,7 @@ sql_key_info_to_key_def(struct sql_key_info *key_info); * @retval 1 if LIKE optimization can be used, 0 otherwise. */ int -sql_is_like_func(struct sql *db, struct Expr *expr, int *is_like_ci); - -int -sqlCreateFunc(struct sql * db, const char *zFunctionName, enum field_type type, - int nArg, int flags, void *pUserData, - int (*xSFunc) (struct func *func, struct port *args, - struct port *ret), - int (*xStep) (struct func *func, struct port *args, - struct port *ret), - int (*xFinal) (struct func *func, struct port *args, - struct port *ret), - struct FuncDestructor * pDestructor); +sql_is_like_func(struct Expr *expr, int *is_like_ci); /** Set OOM error flag. */ static inline void @@ -4456,10 +4296,6 @@ Expr *sqlExprForVectorField(Parse *, Expr *, int); */ extern int sqlSubProgramsRemaining; -/** Register built-in functions to work with ANALYZE data. */ -void -sql_register_analyze_builtins(void); - /** * Generate VDBE code to halt execution with correct error if * the object with specified key is already present (or doesn't diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 92c99877a..eb18d4b7a 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -1722,14 +1722,14 @@ case OP_CollSeq: { case OP_Function0: { sql_context *pCtx; - assert(pOp->p4type==P4_FUNCDEF); + assert(pOp->p4type == P4_FUNC); assert(pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor)); assert(pOp->p5 == 0 || (pOp->p2 > 0 && pOp->p2 + pOp->p5 <= (p->nMem+1 - p->nCursor)+1)); assert(pOp->p3 < pOp->p2 || pOp->p3>=pOp->p2 + pOp->p5); pCtx = sqlDbMallocRawNN(db, sizeof(*pCtx)); if (pCtx==0) goto no_mem; - pCtx->pFunc = pOp->p4.pFunc; + pCtx->func = pOp->p4.func; pCtx->iOp = (int)(pOp - aOp); pCtx->pVdbe = p; pOp->p4type = P4_FUNCCTX; @@ -1764,7 +1764,7 @@ case OP_Function: { } #endif MemSetTypeFlag(pOut, MEM_Null); - rc = (*pCtx->pFunc->xSFunc)(NULL, &args, &ret); + rc = func_call(pCtx->func, &args, &ret); if (rc != 0) goto abort_due_to_error; uint32_t size; @@ -4969,15 +4969,18 @@ case OP_DecrJumpZero: { /* jump, in1 */ case OP_AggStep0: { sql_context *pCtx; - assert(pOp->p4type == P4_FUNCDEF); + assert(pOp->p4type == P4_FUNC); assert(pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor)); assert(pOp->p5 == 0 || (pOp->p2 > 0 && pOp->p2 + pOp->p5 <= (p->nMem+1 - p->nCursor)+1)); - assert(pOp->p3 < pOp->p2 || pOp->p3 >= pOp->p2 + pOp->p5); + assert(pOp->p3p2 || pOp->p3>=pOp->p2 + pOp->p5); + assert(pOp->p4.func != NULL && + pOp->p4.func->def->language == FUNC_LANGUAGE_SQL_BUILTIN && + pOp->p4.func->def->aggregate == FUNC_AGGREGATE_GROUP); pCtx = sqlDbMallocRawNN(db, sizeof(*pCtx)); if (pCtx==0) goto no_mem; pCtx->pMem = 0; - pCtx->pFunc = pOp->p4.pFunc; + pCtx->func = pOp->p4.func; pCtx->iOp = (int)(pOp - aOp); pCtx->pVdbe = p; pOp->p4type = P4_FUNCCTX; @@ -5011,7 +5014,7 @@ case OP_AggStep: { sqlVdbeMemInit(&t, db, MEM_Null); pOut = &t; pCtx->skipFlag = 0; - rc = (*pCtx->pFunc->xSFunc)(NULL, &args, &ret); + rc = func_call(pCtx->func, &args, &ret); region_truncate(region, region_svp); if (rc != 0) goto abort_due_to_error; @@ -5042,7 +5045,7 @@ case OP_AggFinal: { assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor)); pMem = &aMem[pOp->p1]; assert((pMem->flags & ~(MEM_Null|MEM_Agg))==0); - if (sqlVdbeMemFinalize(pMem, pOp->p4.pFunc) != 0) + if (sql_vdbemem_finilize(pMem, pOp->p4.func) != 0) goto abort_due_to_error; UPDATE_MAX_BLOBSIZE(pMem); if (sqlVdbeMemTooBig(pMem)) { diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h index 4f94643cb..31a0a39d7 100644 --- a/src/box/sql/vdbe.h +++ b/src/box/sql/vdbe.h @@ -72,7 +72,8 @@ struct VdbeOp { char *z; /* Pointer to data for string (char array) types */ i64 *pI64; /* Used when p4type is P4_INT64 */ double *pReal; /* Used when p4type is P4_REAL */ - FuncDef *pFunc; /* Used when p4type is P4_FUNCDEF */ + /* Used when p4type is P4_FUNC. */ + struct func *func; sql_context *pCtx; /* Used when p4type is P4_FUNCCTX */ struct coll *pColl; /* Used when p4type is P4_COLLSEQ */ Mem *pMem; /* Used when p4type is P4_MEM */ @@ -122,7 +123,8 @@ struct SubProgram { #define P4_DYNAMIC (-1) /* Pointer to a string obtained from sqlMalloc() */ #define P4_STATIC (-2) /* Pointer to a static string */ #define P4_COLLSEQ (-3) /* P4 is a pointer to a CollSeq structure */ -#define P4_FUNCDEF (-4) /* P4 is a pointer to a FuncDef structure */ +/** P4 is a pointer to a function object. */ +#define P4_FUNC (-4) #define P4_MEM (-7) /* P4 is a pointer to a Mem* structure */ #define P4_TRANSIENT 0 /* P4 is a pointer to a transient string */ #define P4_REAL (-9) /* P4 is a 64-bit floating point value */ diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index 4c8363a22..0d4acb0eb 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -46,6 +46,8 @@ */ typedef struct VdbeOp Op; +struct func; + /* * Boolean values */ @@ -167,7 +169,8 @@ struct Mem { bool b; /* Boolean value used when MEM_Bool is set in flags */ int nZero; /* Used when bit MEM_Zero is set in flags */ void *p; /* Generic pointer */ - FuncDef *pDef; /* Used only when flags==MEM_Agg */ + /** Used only when flags==MEM_Agg. */ + struct func *func; VdbeFrame *pFrame; /* Used when flags==MEM_Frame */ } u; u32 flags; /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */ @@ -291,7 +294,8 @@ mem_apply_numeric_type(struct Mem *record); * (Mem) which are only defined there. */ struct sql_context { - FuncDef *pFunc; /* Pointer to function information */ + /** Pointer to function object. */ + struct func *func; Mem *pMem; /* Memory cell used to store aggregate context */ Vdbe *pVdbe; /* The VM that owns this context */ int iOp; /* Instruction number of OP_Function */ @@ -472,7 +476,17 @@ int sqlVdbeMemNumerify(Mem *); int sqlVdbeMemCast(Mem *, enum field_type type); int sqlVdbeMemFromBtree(BtCursor *, u32, u32, Mem *); void sqlVdbeMemRelease(Mem * p); -int sqlVdbeMemFinalize(Mem *, FuncDef *); + +/** + * Memory cell pMem contains the context of an aggregate function. + * This routine calls the finalize method for that function. The + * result of the aggregate is stored back into pMem. + * + * Returns -1 if the finalizer reports an error. 0 otherwise. + */ +int +sql_vdbemem_finilize(struct Mem *mem, struct func *func); + const char *sqlOpcodeName(int); int sqlVdbeMemGrow(Mem * pMem, int n, int preserve); int sqlVdbeMemClearAndResize(Mem * pMem, int n); diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index d014e8258..c59b740a7 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -370,8 +370,9 @@ sql_step(sql_stmt * pStmt) void * sql_user_data(sql_context * p) { - assert(p && p->pFunc); - return p->pFunc->pUserData; + assert(p != NULL && p->func != NULL && + p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN); + return ((struct func_sql_builtin *)p->func)->user_data; } /* @@ -416,7 +417,7 @@ createAggContext(sql_context * p, int nByte) } else { sqlVdbeMemClearAndResize(pMem, nByte); pMem->flags = MEM_Agg; - pMem->u.pDef = p->pFunc; + pMem->u.func = p->func; if (pMem->z) { memset(pMem->z, 0, nByte); } @@ -432,7 +433,9 @@ createAggContext(sql_context * p, int nByte) void * sql_aggregate_context(sql_context * p, int nByte) { - assert(p && p->pFunc && p->pFunc->xFinalize); + assert(p != NULL && p->func != NULL && + p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN && + p->func->def->aggregate == FUNC_AGGREGATE_GROUP); testcase(nByte < 0); if ((p->pMem->flags & MEM_Agg) == 0) { return createAggContext(p, nByte); diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c index baeeb4624..2c6f5b279 100644 --- a/src/box/sql/vdbeaux.c +++ b/src/box/sql/vdbeaux.c @@ -39,6 +39,8 @@ #include "box/schema.h" #include "box/tuple_format.h" #include "box/txn.h" +#include "box/func.h" +#include "box/func_def.h" #include "msgpuck/msgpuck.h" #include "sqlInt.h" #include "vdbeInt.h" @@ -650,24 +652,11 @@ sqlVdbeJumpHere(Vdbe * p, int addr) sqlVdbeChangeP2(p, addr, p->nOp); } -/* - * If the input FuncDef structure is ephemeral, then free it. If - * the FuncDef is not ephermal, then do nothing. - */ -static void -freeEphemeralFunction(sql * db, FuncDef * pDef) -{ - if ((pDef->funcFlags & SQL_FUNC_EPHEM) != 0) { - sqlDbFree(db, pDef); - } -} - static void vdbeFreeOpArray(sql *, Op *, int); static SQL_NOINLINE void freeP4FuncCtx(sql * db, sql_context * p) { - freeEphemeralFunction(db, p->pFunc); sqlDbFree(db, p); } @@ -690,10 +679,6 @@ freeP4(sql * db, int p4type, void *p4) case P4_KEYINFO: sql_key_info_unref(p4); break; - case P4_FUNCDEF:{ - freeEphemeralFunction(db, (FuncDef *) p4); - break; - } case P4_MEM: sqlValueFree((sql_value *) p4); break; @@ -1150,15 +1135,17 @@ displayP4(Op * pOp, char *zTemp, int nTemp) sqlXPrintf(&x, "(binary)"); break; } - case P4_FUNCDEF:{ - FuncDef *pDef = pOp->p4.pFunc; - sqlXPrintf(&x, "%s(%d)", pDef->zName, pDef->nArg); + case P4_FUNC:{ + struct func *func = pOp->p4.func; + sqlXPrintf(&x, "%s(%d)", func->def->name, + func->def->param_count); break; } #if defined(SQL_DEBUG) || defined(VDBE_PROFILE) case P4_FUNCCTX:{ - FuncDef *pDef = pOp->p4.pCtx->pFunc; - sqlXPrintf(&x, "%s(%d)", pDef->zName, pDef->nArg); + struct func *func = pOp->p4.pCtx->func; + sqlXPrintf(&x, "%s(%d)", func->def->name, + func->def->param_count); break; } #endif diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c index 7bd5cdd71..b785739a6 100644 --- a/src/box/sql/vdbemem.c +++ b/src/box/sql/vdbemem.c @@ -39,6 +39,8 @@ #include "sqlInt.h" #include "vdbeInt.h" #include "tarantoolInt.h" +#include "box/func.h" +#include "box/func_def.h" #include "box/schema.h" #include "box/tuple.h" #include "box/port.h" @@ -306,40 +308,30 @@ sqlVdbeMemStringify(Mem * pMem, u8 bForce) return 0; } -/* - * Memory cell pMem contains the context of an aggregate function. - * This routine calls the finalize method for that function. The - * result of the aggregate is stored back into pMem. - * - * Return -1 if the finalizer reports an error. 0 otherwise. - */ int -sqlVdbeMemFinalize(Mem * pMem, FuncDef * pFunc) +sql_vdbemem_finilize(struct Mem *mem, struct func *func) { int rc = 0; - if (ALWAYS(pFunc && pFunc->xFinalize)) { - assert((pMem->flags & MEM_Null) != 0 || pFunc == pMem->u.pDef); - struct sql_context ctx; - memset(&ctx, 0, sizeof(ctx)); - ctx.pMem = pMem; - ctx.pFunc = pFunc; - struct region *region = &fiber()->gc; - size_t region_svp = region_used(region); - struct port args, ret; - port_vdbemem_create(&args, NULL, 0, &ctx); - rc = pFunc->xFinalize(NULL, &args, &ret); - assert((pMem->flags & MEM_Dyn) == 0); - if (pMem->szMalloc > 0) - sqlDbFree(pMem->db, pMem->zMalloc); - uint32_t size; - struct Mem *mem = (struct Mem *)port_get_vdbemem(&ret, &size); - if (mem != NULL) { - *pMem = mem[0]; - } else { - rc = -1; - } - region_truncate(region, region_svp); - } + assert(func != NULL && func->def->aggregate == FUNC_AGGREGATE_GROUP); + struct sql_context ctx; + memset(&ctx, 0, sizeof(ctx)); + ctx.pMem = mem; + ctx.func = func; + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + struct port args, ret; + port_vdbemem_create(&args, NULL, 0, &ctx); + rc = ((struct func_sql_builtin *)func)->finalize(func, &args, &ret); + assert((mem->flags & MEM_Dyn) == 0); + if (mem->szMalloc > 0) + sqlDbFree(mem->db, mem->zMalloc); + uint32_t size; + struct Mem *val = (struct Mem *)port_get_vdbemem(&ret, &size); + if (val != NULL) + *mem = val[0]; + else + rc = -1; + region_truncate(region, region_svp); return rc; } @@ -357,7 +349,7 @@ vdbeMemClearExternAndSetNull(Mem * p) { assert(VdbeMemDynamic(p)); if (p->flags & MEM_Agg) { - sqlVdbeMemFinalize(p, p->u.pDef); + sql_vdbemem_finilize(p, p->u.func); assert((p->flags & MEM_Agg) == 0); testcase(p->flags & MEM_Dyn); } @@ -1222,8 +1214,8 @@ valueNew(sql * db, struct ValueNewStat4Ctx *p) * to be a scalar SQL function. If * * * all function arguments are SQL literals, - * * one of the SQL_FUNC_CONSTANT or _SLOCHNG function flags is set, and - * * the SQL_FUNC_NEEDCOLL function flag is not set, + * * the function is deterministic and the SQL_FUNC_NEEDCOLL + * flag is not set, * * then this routine attempts to invoke the SQL function. Assuming no * error occurs, output parameter (*ppVal) is set to point to a value @@ -1247,7 +1239,6 @@ valueFromFunction(sql * db, /* The database connection */ { sql_value **apVal = 0; /* Function arguments */ int nVal = 0; /* Size of apVal[] array */ - FuncDef *pFunc = 0; /* Function definition */ sql_value *pVal = 0; /* New value */ int rc = 0; /* Return code */ ExprList *pList = 0; /* Function arguments */ @@ -1258,13 +1249,11 @@ valueFromFunction(sql * db, /* The database connection */ pList = p->x.pList; if (pList) nVal = pList->nExpr; - pFunc = sqlFindFunction(db, p->u.zToken, nVal, 0); - assert(pFunc); - if ((pFunc->funcFlags & (SQL_FUNC_CONSTANT | SQL_FUNC_SLOCHNG)) == - 0 || (pFunc->funcFlags & SQL_FUNC_NEEDCOLL) - ) { + struct func *func = + sql_func_by_signature(p->u.zToken, strlen(p->u.zToken), nVal); + assert(func != NULL); + if ((func->def->sql_flags & SQL_FUNC_NEEDCOLL) != 0) return 0; - } struct region *region = &fiber()->gc; size_t region_svp = region_used(region); @@ -1295,7 +1284,7 @@ valueFromFunction(sql * db, /* The database connection */ struct port args, ret; port_vdbemem_create(&args, (struct sql_value *)apVal, nVal, NULL); - if ((*pFunc->xSFunc)(NULL, &args, &ret) != 0) { + if (func_call(func, &args, &ret) != 0) { rc = -1; goto value_from_function_out; } diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c index befe4ce73..5736a4a0b 100644 --- a/src/box/sql/whereexpr.c +++ b/src/box/sql/whereexpr.c @@ -259,9 +259,9 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix, /* Result code to return. */ int rc; - if (!sql_is_like_func(db, pExpr, pnoCase)) { + if (!sql_is_like_func(pExpr, pnoCase)) return 0; - } + pList = pExpr->x.pList; pLeft = pList->a[1].pExpr; /* Value might be numeric */ diff --git a/src/lib/coll/coll.c b/src/lib/coll/coll.c index fa27bf34e..275b1cfd7 100644 --- a/src/lib/coll/coll.c +++ b/src/lib/coll/coll.c @@ -36,6 +36,7 @@ #include #include #include +#include #include "tt_static.h" struct UCaseMap *icu_ucase_default_map = NULL; diff --git a/test/box/function1.result b/test/box/function1.result index c434b0067..7c3a93577 100644 --- a/test/box/function1.result +++ b/test/box/function1.result @@ -365,6 +365,125 @@ test_run:cmd("setopt delimiter ''"); c:close() --- ... +-- +-- gh-2233: Invoke Lua functions created outside SQL. +-- +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', is_deterministic = true, exports = {'LUA'}}) +--- +... +box.execute('SELECT "function1.divide"()') +--- +- error: wrong number of arguments to function function1.divide() +... +box.func["function1.divide"]:drop() +--- +... +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', is_deterministic = true, exports = {'LUA', 'SQL'}}) +--- +... +box.execute('SELECT "function1.divide"()') +--- +- error: invalid argument +... +box.execute('SELECT "function1.divide"(6)') +--- +- error: wrong number of arguments to function function1.divide() +... +box.execute('SELECT "function1.divide"(6, 3)') +--- +- error: wrong number of arguments to function function1.divide() +... +box.func["function1.divide"]:drop() +--- +... +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', param_list = {'number', 'number'}, is_deterministic = true, exports = {'LUA', 'SQL'}}) +--- +... +box.execute('SELECT "function1.divide"()') +--- +- error: wrong number of arguments to function function1.divide() +... +box.execute('SELECT "function1.divide"(6)') +--- +- error: wrong number of arguments to function function1.divide() +... +box.execute('SELECT "function1.divide"(6, 3, 3)') +--- +- error: wrong number of arguments to function function1.divide() +... +box.execute('SELECT "function1.divide"(6, 3)') +--- +- metadata: + - name: '"function1.divide"(6, 3)' + type: number + rows: + - [2] +... +box.execute('SELECT "function1.divide"(5, 2)') +--- +- metadata: + - name: '"function1.divide"(5, 2)' + type: number + rows: + - [2.5] +... +box.func["function1.divide"]:drop() +--- +... +function SUMMARIZE(a, b) return a + b end +--- +... +box.schema.func.create("SUMMARIZE", {language = 'LUA', returns = 'number', is_deterministic = true, param_list = {'number', 'number'}, exports = {'LUA', 'SQL'}}) +--- +... +box.execute('SELECT summarize(1, 2)') +--- +- metadata: + - name: summarize(1, 2) + type: number + rows: + - [3] +... +box.func.SUMMARIZE:drop() +--- +... +box.schema.func.create("SUMMARIZE", {language = 'LUA', returns = 'number', body = 'function (a, b) return a + b end', is_deterministic = true, param_list = {'number', 'number'}, exports = {'LUA', 'SQL'}}) +--- +... +box.execute('SELECT summarize(1, 2)') +--- +- metadata: + - name: summarize(1, 2) + type: number + rows: + - [3] +... +box.func.SUMMARIZE:drop() +--- +... +-- +-- gh-4113: Valid method to use Lua from SQL +-- +box.execute('SELECT lua(\'return 1 + 1\')') +--- +- metadata: + - name: lua('return 1 + 1') + type: any + rows: + - [2] +... +box.execute('SELECT lua(\'return box.cfg\')') +--- +- error: 'Failed to execute SQL statement: Unsupported type passed from Lua' +... +box.execute('SELECT lua(\'return box.cfg.memtx_memory\')') +--- +- metadata: + - name: lua('return box.cfg.memtx_memory') + type: any + rows: + - [107374182] +... -- Test registered functions interface. function divide(a, b) return a / b end --- diff --git a/test/box/function1.test.lua b/test/box/function1.test.lua index dbbdcf8be..69c2de685 100644 --- a/test/box/function1.test.lua +++ b/test/box/function1.test.lua @@ -128,6 +128,41 @@ identifier.run_test( test_run:cmd("setopt delimiter ''"); c:close() +-- +-- gh-2233: Invoke Lua functions created outside SQL. +-- +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', is_deterministic = true, exports = {'LUA'}}) +box.execute('SELECT "function1.divide"()') +box.func["function1.divide"]:drop() +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', is_deterministic = true, exports = {'LUA', 'SQL'}}) +box.execute('SELECT "function1.divide"()') +box.execute('SELECT "function1.divide"(6)') +box.execute('SELECT "function1.divide"(6, 3)') +box.func["function1.divide"]:drop() +box.schema.func.create("function1.divide", {language = 'C', returns = 'number', param_list = {'number', 'number'}, is_deterministic = true, exports = {'LUA', 'SQL'}}) +box.execute('SELECT "function1.divide"()') +box.execute('SELECT "function1.divide"(6)') +box.execute('SELECT "function1.divide"(6, 3, 3)') +box.execute('SELECT "function1.divide"(6, 3)') +box.execute('SELECT "function1.divide"(5, 2)') +box.func["function1.divide"]:drop() + +function SUMMARIZE(a, b) return a + b end +box.schema.func.create("SUMMARIZE", {language = 'LUA', returns = 'number', is_deterministic = true, param_list = {'number', 'number'}, exports = {'LUA', 'SQL'}}) +box.execute('SELECT summarize(1, 2)') +box.func.SUMMARIZE:drop() + +box.schema.func.create("SUMMARIZE", {language = 'LUA', returns = 'number', body = 'function (a, b) return a + b end', is_deterministic = true, param_list = {'number', 'number'}, exports = {'LUA', 'SQL'}}) +box.execute('SELECT summarize(1, 2)') +box.func.SUMMARIZE:drop() + +-- +-- gh-4113: Valid method to use Lua from SQL +-- +box.execute('SELECT lua(\'return 1 + 1\')') +box.execute('SELECT lua(\'return box.cfg\')') +box.execute('SELECT lua(\'return box.cfg.memtx_memory\')') + -- Test registered functions interface. function divide(a, b) return a / b end box.schema.func.create("divide", {comment = 'Divide two values'}) diff --git a/test/sql-tap/alias.test.lua b/test/sql-tap/alias.test.lua index f9e6fc9fa..75391b305 100755 --- a/test/sql-tap/alias.test.lua +++ b/test/sql-tap/alias.test.lua @@ -25,14 +25,13 @@ test:plan(9) -- counter = 0 -sequence = function() - counter = counter + 1 - return counter -end -- Function is declared as deterministic deliberately. -- Otherwise it would be called as much as it occurs in a query. -box.internal.sql_create_function("sequence", "INT", sequence, 0, true) +box.schema.func.create('SEQUENCE', {language = 'Lua', is_deterministic = true, + returns = 'unsigned', + body = 'function() counter = counter + 1 return counter end', + exports = {'LUA', 'SQL'}}) test:do_test( "alias-1.1", @@ -220,6 +219,6 @@ test:do_test( -- -- -- }) - +box.func.SEQUENCE:drop() test:finish_test() diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua index 98f906cf4..d83aafe2c 100755 --- a/test/sql-tap/check.test.lua +++ b/test/sql-tap/check.test.lua @@ -679,10 +679,12 @@ test:do_execsql_test( -- cannot be tested). -- --reset_db() -local function myfunc(x) - return x < 10 -end -box.internal.sql_create_function("myfunc", "INT", myfunc) + +box.schema.func.create('MYFUNC', {language = 'Lua', + is_deterministic = true, + body = 'function(x) return x < 10 end', + returns = 'boolean', param_list = {'number'}, + exports = {'LUA', 'SQL'}}) test:do_execsql_test( 7.1, @@ -807,5 +809,8 @@ test:do_catchsql_test( -- }) + +box.func.MYFUNC:drop() + test:finish_test() diff --git a/test/sql-tap/func5.test.lua b/test/sql-tap/func5.test.lua index f3706e631..4134d4036 100755 --- a/test/sql-tap/func5.test.lua +++ b/test/sql-tap/func5.test.lua @@ -71,13 +71,25 @@ test:do_execsql_test( global_counter = 0 -counter = function(str) - global_counter = global_counter + 1 - return global_counter -end - -box.internal.sql_create_function("counter1", "INT", counter, -1, false) -box.internal.sql_create_function("counter2", "INT", counter, -1, true) +box.schema.func.create('COUNTER1', {language = 'Lua', is_deterministic = false, + param_list = {'any'}, returns = 'integer', + exports = {'SQL', 'LUA'}, + body = [[ + function(str) + global_counter = global_counter + 1 + return global_counter + end + ]]}) + +box.schema.func.create('COUNTER2', {language = 'Lua', is_deterministic = true, + param_list = {'any'}, returns = 'integer', + exports = {'SQL', 'LUA'}, + body = [[ + function(str) + global_counter = global_counter + 1 + return global_counter + end + ]]}) test:do_execsql_test( "func5-2.2", @@ -229,4 +241,7 @@ test:do_catchsql_test( } ) +box.func.COUNTER1:drop() +box.func.COUNTER2:drop() + test:finish_test() diff --git a/test/sql-tap/lua_sql.test.lua b/test/sql-tap/lua_sql.test.lua index b0913e7f4..6451a2cf5 100755 --- a/test/sql-tap/lua_sql.test.lua +++ b/test/sql-tap/lua_sql.test.lua @@ -1,19 +1,16 @@ #!/usr/bin/env tarantool test = require("sqltester") NULL = require('msgpack').NULL -test:plan(24) - -local function func1(a) - return a -end -local function allways_2(a) - return 2 -end +test:plan(22) test:do_test( "lua_sql-1.0", function () - box.internal.sql_create_function("func1", "INT", allways_2) + box.schema.func.create('FUNC1', {language = 'Lua', + is_deterministic = true, + body = 'function(a) return 2 end', + param_list = {'any'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) return test:execsql("select func1(1)") end, {2}) @@ -22,34 +19,16 @@ test:do_test( test:do_test( "lua_sql-1.1", function () - box.internal.sql_create_function("func1", "INT", func1) + box.func.FUNC1:drop() + box.schema.func.create('FUNC1', {language = 'Lua', + is_deterministic = true, + body = 'function(a) return a end', + param_list = {'scalar'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) return test:execsql("select func1(1)") end, {1}) --- try to loose memory -test:do_test( - "lua_sql-1.2", - function () - for i = 1, 1000000, 1 do - box.internal.sql_create_function("func1", "INT", func1) - end - return test:execsql("select func1(1)") - end, - {1}) - --- check sql polymorphism -test:do_test( - "lua_sql-1.3", - function () - box.internal.sql_create_function("allways_2", "INT", allways_2, 1) -- specify 1 arg - box.internal.sql_create_function("allways_2", "INT", func1) - box.internal.sql_create_function("allways_2", "INT", func1, 2) - box.internal.sql_create_function("allways_2", "INT", func1, 3) - return test:execsql("select allways_2(1)") - end, - {2}) - test:do_catchsql_test( "lua_sql-1.0", "select func3(1)", @@ -72,7 +51,7 @@ for _, val in ipairs({ {result}) end -local from_sql_to_lua = { +from_sql_to_lua = { [1] = {1, 1}, [2] = {"1", 1}, [3] = {"1.5", 1.5}, @@ -81,14 +60,19 @@ local from_sql_to_lua = { [6] = {"x'0500'", "\u{0005}\u{0000}"}, [7] = {"123123123123123", 123123123123123LL}, } -local json = require("json") -local function check_from_sql_to_lua(i, arg) - if from_sql_to_lua[i][2] == arg then - return 1 - end - return 0 -end -box.internal.sql_create_function("check_from_sql_to_lua", "INT", check_from_sql_to_lua) + +box.schema.func.create('CHECK_FROM_SQL_TO_LUA', {language = 'Lua', + is_deterministic = true, + body = [[ + function(i, arg) + if from_sql_to_lua[i][2] == arg then + return 1 + end + return 0 + end + ]], + param_list = {'integer', 'scalar'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) -- check for different types for i = 1, #from_sql_to_lua, 1 do @@ -98,17 +82,23 @@ for i = 1, #from_sql_to_lua, 1 do {1}) end -local from_lua_to_sql = { +from_lua_to_sql = { [1] = {1, 1}, [2] = {"1.5", 1.5}, [3] = {"'1'", "1"}, [4] = {"true", true}, [5] = {"false", false}, } -local function check_from_lua_to_sql(i) - return from_lua_to_sql[i][2] -end -box.internal.sql_create_function("check_from_lua_to_sql", "BLOB", check_from_lua_to_sql) + +box.schema.func.create('CHECK_FROM_LUA_TO_SQL', {language = 'Lua', + is_deterministic = true, + body = [[ + function(i) + return from_lua_to_sql[i][2] + end + ]], + param_list = {'integer'}, returns = 'scalar', + exports = {'LUA', 'SQL'}}) -- check for different types for i = 1, #from_lua_to_sql, 1 do @@ -118,14 +108,20 @@ for i = 1, #from_lua_to_sql, 1 do {1}) end -local from_lua_to_sql_bad = { +from_lua_to_sql_bad = { [1] = NULL, [2] = 12LL, -- it is possible to support this type } -local function check_from_lua_to_sql_bad(i) - return from_lua_to_sql_bad[i] -end -box.internal.sql_create_function("check_from_lua_to_sql_bad", "BLOB", check_from_lua_to_sql_bad) + +box.schema.func.create('CHECK_FROM_LUA_TO_SQL_BAD', {language = 'Lua', + is_deterministic = true, + body = [[ + function(i) + return from_lua_to_sql_bad[i] + end + ]], + param_list = {'integer'}, returns = 'scalar', + exports = {'LUA', 'SQL'}}) for i = 1, #from_lua_to_sql_bad, 1 do test:do_catchsql_test( @@ -134,16 +130,26 @@ for i = 1, #from_lua_to_sql_bad, 1 do {1, "/Unsupported/"}) end -local function allways_error() - error("my_error123") - return 1 -end -box.internal.sql_create_function("allways_error", "INT", allways_error) +box.schema.func.create('ALLWAYS_ERROR', {language = 'Lua', + is_deterministic = true, + body = [[ + function() + error("my_error123") + return 1 + end + ]], + param_list = {}, returns = 'integer', + exports = {'LUA', 'SQL'}}) test:do_catchsql_test( "lua_sql-2.6", "select allways_error()", {1, "/my_error123/"}) +box.func.FUNC1:drop() +box.func.CHECK_FROM_SQL_TO_LUA:drop() +box.func.CHECK_FROM_LUA_TO_SQL:drop() +box.func.CHECK_FROM_LUA_TO_SQL_BAD:drop() +box.func.ALLWAYS_ERROR:drop() test:finish_test() diff --git a/test/sql-tap/subquery.test.lua b/test/sql-tap/subquery.test.lua index 7a3e270dc..743b7141e 100755 --- a/test/sql-tap/subquery.test.lua +++ b/test/sql-tap/subquery.test.lua @@ -710,17 +710,20 @@ test:do_execsql_test( -- for a matching column name did not cause an otherwise static subquery -- to become a dynamic (correlated) subquery. -- -local callcnt = 0 +callcnt = 0 test:do_test( "subquery-5.1", function() - local function callcntproc(n) - callcnt = callcnt + 1 - return n - end - - callcnt = 0 - box.internal.sql_create_function("callcnt", "INT", callcntproc) + box.schema.func.create('CALLCNT', {language = 'Lua', + is_deterministic = true, + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}, + body = [[ + function(n) + callcnt = callcnt + 1 + return n + end + ]]}) return test:execsql [[ CREATE TABLE t4(x TEXT,y INT PRIMARY KEY); INSERT INTO t4 VALUES('one',1); @@ -791,6 +794,8 @@ test:do_test( return callcnt end, 1) +box.func.CALLCNT:drop() + --############ was disable until we get #2652 fixed -- Ticket #2652. Allow aggregate functions of outer queries inside -- a non-aggregate subquery. diff --git a/test/sql-tap/trigger9.test.lua b/test/sql-tap/trigger9.test.lua index e7e170b3d..64dcaff33 100755 --- a/test/sql-tap/trigger9.test.lua +++ b/test/sql-tap/trigger9.test.lua @@ -46,7 +46,10 @@ local function has_rowdata(sql) -- X(41, "X!cmd", [=[["expr","[lsearch [execsql \"explain $sql\"] RowData]>=0"]]=]) end -box.internal.sql_create_function('randstr', 'TEXT', test.randstr, 1) +box.schema.func.create('RANDSTR', {language = 'Lua', + body = 'function(n) return test.randstr(n) end', + param_list = {'integer'}, returns = 'string', + exports = {'LUA', 'SQL'}}) -- MUST_WORK_TEST test:do_execsql_test( @@ -450,5 +453,6 @@ test:do_execsql_test( -- }) +box.func.RANDSTR:drop() test:finish_test() diff --git a/test/sql-tap/where2.test.lua b/test/sql-tap/where2.test.lua index 4116ca913..f267be8e6 100755 --- a/test/sql-tap/where2.test.lua +++ b/test/sql-tap/where2.test.lua @@ -231,7 +231,7 @@ test:do_execsql_test( EXPLAIN SELECT * FROM x1, x2 WHERE x=1 ORDER BY random(); ]], { -- - "/random/" + "/RANDOM/" -- }) @@ -254,7 +254,7 @@ test:do_execsql_test( EXPLAIN SELECT * FROM x1, x2 WHERE x=1 ORDER BY abs(5); ]], { -- - "~/abs/" + "~/ABS/" -- }) diff --git a/test/sql/errinj.result b/test/sql/errinj.result index 8846e5ee8..b4b44dda4 100644 --- a/test/sql/errinj.result +++ b/test/sql/errinj.result @@ -439,31 +439,6 @@ errinj.set("ERRINJ_WAL_DELAY", false) - ok ... -- --- gh-3931: Store regular identifiers in case-normal form --- -errinj = box.error.injection ---- -... -errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", true) ---- -- ok -... -box.execute("CREATE TABLE hello (id INT primary key,x INT,y INT);") ---- -- error: Failed to allocate 6 bytes in sqlDbMallocRawNN for res -... -dummy_f = function(int) return 1 end ---- -... -box.internal.sql_create_function("counter1", "INT", dummy_f, -1, false) ---- -- error: Failed to allocate 9 bytes in region_alloc for res -... -errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) ---- -- ok -... --- -- Tests which are aimed at verifying work of commit/rollback -- triggers on _ck_constraint space. -- diff --git a/test/sql/errinj.test.lua b/test/sql/errinj.test.lua index 48b80a443..1250bf34b 100644 --- a/test/sql/errinj.test.lua +++ b/test/sql/errinj.test.lua @@ -140,16 +140,6 @@ box.execute("UPDATE t SET id = 2;") -- Finish drop space. errinj.set("ERRINJ_WAL_DELAY", false) --- --- gh-3931: Store regular identifiers in case-normal form --- -errinj = box.error.injection -errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", true) -box.execute("CREATE TABLE hello (id INT primary key,x INT,y INT);") -dummy_f = function(int) return 1 end -box.internal.sql_create_function("counter1", "INT", dummy_f, -1, false) -errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) - -- -- Tests which are aimed at verifying work of commit/rollback -- triggers on _ck_constraint space. diff --git a/test/sql/func-recreate.result b/test/sql/func-recreate.result index 73fb03cc4..da6d6e770 100644 --- a/test/sql/func-recreate.result +++ b/test/sql/func-recreate.result @@ -12,7 +12,15 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') fiber = require('fiber') --- ... -box.internal.sql_create_function('WAITFOR', 'INT', function (n) fiber.sleep(n) return n end) +test_run:cmd("setopt delimiter ';'") +--- +- true +... +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); --- ... ch = fiber.channel(1) @@ -24,10 +32,19 @@ _ = fiber.create(function () ch:put(box.execute('select WAITFOR(0.2)')) end) fiber.sleep(0.1) --- ... -box.internal.sql_create_function('WAITFOR', 'INT', function (n) require('fiber').sleep(n) return n end) +box.func.WAITFOR:drop() +--- +... +test_run:cmd("setopt delimiter ';'") +--- +- true +... +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); --- -- error: 'Failed to create function ''WAITFOR'': unable to create function due to - active statements' ... ch:get() --- @@ -37,6 +54,20 @@ ch:get() rows: - [0.2] ... -box.internal.sql_create_function('WAITFOR', 'INT', function (n) require('fiber').sleep(n) return n end) +box.func.WAITFOR:drop() +--- +... +test_run:cmd("setopt delimiter ';'") +--- +- true +... +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); +--- +... +box.func.WAITFOR:drop() --- ... diff --git a/test/sql/func-recreate.test.lua b/test/sql/func-recreate.test.lua index 753e9ca4d..1b6e870f3 100644 --- a/test/sql/func-recreate.test.lua +++ b/test/sql/func-recreate.test.lua @@ -4,14 +4,36 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') -- Check errors during function create process fiber = require('fiber') -box.internal.sql_create_function('WAITFOR', 'INT', function (n) fiber.sleep(n) return n end) + +test_run:cmd("setopt delimiter ';'") +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); ch = fiber.channel(1) _ = fiber.create(function () ch:put(box.execute('select WAITFOR(0.2)')) end) fiber.sleep(0.1) -box.internal.sql_create_function('WAITFOR', 'INT', function (n) require('fiber').sleep(n) return n end) +box.func.WAITFOR:drop() + +test_run:cmd("setopt delimiter ';'") +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); + ch:get() -box.internal.sql_create_function('WAITFOR', 'INT', function (n) require('fiber').sleep(n) return n end) +box.func.WAITFOR:drop() + +test_run:cmd("setopt delimiter ';'") +box.schema.func.create('WAITFOR', {language = 'Lua', + body = 'function (n) fiber.sleep(n) return n end', + param_list = {'integer'}, returns = 'integer', + exports = {'LUA', 'SQL'}}) +test_run:cmd("setopt delimiter ''"); +box.func.WAITFOR:drop() -- 2.21.0