From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id B80A06ECE3; Fri, 14 Jan 2022 16:38:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B80A06ECE3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1642167537; bh=NgimIMoAje3hi8jsqX4Dcv42J1yixHaQFhfT2lL1jv8=; h=To:Cc:Date:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=ypYAyL7dmprUpyFj+WRIaorUnCS4WQXn500kyQznx7yY3c2bpQGAYBN6AbJXjgFh1 vizZRJ3OGdrai56vtgrOYQrzZirlUDtAO8il7feUPVQarfGLYe6S0HwSGgIT5ZoDvr ODlPWwvll95+bZ6R7ffpwfYOb4ghq1H06jWgm7Zs= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 86A316ECE3 for ; Fri, 14 Jan 2022 16:38:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 86A316ECE3 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1n8Mn9-0003ZX-KU; Fri, 14 Jan 2022 16:38:56 +0300 To: v.shpilevoy@tarantool.org Cc: tarantool-patches@dev.tarantool.org Date: Fri, 14 Jan 2022 16:38:55 +0300 Message-Id: <04154369ec1ff8a1eaf7c9ea1ed37e1fcd1a7120.1642167504.git.imeevma@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9CD668969C51240A483C19EF8315A6FF9754A7C2D77786D12182A05F5380850405BACFB9E08DB7FE44C08DA18E5915C51425F59DCEACF8B2811E306C2D44118A1 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE782FBE8B5DCA684DBC2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE7FC9D84217C4FCDC2EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BEBC5CAB6D411FFA6AC4FA533D60FB075CB7CFAAE60B63454CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F6B57BC7E64490618DEB871D839B7333395957E7521B51C2DFABB839C843B9C08941B15DA834481F8AA50765F7900637F6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B3241B8C6B78F28F975ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975CBF0E6906D4BA60DF08FA542852BD03396B0897C4B6D114FE9C2B6934AE262D3EE7EAB7254005DCED17312A81722357511E0A4E2319210D9B64D260DF9561598F01A9E91200F654B00191F08AA555D0B08E8E86DC7131B365E7726E8460B7C23C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A150564C13026C18B9FDC2777B3664239C2159D0961430052E712EC8A416920F2DD703D6BCDF58701D7E09C32AA3244CDB8A50E73C6BE63947D750DD9CF991D0408A6A02710B7304729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojieoUMt/u5LjVwaHkJTjeqQ== X-Mailru-Sender: 689FA8AB762F739339CABD9B3CA9A7D6428E4D7258AD99953569A1411B566A3383D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E3365FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v1 1/1] sql: introduce user-defined aggregate functions X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: imeevma@tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This patch introduces user-defined SQL aggregate functions and an interface for creating them. Closes #2579 @TarantoolBot document Title: User-defined aggregate functions User-defined aggregate functions are now available. There are three types of user-defined aggregate functions: persistent functions, non-persistent functions, and C-functions. All user-defined aggregate functions consist of two functions: a "step" function, which is executed for each record in the group, and a "finalize" function, which is executed after all records in the group have been processed. The "Step" function has one more argument than specified in "param_list" in the definition. This argument is the current state of the aggregate function and must be the first argument. The "Finalize" function has exactly one argument, which is the current state of the aggregate function. To use a C user-defined aggregate function and a non-persistent user-defined aggregate function, you must define a "step" function with a name equal to the function name, and a "finalize" function with a name equal to the function name with "_finalize" appended. For non-persistent functions, both functions must be global in Lua, and for a C function, the functions with the these names must be defined in the appropriate .so file. Example of non-persistent aggregate function: ``` F0 = function(x, y) if x == nil then x = {sum = 0, count = 0} end x.sum = x.sum + y x.count = x.count + 1 return x end F0_finalize = function(x) return x.sum / x.count end box.schema.func.create('F0', {aggregate = 'group', returns = 'number', param_list = {'integer'}, exports = {'SQL'}}) box.execute([[SELECT f0(column_3) FROM (VALUES(1), (2), (3));]]) ``` To use a persistent user-defined aggregate function, you must define the "body" field in the function definition as a map with two fields, "step" and "finalize" according to the rules defined above. Example of persistent aggregate function: ``` body = { step = [[function(x, y) if x == nil then x = {sum = 0, count = 0} end x.sum = x.sum + y x.count = x.count + 1 return x end]], finalize = [[function(x) return x.sum / x.count end]] } box.schema.func.create('F1', {aggregate = 'group', returns = 'number', body = body, param_list = {'integer'}, exports = {'SQL'}}) box.execute([[SELECT f1(column_3) FROM (VALUES(1), (2), (3));]]) ``` --- https://github.com/tarantool/tarantool/issues/2579 https://github.com/tarantool/tarantool/tree/imeevma/gh-2579-introduce-custom-aggregate-functions ...79-introduce-custom-aggregate-functions.md | 4 + src/box/alter.cc | 161 +++++++++++- src/box/bootstrap.snap | Bin 4891 -> 4915 bytes src/box/lua/schema.lua | 3 +- src/box/lua/upgrade.lua | 16 ++ src/box/schema.cc | 37 +++ src/box/schema.h | 9 + src/box/sql/expr.c | 19 +- src/box/sql/func.c | 7 +- src/box/sql/select.c | 33 ++- src/box/sql/vdbe.c | 87 +++++++ test/box-py/bootstrap.result | 4 +- test/sql-tap/CMakeLists.txt | 2 + test/sql-tap/gh-2579-custom-aggregate.c | 48 ++++ .../sql-tap/gh-2579-custom-aggregate.test.lua | 232 ++++++++++++++++++ 15 files changed, 638 insertions(+), 24 deletions(-) create mode 100644 changelogs/unreleased/gh-2579-introduce-custom-aggregate-functions.md create mode 100644 test/sql-tap/gh-2579-custom-aggregate.c create mode 100755 test/sql-tap/gh-2579-custom-aggregate.test.lua diff --git a/changelogs/unreleased/gh-2579-introduce-custom-aggregate-functions.md b/changelogs/unreleased/gh-2579-introduce-custom-aggregate-functions.md new file mode 100644 index 000000000..68adad478 --- /dev/null +++ b/changelogs/unreleased/gh-2579-introduce-custom-aggregate-functions.md @@ -0,0 +1,4 @@ +## feature/sql + +* Custom SQL aggregate function and an interface to create them is now + available (gh-2579). diff --git a/src/box/alter.cc b/src/box/alter.cc index 65c1cb952..f1dfcd807 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -3276,6 +3276,85 @@ func_def_get_ids_from_tuple(struct tuple *tuple, uint32_t *fid, uint32_t *uid) return tuple_field_u32(tuple, BOX_FUNC_FIELD_UID, uid); } +static int +func_def_check_body(struct tuple *tuple) +{ + assert(tuple_field_count(tuple) > BOX_FUNC_FIELD_BODY); + const char *field = tuple_field(tuple, BOX_FUNC_FIELD_BODY); + assert(field != NULL); + enum mp_type type = mp_typeof(*field); + if (type == MP_STR) + return 0; + if (type != MP_MAP) { + diag_set(ClientError, ER_FIELD_TYPE, "map or array", + mp_type_strs[type]); + return -1; + } + const char *agg = tuple_field_cstr(tuple, BOX_FUNC_FIELD_AGGREGATE); + if (agg == NULL) + return -1; + if (STR2ENUM(func_aggregate, agg) != FUNC_AGGREGATE_GROUP) { + const char *name = tuple_field_cstr(tuple, BOX_FUNC_FIELD_NAME); + diag_set(ClientError, ER_CREATE_FUNCTION, name, + "only aggregate functions can have map as body"); + return -1; + } + + bool has_step = false; + bool has_finalize = false; + if (mp_decode_map(&field) != 2) + goto error; + for (int i = 0; i < 2; ++i) { + const char *value; + uint32_t size; + if (mp_typeof(*field) != MP_STR) + goto error; + value = mp_decode_str(&field, &size); + if (size == strlen("step")) { + if (strncmp(value, "step", size) != 0) + goto error; + has_step = true; + } else if (size == strlen("finalize")) { + if (strncmp(value, "finalize", size) != 0) + goto error; + has_finalize = true; + } + if (mp_typeof(*field) != MP_STR) + goto error; + mp_next(&field); + } + if (has_step && has_finalize) + return 0; +error: + const char *name = tuple_field_cstr(tuple, BOX_FUNC_FIELD_NAME); + diag_set(ClientError, ER_CREATE_FUNCTION, name, + "body of aggregate function should be map that contains " + "exactly two string fields: 'step' and 'finalize'"); + return -1; +} + +static const char * +func_def_get_agg_body(struct tuple *tuple, uint32_t *body_len, bool is_step) +{ + const char *field = tuple_field(tuple, BOX_FUNC_FIELD_BODY); + assert(field != NULL); + if (mp_typeof(*field) == MP_STR) { + if (is_step) + return mp_decode_str(&field, body_len); + *body_len = 0; + return field; + } + assert(mp_typeof(*field) == MP_MAP); + mp_decode_map(&field); + uint32_t size; + mp_decode_str(&field, &size); + if (size == (is_step ? strlen("step") : strlen("finalize"))) + return mp_decode_str(&field, body_len); + mp_next(&field); + mp_next(&field); + return mp_decode_str(&field, body_len); +} + /** Create a function definition from tuple. */ static struct func_def * func_def_new_from_tuple(struct tuple *tuple) @@ -3295,7 +3374,9 @@ func_def_new_from_tuple(struct tuple *tuple) if (identifier_check(name, name_len) != 0) return NULL; if (field_count > BOX_FUNC_FIELD_BODY) { - body = tuple_field_str(tuple, BOX_FUNC_FIELD_BODY, &body_len); + if (func_def_check_body(tuple) != 0) + return NULL; + body = func_def_get_agg_body(tuple, &body_len, true); if (body == NULL) return NULL; comment = tuple_field_str(tuple, BOX_FUNC_FIELD_COMMENT, @@ -3498,6 +3579,11 @@ func_def_new_from_tuple(struct tuple *tuple) def->exports.lua = true; def->param_count = 0; } + if (def->aggregate == FUNC_AGGREGATE_GROUP && def->exports.lua != 0) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "aggregate function can only be accessed in SQL"); + return NULL; + } if (func_def_check(def) != 0) return NULL; def_guard.is_active = false; @@ -3536,6 +3622,49 @@ on_drop_func_rollback(struct trigger *trigger, void * /* event */) return 0; } +struct func_def * +func_fin_def_new(struct tuple *tuple, const struct func_def *def) +{ + const char *name = tt_sprintf("%s_finalize", def->name); + size_t name_len = strlen(name); + uint32_t body_len; + const char *body = func_def_get_agg_body(tuple, &body_len, false); + if (body == NULL) + return NULL; + if ((body_len == 0 && def->body != NULL) || + (def->body == NULL && body_len != 0)) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "step or finalize of aggregate function is undefined"); + return NULL; + } + uint32_t len = sizeof(struct func_def) + name_len + 1; + if (body_len > 0) + len += body_len + 1; + struct func_def *fin_def = (struct func_def *)malloc(len); + fin_def->fid = def->fid; + fin_def->uid = def->uid; + if (body_len > 0) { + fin_def->body = fin_def->name + name_len + 1; + memcpy(fin_def->body, body, body_len); + fin_def->body[body_len] = '\0'; + } else { + fin_def->body = NULL; + } + fin_def->comment = NULL; + fin_def->setuid = def->setuid; + fin_def->is_deterministic = def->is_deterministic; + fin_def->is_sandboxed = def->is_sandboxed; + fin_def->param_count = 0; + fin_def->returns = def->returns; + fin_def->aggregate = FUNC_AGGREGATE_GROUP; + fin_def->language = def->language; + fin_def->name_len = name_len; + fin_def->exports = def->exports; + fin_def->opts = def->opts; + memcpy(fin_def->name, name, name_len + 1); + return fin_def; +} + /** * A trigger invoked on replace in a space containing * functions on which there were defined any grants. @@ -3568,6 +3697,16 @@ on_replace_dd_func(struct trigger * /* trigger */, void *event) struct func *func = func_new(def); if (func == NULL) return -1; + if (def->aggregate == FUNC_AGGREGATE_GROUP) { + struct func_def *fin_def = + func_fin_def_new(new_tuple, def); + if (fin_def == NULL) + return -1; + struct func *func_fin = func_new(fin_def); + if (func_fin == NULL) + return -1; + func_fin_cache_insert(func_fin); + } def_guard.is_active = false; func_cache_insert(func); on_rollback->data = func; @@ -3610,6 +3749,8 @@ on_replace_dd_func(struct trigger * /* trigger */, void *event) txn_alter_trigger_new(on_drop_func_rollback, old_func); if (on_commit == NULL || on_rollback == NULL) return -1; + if (old_func->def->aggregate == FUNC_AGGREGATE_GROUP) + func_fin_cache_delete(old_func->def->fid); func_cache_delete(old_func->def->fid); txn_stmt_on_commit(stmt, on_commit); txn_stmt_on_rollback(stmt, on_rollback); @@ -3622,9 +3763,13 @@ on_replace_dd_func(struct trigger * /* trigger */, void *event) * definition to support upgrade script. */ struct func_def *old_def = NULL, *new_def = NULL; - auto guard = make_scoped_guard([&old_def, &new_def] { + struct func_def *new_fin_def = NULL; + struct func_def *old_fin_def = NULL; + auto guard = make_scoped_guard([=] { free(old_def); free(new_def); + free(new_fin_def); + free(old_fin_def); }); old_def = func_def_new_from_tuple(old_tuple); new_def = func_def_new_from_tuple(new_tuple); @@ -3635,6 +3780,18 @@ on_replace_dd_func(struct trigger * /* trigger */, void *event) "alter"); return -1; } + if (new_def->aggregate != FUNC_AGGREGATE_GROUP) + return 0; + new_fin_def = func_fin_def_new(new_tuple, new_def); + old_fin_def = func_fin_def_new(old_tuple, old_def); + if (new_fin_def == NULL || old_fin_def == NULL) + return -1; + if (func_def_cmp(new_fin_def, old_fin_def) != 0) { + diag_set(ClientError, ER_UNSUPPORTED, "function", + "alter"); + return -1; + } + return 0; } return 0; } diff --git a/src/box/bootstrap.snap b/src/box/bootstrap.snap index 018670d2a6558e64ba691ce0002320b76ac1c97b..610e513fdefaa8a3bd60e95a45219c3bee426b71 100644 GIT binary patch delta 4898 zcmV+-6W#2aCbK4x6@M`>E-)=(WprUOEi*PPXJchFFf}zWIW`JOZgX^DZewLSATcv# zWHVtoH7#Z`VKOZ=H#TA|IW%T7Eo5XgHZ@~mHDx(BWC~V8Y;R+0Iv{&}3JTS_3%bn` zvjEQ5vG$6k0000004TLD{Qy{1l>o}86eUIwh*;YI000000Do{W@Qe>$sJGs4T?!dY zzDXmH-vPG~d#7ZSnIt=M<>tzn&iE!LK>3ftTy=&BI`y~-#(MoXklg2mI+0ESUIGmQ znan4@m3rSw^SqU}l1Zm!Hq)|XO3Sn)b6JjMSz@v*S8*)M5SOJ*EY}-i?12qq@M7G7 z2elptAg#AS2Y*ht%sJh%rvV3Ld1iyMG^0UTt{I$3Nes-S-6ZaE$Z-$46B-=9mf+#ts}|JcSCzs}u)pzE_)K^F5jrn-_C7 z-<#EHcVsl$od#x0NwcM-J5gk{E+`Gqx)_LJ@(@hHEm}Y%FuL6mD7xJcP{1FMfcFL<;C%rH+;EK3++?#bn46(8T)zn0PNlzz!N5ImI#1tKmlMIwkcVH!zBU2;gBf7 z;eL{Me*}s5Mt;06a-?@6MtUDoq<0}i&J%zk=YJVQ$a&rs2mrMoLB!2N?6YV-fQ;8_ z=E3oTYmXndes{L*e0-d_qr4n~TCg5IuvgTr-~2oNSmh@-?%!iUy(yE$OU`1Ehk2(w zTNMNH@qxXf(!4P;EIW%g-&e1y7?2O_6=j9MHqX9emB+^Vz+O>%>`%<#(>-@n zv40=?dg=L5BU7`*8B0%7Bqh-U_EHlo9(!uNJMy6Q?7-tX<*rk&jyt$YhaFs17F z)hL$^7sVlmi^}1;=tbk;(0NZz=YMOQ&O3+GJNednaxFL~KY7*`mKHC|=XcKcc34}- z+xu#_cTQ*9R#&!tL(aBs`;FDN;fDWfc=JCEHuM>4=v@XHdXHfS?M4QzVTeKNHg3$b z3I?h+=4qa6s#J9il{&>nRi)Pa|H9Pjf|RyeTX7F^hRg%-BnS_KwTSbu@mLqP?f zsdJ|8osdG!1QdMcTEYoFb0xuq+6cZ9Xi+5ixS8vz8NpYBit?Fhd}@5A2v9!LTti6X z+a?GiY;&7vLeS<1AZQbZMw{BaePEhTIMZC6Zl;yFs!x|Gb-GMny-b(Mr#|=ZbCdVk zXS9=`{mCoJv^?ko|J8x_>DLr(q`!Jch0v@h12+jMxtWq3o~@ArG% z)7|;Zbxh*FGY`ezU%A;hZNeVRvP_z^Q9+&+V)F0V`M4vm9(8}GJOo9J^SHG$x$&+I zxU)e7IN_}No9C(EmVaF`0D8}y|0ut@A=f+f_B-BATxV6adHz>x9)emHNGK21wySlz zL7%s9s;{cM?VF3bVKYbW+`r$cnM(6|`X@eh_rzuBL7x4wzuWuOVU4C=x4q7Zc$3-h z6bqxbe5xK*^G{#3cbjqU0wDHhKU42$I?yBR{k`MULd#&gdK>MrvI{iqN`*5b4oo zHJXj=0$@muP7on9Aogc}B`(e_r;ynO()kgj#Q6YH;(GijaV5^jM-66w_D8e!=X1Ng z?9cvcdwf+fAbi z3*7e znTly4$}&ZSWhiOr%8&%5zh#pMm{?%~0h0%Ws(%O}2_3;;K~(IN6doJoK+u#WQkK-L zr2)-eQ&M?w+LU*?gTlD~tJjbHRmF&8`{wf8`t2M&@y~*S;V`Qmd!8qE{vUgA^XxtL zAPyEy13auXTlPoMw!1OtvzExitM7AFA=tAT%9iG#))moo^Q`dSQ@6Eozi5`Ia~Zn% zJ67NR{pKNPj$qG<3|#a|*YUwsgnoI6gaK8^T`S{*9k{@SO@0w@E>$Rp5GKf1o1!Q;jbvZpN7*ke{I|orneMPM^`#G6KK;{UK)7tyOFC>()1iNS|I# zrSVy@cCKLDs909omA}_BdDG>GEf8?l5~^cqX~kZWZ>iS?JtA;^7S1emHl-HP0w|QL z8YCaZ*Xs&NnrQsaW000Jr!Kq;=Da!bsJgR}54f1Oe~mI}I-O&9mthywH&^Oi^AFQT zrSuXn-w8LdyREMLpumn|nh78V3oD=9btMeQ5W+~N2N75AAA-h!%%U;WWlMq~Ja$2_ z^{h0PHcb$}fi@|3KL>)r3KEUzuN{+Ui+9?jTN&LcZ$1z?%M3q^GxrTk>G9YRUvN5t z*E|4ae{N}mT@;|68daE9KXBpZceWmfP(Z1=(JFE^H=h0(aNmWg7<4VnUztE;7Fzin z_2TI%XO`s{h$}Nu032}0%~9M=X*CP-;dX9&3wan9C)|#wv+VGYLpLePwZvDj&AB7r z-^|u&YoO4%D9R*#oNUHnw~IZuiX;Nv6z))ve_gH2qn1!w*8(Xw;MiIdA429K=Q%Xn z$bXtcVlf6qDC2cE2_~&}in!ZAP6ja8Jo|kyiQpvDh#?`-q8nV#;_u8oO8|>Qjt9z9j$ShJ9pbcdW?^;y{vmq_?LT|)5Id+~ zegd3Qo&s1sPM12EM{J<1C-{vWMdrzH!QU2O)OOR09zb4=fh#Zc8bg}#ZF0Y7JIF97 zN#o{pGrp}RUf6TG8?;o?fZxs)o?!qie>BUbAHUbQeA1Vq?yK88+q*?+xwhd9nXQ8{ zf;!ICfOe>$+&M)9qrKEI#OY(0hB?`b9t!DuoEmX818>SnOGVAT`erbXdAU3{A2al$hB^#E@gle`Kf> z8xS=729m!KW@4p47sQ2mzepSAQAJsM*2R#_5XMCuLtFr`;o$Z_76B6eL>Vme76BAJ zBO}R_nfU2ivts?7f#gEk0$n-HNouhq6zP;8xUndL8l7grNMu73qC_THY{^@4g4T0{ z%#+SgLkJPU2M@#e$oYeJ8p3M(fBy6S6r$rlzw2hs+sVsC^Aj8i?F~@^lPp$?_(*ZW z+FnH@7;b>$sY%GZQB(tuXgnu>mx$&U^#|fg7O>T5SVKC))4w6U@>Uy-LQ^x@!cTzm z2T>B2jG#x9zlpNQ&Tbd@^dn%v4AI8xg@eR4jvu7~Wg3Tz8{x>4= zl8$Xg!WzODmhug8i?`cg)Fhix7X5BS!Vz6;hQu_25smU2a#wHX!Keu{Qyu+{5SdWw ze=7-SK}6C5kV9Uzov~@=f9X5obrcb~9s<*Sf^g$f7-~4m1d+kbBl2|g2#pRx*GU1> z8`|JgB7t=YIIiIsFBKDBKtQ&V3?fb$BY0@;ib;YN2kLJPh+JP&fYmbxV`I<2X-l8f zpb)J?jwE@~0csFQBJjutQ|IIqz99;2)MI#;wAtO+8 zZ4ga?kjm#F^a;sVtYjcS$Sgvfu5cCs1OAP;qD%(5N( zWM#B`<9p8+Q9RE!XVCnn7!B(HFv5JzMrkU5LyI05GBMaGf4!=l|8SW}Ea)GWbvMx* z5@`5~26mg}$ERZpVi5H2P%7P+SnMRDI_#XkTJ#g;vE_}N@Bj>Gf zlE+5$il}T_;Sq0i4DE5iGJ=+8Ba=r*@Qu4TizJUX?6-7&yA2El3kjqSuYnvE-_}~* z4YQYh@3Bo`e@)|HeS9*?RLu@$<*4&$vvvn~iXgbG;94&REYY0@K?_a>{Z-v;H;srM zb)%9$1wXrtoWyHY0a6QR)N#H{JVZQ51cj2j1wqfk(XIeZ4+bF$?~2@Wkcy5)KVXy^ zXpbn|Y?%KHB>X^Tor1stc8oXpGuLF0j-VR9_AVF+fAMKJ_)o=>2Rp12mP_&n+VX8s z6fGR$1Lq<55rL3i*+i?aGUOp5AsJ22=!)@H@281qmxp3^S0u~S#sb9>x=u_|6c?zM>N-(L zvALjLK&tA5OUmYgGD|Yg*5OOt7lWiMhzY|&6w6#wT$SsyBxi2n$SgAJi%GAu}Pm}CeLwfI>&Pq^Eig%IAU=eQ!pII509e_9Mk(@?12qqdok|7gBp(mOXF?O z!P6aaPj}>Lz<9WEa+t%E zOP4U*GC>$>;Sz>vmM#Bd(enRNmjBZ$SZaABSc>2Zcz^r%SkB!%{|w&#t@SvM6$v1S z0OJR&U|as@&#O_fc^++w&4W0b=gn%hJV-QJ z9tLJ*mS$y^@<>6dc|~D>=Cyznl7IHJTeMgVc9ERh@aCgwE%t&E<$?+ ze+0;TBS?BLa-?@6MtUDoq%4IHDa#;3%JQZ_gn!h21OYeyu-Bsb021D-nYZD!X^$Vb zes{L%eteXv!W-~2oNS>qQs?%!ug#W9n`3$9|2e|ev~Y9$N$`H8{U zsEKoC%u*OHSacO{zOi6dvY?+Bj7=31+dTV@(Vd(66N9noQ+r^xP`A`k$$%;h#s_H4 z3V&{F@x&G^Q!FLXBL-U&dC+)v;BlXD_lZ}>9UP^@4vx-I2S+(72OSxWV~&i%AxB0t z`bHf6bOVllw(&+k`^knI=eb53=cxu8XPU+uwJtQ&pmm**#@Bfzo?T|3(SI3d^iPHv z{r5jc8UKqx#{XfA@qY@282^6}#{a1oV1HbtU3@WqF1#537F~>g{917Fe->N(kA)Wh zr&nZgzZF>AU&R&o-%o`VwL4Q(LAxUb6{x2J;FXNT*r_sYZj9X!0-@#3BLI(M|GgB@+=P)D2E?9$ea(E@@N=v>ar+?%n*W$|B+G&0Hjq^NR*3Rwr#?tML%h~o7mTljF zvu)dsbMGu_57$lQ`a*5x#0tY`#b}Uu(Wd5v0CIG7Uk^*Cs&_VUwFo6M`l` z06~*cX*8+H+Xo~G6-AQi(~Z0$Qxxhpolm#vqqpfc`P1kAeRlFb`)qbGb0;xxQ_Nxx zf8Q?d(A-U%t4Rn{b?i^n>VN9&fbpNVs$+lJTtza#&9~1GwYqv#K*wKS`~B5&{>A;< z;j-l4j+;<;r}h1;*5?K>6t{uV8+O10T$` z_ExJlaqh4 z=fBCZ?#H#xy#0>%6MxTLS#6&G)tY~Hs|F42!P<7UKDYIm3!nO;x!b-tHaAS-#GU*1 zx;0d4U{JrrW$u=E{Jh;&pxV2=-yGKF`E%QA9Ei7={XVm>dHYykw)nfI&%)*1SZmr; zFkAEFUCfe#Va-SKyV|COKh_wuAnqN97_|DfGivSBT6=F~LVp1S0kDM0IYuc7O0on< z2_|{}9!Zi~og+ZBI!2J>+s8UzSnWNf zc=W09?g$koReyUw6$lY*gePh02UmWgVqb1bP_w5aRCW^mAzzvFSWxZdoR-WQt5ve3 z+UwJ&51+b)VAywz=6zvBo2Sp!C*+s4i91*QX~LR!tS@WZY3gdMN`_Q> zJD1?i)mkM(8m)-oXx_YD0>kBlA_fh9x`p{jCYUFTj(;c*vpf1I<{dvRsgC`L^*uht zT!mH1kVGaiWN=hd;KoHqMn~Y~+UR%e{Vggw!XW>QrckkDh6z`f!Wxw-RAyCbv!WVO zlq94mGa3>thd64M%rN1m_UO@rH-BNDVOE?aGfX%b4|kT*u|K6qf#NnDYFz@mLS}^1 zfUyu)|9|&w`c}{bD$r)t4$5%wWrNy9gDz<5l`~f|P{Kgz0wpt4x>!;fSQ42Zsy0bu zW@2U*%ql<%gG2!m(-D}XFH=WejJg<2FlEF=wuMTjr6>zk5f-8)qANrajQ);JB4}cz z2?R|Z7^*U)G?WyJ2{}-yru0;y285_EnZl%2D}OC$2BVtVgHNZtPn{IT{bRv^YR^hm zgtRYi&#m9i%@ThtsTlq;+Ogkxap(WBx0`40v9~zXGz{#r+Hln`MceMi*5@sde^}q+ ztUk-DItrKO-_`}qb91fl-!jLwalfV(sPp*2`8(hK{pOz~L(8k+0@uXKb-b}?eU`&j zHASbEfs&)b|9<=I-)hw_3bYZ$+`rALU5-vHs_#$z*{piS2+iG8NALsjRY$H-m0bRfMT|sJ4UEJt@uPwDT~9!ywG1^4Jn3(BAki9k;f5#BOg|0 zpuI~7_J}XGIQL+QU!-hT4HB7dsg~?E$ImTD5ty7?<%Y`B_J0pF02~sJK8*N(4hIe^ z44yLjC7vFRSrzsJtUS}Li-4$?r=apVRqirnCZ1HRuXtn&+LndQW*r1G2>Sys@!PVG z@4kH|tqH{U8M=_*II}I*9AY)YXJ6qk-l`cczmNEkHx#7L zA7s!IaI-6ibXjfw0g5IK4V9TIy7$_r0mBvCe}=TUe7%>c(L-cVk@=~AM!OU@zhNQPtQ_myjQFpD;Vc0mX&tp-jSKCsq(`H z2smpA^RcwFA}Hy%+-rmT5jZakM;AJiQj3@Y6q>6VFdv1$>mnpoH1;lGkd`B-E|Lr8 zygBlyy1$W+xtM2;GG97>o#T0zkrz}qR_bH(4+BTFv=Xo237@gMt=#;e@Q%Wp37`iH zi=W-~Bn-$H!pNrw$ye_mX2yWbvN6)UF0i_K{+sf74c=~6+Js759&^3a8C2J9O4%@Nmx{l_^vM8IvJhTme20QJv_YWgToh#3KJGQ^uBPN zX9Q0mnC@2EkxLMNm1>!M8jxdyiO=~kh-Vy{9pp33fn_mJBmz` z>o4prnnP^UL@&ns8y?*=|5xNdkU9_dLT4 zSZIbzKYp)&arvxYih8JSqip9kC8x@kGh~|%RttKZs)6mCf^y^(4UP6vFA=YgVH$*F zFL@~R8>Me=5|qpB1Tr1b!3PI7x`RSt*-y1CO(Bm!H#OTr(gQ>iyok^R@Dqn~2t31= z3BxAGilVN2iXdUA1~C-PLXgBp9}t@yYbHaN$VAhB_#c4y#h8ed0$m&z;{7UXm}eDb z?bsIcWJDMjF${45!iMA98#x3>_!VKW%v&W`^{k8}IVGZ}Yt4`KW(JZAw*?yEG$*Oe zl5n6?hM>ow3TAYfM6t+*MhKFbli1R?QNzAE8E#o7_gtWaCkzkntjHf0b^G0zEJfrcP{6ZpH zUc_IBE=kB1BO?Xnrce8dIO41}7!}P%6pKHRbhHZ zr9b;5+s0vfz;g7m#z{o1ecsgq&_OAt@n0Z+^dQYsy)1^zG^&Y)iYxM{v-)6Elo=@< zy*D89A+6brMk|aLoaP(yv189{CVK<YCC7s#L{-e=_nC@ zGA%@|`2?Zor830Ogb1O@n})<`=mEM8Vyl$~hBGwb(=vn0QgB?u89tpeyo7`-l8gdQ z8By@7|B6Y1tqzr2IsjZ>a|p|4PR0hF!D$Ph)SwWlL58H`PyuFenMAP2CR6uh6uKb} zZP!2~I+S6B-u+J{Qd~wcn2_ZsjBb5@1n_zfKfq2oYBY@D5>zGqK0-BgV%cWojakx% zBy&+84^+9dRz@tBLue>W_*d+%=wqnU*uLp_`CjL$FpPn4?DW>d?ynn9@mv@P`BJE0 zk1DPPnBc^N(A6oj#M`_iT286;f>kucCxS|eB+lmADi{k!{w#=uK##G>x(<4O(HNJ4 znGlrRn@rci7mWFkATk1vXREUs!jjQG2_q+vV{Ot-gRsh{nDPl3S7bK6XO1tVe4+KV zhO0~jwdRs)iAJ@`aRTJKMl;!tW{`$DG-gSTeX=rIzVV*p4Wg9OJ$3k10CAG?wA2Jb6bRNEETqR~En+ zGsO$!Ok_a`t4t%tQv8bm*B&_cco)8Xp3(ExIL%{Yctun;t#Hc+I)-{UXbC}!vx&*0 zBlyNC&P9^fTlO0|x7`LonTG{Z=l1{(i*H*k^oG&Pzx2o^peAv!);<}3D^;`RSvl$) z+N|9Hp8`rJBe>qX0Yr4=Nf1R6!F#HU>`g19uTE4lPXW&^At&}4T7cEU8Fk2)gNK9% ziA3SD?m^JEaI_bo;lUsT;Z2!)4r1A%=ocBa2HFvZpAF-mfdn7OtkV!Uz>XmXf99GD z(g9TC*UkeYAwCHO@2OjVvuML)!g5XiNL$_xi?W46{E$ipX>6IZL`-6eQUZ6ZtJMMM zI)RubNZ{;}2HBMfq`_Y~#ybK%(%&`(lcyt>lMA?h6(TZLtD9BTRR%0fBqZ1L8eTIV zb#5Anl<+_@vq;4uXsk0>WY#qJSeKknR wfS53>M6uj8!Bx3FOG@Tu8`(wnzX&*h$1oxFf-Z2tbAi;#qL#HEy44V^?FR`@!vFvP diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 99b65b0f5..8c328852d 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -2597,7 +2597,8 @@ box.schema.func.create = function(name, opts) opts = opts or {} check_param_table(opts, { setuid = 'boolean', if_not_exists = 'boolean', - language = 'string', body = 'string', + language = 'string', body = 'any', + aggregate = 'string', is_deterministic = 'boolean', is_sandboxed = 'boolean', is_multikey = 'boolean', diff --git a/src/box/lua/upgrade.lua b/src/box/lua/upgrade.lua index f9e6543c2..0e2c4a091 100644 --- a/src/box/lua/upgrade.lua +++ b/src/box/lua/upgrade.lua @@ -1212,6 +1212,21 @@ local function upgrade_to_2_9_1() remove_sql_builtin_functions_from_func() end +-------------------------------------------------------------------------------- +-- Tarantool 2.10.1 +-------------------------------------------------------------------------------- +local function change_func_body_field_type() + local _func = box.space._func + local format = _func:format() + -- Change type of 'body' field from 'string' to 'any'. + format[6].type = 'any' + _func:format(format) +end + +local function upgrade_to_2_10_1() + change_func_body_field_type() +end + -------------------------------------------------------------------------------- local handlers = { @@ -1229,6 +1244,7 @@ local handlers = { {version = mkversion(2, 3, 1), func = upgrade_to_2_3_1, auto = true}, {version = mkversion(2, 7, 1), func = upgrade_to_2_7_1, auto = true}, {version = mkversion(2, 9, 1), func = upgrade_to_2_9_1, auto = true}, + {version = mkversion(2, 10, 1), func = upgrade_to_2_10_1, auto = true}, } -- Schema version of the snapshot. diff --git a/src/box/schema.cc b/src/box/schema.cc index c248d4ee4..eb1b4a7dd 100644 --- a/src/box/schema.cc +++ b/src/box/schema.cc @@ -60,6 +60,7 @@ static struct mh_i32ptr_t *spaces; static struct mh_strnptr_t *spaces_by_name; static struct mh_i32ptr_t *funcs; static struct mh_strnptr_t *funcs_by_name; +static struct mh_i32ptr_t *funcs_fin; static struct mh_i32ptr_t *sequences; /** Public change counter. On its update clients need to fetch * new space data from the instance. */ @@ -379,6 +380,7 @@ schema_init(void) spaces_by_name = mh_strnptr_new(); funcs = mh_i32ptr_new(); funcs_by_name = mh_strnptr_new(); + funcs_fin = mh_i32ptr_new(); sequences = mh_i32ptr_new(); /* * Create surrogate space objects for the mandatory system @@ -540,6 +542,14 @@ schema_free(void) func_delete(func); } mh_i32ptr_delete(funcs); + while (mh_size(funcs_fin) > 0) { + mh_int_t i = mh_first(funcs_fin); + struct func *func = + (struct func *)mh_i32ptr_node(funcs_fin, i)->val; + func_cache_delete(func->def->fid); + func_delete(func); + } + mh_i32ptr_delete(funcs_fin); while (mh_size(sequences) > 0) { mh_int_t i = mh_first(sequences); @@ -597,6 +607,33 @@ func_by_name(const char *name, uint32_t name_len) return (struct func *) mh_strnptr_node(funcs_by_name, func)->val; } +void +func_fin_cache_insert(struct func *func) +{ + uint32_t fid = func->def->fid; + assert(func_fin_by_id(fid) == NULL); + const struct mh_i32ptr_node_t node = {fid, func}; + mh_i32ptr_put(funcs_fin, &node, NULL, NULL); +} + +void +func_fin_cache_delete(uint32_t fid) +{ + mh_int_t k = mh_i32ptr_find(funcs_fin, fid, NULL); + if (k == mh_end(funcs_fin)) + return; + mh_i32ptr_del(funcs_fin, k, NULL); +} + +struct func * +func_fin_by_id(uint32_t fid) +{ + mh_int_t func = mh_i32ptr_find(funcs_fin, fid, NULL); + if (func == mh_end(funcs_fin)) + return NULL; + return (struct func *)mh_i32ptr_node(funcs_fin, func)->val; +} + int schema_find_grants(const char *type, uint32_t id, bool *out) { diff --git a/src/box/schema.h b/src/box/schema.h index d3bbdd590..2f020ee96 100644 --- a/src/box/schema.h +++ b/src/box/schema.h @@ -102,6 +102,15 @@ func_by_id(uint32_t fid); struct func * func_by_name(const char *name, uint32_t name_len); +void +func_fin_cache_insert(struct func *func); + +void +func_fin_cache_delete(uint32_t fid); + +struct func * +func_fin_by_id(uint32_t fid); + /** Call a visitor function on every space in the space cache. */ int space_foreach(int (*func)(struct space *sp, void *udata), void *udata); diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c index eb169aeb8..46ffe377c 100644 --- a/src/box/sql/expr.c +++ b/src/box/sql/expr.c @@ -5469,7 +5469,18 @@ analyzeAggregate(Walker * pWalker, Expr * pExpr) (pExpr, EP_xIsSelect)); pItem = &pAggInfo->aFunc[i]; pItem->pExpr = pExpr; - pItem->iMem = ++pParse->nMem; + int n = pExpr->x.pList == NULL ? + 0 : pExpr->x.pList->nExpr; + /* + * Allocate 1 MEM for + * accumulator and next n MEMs + * for arguments. This makes it + * easier to pass these n + 1 + * MEMs to the user-defined + * aggregate function. + */ + pItem->iMem = pParse->nMem + 1; + pParse->nMem += n + 1; assert(!ExprHasProperty (pExpr, EP_IntValue)); pItem->func = @@ -5479,12 +5490,6 @@ analyzeAggregate(Walker * pWalker, Expr * pExpr) true; return WRC_Abort; } - assert(pItem->func->def-> - language == - FUNC_LANGUAGE_SQL_BUILTIN && - pItem->func->def-> - aggregate == - FUNC_AGGREGATE_GROUP); if (pExpr->flags & EP_Distinct) { pItem->iDistinct = pParse->nTab++; diff --git a/src/box/sql/func.c b/src/box/sql/func.c index b69bf7fd6..493b69a4c 100644 --- a/src/box/sql/func.c +++ b/src/box/sql/func.c @@ -2072,9 +2072,12 @@ uint32_t sql_func_flags(const char *name) { struct sql_func_dictionary *dict = built_in_func_get(name); - if (dict == NULL) + if (dict != NULL) + return dict->flags; + struct func *func = func_by_name(name, strlen(name)); + if (func == NULL || func->def->aggregate != FUNC_AGGREGATE_GROUP) return 0; - return dict->flags; + return SQL_FUNC_AGG; } static struct func_vtab func_sql_builtin_vtab; diff --git a/src/box/sql/select.c b/src/box/sql/select.c index 2e33a31d2..c77cd1c9f 100644 --- a/src/box/sql/select.c +++ b/src/box/sql/select.c @@ -4648,8 +4648,6 @@ is_simple_count(struct Select *select, struct AggInfo *agg_info) return NULL; if (NEVER(agg_info->nFunc == 0)) return NULL; - assert(agg_info->aFunc->func->def->language == - FUNC_LANGUAGE_SQL_BUILTIN); if (memcmp(agg_info->aFunc->func->def->name, "COUNT", 5) != 0 || (agg_info->aFunc->pExpr->x.pList != NULL && agg_info->aFunc->pExpr->x.pList->nExpr > 0)) @@ -5575,9 +5573,17 @@ finalizeAggFunctions(Parse * pParse, AggInfo * pAggInfo) for (i = 0, pF = pAggInfo->aFunc; i < pAggInfo->nFunc; i++, pF++) { ExprList *pList = pF->pExpr->x.pList; assert(!ExprHasProperty(pF->pExpr, EP_xIsSelect)); - sqlVdbeAddOp2(v, OP_AggFinal, pF->iMem, - pList ? pList->nExpr : 0); - sqlVdbeAppendP4(v, pF->func, P4_FUNC); + if (pF->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN) { + sqlVdbeAddOp2(v, OP_AggFinal, pF->iMem, + pList != NULL ? pList->nExpr : 0); + sqlVdbeAppendP4(v, pF->func, P4_FUNC); + } else { + sqlVdbeAddOp1(v, OP_UserAggFinal, pF->iMem); + const char *name = pF->func->def->name; + int len = pF->func->def->name_len; + char *str = sqlDbStrNDup(pParse->db, name, len); + sqlVdbeAppendP4(v, str, P4_DYNAMIC); + } } } @@ -5604,7 +5610,7 @@ updateAccumulator(Parse * pParse, AggInfo * pAggInfo) assert(!ExprHasProperty(pF->pExpr, EP_xIsSelect)); if (pList) { nArg = pList->nExpr; - regAgg = sqlGetTempRange(pParse, nArg); + regAgg = pF->iMem + 1; sqlExprCodeExprList(pParse, pList, regAgg, 0, SQL_ECEL_DUP); } else { @@ -5644,10 +5650,17 @@ updateAccumulator(Parse * pParse, AggInfo * pAggInfo) pParse->is_aborted = true; return; } - sqlVdbeAddOp3(v, OP_AggStep, nArg, regAgg, pF->iMem); - sqlVdbeAppendP4(v, ctx, P4_FUNCCTX); - sql_expr_type_cache_change(pParse, regAgg, nArg); - sqlReleaseTempRange(pParse, regAgg, nArg); + if (pF->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN) { + sqlVdbeAddOp3(v, OP_AggStep, nArg, regAgg, pF->iMem); + sqlVdbeAppendP4(v, ctx, P4_FUNCCTX); + } else { + sqlVdbeAddOp3(v, OP_UserAggStep, nArg, regAgg, + pF->iMem); + const char *name = pF->func->def->name; + int len = pF->func->def->name_len; + char *str = sqlDbStrNDup(pParse->db, name, len); + sqlVdbeAppendP4(v, str, P4_DYNAMIC); + } if (addrNext) { sqlVdbeResolveLabel(v, addrNext); sqlExprCacheClear(pParse); diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 24cb28260..873b9f9f4 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -1289,6 +1289,93 @@ case OP_FunctionByName: { break; } +/* + * Opcode: UserAggStep P1 P2 P3 P4 * + * Synopsis: accum=r[P3] step(r[P2@P1]) + * + * Execute the step function for a user-defined aggregate function. The step + * function has P1 arguments. P4 is a pointer to a function object that defines + * the aggregate function. Register P3 is the accumulator. + * + * The P1 arguments are taken from register P2 and its successors. The P2 + * register must be immediately after the P3 register. + */ +case OP_UserAggStep: { + assert(pOp->p4type == P4_DYNAMIC); + assert(pOp->p1 == 0 || pOp->p3 == pOp->p2 - 1); + struct func *func = func_by_name(pOp->p4.z, strlen(pOp->p4.z)); + if (func == NULL) { + diag_set(ClientError, ER_NO_SUCH_FUNCTION, pOp->p4.z); + goto abort_due_to_error; + } + + struct Mem *argv = &aMem[pOp->p3]; + struct port args; + struct port ret; + + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + port_vdbemem_create(&args, (struct sql_value *)argv, pOp->p1 + 1); + if (func->vtab->call(func, &args, &ret) != 0) { + region_truncate(region, region_svp); + goto abort_due_to_error; + } + + uint32_t size; + struct Mem *mem = (struct Mem *)port_get_vdbemem(&ret, &size); + assert(size == 1); + bool is_error = mem == NULL || mem_copy(&aMem[pOp->p3], mem) != 0; + port_destroy(&ret); + region_truncate(region, region_svp); + if (is_error) + goto abort_due_to_error; + break; +} + +/* Opcode: UserAggFinal P1 * * P4 * + * Synopsis: r[P1] = finalize(r[P1]) + * + * Execute the finalize function for a user-defined aggregate function. P4 is a + * pointer to a function object that defines the aggregate function. Register P1 + * is the accumulator. The result will be written to register P1. + */ +case OP_UserAggFinal: { + assert(pOp->p1 > 0 && pOp->p1 <= (p->nMem + 1 - p->nCursor)); + assert(pOp->p4type == P4_DYNAMIC); + struct func *step_func = func_by_name(pOp->p4.z, strlen(pOp->p4.z)); + if (step_func == NULL) { + diag_set(ClientError, ER_NO_SUCH_FUNCTION, pOp->p4.z); + goto abort_due_to_error; + } + struct func *func = func_fin_by_id(step_func->def->fid); + if (func == NULL) { + diag_set(ClientError, ER_NO_SUCH_FUNCTION, pOp->p4.z); + goto abort_due_to_error; + } + + struct Mem *argv = &aMem[pOp->p1]; + struct port args; + struct port ret; + + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + port_vdbemem_create(&args, (struct sql_value *)argv, 1); + if (func->vtab->call(func, &args, &ret) != 0) { + region_truncate(region, region_svp); + goto abort_due_to_error; + } + + uint32_t size; + struct Mem *mem = (struct Mem *)port_get_vdbemem(&ret, &size); + assert(size == 1); + bool is_error = mem == NULL || mem_copy(&aMem[pOp->p1], mem) != 0; + port_destroy(&ret); + region_truncate(region, region_svp); + if (is_error) + goto abort_due_to_error; + break; +} + /* Opcode: BitAnd P1 P2 P3 * * * Synopsis: r[P3]=r[P1]&r[P2] * diff --git a/test/box-py/bootstrap.result b/test/box-py/bootstrap.result index cea440c64..78d444bb0 100644 --- a/test/box-py/bootstrap.result +++ b/test/box-py/bootstrap.result @@ -4,7 +4,7 @@ box.internal.bootstrap() box.space._schema:select{} --- - - ['max_id', 511] - - ['version', 2, 9, 1] + - ['version', 2, 10, 1] ... box.space._cluster:select{} --- @@ -54,7 +54,7 @@ box.space._space:select{} - [296, 1, '_func', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', 'type': 'unsigned'}, {'name': 'language', 'type': 'string'}, {'name': 'body', - 'type': 'string'}, {'name': 'routine_type', 'type': 'string'}, {'name': 'param_list', + 'type': 'any'}, {'name': 'routine_type', 'type': 'string'}, {'name': 'param_list', 'type': 'array'}, {'name': 'returns', 'type': 'string'}, {'name': 'aggregate', 'type': 'string'}, {'name': 'sql_data_access', 'type': 'string'}, {'name': 'is_deterministic', 'type': 'boolean'}, {'name': 'is_sandboxed', 'type': 'boolean'}, {'name': 'is_null_call', diff --git a/test/sql-tap/CMakeLists.txt b/test/sql-tap/CMakeLists.txt index c4ec1214a..136a517d4 100644 --- a/test/sql-tap/CMakeLists.txt +++ b/test/sql-tap/CMakeLists.txt @@ -3,6 +3,8 @@ build_module(gh-5938-wrong-string-length gh-5938-wrong-string-length.c) target_link_libraries(gh-5938-wrong-string-length msgpuck) build_module(gh-6024-funcs-return-bin gh-6024-funcs-return-bin.c) target_link_libraries(gh-6024-funcs-return-bin msgpuck) +build_module(gh-2579-custom-aggregate gh-2579-custom-aggregate.c) +target_link_libraries(gh-2579-custom-aggregate msgpuck) build_module(sql_uuid sql_uuid.c) target_link_libraries(sql_uuid msgpuck core) build_module(decimal decimal.c) diff --git a/test/sql-tap/gh-2579-custom-aggregate.c b/test/sql-tap/gh-2579-custom-aggregate.c new file mode 100644 index 000000000..5552effc5 --- /dev/null +++ b/test/sql-tap/gh-2579-custom-aggregate.c @@ -0,0 +1,48 @@ +#include "msgpuck.h" +#include "module.h" +#include "mp_decimal.h" +#include "lua/tnt_msgpuck.h" +#include "mp_uuid.h" + +enum { + BUF_SIZE = 512, +}; + +int +S3(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)args_end; + uint32_t arg_count = mp_decode_array(&args); + if (arg_count != 2) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, + "invalid argument count"); + } + int sum = 0; + if (mp_typeof(*args) != MP_UINT) + mp_decode_nil(&args); + else + sum = mp_decode_uint(&args); + sum += mp_decode_uint(&args); + char res[BUF_SIZE]; + char *end = mp_encode_uint(res, sum); + box_return_mp(ctx, res, end); + return 0; +} + +int +S3_finalize(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)args_end; + uint32_t arg_count = mp_decode_array(&args); + if (arg_count != 1) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, + "invalid argument count"); + } + int sum = 0; + if (mp_typeof(*args) == MP_UINT) + sum = mp_decode_uint(&args); + char res[BUF_SIZE]; + char *end = mp_encode_double(res, sum / 10.0); + box_return_mp(ctx, res, end); + return 0; +} diff --git a/test/sql-tap/gh-2579-custom-aggregate.test.lua b/test/sql-tap/gh-2579-custom-aggregate.test.lua new file mode 100755 index 000000000..d5b845761 --- /dev/null +++ b/test/sql-tap/gh-2579-custom-aggregate.test.lua @@ -0,0 +1,232 @@ +#!/usr/bin/env tarantool +local build_path = os.getenv("BUILDDIR") +package.cpath = build_path..'/test/sql-tap/?.so;'..build_path..'/test/sql-tap/?.dylib;'..package.cpath + +local test = require("sqltester") +test:plan(13) + +-- Make sure that non-persistent aggregate functions are working as expected. +local step = function(x, y) + if x == nil then + x = {sum = 0, count = 0} + end + x.sum = x.sum + y + x.count = x.count + 1 + return x +end + +local finalize = function(x) + return x.sum / x.count +end + +rawset(_G, 'S1', step) +rawset(_G, 'S1_finalize', finalize) +box.schema.func.create('S1', {aggregate = 'group', returns = 'number', + param_list = {'integer'}, exports = {'SQL'}}) + +test:execsql([[ + CREATE TABLE t(i INTEGER PRIMARY KEY AUTOINCREMENT, a INT); + INSERT INTO t(a) VALUES(1), (1), (2), (2), (3); +]]) + +test:do_execsql_test( + "gh-2579-1", + [[ + SELECT avg(a), s1(a) FROM t; + ]], { + 1, 1.8 + }) + +test:do_execsql_test( + "gh-2579-2", + [[ + SELECT avg(distinct a), s1(distinct a) FROM t; + ]], { + 2, 2 + }) + +-- Make sure that persistent aggregate functions are working as expected. +local body_s2 = { + step = [[function(x, y) + if x == nil then + x = {sum = 0, count = 0} + end + x.sum = x.sum + y + x.count = x.count + 1 + return x + end]], + finalize = [[function(x) + return x.sum / x.count + end]] +} + +box.schema.func.create('S2', {aggregate = 'group', returns = 'number', + body = body_s2, param_list = {'integer'}, + exports = {'SQL'}}) + +test:do_execsql_test( + "gh-2579-3", + [[ + SELECT avg(a), s2(a) FROM t; + ]], { + 1, 1.8 + }) + +test:do_execsql_test( + "gh-2579-4", + [[ + SELECT avg(distinct a), s2(distinct a) FROM t; + ]], { + 2, 2 + }) + +-- Make sure that C aggregate functions are working as expected. +box.schema.func.create("gh-2579-custom-aggregate.S3", { + language = "C", aggregate = 'group', param_list = {"integer"}, + returns = "number", exports = {"SQL"}, +}) + +test:do_execsql_test( + "gh-2579-5", + [[ + SELECT "gh-2579-custom-aggregate.S3"(a) FROM t; + ]], { + 0.9 + }) + +test:do_execsql_test( + "gh-2579-6", + [[ + SELECT "gh-2579-custom-aggregate.S3"(distinct a) FROM t; + ]], { + 0.6 + }) + +-- +-- Make sure that user-defined aggregate functions can have more than one +-- argument. +-- +local body_s4 = { + step = [[function(x, y, z) + if x == nil then + x = {sum = 0, count = 0} + end + x.sum = x.sum + y + z + x.count = x.count + 1 + return x + end]], + finalize = [[function(x) + return x.sum / x.count + end]] +} +box.schema.func.create('S4', {aggregate = 'group', returns = 'number', + param_list = {'integer', 'integer'}, + body = body_s4, exports = {'SQL'}}) + +test:do_execsql_test( + "gh-2579-7", + [[ + SELECT s4(a, i) FROM t; + ]], { + 4.8 + }) + +-- +-- Make sure that user-defined aggregate functions with more than one argument +-- cannot work with DISTINCT. +-- +test:do_catchsql_test( + "gh-2579-8", + [[ + SELECT s4(distinct a, i) FROM t; + ]], { + 1, "DISTINCT aggregates must have exactly one argument" + }) + +-- Make sure user-defined aggregate functions are not available in Lua. +test:do_test( + "gh-2579-9", + function() + local def = {aggregate = 'group', returns = 'number', + param_list = {'integer'}, exports = {'LUA', 'SQL'}} + local res = {pcall(box.schema.func.create, 'S5', def)} + return {tostring(res[2])} + end, { + "Failed to create function 'S5': aggregate function can only be ".. + "accessed in SQL" + }) + +-- +-- Make sure user-defined aggregate functions should have both STEP and FINALIZE +-- persistent or non-persistent. +-- +test:do_test( + "gh-2579-10", + function() + local body = "function(x) return x / 1000 end" + local def = {aggregate = 'group', returns = 'number', body = body, + param_list = {'integer'}, exports = {'SQL'}} + local res = {pcall(box.schema.func.create, 'S6', def)} + return {tostring(res[2])} + end, { + "Failed to create function 'S6': step or finalize of aggregate ".. + "function is undefined" + }) + +-- +-- Make sure body of user-defined aggregate function should have exactly two +-- fields: "step" and "finalize". +-- +test:do_test( + "gh-2579-11", + function() + local body = {step = "function(x) return x / 1000 end"} + local def = {aggregate = 'group', returns = 'number', body = body, + param_list = {'integer'}, exports = {'SQL'}} + local res = {pcall(box.schema.func.create, 'S7', def)} + return {tostring(res[2])} + end, { + "Failed to create function 'S7': body of aggregate function should ".. + "be map that contains exactly two string fields: 'step' and 'finalize'" + }) + +test:do_test( + "gh-2579-12", + function() + local body = {finalize = "function(x) return x / 1000 end"} + local def = {aggregate = 'group', returns = 'number', body = body, + param_list = {'integer'}, exports = {'SQL'}} + local res = {pcall(box.schema.func.create, 'S8', def)} + return {tostring(res[2])} + end, { + "Failed to create function 'S8': body of aggregate function should ".. + "be map that contains exactly two string fields: 'step' and 'finalize'" + }) + +test:do_test( + "gh-2579-13", + function() + local body = { + step = [[function(x, y, z) + if x == nil then + x = {sum = 0, count = 0} + end + x.sum = x.sum + y + z + x.count = x.count + 1 + return x + end]], + finalize = [[function(x) + return x.sum / x.count + end]], + something = 0 + } + local def = {aggregate = 'group', returns = 'number', body = body, + param_list = {'integer'}, exports = {'SQL'}} + local res = {pcall(box.schema.func.create, 'S9', def)} + return {tostring(res[2])} + end, { + "Failed to create function 'S9': body of aggregate function should ".. + "be map that contains exactly two string fields: 'step' and 'finalize'" + }) + +test:finish_test() -- 2.25.1