From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 843C32106F for ; Wed, 10 Jul 2019 07:01:21 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CWh_0GdBh4DQ for ; Wed, 10 Jul 2019 07:01:21 -0400 (EDT) Received: from smtp60.i.mail.ru (smtp60.i.mail.ru [217.69.128.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id BC9B2248C5 for ; Wed, 10 Jul 2019 07:01:20 -0400 (EDT) From: Kirill Shcherbatov Subject: [tarantool-patches] [PATCH v2 09/12] box: introduce Lua persistent functions Date: Wed, 10 Jul 2019 14:01:08 +0300 Message-Id: <2f63c2c9773c448399d5b8fac08e282dabf57619.1562756438.git.kshcherbatov@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org, korablev@tarantool.org Cc: kostja@tarantool.org, Kirill Shcherbatov Closes #4182 Closes #4219 Needed for #1260 @TarantoolBot document Title: Persistent Lua functions Now Tarantool supports 'persistent' Lua functions. Such functions are stored in snapshot and are available after restart. To create a persistent Lua function, specify a function body in box.schema.func.create call: e.g. body = "function(a, b) return a + b end" A Lua persistent function may be 'sandboxed'. The 'sandboxed' function is executed in isolated environment: a. only limited set of Lua functions and modules are available: -assert -error -pairs -ipairs -next -pcall -xpcall -type -print -select -string -tonumber -tostring -unpack -math -utf8; b. global variables are forbidden Finally, the new 'is_deterministic' flag allows to mark a registered function as deterministic, i.e. the function that can produce only one result for a given list of parameters. The new box.schema.func.create interface is: box.schema.func.create('funcname', , , , , , , ) This schema change is also reserves names for sql builtin functions: TRIM, TYPEOF, PRINTF, UNICODE, CHAR, HEX, VERSION, QUOTE, REPLACE, SUBSTR, GROUP_CONCAT, JULIANDAY, DATE, TIME, DATETIME, STRFTIME, CURRENT_TIME, CURRENT_TIMESTAMP, CURRENT_DATE, LENGTH, POSITION, ROUND, UPPER, LOWER, IFNULL, RANDOM, CEIL, CEILING, CHARACTER_LENGTH, CHAR_LENGTH, FLOOR, MOD, OCTET_LENGTH, ROW_COUNT, COUNT, LIKE, ABS, EXP, LN, POWER, SQRT, SUM, TOTAL, AVG, RANDOMBLOB, NULLIF, ZEROBLOB, MIN, MAX, COALESCE, EVERY, EXISTS, EXTRACT, SOME, GREATER, LESSER, _sql_stat_get, _sql_stat_push, _sql_stat_init, LUA A new Lua persistent function LUA is introduced to evaluate LUA strings from SQL in future. This names could not be used for user-defined functions. Example: lua_code = [[function(a, b) return a + b end]] box.schema.func.create('summarize', {body = lua_code, is_deterministic = true, is_sandboxed = true}) box.func.summarize --- - aggregate: none returns: any exports: lua: true sql: false id: 60 is_sandboxed: true setuid: false is_deterministic: true body: function(a, b) return a + b end name: summarize language: LUA ... box.func.summarize:call({1, 3}) --- - 4 ... --- src/box/alter.cc | 154 +++++++++++++++++-- src/box/bootstrap. | Bin 0 -> 5528 bytes src/box/bootstrap.snap | Bin 4475 -> 5794 bytes src/box/func.c | 22 ++- src/box/func_def.c | 24 ++- src/box/func_def.h | 59 ++++++- src/box/lua/call.c | 241 ++++++++++++++++++++++++++++- src/box/lua/schema.lua | 19 ++- src/box/lua/upgrade.lua | 67 +++++++- src/box/schema_def.h | 14 ++ src/box/sql.h | 5 + src/box/sql/func.c | 43 ++++++ test-run | 2 +- test/box-py/bootstrap.result | 76 ++++++++- test/box-py/bootstrap.test.py | 2 +- test/box/access.result | 2 +- test/box/access.test.lua | 2 +- test/box/access_bin.result | 2 +- test/box/access_bin.test.lua | 2 +- test/box/access_misc.result | 133 +++++++++++++++- test/box/access_sysview.result | 8 +- test/box/alter.result | 2 +- test/box/function1.result | 273 ++++++++++++++++++++++++++++++++- test/box/function1.test.lua | 98 +++++++++++- test/wal_off/func_max.result | 8 +- 25 files changed, 1199 insertions(+), 59 deletions(-) create mode 100644 src/box/bootstrap. diff --git a/src/box/alter.cc b/src/box/alter.cc index ce0cf2d9b..c92a1f710 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -2624,35 +2624,88 @@ func_def_get_ids_from_tuple(struct tuple *tuple, uint32_t *fid, uint32_t *uid) static struct func_def * func_def_new_from_tuple(struct tuple *tuple) { - uint32_t len; - const char *name = tuple_field_str_xc(tuple, BOX_FUNC_FIELD_NAME, - &len); - if (len > BOX_NAME_MAX) + uint32_t field_count = tuple_field_count(tuple); + uint32_t name_len, body_len, comment_len; + const char *name, *body, *comment; + name = tuple_field_str_xc(tuple, BOX_FUNC_FIELD_NAME, &name_len); + if (name_len > BOX_NAME_MAX) { tnt_raise(ClientError, ER_CREATE_FUNCTION, tt_cstr(name, BOX_INVALID_NAME_MAX), "function name is too long"); - identifier_check_xc(name, len); - struct func_def *def = (struct func_def *) malloc(func_def_sizeof(len)); + } + identifier_check_xc(name, name_len); + if (field_count > BOX_FUNC_FIELD_BODY) { + body = tuple_field_str_xc(tuple, BOX_FUNC_FIELD_BODY, + &body_len); + comment = tuple_field_str_xc(tuple, BOX_FUNC_FIELD_COMMENT, + &comment_len); + uint32_t len; + const char *routine_type = tuple_field_str_xc(tuple, + BOX_FUNC_FIELD_ROUTINE_TYPE, &len); + if (len != strlen("function") || + strncasecmp(routine_type, "function", len) != 0) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, name, + "unsupported routine_type value"); + } + const char *sql_data_access = tuple_field_str_xc(tuple, + BOX_FUNC_FIELD_SQL_DATA_ACCESS, &len); + if (len != strlen("none") || + strncasecmp(sql_data_access, "none", len) != 0) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, name, + "unsupported sql_data_access value"); + } + bool is_null_call = tuple_field_bool_xc(tuple, + BOX_FUNC_FIELD_IS_NULL_CALL); + if (is_null_call != true) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, name, + "unsupported is_null_call value"); + } + } else { + body = NULL; + body_len = 0; + comment = NULL; + comment_len = 0; + } + uint32_t body_offset, comment_offset; + uint32_t def_sz = func_def_sizeof(name_len, body_len, comment_len, + &body_offset, &comment_offset); + struct func_def *def = + (struct func_def *) malloc(def_sz); if (def == NULL) - tnt_raise(OutOfMemory, func_def_sizeof(len), "malloc", "def"); + tnt_raise(OutOfMemory, def_sz, "malloc", "def"); auto def_guard = make_scoped_guard([=] { free(def); }); func_def_get_ids_from_tuple(tuple, &def->fid, &def->uid); if (def->fid > BOX_FUNCTION_MAX) { tnt_raise(ClientError, ER_CREATE_FUNCTION, - tt_cstr(name, len), "function id is too big"); + tt_cstr(name, name_len), "function id is too big"); + } + memcpy(def->name, name, name_len); + def->name[name_len] = 0; + def->name_len = name_len; + if (body_len > 0) { + def->body = (char *)def + body_offset; + memcpy(def->body, body, body_len); + def->body[body_len] = 0; + } else { + def->body = NULL; } - memcpy(def->name, name, len); - def->name[len] = 0; - def->name_len = len; - if (tuple_field_count(tuple) > BOX_FUNC_FIELD_SETUID) + if (comment_len > 0) { + def->comment = (char *)def + comment_offset; + memcpy(def->comment, comment, comment_len); + def->comment[comment_len] = 0; + } else { + def->comment = NULL; + } + if (field_count > BOX_FUNC_FIELD_SETUID) def->setuid = tuple_field_u32_xc(tuple, BOX_FUNC_FIELD_SETUID); else def->setuid = false; - if (tuple_field_count(tuple) > BOX_FUNC_FIELD_LANGUAGE) { + if (field_count > BOX_FUNC_FIELD_LANGUAGE) { const char *language = tuple_field_cstr_xc(tuple, BOX_FUNC_FIELD_LANGUAGE); def->language = STR2ENUM(func_language, language); - if (def->language == func_language_MAX) { + if (def->language == func_language_MAX || + def->language == FUNC_LANGUAGE_SQL) { tnt_raise(ClientError, ER_FUNCTION_LANGUAGE, language, def->name); } @@ -2660,6 +2713,79 @@ func_def_new_from_tuple(struct tuple *tuple) /* Lua is the default. */ def->language = FUNC_LANGUAGE_LUA; } + if (field_count > BOX_FUNC_FIELD_BODY) { + def->is_deterministic = + tuple_field_bool_xc(tuple, + BOX_FUNC_FIELD_IS_DETERMINISTIC); + def->is_sandboxed = + tuple_field_bool_xc(tuple, + BOX_FUNC_FIELD_IS_SANDBOXED); + const char *returns = + tuple_field_cstr_xc(tuple, BOX_FUNC_FIELD_RETURNS); + def->returns = STR2ENUM(field_type, returns); + if (def->returns == field_type_MAX) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, + def->name, "invalid returns value"); + } + def->exports.all = 0; + const char *exports = + tuple_field_with_type_xc(tuple, BOX_FUNC_FIELD_EXPORTS, + MP_ARRAY); + uint32_t cnt = mp_decode_array(&exports); + for (uint32_t i = 0; i < cnt; i++) { + if (mp_typeof(*exports) != MP_STR) { + tnt_raise(ClientError, ER_FIELD_TYPE, + int2str(BOX_FUNC_FIELD_EXPORTS + 1), + mp_type_strs[MP_STR]); + } + uint32_t len; + const char *str = mp_decode_str(&exports, &len); + switch (STRN2ENUM(func_language, str, len)) { + case FUNC_LANGUAGE_LUA: + def->exports.lua = true; + break; + case FUNC_LANGUAGE_SQL: + def->exports.sql = true; + break; + default: + tnt_raise(ClientError, ER_CREATE_FUNCTION, + def->name, "invalid exports value"); + } + } + const char *aggregate = + tuple_field_cstr_xc(tuple, BOX_FUNC_FIELD_AGGREGATE); + def->aggregate = STR2ENUM(func_aggregate, aggregate); + if (def->aggregate == func_aggregate_MAX) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, + def->name, "invalid aggregate value"); + } + const char *param_list = + tuple_field_with_type_xc(tuple, + BOX_FUNC_FIELD_PARAM_LIST, MP_ARRAY); + uint32_t argc = mp_decode_array(¶m_list); + for (uint32_t i = 0; i < argc; i++) { + if (mp_typeof(*param_list) != MP_STR) { + tnt_raise(ClientError, ER_FIELD_TYPE, + int2str(BOX_FUNC_FIELD_PARAM_LIST + 1), + mp_type_strs[MP_STR]); + } + uint32_t len; + const char *str = mp_decode_str(¶m_list, &len); + if (STRN2ENUM(field_type, str, len) == field_type_MAX) { + tnt_raise(ClientError, ER_CREATE_FUNCTION, + def->name, "invalid argument type"); + } + } + def->param_count = argc; + } else { + def->is_deterministic = false; + def->is_sandboxed = false; + def->returns = FIELD_TYPE_ANY; + def->aggregate = FUNC_AGGREGATE_NONE; + def->exports.all = 0; + def->exports.lua = true; + def->param_count = 0; + } def_guard.is_active = false; return def; } diff --git a/src/box/bootstrap. b/src/box/bootstrap. new file mode 100644 index 0000000000000000000000000000000000000000..c1fd6a6f96746d5de84be79822b11a7b4dd4040d GIT binary patch literal 5528 zcmV;J6=&*GPC-x#FfK7O3RY!ub7^mGIv_GGGA=MJH83|VXE--CHf1$3HZlrHZgX^D zZewLSAY(XVVlZMeI4xu~W@0TgGh;F>Vq!HhEjeOlHZ);5G&DA5WC~V8Y;R+0Iv{&} z3JTS_3%bn}9RSW`n{|q%0000004TLD{Qyv|NsC0|AhYCGp|Cp zDLnH$7i5tTovO&>kEORvNl_|Uo0F6>NlJh>D~+kHWn-XtYY$}WkyGVotR)mB|2b!G zh=~NH0>lDx0$c5Gs1)PDmMqNE)(^~ccI;DG&QumSsX z(t!Q3KVb&HUz5}Cmt+$DsD8;{Q~vTYzYKDe$FTg;i zw7`rhE4Y9uuS2H1Dk-pFUAvD56=yWF zr)qS(G|jS0Qzc!R%w=JfIHi~}DaY<~sJ!81T3C{HdgZp$G+WDtWv572c5218)14-( zAf;)|N}3*3Nz){)_P^25|1;YENh(INHxtYxpn~T9y~fJC-G8IGf7A5(O#~A_Ac58o zM4j(A_hi1$Qv zydP4ecOgW24>F{8AVPpZ0D=I4KY##%Iq3rcS&ttmZ%^&3Vmy4LnQF+JX*QEPdI)>w z+h@$-0o>g^?$fk-?Z|n#o`gN`-}OhXU*)=gj|_vRN3#vFM%#1q_lT!pJ2^Qo*Ao~t zU34>?&B%x~Z#mI51>4Daxt{Rr#oO(>4j#+koR{k<4ql?$Y|21m6if%ZTt(=J5ka@5 z*ns53jEEgCmzexTYFGG)=r;A4U60hj%!y*iV04@A+jbQh#--VI{hgWhRq3x;jhd+L zU8fo~Ss~v+lMTjrX$Hn4z)k>2lDw+rKYjM7D;APHfGPB3sC}yG95R znO2Y?F)biMl(8XM%%^Gr5Jbl22auQ#yu{1ZDNSCZsM}|<>G5NA;^AX;+RUs6M;nAg zjyCv49Br@-x&g-yw(-Uevf;)K?%*12^j8fw`kTfY{p~LrYP|0mX}qr)XuQcX&X8r1 zVMZ)#j54-fD)r40gN**e7^8nM#OS~OFT(iW3o!og;*0;2TzK*S7G3zLJ;A75JPwmgb?2U5QO(W0O6B8{2+6~gAXw`aC$g5>aqH{QPbSs z%?>}uvVUVk_HJs(K6|Y1}VZl)&~X32d#ffGx1CxhQ;tx=){hz4#UgsorlgAimMY=IDa`XE3UuqH+?h(awu z;tF^aT=9aDx*&1ERJKsjQq(Kr744#oFCv|bC`(Qh86u^j+z<|GC`srhgau0IN{}rG zDagvJ>h%Ov1*irEbOfVC!Hj-pQ9o3CCh(EKCxX@IbqNGY9#AqxNl_0`kCYw~ zI0_05=4pUOmL!&B2;&{09R^%cP(0{&Y4g_@M(h5GE*-p_g8e}0-w)CDBz`}XI!z^J zpIou+X_Zm;y{^-ul9 z*IsGeexAE}4DRllMNE`{RM-9unnpZ?!1~`T)wO@kA{IKJ%*o$0Xc}>l0bPGKo8M;^ z>u251qXq)o^Vqsg`pB~RG+FlLsBUU~D9_&X8|&QPZ@z(De~p@ctMU3gv&k;`G(W4l zt7@+DyuV-fqu;Cv%ssnSrsn&EvOTmcnGESU0Mc^TiOZ%>ao4-h^9WNJ_f)T~g zwfp)aW`gG2tgHJK&VfO*Z6}_a^}VhUzocM7FW@ELKr|OImVybn5P_PmuF$hHELF}X zS-fZ_A?^#gSeSHh4cai2SO3J#>sb}1YyT3z*RNW{E-9E$3%4x0VHC>sPY@8ATlVKC zNTclunib-GheIb&)+9;kqT&UPu)#+$(vW0$V_fW#8RecGaZjh z?T8(a;_Yl)YDOHTjIO4oR>TdE+0|~d91V$OQM05WjzQUyhInX9PFM4S#6l_9ZnqxJ zh#5e^D0efm!cb1oG8^J2Cd_y_UJ*Mfp|>02CM4*x?1*@Q5tNEJNQ4^|1`2^oZHN^B zVP`ZO;sidxV!IJB0v;6c0Z+JIj%IGj>8h{+Vt6$kEr*2*2$M@1Vi=fQ?pCFUe>#{U z(XKEP4oqE*N1N$pNHjIrX9FuZ+s#&^9q|$kY&9AYE6E@!cZ8eijFm4xcTVP9N!q6jd!#4jLch>-4#x<>3Br^%0;p% zMX@PDLq9}9PoxMvDH;-r&432Xtalqi@c;*o=It2%rbsli-ff86ly4|q`xgq7rq)LT z#zC zm_!)8p{Z)-ZANC(49KPgvIy7yy5Y@8Js%5iT-Y;}1sSchtN?#MzfRT)8sVOZl zd(6M&!&UXFG=Qp(W{80MIo+D8_;337Tw!-ShDi!3{*dznasoycE>W+6KbN-DoO75+&5gi^(h8=VYLD5X%{4l>z9|;|KXd<7bx{~ z^A_QHy=Y*2893g@Q@YvP@Ug$aTdCb>q2aaBHGTlv(IlW_tHb?jBgI-4WC`8pN~sZe zT(|iN4QPkTKUfi==7QZ7#}GH%zUQZVI@XdqfH^cbvrV4jdXQA*;fby?9m2c{`Qn!+ z8aP&xGwTTi1*HdmLiq_oeJ_d7%hog@#IqJyg4 zmJG&2BHDPWUX@=6q9O-}TdW{>jXWGq_r&Lo3#=Q^JEU2}aF#*A;bi$Gkx4lO?BzWW z_P}64lP1`omo<&49mkQNhh0f!!>?G~RnXm46lZtgx|5mD4f$Ig7?$;6$g#>=6cDRd zmsfqPBe*o1yAl46Ql-oQD#>-enJ+5$S|WWC4aQFmQg-0%sJ`HwG~A7n(Km__7r~b! zI%9JUW>U z)7Lt;#A^>$VLS8Ry0+OK(Y7nlx%+rF`k*Y=VK;x{x=OW?jvmQLFy&U7v`Y~0)Ut#P zy?ANj7=2h~r9=}t=Tu>ci)g1q0qiW2;6CM1#O5sJ>|cWIB6o{%mJ{4S+_ks`)<0V( zOhDn8U(EU~G&(sw$R#1>B4G#%1P@ZE_!^ePa@(7Ty)TYhTg*~Nf~sJM<6gQ~ZN;C4 z>VI%+$1@d}6{2D1wygxW5LcAIBPeg}Ts?~)0At%LXK@BH1Xc1r z@X)+nRbo8xM$}MkEGOEC%#I2O6RAoBZ^QL&h+t8G6)yKiVU`3{f&YTrQBtE+;R#Ke z1Z997p0BB?01S=PVmpEi(90$lJMXega=sk)}`D0Q13QgUMh@JH)Acy zlEP;ue^{4LxGBj#=7yP2s%$$G`J2t`r-VT#C{V zelrCgj2+5DL5C;xhLx#F`?J)d^E5Sm(J z#nhmE^A_A6lLyl7hUeO_&6F#}h

VK4d^9#KW)MGG3k1e4h*)e9hZV#xzHkSl1L3Ug@VEilgLI%^N=>qP|1rQy?*Ez)AOr{vIf%L+Gc?NMR1>T-_LF$@l8xIr+Vh0GPa*fd`6XRB zr4ATcm*L+G9YE|tAvEsX95MEZ4X6-(W$a}7QY;w$^%P);9~ggG3lA5J9^~f^LI)w7 zcjxAau}^G3h3G3|C)1ZgmxPsx!3SK?rFxFFte`uF&aWpF0&OdfBYLzFSgOoRM80%b zPR_&`pdOoJ^6`N(QRYt!cFfB4TfcR4O7(7-u_t^$h3G3|H|wXAQ1S}XY%=JqYAd=P zYvzQd)&?9gTc~odATR5;JWz~X&g`B9v52`vF^?=Kc^YOFk6F;7RnElFy@ragbXzap zjB1`3aD1scob!qzWzqbMWkz3k1_$E5j#|qeJB#;j$A;W7Vc1^;A=tL^dZPD;dRw{Y zfF}2?B|P2UowOX?W~%{*S?JR&qzckjR^A4JNtRtF>S?$9-eVKQSZ6Z2POzP^5yVfS z!^7I9FoCFamEwDDMfldMW6e@)GK6{lXmij#;7bZ53CJYEdghTT- z5VeC8)%cs45ITzbOUs-OiL3>J>!9n|K+gS5qG*69qhd6EHDQ}!pUg`y9-HisOZ0h7 z>zTQz{(8Ef>g21W@`l(uopaEppCB^BTE0^LT|$YA!vxPANTJTF)HF=4Ge+my;|km5 z=D&t!Xs7q^N^KkR>W9!>_`#5>Hi-7pJ`M7z&5CK8310RpAfPft@4-dAIyN|Dx zyN#U)$c~?%hsU0e2h7K3F9;Uk;THxA+gjOz1q5uYtZnV>Z0v1#vGrwLJ#1V=Kq2q3 zu@}2=x~9n3kbYjP7jb_<`d`?;zC|Eu-ZUb0jn9CUfS$&FJd->_xcrkyq9{DVJ;Tdc zRVTqr{}z7%lfJMwhH~m?XDI)wuUuhvk`cPS^_&KVNE{Yde_Uuh4EgjQ40V}-6!|ojmWqG^zoIgBs64|f2jWUr zahYrIrubnbl(qPxVH5F0yN9@4I~&}P#}W#NG3kKUdD(S((Gn%sc!>}7p3wE41n-+| z>UAtZI_hi4>Roj3>RqarNGt273Y^`LG+hK-%wPZkHo)PeuoOT$NL+bukpl^E;%pD| z12gq^36AFk#n}Alk>Ry9-{yQy{xo|5&z(I&e1uT1M)?NzTJosArHI}^%kSo?)xm)H zNSrOxii!A$^&_ZU&lYUY)RDN3A`f074*YG&u?3MHO+fz zSb6bq-L=F_?RifMS`Al@SRfoEW1HF1n@p}c`@y;6FPwGmUrAZ( zYPg)#MiPEaap=K*E0rN%|6Cy{F-Vd6t6MsI$vkBr^qKv^S-ID^h+RqGExWe8xkf4O z8p-=v6XNQ{daE^Lh=&2W+}CVkb|be-<{Cxy=Dfu+R(V!M`9B|-%CxYEi*4a|iW!07 z%u4F{e9XdbRJ7wx`s(@ewypJyWkv>sOe$^EZf4Mg=TyeuI1D@Dvp@;v=qnJk0EKc+ z>Jx~H0u_n^`4xtb~OIPM5{qiNbo82FmVeM1w~^mEBPb$3v;##Sc>n z#SgE4wu%xgi~X^t8&a2xOEpmNWp~z6KA+c~B4nSs@;OapOhuD; z;r~>hs3LUFhHf&CA-R~WZdW>|j2>;pBO&GNtzTlvtsZBL%WKU80hm16`S3dJK6HGq zj8kCPM1*{y;uL7C(pVU_Wj%)691@{yJDiKNK~8!l$HhZ*A>{7U7UllQ^i}S<6)QKY zYLhfo<#_r+$GvXkeDnPRYxP3{EmB|vq()D7@!Ls>zSgg?0K%_PykWg*cxYJ2m$HXu zRiUUi_kdklg0&a%6uMRsQJB4-@4sdF-VG69|Ct`o!-nO zvWSZkTdb|d^Egvr5zXrp43N(fvFC6gw`i5)?r}^s@tq}GDB$2fLt#0hz1{A(1&PB9`KNj9tE+bErWrs2eiRix-4e$a2@IFT?{z}KZ>^_t z&+(hJ9J)N9S$)5R3-Nu6g12#%`|0Vtm=F5!Rzgc6)*&P1k|#9iRR;HWH&WCcxMxco zhdZ!O2B)_Q8Am`YL_cSHwoIl=Z+daQuJrIWPj{}o#*KYjQ3Ia!Ar2H-~7w4m@>eGbt+(@H-ewp+RF3BUXg}q!;&=7 ztv*5DJz=C;@U^AM*<5qP9s;%;TEXVt;zA?y%xWRl1p&|iymQ>$)Vh?#M6XCIhn~GO z=c8y-F-LCOY&6qx!%eO%M?Z+pu+_%dm9P>rnb}p9m0Ey1B~=LT(Q5JFYscTs+a9Tu z6}J<#Uevpy%1bZFM0V9^39t)-WrAiV6ho34G{Y)Xo(m)iKyS`~x?RE87TBgO-s+L} zAtzY>$1)K?8_E7?^%POYoVt?1m+OoS6=a>vYc#C5-RNDGa#9nYg}p<4zXj$Nqwx-k z#oDAc5;RRzskE%cgRhEYjE3>141>^CsBKK8GM69}%|!2LKvpN#QHGy^B`NA-f1vh=PQF;`IDD49?6985@eVt2Q(c#H6cvE-ry>@4fo*+C~x_Q-~r06aU()bDJQ zv3gip`Pb`yO+6x!)k~X5h$B2oP8Xh*>Wf%O3M)3MFZR>O&Qo-qw>wckVwu<{v0KTm z39d$~77JPMsP_%;dtmiPd`);%8$A)2VyE6ac;;F#W1loGy1P5+z0Q||cRCp0S3kUX ztaTmICpdHdM%#l8|k%Q&}W$?L?ZQ|*@mnqD{ALVzcTb6as_z3y+IcJTkkM@8c>pF;tAthbz2pkmb&2>2-bI zgk(FX8yWU&NVd4cuIP%KmON>~`K{uM~J>|wuUAJ#9LlD^M6#gtKJ0SA% zfYxg5PC*$F8Eu~lIz;`qX?b}?W4HM-4l&1-Esx$7$HvDcKWi?^P%t$7h0d-_Y<1XR z>pHl-fQa8}t7aq6JS*^Msy=R5ztI&^lrrfCuQ7cHe#a-oQoa)#K zW1XjWb`f4z0uVyh6vCgUoYfuVZXL>kUvx)S8`SAym|YNHTY8 zNz|H+=-=RVYKIc4rcx5BG&c^txYWM?uqw@=R~7BWNA&(JZ^hc^2xj$W`Mb518D7yM zGWpW)0aq*jruT|@8}oz{GpGKY+)A#kQd8q?G0E((OLqyAWCNe(*#SAMz}ao@j(j(G zA_&&KlA1FH(CB70ORgA}t@Q3JuE$Y_$54!(dMl|4_(erNvRtIad@Q78m0s?r@ zKVxE|zo4ut$dPl$`rkydUjvWMf*U_$Wf1;iZ%yBg&Mf`8JmKK|z~5$!fBfBKhkC93 zDMrbfy2d^3j9zR^Z^R}>fQccpmy5Bb=US!QJe>jif}rw1>vuHlugTgx zbzTuNZe=8@E~X`#YwT$AROJ898eJ2OE9m;+Tr#cpL^~!Cbu7NG;@yBfNt$2`h?I#` zn#_VyYdt&Hwep}UNe779P(%8(RR&vt(FWEY8v8!q44<>TVqSd_c$#aPAUE|Ky(>6s zq^I*cLMulZAV9zLI+2wV`9s($Ol7pOB}&Rhk16(L_!hmy7MF+=Rd4>TpmgwD5J;Aw z!45)UgEq7v^5(x)0BQAYF*^};{hS0kC^<8B86 zDpM57_zKSp@0T`V1L^C_-n)C8kT9~=G6IfK{kW&aw)`=0JmYes<0?$Qo^x7uD3hVR z)oR$@WDaXK1iVcoTtWLFa+YaZR5an zccIaZ^d=g5{N9CevnWg$1vaq!fnM>VriVP1)Ps@!A>BC^QW!L{`2bJY82(!p#Sez>Z$VeynDN);ulWENX1$uO z;9Hj_3ib7Ztgyx}rCjD|ew7qrHHN3%(FZ5Ejcw1GD98?W(*CFF;6w2v{-dcjA5{GD z7fYOql2$qU{cNA5IH%Kc;#D6eDkv%0Wu5EtX21v!G0k|CUUkSCYJ{FWQa%a zWwS$EW>}Ln3(Q8Q9aSiBVtm=%8DLQpmeQNQHiqHDDEU@}by%HrXSlqjxB;DABxw&l zz(#SpCg@45&1aQa9HBBT@yzVh;8;-XphoeLyDVhaGSXw(QLwsPVdBaO=r$&ya>}_H zq&zVx`MFu)UR)$foaBq26O)E#(%QW~=uQIhyg8s`9yb@w)cXv5TX9>`r>qk?Py^D@xhMk9uk3D<&dg6hIcrimS*cQms>>ev*C?j zPw2E$6fIk1hNyEniynPkIiC)yy?lcct`3Q!U%^ZNdWA1tU!v6<0p2oaY!Y&=EokFq z3cxcm^E)_MfQbq1`g&0PkN<>GIvvpQe#)l>KvsA-r)RNxZhEAm<$i2G$gEZVkLNpp z&})*H0wW~^;^y!Pb=y1P_JjFC6{BJ9$hD!8^8P1BhlM^3A-GgSp`$hjRY-xCO($Vj z@>v6&lMdB6d)|%m_9Z(0OT+wtc{8W zb|Ou9^4=pMArvRiS>tX?N<^XC7u;j6TxufCJfv@}lVCi)gA_7%MR=S@;^t}hB-CQK zN7cose->2HYHHm_`3S!BY;PoVqSa1jAODtZ-kVc z;6SxJ*#DjxHy+0jJ!OCmb{K_&Y}6UWR#te4g8cXp8C`yQI(}$?Ktg&SDqX_qAI5jB zB?*Aio2JfWm`P9~GU$j{C z?8Vp*QDRvhj!!1zBE@AoQX7gyOXUtT)BCT7SvhK_l-JXu->;9N<08I%zwM0M?q+jW zf3D~*Ibu+x2QYcFWNkD310W%|;v0>4M}B--&QmN~qtM?FB{5)Rd6SS^aR1P0$&GxZ zF*iiXT3*0J{S)eJz!51C_}Tx*-iLS-?{SQ@U1Oi&@{cUGdbSMu(fh1$6ddTl`&}T9 zJ}!NOoJ{i@PdjBO(kILT>%UG5T0xeIAfOiJKr_PW@Kc~5Gle(yoDWSl$wdaTwA?sl zOwM`~D*7abPTMmM9tGi4T(r-J{-Ds`S?KiAN-B)aF>;t$W*V|7KWt=?0nv=OiHHg< zg}n#{LTM-ibmBXIma_UnW4$BTt5=ZHPZy)G0VgZDk`gzFl7Jmr1vQudXv(Xa=R}1! zJ(fh6jB%`)D-ty*$d0ARE-?k3z|Y2g2}#31rT@-Wqnj4{#4@U{JH58!^^HO66|+}3U5gYDjoaB@bJno%o;UiNTK+ZB9E&~#chJx1*9 ze*Gw-dW=&!P446ujPmZQWH&Gj_e;*T;m$*Bb`sT(pwXAu!kA?WR^i<6g@4BWap4Uf zcnWzNHL?cB^0Z@~uqp|eLR~ixJC^D40ar_ZZ3`Lo6ugOa3US_C3{>n%@(=D0cxzA@ znJV~?%hL}IiNN&`VLi#?9xB3O$#bY2-J8V0jyUv4e*va!E*)|CE^gmAoT&)RY&Pd< zelwKh6Iq>%%T1ZX_d!J$KvRIlGP?DZkWmO>8>Kcb+HcHtY(XurZSKN1f#O%u($qv!V@FqLj}4Vuw;`H3?Iv94Yl%jt9D^Rl3Jr zx$&DUtsHvlxeurB>~0GgTNI|I3R}v5LL#^GIdxTSXCCcmS#1;o$U1Y$b)C7Nlu}N| zcSYKD$;@tT%sa+3$yhrn1KDI(Yx_#BY(aX~wOSyVoBcc%10PJXt$d5*N$16I2*&4X ZK?CwIx&xG3pi5rpnjPC-x#FfK7O3RY!ub7^mGIv_GGGA=MJGdMUcXEb97;D zV`VxZHDzKqVmUQrEipD@GA%S>HexMdGdVdeV=*-`WMN@9Ha9b53RXjGZ)0mZAbWiZ z3e~y`y3G*+0M73H2F|4b00000D77#B08q_`0LqncAx038*f{_U01NL;9HZ zjmE{UXYp(U&L|KfBvUs;a2N;{-0OK+nV4P}T<{UF9bB-8jaE|9lCIi|WCk$qD zE+CU@y7F~#&KM_A6mx+nh8ZKK7KqV1Fkxo93M|lB#O=O-p?P_!Ad6kTWKmA`;^HNX zTe@VC0xnt1Vyb0}C}yER6tQR##q>&+|E*y8zv;^VDU%i2*91*fm_pmXzj5s5`RCB~ z|9$>`QJFx3D)&Y}OG;&F^SaYJN9spyoqd z&F_+Ev@fLD?2CaUr=}#QrY|W-G_NQO%)AzmLUQmszbsh38+MVL*Y4)n{J!f2C5jCd zSipuyP;6*JBL!>;9NBCL7};zGB;XA|!21FY@cxMb7EEOYSTHSmV!n40D^7}NPt`W3EAlF2-ia6~O8|1hI9yb4}L+q|DvS%HRhl_M;~To3eGAf4^S~|8d{`y%w4`WUg?*)f;j!?=!1f z$$olzVz8+*Z_12m0>j0MuHMZz7R*Za(-VVD)mUuv?E6Kt*wmgFY${Lfh53`RrH)GG zQ(>?se9}CzQsPLMDpoWx@goLH6aRDJ#sBFRUD&S$7xrhdh5hzpp#{y(60tufMaRluS9wTOVh$sTgp^wwS z9DZN+cZR@|y%0oXAH)#ZXAguB*Z&a2^*#V`{q;TkVEva5*5B#Ddh5Nuc(-$h_c~s@ z^G?T(HhHL{%^m1yQ=45pSW|}`tU0I4n%0z~L+86Wou_g-?>x?~GsCuYWoVSwsXMMc5T18`Z(n9{|#^cw*iMA=>t8&TAU9T?|?< z_q+W*CuMgoas7_>@0?}c@3GwMdqQ3RZPlSBZO|{P3JrZ*b}s5htHs>kCkMY!-z;+N zJnpM&^XY030Z})j-sZE=kE;R=VA>Mrzsa#I$hFS2{eE{8&s|w$+39Km}_61AVcsCh!A}H5d`1xtp^A#R(rKKtF_l- zv%zYw_Go!{RTl3~XGW0diKDh3%>X&7VR4~7`{e-Q@$_K)_xBug6;>rfvQ1>Z;3^k)meQ|3 zrAU$DH6CiMk{NxVzZ!Z#6?ULiIdIybi_*%d08m7*r9A||2;g!&N1xIzh=%-EYBt_Wy0wE()v}$K3y?Rks|SSXAGi z8vL{>lM$M`xs?# z*PmamA{oF2+^2|sx_VT=`LAaK?o<47)#!;8`ejv_tXngeaaK|{42EW?1Ei-KLL(yp z000mO08<-C5LavFk_;07fI-56v9d8d7K;Qi44CUN%hUi6fPe)FiU6g6fZfRuF2g7V}|o1*<23+CtwWu!{5^Rx=BZ30}>glkX-z%=23-#H(K5|OtV&h{EVErp?uMwM$W!qgE_FRJ^m|v_FE2XdtwM$6W+9DP zM%jbY``wKG-K9JW@C$_0v~`iew{hDnI!8aN%GnPhpHSuK2hmR`EbvO1Sk|Y7@TN-n zr&2S|iaS(O6tXPr|4_E<%5BkuFtV)92vJL+N-&6H(Fm2#F>oKDs&X`kQT&8gm7|^2 zaQ^Uy0_)`S4ycLr!^kJvF&baVWDtk~vEF|SJzxNss)?=VA5=tsaJJR&qdXPGwm@R) zRLFf)idygDn*?U=ImxqCP!!gr-i1_BH>DM4^y2+6j@MS1(oa5CCKTP{i7n1GT2^HAR?0alBgJiU|is~FZ>}P#cbH-tZ-VVEpH<6B6ZmHly zwPL+mV)~$#lh}|Cr<3?jk6>XAxaKXjQy*~`L4Ab+Tw<5O_JD&TrW2l%b^;ud)nzeU z;4IvTLg=?W{j(XuyhEJ%6W5QHS0{|cp(Vu6ONhn41gDx)1~eMRBD`;d z&n>K|s+T&=sT-uU3R;O?e@YJdPsWdjLSv!} z=z#=yshvQ6NfbFQ7qbK=xBFOuXLGUEjm7NFr^Fe40W?K5K&?U7w6 z0EDFHa5Wnx!QstW-2$Q(7~a_^2o6-vYT`bWHlo%B=&s{LUI3Ym#W6`%17sN&Mueg3 z!4UrhHhh1|^dW@okEllPKtM5EFXy4Cp*6c}HGsqkfYh$q!c5lgNvFip)sgr|m~1a; zMo(biwy0L@L)lYn*Suvg2pLA0dpJ$TPGI1+h%4izRu4KOJJo?x;~j>($Dlr-!{P!Y zFMn8^a7$izS8asp)4Jk==X~2eN+n*x3uA#15C^cRhVdim9BvX3t1dnx!UCyPV$8BX zvsYpR2zoqzXQLoEP&uoK`%v15+OSKM07xw>YHcj^xK11ac=T_-M3?pgvmb}y9(;7M z{F9*BGXKH?&;i>_BV+8*jy(NQp^VsQA*>wcI0QQwOK*e*#$ogLV-VcnczPo+Fo2rJ z9E0EvN7EaDfdSM!<`@KbIGWxF3=E*=F~=ad!_o9cU|;|>k2wax9ge0q0s{l6dCV~g z?r=1{5f~Ui&0{i`236vWZHv5Y1_q!Aq-sZHlb>s(bJ1yQWO|Jb@gV5X@6ru};6UZ9 zA~70GGeeTc(ZKs7z)jYB$RHOs_O3>rtJv)xI3vJySqI;^p9=_i*fz*zLNuB^Ln6SV z9eG-B#Pe)G2K%fAbxWaO1RIaj(I^WH4aRFS>)R_>yCig5I|6_+W6HfNvJoj!Nin!J zpU^zhHMV$NeU@#J*jB_7P=*FG2D5Tmz-?Et5d;1lGeYF-yZ~wTJTl~3 zl4I6-1xjS`gTZCP0>#rt{NqC^{7A}rhtrwKtEOV#hfv!PJcpuVMsxs13~_9egyn>?m;TEAwYjej`?U$oF}AJV`1h( zfSLTiL;24+#4`k*V8^Nf)cj4V=Sc;_L(R=jJmN#3R4EtCKkgWYFf3$Q?Z^;u@fTbGf+p0egF}x~JbOq` N%5QO4t`F4^t?kB;gJb{z diff --git a/src/box/func.c b/src/box/func.c index 8227527ec..8d93a83b2 100644 --- a/src/box/func.c +++ b/src/box/func.c @@ -34,7 +34,9 @@ #include "assoc.h" #include "lua/utils.h" #include "lua/call.h" +#include "lua/lua_sql.h" #include "error.h" +#include "sql.h" #include "diag.h" #include "port.h" #include "schema.h" @@ -385,11 +387,18 @@ struct func * func_new(struct func_def *def) { struct func *func; - if (def->language == FUNC_LANGUAGE_C) { + switch (def->language) { + case FUNC_LANGUAGE_C: func = func_c_new(def); - } else { - assert(def->language == FUNC_LANGUAGE_LUA); + break; + case FUNC_LANGUAGE_LUA: func = func_lua_new(def); + break; + case FUNC_LANGUAGE_SQL_BUILTIN: + func = func_sql_builtin_new(def); + break; + default: + unreachable(); } if (func == NULL) return NULL; @@ -416,8 +425,13 @@ static struct func_vtab func_c_vtab; static struct func * func_c_new(struct func_def *def) { - (void) def; assert(def->language == FUNC_LANGUAGE_C); + if (def->body != NULL || def->is_sandboxed) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "body and is_sandboxed options are not compatible " + "with C language"); + return NULL; + } struct func_c *func = (struct func_c *) malloc(sizeof(struct func_c)); if (func == NULL) { diag_set(OutOfMemory, sizeof(*func), "malloc", "func"); diff --git a/src/box/func_def.c b/src/box/func_def.c index 2b135e2d7..fb9f77df8 100644 --- a/src/box/func_def.c +++ b/src/box/func_def.c @@ -1,7 +1,9 @@ #include "func_def.h" #include "string.h" -const char *func_language_strs[] = {"LUA", "C"}; +const char *func_language_strs[] = {"LUA", "C", "SQL", "SQL_BUILTIN"}; + +const char *func_aggregate_strs[] = {"none", "group"}; int func_def_cmp(struct func_def *def1, struct func_def *def2) @@ -14,7 +16,27 @@ func_def_cmp(struct func_def *def1, struct func_def *def2) return def1->setuid - def2->setuid; if (def1->language != def2->language) return def1->language - def2->language; + if (def1->is_deterministic != def2->is_deterministic) + return def1->is_deterministic - def2->is_deterministic; + if (def1->is_sandboxed != def2->is_sandboxed) + return def1->is_sandboxed - def2->is_sandboxed; if (strcmp(def1->name, def2->name) != 0) return strcmp(def1->name, def2->name); + if ((def1->body != NULL) != (def2->body != NULL)) + return def1->body - def2->body; + if (def1->body != NULL && strcmp(def1->body, def2->body) != 0) + return strcmp(def1->body, def2->body); + if (def1->returns != def2->returns) + return def1->returns - def2->returns; + if (def1->exports.all != def2->exports.all) + return def1->exports.all - def2->exports.all; + if (def1->aggregate != def2->aggregate) + return def1->aggregate - def2->aggregate; + if (def1->param_count != def2->param_count) + return def1->param_count - def2->param_count; + if ((def1->comment != NULL) != (def2->comment != NULL)) + return def1->comment - def2->comment; + if (def1->comment != NULL && strcmp(def1->comment, def2->comment) != 0) + return strcmp(def1->comment, def2->comment); return 0; } diff --git a/src/box/func_def.h b/src/box/func_def.h index 866d425a1..508580f78 100644 --- a/src/box/func_def.h +++ b/src/box/func_def.h @@ -32,6 +32,7 @@ */ #include "trivia/util.h" +#include "field_def.h" #include #ifdef __cplusplus @@ -44,11 +45,21 @@ extern "C" { enum func_language { FUNC_LANGUAGE_LUA, FUNC_LANGUAGE_C, + FUNC_LANGUAGE_SQL, + FUNC_LANGUAGE_SQL_BUILTIN, func_language_MAX, }; extern const char *func_language_strs[]; +enum func_aggregate { + FUNC_AGGREGATE_NONE, + FUNC_AGGREGATE_GROUP, + func_aggregate_MAX, +}; + +extern const char *func_aggregate_strs[]; + /** * Definition of a function. Function body is not stored * or replicated (yet). @@ -58,17 +69,46 @@ struct func_def { uint32_t fid; /** Owner of the function. */ uint32_t uid; + /** Definition of the persistent function. */ + char *body; + /** User-defined comment for a function. */ + char *comment; /** * True if the function requires change of user id before * invocation. */ bool setuid; + /** + * Whether this function is deterministic (can produce + * only one result for a given list of parameters). + */ + bool is_deterministic; + /** + * Whether the routine must be initialized with isolated + * sandbox where only a limited number if functions is + * available. + */ + bool is_sandboxed; + /** The count of function's input arguments. */ + int param_count; + /** The type of the value returned by function. */ + enum field_type returns; + /** Function aggregate option. */ + enum func_aggregate aggregate; /** * The language of the stored function. */ enum func_language language; /** The length of the function name. */ uint32_t name_len; + /** Frontends where function must be available. */ + union { + struct { + bool lua : 1; + bool sql : 1; + }; + uint8_t all; + } exports; /** Function name. */ char name[0]; }; @@ -76,19 +116,32 @@ struct func_def { /** * @param name_len length of func_def->name * @returns size in bytes needed to allocate for struct func_def - * for a function of length @a a name_len. + * for a function of length @a a name_len, body @a body_len and + * with comment @a comment_len. */ static inline size_t -func_def_sizeof(uint32_t name_len) +func_def_sizeof(uint32_t name_len, uint32_t body_len, uint32_t comment_len, + uint32_t *body_offset, uint32_t *comment_offset) { /* +1 for '\0' name terminating. */ - return sizeof(struct func_def) + name_len + 1; + size_t sz = sizeof(struct func_def) + name_len + 1; + *body_offset = sz; + if (body_len > 0) + sz += body_len + 1; + *comment_offset = sz; + if (comment_len > 0) + sz += comment_len + 1; + return sz; } /** Compare two given function definitions. */ int func_def_cmp(struct func_def *def1, struct func_def *def2); +/** Duplicate a given function defintion object. */ +struct func_def * +func_def_dup(struct func_def *def); + /** * API of C stored function. */ diff --git a/src/box/lua/call.c b/src/box/lua/call.c index 38f2f696b..95fac4834 100644 --- a/src/box/lua/call.c +++ b/src/box/lua/call.c @@ -294,6 +294,7 @@ port_lua_create(struct port *port, struct lua_State *L) } struct execute_lua_ctx { + int lua_ref; const char *name; uint32_t name_len; struct port *args; @@ -323,6 +324,24 @@ execute_lua_call(lua_State *L) return lua_gettop(L); } +static int +execute_lua_call_by_ref(lua_State *L) +{ + struct execute_lua_ctx *ctx = + (struct execute_lua_ctx *) lua_topointer(L, 1); + lua_settop(L, 0); /* clear the stack to simplify the logic below */ + + lua_rawgeti(L, LUA_REGISTRYINDEX, ctx->lua_ref); + + /* Push the rest of args (a tuple). */ + int top = lua_gettop(L); + port_dump_lua(ctx->args, L, true); + int arg_count = lua_gettop(L) - top; + + lua_call(L, arg_count, LUA_MULTRET); + return lua_gettop(L); +} + static int execute_lua_eval(lua_State *L) { @@ -534,22 +553,168 @@ box_lua_eval(const char *expr, uint32_t expr_len, struct func_lua { /** Function object base class. */ struct func base; + /** + * For a persistent function: a reference to the + * function body. Otherwise LUA_REFNIL. + */ + int lua_ref; }; static struct func_vtab func_lua_vtab; +static struct func_vtab func_persistent_lua_vtab; + +static const char *default_sandbox_exports[] = { + "assert", "error", "ipairs", "math", "next", "pairs", "pcall", "print", + "select", "string", "table", "tonumber", "tostring", "type", "unpack", + "xpcall", "utf8", +}; + +/** + * Assemble a new sandbox with given exports table on the top of + * a given Lua stack. All modules in exports list are copied + * deeply to ensure the immutability of this system object. + */ +static int +prepare_lua_sandbox(struct lua_State *L, const char *exports[], + int export_count) +{ + lua_createtable(L, export_count, 0); + if (export_count == 0) + return 0; + int rc = -1; + const char *deepcopy = "table.deepcopy"; + int luaL_deepcopy_func_ref = LUA_REFNIL; + int ret = box_lua_find(L, deepcopy, deepcopy + strlen(deepcopy)); + if (ret < 0) + goto end; + luaL_deepcopy_func_ref = luaL_ref(L, LUA_REGISTRYINDEX); + assert(luaL_deepcopy_func_ref != LUA_REFNIL); + for (int i = 0; i < export_count; i++) { + uint32_t name_len = strlen(exports[i]); + ret = box_lua_find(L, exports[i], exports[i] + name_len); + if (ret < 0) + goto end; + switch (lua_type(L, -1)) { + case LUA_TTABLE: + lua_rawgeti(L, LUA_REGISTRYINDEX, + luaL_deepcopy_func_ref); + lua_insert(L, -2); + lua_call(L, 1, 1); + break; + case LUA_TFUNCTION: + break; + default: + unreachable(); + } + lua_setfield(L, -2, exports[i]); + } + rc = 0; +end: + luaL_unref(tarantool_L, LUA_REGISTRYINDEX, luaL_deepcopy_func_ref); + return rc; +} + +/** + * Assemble a Lua function object by user-defined function body. + */ +static int +func_persistent_lua_load(struct func_lua *func) +{ + int rc = -1; + int top = lua_gettop(tarantool_L); + struct region *region = &fiber()->gc; + size_t region_svp = region_used(region); + const char *load_pref = "return "; + uint32_t load_str_sz = + strlen(load_pref) + strlen(func->base.def->body) + 1; + char *load_str = region_alloc(region, load_str_sz); + if (load_str == NULL) { + diag_set(OutOfMemory, load_str_sz, "region", "load_str"); + return -1; + } + sprintf(load_str, "%s%s", load_pref, func->base.def->body); + + /* + * Perform loading of the persistent Lua function + * in a new sandboxed Lua thread. The sandbox is + * required to guarantee the safety of executing + * an arbitrary user-defined code + * (e.g. body = 'fiber.yield()'). + */ + struct lua_State *coro_L = lua_newthread(tarantool_L); + if (!func->base.def->is_sandboxed) { + /* + * Keep an original env to apply for non-sandboxed + * persistent function. It is required because + * built object inherits parent env. + */ + lua_getfenv(tarantool_L, -1); + lua_insert(tarantool_L, -2); + } + if (prepare_lua_sandbox(tarantool_L, NULL, 0) != 0) + unreachable(); + lua_setfenv(tarantool_L, -2); + int coro_ref = luaL_ref(tarantool_L, LUA_REGISTRYINDEX); + if (luaL_loadstring(coro_L, load_str) != 0 || + lua_pcall(coro_L, 0, 1, 0) != 0) { + diag_set(ClientError, ER_LOAD_FUNCTION, func->base.def->name, + luaT_tolstring(coro_L, -1, NULL)); + goto end; + } + if (!lua_isfunction(coro_L, -1)) { + diag_set(ClientError, ER_LOAD_FUNCTION, func->base.def->name, + "given body doesn't define a function"); + goto end; + } + lua_xmove(coro_L, tarantool_L, 1); + if (func->base.def->is_sandboxed) { + if (prepare_lua_sandbox(tarantool_L, default_sandbox_exports, + nelem(default_sandbox_exports)) != 0) { + diag_set(ClientError, ER_LOAD_FUNCTION, + func->base.def->name, + diag_last_error(diag_get())->errmsg); + goto end; + } + } else { + lua_insert(tarantool_L, -2); + } + lua_setfenv(tarantool_L, -2); + func->lua_ref = luaL_ref(tarantool_L, LUA_REGISTRYINDEX); + rc = 0; +end: + lua_settop(tarantool_L, top); + region_truncate(region, region_svp); + luaL_unref(tarantool_L, LUA_REGISTRYINDEX, coro_ref); + return rc; +} struct func * func_lua_new(struct func_def *def) { - (void) def; assert(def->language == FUNC_LANGUAGE_LUA); + if (def->is_sandboxed && def->body == NULL) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "is_sandboxed option may be set only for persistent " + "Lua function (when body option is set)"); + return NULL; + } struct func_lua *func = (struct func_lua *) malloc(sizeof(struct func_lua)); if (func == NULL) { diag_set(OutOfMemory, sizeof(*func), "malloc", "func"); return NULL; } - func->base.vtab = &func_lua_vtab; + if (def->body != NULL) { + func->base.def = def; + func->base.vtab = &func_persistent_lua_vtab; + if (func_persistent_lua_load(func) != 0) { + free(func); + return NULL; + } + } else { + func->lua_ref = LUA_REFNIL; + func->base.vtab = &func_lua_vtab; + } return &func->base; } @@ -574,6 +739,42 @@ static struct func_vtab func_lua_vtab = { .destroy = func_lua_destroy, }; +static void +func_persistent_lua_unload(struct func_lua *func) +{ + luaL_unref(tarantool_L, LUA_REGISTRYINDEX, func->lua_ref); +} + +static void +func_persistent_lua_destroy(struct func *base) +{ + assert(base != NULL && base->def->language == FUNC_LANGUAGE_LUA && + base->def->body != NULL); + assert(base->vtab == &func_persistent_lua_vtab); + struct func_lua *func = (struct func_lua *) base; + func_persistent_lua_unload(func); + free(func); +} + +static inline int +func_persistent_lua_call(struct func *base, struct port *args, struct port *ret) +{ + assert(base != NULL && base->def->language == FUNC_LANGUAGE_LUA && + base->def->body != NULL); + assert(base->vtab == &func_persistent_lua_vtab); + struct func_lua *func = (struct func_lua *)base; + struct execute_lua_ctx ctx; + ctx.lua_ref = func->lua_ref; + ctx.args = args; + return box_process_lua(execute_lua_call_by_ref, &ctx, ret); + +} + +static struct func_vtab func_persistent_lua_vtab = { + .call = func_persistent_lua_call, + .destroy = func_persistent_lua_destroy, +}; + static int lbox_module_reload(lua_State *L) { @@ -667,6 +868,40 @@ lbox_func_new(struct lua_State *L, struct func *func) lua_pushstring(L, "language"); lua_pushstring(L, func_language_strs[func->def->language]); lua_settable(L, top); + lua_pushstring(L, "returns"); + lua_pushstring(L, field_type_strs[func->def->returns]); + lua_settable(L, top); + lua_pushstring(L, "aggregate"); + lua_pushstring(L, func_aggregate_strs[func->def->aggregate]); + lua_settable(L, top); + lua_pushstring(L, "body"); + if (func->def->body != NULL) + lua_pushstring(L, func->def->body); + else + lua_pushnil(L); + lua_settable(L, top); + lua_pushstring(L, "comment"); + if (func->def->comment != NULL) + lua_pushstring(L, func->def->comment); + else + lua_pushnil(L); + lua_settable(L, top); + lua_pushstring(L, "exports"); + lua_newtable(L); + lua_pushboolean(L, func->def->exports.lua); + lua_setfield(L, -2, "lua"); + lua_pushboolean(L, func->def->exports.sql); + lua_setfield(L, -2, "sql"); + lua_settable(L, -3); + lua_pushstring(L, "is_deterministic"); + lua_pushboolean(L, func->def->is_deterministic); + lua_settable(L, top); + lua_pushstring(L, "is_sandboxed"); + if (func->def->body != NULL) + lua_pushboolean(L, func->def->is_sandboxed); + else + lua_pushnil(L); + lua_settable(L, top); /* Bless func object. */ lua_getfield(L, LUA_GLOBALSINDEX, "box"); @@ -712,6 +947,8 @@ lbox_func_new_or_delete(struct trigger *trigger, void *event) { struct lua_State *L = (struct lua_State *) trigger->data; struct func *func = (struct func *)event; + if (!func->def->exports.lua) + return; if (func_by_id(func->def->fid) != NULL) lbox_func_new(L, func); else diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 084addc2c..aadcd3fa9 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -2107,7 +2107,9 @@ box.schema.func.create = function(name, opts) opts = opts or {} check_param_table(opts, { setuid = 'boolean', if_not_exists = 'boolean', - language = 'string'}) + language = 'string', body = 'string', + is_deterministic = 'boolean', + is_sandboxed = 'boolean', comment = 'string' }) local _func = box.space[box.schema.FUNC_ID] local _vfunc = box.space[box.schema.VFUNC_ID] local func = _vfunc.index.name:get{name} @@ -2117,10 +2119,21 @@ box.schema.func.create = function(name, opts) end return end - opts = update_param_table(opts, { setuid = false, language = 'lua'}) + local datetime = os.date("%Y-%m-%d %H:%M:%S") + opts = update_param_table(opts, { setuid = false, language = 'lua', + body = '', routine_type = 'function', returns = 'any', + param_list = {}, aggregate = 'none', sql_data_access = 'none', + is_deterministic = false, is_sandboxed = false, + is_null_call = true, exports = {'LUA'}, opts = setmap{}, + comment = '', created = datetime, last_altered = datetime}) opts.language = string.upper(opts.language) opts.setuid = opts.setuid and 1 or 0 - _func:auto_increment{session.euid(), name, opts.setuid, opts.language} + _func:auto_increment{session.euid(), name, opts.setuid, opts.language, + opts.body, opts.routine_type, opts.param_list, + opts.returns, opts.aggregate, opts.sql_data_access, + opts.is_deterministic, opts.is_sandboxed, + opts.is_null_call, opts.exports, opts.opts, + opts.comment, opts.created, opts.last_altered} end box.schema.func.drop = function(name, opts) diff --git a/src/box/lua/upgrade.lua b/src/box/lua/upgrade.lua index 3385b8e17..a27240815 100644 --- a/src/box/lua/upgrade.lua +++ b/src/box/lua/upgrade.lua @@ -152,6 +152,7 @@ local function initial_1_7_5() local _cluster = box.space[box.schema.CLUSTER_ID] local _truncate = box.space[box.schema.TRUNCATE_ID] local MAP = setmap({}) + local datetime = os.date("%Y-%m-%d %H:%M:%S") -- -- _schema @@ -326,7 +327,9 @@ local function initial_1_7_5() -- create "box.schema.user.info" function log.info('create function "box.schema.user.info" with setuid') - _func:replace{1, ADMIN, 'box.schema.user.info', 1, 'LUA'} + _func:replace({1, ADMIN, 'box.schema.user.info', 1, 'LUA', '', 'function', + {}, 'any', 'none', 'none', false, false, true, {'LUA'}, + MAP, '', datetime, datetime}) -- grant 'public' role access to 'box.schema.user.info' function log.info('grant execute on function "box.schema.user.info" to public') @@ -820,10 +823,72 @@ local function create_vcollation_space() box.space[box.schema.VCOLLATION_ID]:format(format) end +local function upgrade_func_to_2_2_1() + log.info("Update _func format") + local _func = box.space[box.schema.FUNC_ID] + local _priv = box.space[box.schema.PRIV_ID] + local datetime = os.date("%Y-%m-%d %H:%M:%S") + for _, v in box.space._func:pairs() do + box.space._func:replace({v.id, v.owner, v.name, v.setuid, v[5] or 'LUA', + '', 'function', {}, 'any', 'none', 'none', + false, false, true, v[15] or {'LUA'}, + setmap({}), '', datetime, datetime}) + end + local sql_builtin_list = { + "TRIM", "TYPEOF", "PRINTF", "UNICODE", "CHAR", "HEX", "VERSION", + "QUOTE", "REPLACE", "SUBSTR", "GROUP_CONCAT", "JULIANDAY", "DATE", + "TIME", "DATETIME", "STRFTIME", "CURRENT_TIME", "CURRENT_TIMESTAMP", + "CURRENT_DATE", "LENGTH", "POSITION", "ROUND", "UPPER", "LOWER", + "IFNULL", "RANDOM", "CEIL", "CEILING", "CHARACTER_LENGTH", + "CHAR_LENGTH", "FLOOR", "MOD", "OCTET_LENGTH", "ROW_COUNT", "COUNT", + "LIKE", "ABS", "EXP", "LN", "POWER", "SQRT", "SUM", "TOTAL", "AVG", + "RANDOMBLOB", "NULLIF", "ZEROBLOB", "MIN", "MAX", "COALESCE", "EVERY", + "EXISTS", "EXTRACT", "SOME", "GREATER", "LESSER", + "_sql_stat_get", "_sql_stat_push", "_sql_stat_init", + } + for _, v in pairs(sql_builtin_list) do + local t = _func:auto_increment({ADMIN, v, 1, 'SQL_BUILTIN', '', + 'function', {}, 'any', 'none', 'none', + false, false, true, {}, setmap({}), '', + datetime, datetime}) + _priv:replace{ADMIN, PUBLIC, 'function', t.id, box.priv.X} + end + local t = _func:auto_increment({ADMIN, 'LUA', 1, 'LUA', + 'function(code) return assert(loadstring(code))() end', + 'function', {'string'}, 'any', 'none', 'none', + false, false, true, {'LUA', 'SQL'}, + setmap({}), '', datetime, datetime}) + _priv:replace{ADMIN, PUBLIC, 'function', t.id, box.priv.X} + local format = {} + format[1] = {name='id', type='unsigned'} + format[2] = {name='owner', type='unsigned'} + format[3] = {name='name', type='string'} + format[4] = {name='setuid', type='unsigned'} + format[5] = {name='language', type='string'} + format[6] = {name='body', type='string'} + format[7] = {name='routine_type', type='string'} + format[8] = {name='param_list', type='array'} + format[9] = {name='returns', type='string'} + format[10] = {name='aggregate', type='string'} + format[11] = {name='sql_data_access', type='string'} + format[12] = {name='is_deterministic', type='boolean'} + format[13] = {name='is_sandboxed', type='boolean'} + format[14] = {name='is_null_call', type='boolean'} + format[15] = {name='exports', type='array'} + format[16] = {name='opts', type='map'} + format[17] = {name='comment', type='string'} + format[18] = {name='created', type='string'} + format[19] = {name='last_altered', type='string'} + _func:format(format) + _func.index.name:alter({parts = {{'name', 'string', + collation = 'unicode_ci'}}}) +end + local function upgrade_to_2_2_1() upgrade_sequence_to_2_2_1() upgrade_ck_constraint_to_2_2_1() create_vcollation_space() + upgrade_func_to_2_2_1() end -------------------------------------------------------------------------------- diff --git a/src/box/schema_def.h b/src/box/schema_def.h index 88b5502b8..a97b6d531 100644 --- a/src/box/schema_def.h +++ b/src/box/schema_def.h @@ -167,6 +167,20 @@ enum { BOX_FUNC_FIELD_NAME = 2, BOX_FUNC_FIELD_SETUID = 3, BOX_FUNC_FIELD_LANGUAGE = 4, + BOX_FUNC_FIELD_BODY = 5, + BOX_FUNC_FIELD_ROUTINE_TYPE = 6, + BOX_FUNC_FIELD_PARAM_LIST = 7, + BOX_FUNC_FIELD_RETURNS = 8, + BOX_FUNC_FIELD_AGGREGATE = 9, + BOX_FUNC_FIELD_SQL_DATA_ACCESS = 10, + BOX_FUNC_FIELD_IS_DETERMINISTIC = 11, + BOX_FUNC_FIELD_IS_SANDBOXED = 12, + BOX_FUNC_FIELD_IS_NULL_CALL = 13, + BOX_FUNC_FIELD_EXPORTS = 14, + BOX_FUNC_FIELD_OPTS = 15, + BOX_FUNC_FIELD_COMMENT = 16, + BOX_FUNC_FIELD_CREATED = 17, + BOX_FUNC_FIELD_LAST_ALTERED = 18, }; /** _collation fields. */ diff --git a/src/box/sql.h b/src/box/sql.h index 9ccecf28c..a078bfdec 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -70,6 +70,7 @@ struct Select; struct Table; struct sql_trigger; struct space_def; +struct func_def; /** * Perform parsing of provided expression. This is done by @@ -404,6 +405,10 @@ void vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref, struct tuple *tuple); +/** Construct a SQL builtin function object. */ +struct func * +func_sql_builtin_new(struct func_def *def); + #if defined(__cplusplus) } /* extern "C" { */ #endif diff --git a/src/box/sql/func.c b/src/box/sql/func.c index 21ce78c24..d59aba9ee 100644 --- a/src/box/sql/func.c +++ b/src/box/sql/func.c @@ -38,6 +38,7 @@ #include "vdbeInt.h" #include "version.h" #include "coll/coll.h" +#include "box/func.h" #include "tarantoolInt.h" #include "box/session.h" #include @@ -1824,3 +1825,45 @@ sqlRegisterBuiltinFunctions(void) } #endif } + +struct func_sql_builtin { + /** Function object base class. */ + struct func base; +}; + +static struct func_vtab func_sql_builtin_vtab; + +struct func * +func_sql_builtin_new(struct func_def *def) +{ + assert(def->language == FUNC_LANGUAGE_SQL_BUILTIN); + if (def->body != NULL || def->is_sandboxed) { + diag_set(ClientError, ER_CREATE_FUNCTION, def->name, + "body and is_sandboxed options are not compatible " + "with SQL language"); + return NULL; + } + struct func_sql_builtin *func = + (struct func_sql_builtin *) malloc(sizeof(*func)); + if (func == NULL) { + diag_set(OutOfMemory, sizeof(*func), "malloc", "func"); + return NULL; + } + /** Don't export SQL builtins in Lua for now. */ + def->exports.lua = false; + func->base.vtab = &func_sql_builtin_vtab; + return &func->base; +} + +static void +func_sql_builtin_destroy(struct func *base) +{ + assert(base->vtab == &func_sql_builtin_vtab); + assert(base != NULL && base->def->language == FUNC_LANGUAGE_SQL_BUILTIN); + free(base); +} + +static struct func_vtab func_sql_builtin_vtab = { + .call = NULL, + .destroy = func_sql_builtin_destroy, +}; diff --git a/test-run b/test-run index 37d15bd78..d9b9c6382 160000 --- a/test-run +++ b/test-run @@ -1 +1 @@ -Subproject commit 37d15bd781ddfb41dfd75d9b761c180395b4b53f +Subproject commit d9b9c6382453dfb4d4663909ae4dc3a535826889 diff --git a/test/box-py/bootstrap.result b/test/box-py/bootstrap.result index b20dc41e5..2ef2200bf 100644 --- a/test/box-py/bootstrap.result +++ b/test/box-py/bootstrap.result @@ -53,7 +53,14 @@ box.space._space:select{} 'type': 'string'}, {'name': 'opts', 'type': 'map'}, {'name': 'parts', 'type': 'array'}]] - [296, 1, '_func', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', - 'type': 'unsigned'}]] + 'type': 'unsigned'}, {'name': 'language', 'type': 'string'}, {'name': 'body', + 'type': 'string'}, {'name': 'routine_type', 'type': 'string'}, {'name': 'param_list', + 'type': 'array'}, {'name': 'returns', 'type': 'string'}, {'name': 'aggregate', + 'type': 'string'}, {'name': 'sql_data_access', 'type': 'string'}, {'name': 'is_deterministic', + 'type': 'boolean'}, {'name': 'is_sandboxed', 'type': 'boolean'}, {'name': 'is_null_call', + 'type': 'boolean'}, {'name': 'exports', 'type': 'array'}, {'name': 'opts', + 'type': 'map'}, {'name': 'comment', 'type': 'string'}, {'name': 'created', + 'type': 'string'}, {'name': 'last_altered', 'type': 'string'}]] - [297, 1, '_vfunc', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', 'type': 'unsigned'}]] @@ -113,7 +120,7 @@ box.space._index:select{} - [289, 2, 'name', 'tree', {'unique': true}, [[0, 'unsigned'], [2, 'string']]] - [296, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned']]] - [296, 1, 'owner', 'tree', {'unique': false}, [[1, 'unsigned']]] - - [296, 2, 'name', 'tree', {'unique': true}, [[2, 'string']]] + - [296, 2, 'name', 'tree', {'unique': true}, [{'field': 2, 'collation': 2, 'type': 'string'}]] - [297, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned']]] - [297, 1, 'owner', 'tree', {'unique': false}, [[1, 'unsigned']]] - [297, 2, 'name', 'tree', {'unique': true}, [[2, 'string']]] @@ -150,9 +157,10 @@ box.space._user:select{} - [3, 1, 'replication', 'role', {}] - [31, 1, 'super', 'role', {}] ... -box.space._func:select{} +for _, v in box.space._func:pairs{} do r = {} table.insert(r, v:update({{"=", 18, ""}, {"=", 19, ""}})) return r end --- -- - [1, 1, 'box.schema.user.info', 1, 'LUA'] +- - [1, 1, 'box.schema.user.info', 1, 'LUA', '', 'function', [], 'any', 'none', 'none', + false, false, true, ['LUA'], {}, '', '', ''] ... box.space._priv:select{} --- @@ -160,6 +168,66 @@ box.space._priv:select{} - [1, 0, 'universe', 0, 24] - [1, 1, 'universe', 0, 4294967295] - [1, 2, 'function', 1, 4] + - [1, 2, 'function', 2, 4] + - [1, 2, 'function', 3, 4] + - [1, 2, 'function', 4, 4] + - [1, 2, 'function', 5, 4] + - [1, 2, 'function', 6, 4] + - [1, 2, 'function', 7, 4] + - [1, 2, 'function', 8, 4] + - [1, 2, 'function', 9, 4] + - [1, 2, 'function', 10, 4] + - [1, 2, 'function', 11, 4] + - [1, 2, 'function', 12, 4] + - [1, 2, 'function', 13, 4] + - [1, 2, 'function', 14, 4] + - [1, 2, 'function', 15, 4] + - [1, 2, 'function', 16, 4] + - [1, 2, 'function', 17, 4] + - [1, 2, 'function', 18, 4] + - [1, 2, 'function', 19, 4] + - [1, 2, 'function', 20, 4] + - [1, 2, 'function', 21, 4] + - [1, 2, 'function', 22, 4] + - [1, 2, 'function', 23, 4] + - [1, 2, 'function', 24, 4] + - [1, 2, 'function', 25, 4] + - [1, 2, 'function', 26, 4] + - [1, 2, 'function', 27, 4] + - [1, 2, 'function', 28, 4] + - [1, 2, 'function', 29, 4] + - [1, 2, 'function', 30, 4] + - [1, 2, 'function', 31, 4] + - [1, 2, 'function', 32, 4] + - [1, 2, 'function', 33, 4] + - [1, 2, 'function', 34, 4] + - [1, 2, 'function', 35, 4] + - [1, 2, 'function', 36, 4] + - [1, 2, 'function', 37, 4] + - [1, 2, 'function', 38, 4] + - [1, 2, 'function', 39, 4] + - [1, 2, 'function', 40, 4] + - [1, 2, 'function', 41, 4] + - [1, 2, 'function', 42, 4] + - [1, 2, 'function', 43, 4] + - [1, 2, 'function', 44, 4] + - [1, 2, 'function', 45, 4] + - [1, 2, 'function', 46, 4] + - [1, 2, 'function', 47, 4] + - [1, 2, 'function', 48, 4] + - [1, 2, 'function', 49, 4] + - [1, 2, 'function', 50, 4] + - [1, 2, 'function', 51, 4] + - [1, 2, 'function', 52, 4] + - [1, 2, 'function', 53, 4] + - [1, 2, 'function', 54, 4] + - [1, 2, 'function', 55, 4] + - [1, 2, 'function', 56, 4] + - [1, 2, 'function', 57, 4] + - [1, 2, 'function', 58, 4] + - [1, 2, 'function', 59, 4] + - [1, 2, 'function', 60, 4] + - [1, 2, 'function', 61, 4] - [1, 2, 'space', 276, 2] - [1, 2, 'space', 277, 1] - [1, 2, 'space', 281, 1] diff --git a/test/box-py/bootstrap.test.py b/test/box-py/bootstrap.test.py index 4f2f55a7c..63c13e8a4 100644 --- a/test/box-py/bootstrap.test.py +++ b/test/box-py/bootstrap.test.py @@ -4,7 +4,7 @@ server.admin('box.space._cluster:select{}') server.admin('box.space._space:select{}') server.admin('box.space._index:select{}') server.admin('box.space._user:select{}') -server.admin('box.space._func:select{}') +server.admin('for _, v in box.space._func:pairs{} do r = {} table.insert(r, v:update({{"=", 18, ""}, {"=", 19, ""}})) return r end') server.admin('box.space._priv:select{}') # Cleanup diff --git a/test/box/access.result b/test/box/access.result index ca2531f0e..ecb85a563 100644 --- a/test/box/access.result +++ b/test/box/access.result @@ -691,7 +691,7 @@ box.schema.func.exists(1) --- - true ... -box.schema.func.exists(2) +box.schema.func.exists(62) --- - false ... diff --git a/test/box/access.test.lua b/test/box/access.test.lua index c1ba00211..1341cf67b 100644 --- a/test/box/access.test.lua +++ b/test/box/access.test.lua @@ -276,7 +276,7 @@ box.schema.user.exists{} box.schema.func.exists('nosuchfunc') box.schema.func.exists('guest') box.schema.func.exists(1) -box.schema.func.exists(2) +box.schema.func.exists(62) box.schema.func.exists('box.schema.user.info') box.schema.func.exists() box.schema.func.exists(nil) diff --git a/test/box/access_bin.result b/test/box/access_bin.result index df8ef8dee..395afbcf3 100644 --- a/test/box/access_bin.result +++ b/test/box/access_bin.result @@ -299,7 +299,7 @@ box.schema.user.grant('guest', 'execute', 'universe') function f1() return box.space._func:get(1)[4] end --- ... -function f2() return box.space._func:get(2)[4] end +function f2() return box.space._func:get(62)[4] end --- ... box.schema.func.create('f1') diff --git a/test/box/access_bin.test.lua b/test/box/access_bin.test.lua index e77d8c0a8..394cc49be 100644 --- a/test/box/access_bin.test.lua +++ b/test/box/access_bin.test.lua @@ -113,7 +113,7 @@ test:drop() -- notice that guest can execute stuff, but can't read space _func box.schema.user.grant('guest', 'execute', 'universe') function f1() return box.space._func:get(1)[4] end -function f2() return box.space._func:get(2)[4] end +function f2() return box.space._func:get(62)[4] end box.schema.func.create('f1') box.schema.func.create('f2',{setuid=true}) c = net.connect(box.cfg.listen) diff --git a/test/box/access_misc.result b/test/box/access_misc.result index 53d366106..be793708b 100644 --- a/test/box/access_misc.result +++ b/test/box/access_misc.result @@ -793,7 +793,14 @@ box.space._space:select() 'type': 'string'}, {'name': 'opts', 'type': 'map'}, {'name': 'parts', 'type': 'array'}]] - [296, 1, '_func', 'memtx', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', - 'type': 'unsigned'}]] + 'type': 'unsigned'}, {'name': 'language', 'type': 'string'}, {'name': 'body', + 'type': 'string'}, {'name': 'routine_type', 'type': 'string'}, {'name': 'param_list', + 'type': 'array'}, {'name': 'returns', 'type': 'string'}, {'name': 'aggregate', + 'type': 'string'}, {'name': 'sql_data_access', 'type': 'string'}, {'name': 'is_deterministic', + 'type': 'boolean'}, {'name': 'is_sandboxed', 'type': 'boolean'}, {'name': 'is_null_call', + 'type': 'boolean'}, {'name': 'exports', 'type': 'array'}, {'name': 'opts', + 'type': 'map'}, {'name': 'comment', 'type': 'string'}, {'name': 'created', + 'type': 'string'}, {'name': 'last_altered', 'type': 'string'}]] - [297, 1, '_vfunc', 'sysview', 0, {}, [{'name': 'id', 'type': 'unsigned'}, {'name': 'owner', 'type': 'unsigned'}, {'name': 'name', 'type': 'string'}, {'name': 'setuid', 'type': 'unsigned'}]] @@ -829,7 +836,129 @@ box.space._space:select() ... box.space._func:select() --- -- - [1, 1, 'box.schema.user.info', 1, 'LUA'] +- - [1, 1, 'box.schema.user.info', 1, 'LUA', '', 'function', [], 'any', 'none', 'none', + false, false, true, ['LUA'], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [2, 1, 'TRIM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [3, 1, 'TYPEOF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [4, 1, 'PRINTF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [5, 1, 'UNICODE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [6, 1, 'CHAR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [7, 1, 'HEX', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [8, 1, 'VERSION', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [9, 1, 'QUOTE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [10, 1, 'REPLACE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [11, 1, 'SUBSTR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [12, 1, 'GROUP_CONCAT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [13, 1, 'JULIANDAY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [14, 1, 'DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [15, 1, 'TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [16, 1, 'DATETIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [17, 1, 'STRFTIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [18, 1, 'CURRENT_TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [19, 1, 'CURRENT_TIMESTAMP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', + 'none', false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [20, 1, 'CURRENT_DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [21, 1, 'LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [22, 1, 'POSITION', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [23, 1, 'ROUND', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [24, 1, 'UPPER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [25, 1, 'LOWER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [26, 1, 'IFNULL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [27, 1, 'RANDOM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [28, 1, 'CEIL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [29, 1, 'CEILING', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [30, 1, 'CHARACTER_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', + 'none', false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [31, 1, 'CHAR_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [32, 1, 'FLOOR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [33, 1, 'MOD', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [34, 1, 'OCTET_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [35, 1, 'ROW_COUNT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [36, 1, 'COUNT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [37, 1, 'LIKE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [38, 1, 'ABS', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [39, 1, 'EXP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [40, 1, 'LN', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [41, 1, 'POWER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [42, 1, 'SQRT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [43, 1, 'SUM', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [44, 1, 'TOTAL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [45, 1, 'AVG', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [46, 1, 'RANDOMBLOB', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [47, 1, 'NULLIF', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [48, 1, 'ZEROBLOB', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [49, 1, 'MIN', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [50, 1, 'MAX', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [51, 1, 'COALESCE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [52, 1, 'EVERY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [53, 1, 'EXISTS', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [54, 1, 'EXTRACT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [55, 1, 'SOME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, + false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [56, 1, 'GREATER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [57, 1, 'LESSER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', + false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [58, 1, '_sql_stat_get', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', + 'none', false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [59, 1, '_sql_stat_push', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', + 'none', false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [60, 1, '_sql_stat_init', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', + 'none', false, false, true, [], {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] + - [61, 1, 'LUA', 1, 'LUA', 'function(code) return assert(loadstring(code))() end', + 'function', ['string'], 'any', 'none', 'none', false, false, true, ['LUA', 'SQL'], + {}, '', '2019-07-10 13:38:07', '2019-07-10 13:38:07'] ... session = nil --- diff --git a/test/box/access_sysview.result b/test/box/access_sysview.result index 69eb6d191..752f0102a 100644 --- a/test/box/access_sysview.result +++ b/test/box/access_sysview.result @@ -258,11 +258,11 @@ box.session.su('guest') ... #box.space._vpriv:select{} --- -- 16 +- 76 ... #box.space._vfunc:select{} --- -- 1 +- 61 ... #box.space._vcollation:select{} --- @@ -290,11 +290,11 @@ box.session.su('guest') ... #box.space._vpriv:select{} --- -- 16 +- 76 ... #box.space._vfunc:select{} --- -- 1 +- 61 ... #box.space._vsequence:select{} --- diff --git a/test/box/alter.result b/test/box/alter.result index eb59f05fb..a6db011ff 100644 --- a/test/box/alter.result +++ b/test/box/alter.result @@ -190,7 +190,7 @@ _index:select{} - [289, 2, 'name', 'tree', {'unique': true}, [[0, 'unsigned'], [2, 'string']]] - [296, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned']]] - [296, 1, 'owner', 'tree', {'unique': false}, [[1, 'unsigned']]] - - [296, 2, 'name', 'tree', {'unique': true}, [[2, 'string']]] + - [296, 2, 'name', 'tree', {'unique': true}, [{'field': 2, 'collation': 2, 'type': 'string'}]] - [297, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned']]] - [297, 1, 'owner', 'tree', {'unique': false}, [[1, 'unsigned']]] - [297, 2, 'name', 'tree', {'unique': true}, [[2, 'string']]] diff --git a/test/box/function1.result b/test/box/function1.result index ec1ab5e6b..c434b0067 100644 --- a/test/box/function1.result +++ b/test/box/function1.result @@ -16,7 +16,40 @@ c = net.connect(os.getenv("LISTEN")) box.schema.func.create('function1', {language = "C"}) --- ... -box.space._func:replace{2, 1, 'function1', 0, 'LUA'} +id = box.func["function1"].id +--- +... +function setmap(tab) return setmetatable(tab, { __serialize = 'map' }) end +--- +... +datetime = os.date("%Y-%m-%d %H:%M:%S") +--- +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'procedure', {}, 'any', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': unsupported routine_type value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'reads', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': unsupported sql_data_access value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, false, {"LUA"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': unsupported is_null_call value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'data', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': invalid returns value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, true, {"LUA", "C"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': invalid exports value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'aggregate', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +--- +- error: 'Failed to create function ''function1'': invalid aggregate value' +... +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} --- - error: function does not support alter ... @@ -59,10 +92,16 @@ c:call('function1.args', { 15 }) ... box.func["function1.args"] --- -- language: C +- aggregate: none + returns: any + exports: + lua: true + sql: false + id: 62 setuid: false + is_deterministic: false name: function1.args - id: 2 + language: C ... box.func["function1.args"]:call() --- @@ -330,7 +369,7 @@ c:close() function divide(a, b) return a / b end --- ... -box.schema.func.create("divide") +box.schema.func.create("divide", {comment = 'Divide two values'}) --- ... func = box.func.divide @@ -372,10 +411,17 @@ func:drop() ... func --- -- language: LUA +- aggregate: none + returns: any + exports: + lua: true + sql: false + id: 62 setuid: false + is_deterministic: false + comment: Divide two values name: divide - id: 2 + language: LUA ... func.drop() --- @@ -436,10 +482,16 @@ box.func["function1.divide"] ... func --- -- language: C +- aggregate: none + returns: any + exports: + lua: true + sql: false + id: 62 setuid: false + is_deterministic: false name: function1.divide - id: 2 + language: C ... func:drop() --- @@ -526,6 +578,177 @@ box.schema.func.drop('secret_leak') box.schema.func.drop('secret') --- ... +-- +-- gh-4182: Introduce persistent Lua functions. +-- +test_run:cmd("setopt delimiter ';'") +--- +- true +... +body = [[function(tuple) + if type(tuple.address) ~= 'string' then + return nil, 'Invalid field type' + end + local t = tuple.address:upper():split() + for k,v in pairs(t) do t[k] = v end + return t + end +]] +test_run:cmd("setopt delimiter ''"); +--- +... +box.schema.func.create('addrsplit', {body = body, language = "C"}) +--- +- error: 'Failed to create function ''addrsplit'': body and is_sandboxed options are + not compatible with C language' +... +box.schema.func.create('addrsplit', {is_sandboxed = true, language = "C"}) +--- +- error: 'Failed to create function ''addrsplit'': body and is_sandboxed options are + not compatible with C language' +... +box.schema.func.create('addrsplit', {is_sandboxed = true}) +--- +- error: 'Failed to create function ''addrsplit'': is_sandboxed option may be set + only for persistent Lua function (when body option is set)' +... +box.schema.func.create('invalid', {body = "function(tuple) ret tuple"}) +--- +- error: 'Failed to dynamically load function ''invalid'': [string "return function(tuple) + ret tuple"]:1: ''='' expected near ''tuple''' +... +box.schema.func.create('addrsplit', {body = body, is_deterministic = true}) +--- +... +box.schema.user.grant('guest', 'execute', 'function', 'addrsplit') +--- +... +conn = net.connect(box.cfg.listen) +--- +... +conn:call('addrsplit', {{address = "Moscow Dolgoprudny"}}) +--- +- ['MOSCOW', 'DOLGOPRUDNY'] +... +box.func.addrsplit:call({{address = "Moscow Dolgoprudny"}}) +--- +- - MOSCOW + - DOLGOPRUDNY +... +conn:close() +--- +... +box.snapshot() +--- +- ok +... +test_run:cmd("restart server default") +test_run = require('test_run').new() +--- +... +test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'") +--- +- true +... +net = require('net.box') +--- +... +conn = net.connect(box.cfg.listen) +--- +... +conn:call('addrsplit', {{address = "Moscow Dolgoprudny"}}) +--- +- ['MOSCOW', 'DOLGOPRUDNY'] +... +box.func.addrsplit:call({{address = "Moscow Dolgoprudny"}}) +--- +- - MOSCOW + - DOLGOPRUDNY +... +conn:close() +--- +... +box.schema.user.revoke('guest', 'execute', 'function', 'addrsplit') +--- +... +box.func.addrsplit:drop() +--- +... +-- Test sandboxed functions. +test_run:cmd("setopt delimiter ';'") +--- +- true +... +body = [[function(number) + math.abs = math.log + return math.abs(number) + end]] +test_run:cmd("setopt delimiter ''"); +--- +... +box.schema.func.create('monkey', {body = body, is_sandboxed = true}) +--- +... +box.func.monkey:call({1}) +--- +- 0 +... +math.abs(1) +--- +- 1 +... +box.func.monkey:drop() +--- +... +sum = 0 +--- +... +function inc_g(val) sum = sum + val end +--- +... +box.schema.func.create('call_inc_g', {body = "function(val) inc_g(val) end"}) +--- +... +box.func.call_inc_g:call({1}) +--- +... +assert(sum == 1) +--- +- true +... +box.schema.func.create('call_inc_g_safe', {body = "function(val) inc_g(val) end", is_sandboxed = true}) +--- +... +box.func.call_inc_g_safe:call({1}) +--- +- error: '[string "return function(val) inc_g(val) end"]:1: attempt to call global + ''inc_g'' (a nil value)' +... +assert(sum == 1) +--- +- true +... +box.func.call_inc_g:drop() +--- +... +box.func.call_inc_g_safe:drop() +--- +... +-- Test persistent function assemble corner cases +box.schema.func.create('compiletime_tablef', {body = "{}"}) +--- +- error: 'Failed to dynamically load function ''compiletime_tablef'': given body doesn''t + define a function' +... +box.schema.func.create('compiletime_call_inc_g', {body = "inc_g()"}) +--- +- error: 'Failed to dynamically load function ''compiletime_call_inc_g'': [string + "return inc_g()"]:1: attempt to call global ''inc_g'' (a nil value)' +... +assert(sum == 1) +--- +- true +... test_run:cmd("clear filter") --- - true @@ -564,3 +787,37 @@ box.func.test ~= nil box.func.test:drop() --- ... +-- Check SQL builtins +test_run:cmd("setopt delimiter ';'") +--- +- true +... +sql_builtin_list = { + "TRIM", "TYPEOF", "PRINTF", "UNICODE", "CHAR", "HEX", "VERSION", + "QUOTE", "REPLACE", "SUBSTR", "GROUP_CONCAT", "JULIANDAY", "DATE", + "TIME", "DATETIME", "STRFTIME", "CURRENT_TIME", "CURRENT_TIMESTAMP", + "CURRENT_DATE", "LENGTH", "POSITION", "ROUND", "UPPER", "LOWER", + "IFNULL", "RANDOM", "CEIL", "CEILING", "CHARACTER_LENGTH", + "CHAR_LENGTH", "FLOOR", "MOD", "OCTET_LENGTH", "ROW_COUNT", "COUNT", + "LIKE", "ABS", "EXP", "LN", "POWER", "SQRT", "SUM", "TOTAL", "AVG", + "RANDOMBLOB", "NULLIF", "ZEROBLOB", "MIN", "MAX", "COALESCE", "EVERY", + "EXISTS", "EXTRACT", "SOME", "GREATER", "LESSER", "_sql_stat_get", + "_sql_stat_push", "_sql_stat_init", +} +test_run:cmd("setopt delimiter ''"); +--- +... +ok = true +--- +... +for _, v in pairs(sql_builtin_list) do ok = ok and (box.space._func.index.name:get(v) ~= nil) end +--- +... +ok == true +--- +- true +... +box.func.LUA:call({"return 1 + 1"}) +--- +- 2 +... diff --git a/test/box/function1.test.lua b/test/box/function1.test.lua index a891e1921..dbbdcf8be 100644 --- a/test/box/function1.test.lua +++ b/test/box/function1.test.lua @@ -7,7 +7,16 @@ net = require('net.box') c = net.connect(os.getenv("LISTEN")) box.schema.func.create('function1', {language = "C"}) -box.space._func:replace{2, 1, 'function1', 0, 'LUA'} +id = box.func["function1"].id +function setmap(tab) return setmetatable(tab, { __serialize = 'map' }) end +datetime = os.date("%Y-%m-%d %H:%M:%S") +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'procedure', {}, 'any', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'reads', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, false, {"LUA"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'data', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, true, {"LUA", "C"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'aggregate', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} +box.space._func:replace{id, 1, 'function1', 0, 'LUA', '', 'function', {}, 'any', 'none', 'none', false, false, true, {"LUA"}, setmap({}), '', datetime, datetime} box.schema.user.grant('guest', 'execute', 'function', 'function1') _ = box.schema.space.create('test') _ = box.space.test:create_index('primary') @@ -121,7 +130,7 @@ c:close() -- Test registered functions interface. function divide(a, b) return a / b end -box.schema.func.create("divide") +box.schema.func.create("divide", {comment = 'Divide two values'}) func = box.func.divide func.call({4, 2}) func:call(4, 2) @@ -184,6 +193,70 @@ box.schema.user.revoke('guest', 'execute', 'function', 'secret_leak') box.schema.func.drop('secret_leak') box.schema.func.drop('secret') +-- +-- gh-4182: Introduce persistent Lua functions. +-- +test_run:cmd("setopt delimiter ';'") +body = [[function(tuple) + if type(tuple.address) ~= 'string' then + return nil, 'Invalid field type' + end + local t = tuple.address:upper():split() + for k,v in pairs(t) do t[k] = v end + return t + end +]] +test_run:cmd("setopt delimiter ''"); +box.schema.func.create('addrsplit', {body = body, language = "C"}) +box.schema.func.create('addrsplit', {is_sandboxed = true, language = "C"}) +box.schema.func.create('addrsplit', {is_sandboxed = true}) +box.schema.func.create('invalid', {body = "function(tuple) ret tuple"}) +box.schema.func.create('addrsplit', {body = body, is_deterministic = true}) +box.schema.user.grant('guest', 'execute', 'function', 'addrsplit') +conn = net.connect(box.cfg.listen) +conn:call('addrsplit', {{address = "Moscow Dolgoprudny"}}) +box.func.addrsplit:call({{address = "Moscow Dolgoprudny"}}) +conn:close() +box.snapshot() +test_run:cmd("restart server default") +test_run = require('test_run').new() +test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'") +net = require('net.box') +conn = net.connect(box.cfg.listen) +conn:call('addrsplit', {{address = "Moscow Dolgoprudny"}}) +box.func.addrsplit:call({{address = "Moscow Dolgoprudny"}}) +conn:close() +box.schema.user.revoke('guest', 'execute', 'function', 'addrsplit') +box.func.addrsplit:drop() + +-- Test sandboxed functions. +test_run:cmd("setopt delimiter ';'") +body = [[function(number) + math.abs = math.log + return math.abs(number) + end]] +test_run:cmd("setopt delimiter ''"); +box.schema.func.create('monkey', {body = body, is_sandboxed = true}) +box.func.monkey:call({1}) +math.abs(1) +box.func.monkey:drop() + +sum = 0 +function inc_g(val) sum = sum + val end +box.schema.func.create('call_inc_g', {body = "function(val) inc_g(val) end"}) +box.func.call_inc_g:call({1}) +assert(sum == 1) +box.schema.func.create('call_inc_g_safe', {body = "function(val) inc_g(val) end", is_sandboxed = true}) +box.func.call_inc_g_safe:call({1}) +assert(sum == 1) +box.func.call_inc_g:drop() +box.func.call_inc_g_safe:drop() + +-- Test persistent function assemble corner cases +box.schema.func.create('compiletime_tablef', {body = "{}"}) +box.schema.func.create('compiletime_call_inc_g', {body = "inc_g()"}) +assert(sum == 1) + test_run:cmd("clear filter") -- @@ -198,3 +271,24 @@ box.begin() box.space._func:delete{f.id} f = box.func.test box.rollback() f == nil box.func.test ~= nil box.func.test:drop() + +-- Check SQL builtins +test_run:cmd("setopt delimiter ';'") +sql_builtin_list = { + "TRIM", "TYPEOF", "PRINTF", "UNICODE", "CHAR", "HEX", "VERSION", + "QUOTE", "REPLACE", "SUBSTR", "GROUP_CONCAT", "JULIANDAY", "DATE", + "TIME", "DATETIME", "STRFTIME", "CURRENT_TIME", "CURRENT_TIMESTAMP", + "CURRENT_DATE", "LENGTH", "POSITION", "ROUND", "UPPER", "LOWER", + "IFNULL", "RANDOM", "CEIL", "CEILING", "CHARACTER_LENGTH", + "CHAR_LENGTH", "FLOOR", "MOD", "OCTET_LENGTH", "ROW_COUNT", "COUNT", + "LIKE", "ABS", "EXP", "LN", "POWER", "SQRT", "SUM", "TOTAL", "AVG", + "RANDOMBLOB", "NULLIF", "ZEROBLOB", "MIN", "MAX", "COALESCE", "EVERY", + "EXISTS", "EXTRACT", "SOME", "GREATER", "LESSER", "_sql_stat_get", + "_sql_stat_push", "_sql_stat_init", +} +test_run:cmd("setopt delimiter ''"); +ok = true +for _, v in pairs(sql_builtin_list) do ok = ok and (box.space._func.index.name:get(v) ~= nil) end +ok == true + +box.func.LUA:call({"return 1 + 1"}) diff --git a/test/wal_off/func_max.result b/test/wal_off/func_max.result index ab4217845..07efc6ab1 100644 --- a/test/wal_off/func_max.result +++ b/test/wal_off/func_max.result @@ -42,11 +42,11 @@ test_run:cmd("setopt delimiter ''"); ... func_limit() --- -- error: 'Failed to create function ''func32000'': function id is too big' +- error: 'Failed to create function ''func31940'': function id is too big' ... drop_limit_func() --- -- error: Function 'func32000' does not exist +- error: Function 'func31940' does not exist ... box.schema.user.create('testuser') --- @@ -62,11 +62,11 @@ session.su('testuser') ... func_limit() --- -- error: 'Failed to create function ''func32000'': function id is too big' +- error: 'Failed to create function ''func31940'': function id is too big' ... drop_limit_func() --- -- error: Function 'func32000' does not exist +- error: Function 'func31940' does not exist ... session.su('admin') --- -- 2.21.0