From alexander.turenko at tarantool.org Mon Sep 7 01:42:15 2020 From: alexander.turenko at tarantool.org (Alexander Turenko) Date: Mon, 7 Sep 2020 01:42:15 +0300 Subject: [Tarantool-discussions] Consider exporting symbols from libraries: small, msgpuck Message-ID: <20200906224215.plpdhgav64zpeyop@tkn_work_nb> I was accumulating thoughts around ABI compatibility for myself during some time and want to share them. The main question that I bring into attention here: whether it worth to expose msgpuck, small and other libraries APIs into tarantool's module API. Problem ------- A tarantool module (say, memcached) uses a library, which is also used in tarantool (say, small). Let's assume that tarantool and the module use different versions of the library. Say, a layout of some structure was changed: a non-last field was removed or a field was added to the middle. | tarantool executable | -------------------- | | /* foo.h */ | | struct foo { | uint64_t bar; | uint64_t baz; | struct foo *next; | } | | void | foo_create(struct foo *foo, struct foo *next); | | /* foo.c */ | | void | foo_create(struct foo *foo, struct foo *next) | { | foo->bar = 0; | foo->baz = 0; | foo->next = next; | } | module dynamic library | ---------------------- | | /* foo.h */ | | struct foo { | /* !! no bar !! */ | uint64_t baz; | struct foo *next; | } | | void | foo_create(struct foo *foo, struct foo *next); | | /* foo.c */ | | void | foo_create(struct foo *foo, struct foo *next) | { | /* !! no foo->bar = 0 !! */ | foo->baz = 0; | foo->next = next; | } Let's look how a breakage may occur. After unhiding internal symbols in tarantool executable (see [1]), a call of foo_create() from the module will actually call the function from tarantool executable, which will set foo->next to NULL (`foo->baz = 0;`) and will access a memory out of the structure bounds (`foo->next = next;`). Note for myself: I would take extra care to inline functions in public headers, however I have no example of a possible breakage in the mind. Noted here to think around it later. Note: Some msgpuck symbols were exposed even before [1]. I guess it was to use them using LuaJIT FFI. [1]: https://github.com/tarantool/tarantool/issues/2971 Background ---------- - Default on Linux: use a symbol from executable file. - MacOS behaviour is like RTLD_DEEPBIND is used (from Vlad Sh.) - See dlopen(3): RTLD_DEEPBIND (place a symbol from a library before global one in the lookup order). Known cases ----------- - LTO and ASAN complains about this. https://github.com/tarantool/tarantool/issues/5001 LTO fix: https://github.com/tarantool/tarantool/commit/36927e540549fbdfd156ac3518616dbf4642711f ASAN fix: https://github.com/tarantool/tarantool/commit/e8c72d4fe66ea94e357af2e527cb5cc4727f09da - memcached fails on some tarantool versions. This case is almost same as the abstract one described above: the symbol unhiding patch leads to the breakage. https://github.com/tarantool/memcached/issues/59 - box_txn_alloc() changes its behaviour. Not strictly related to the problem described above, but it is another tarantool public C API breakage. So it is related to the question below: how to test the API to prevent this kind of breakage. https://github.com/tarantool/memcached/issues/53 My questions ------------ - Should not we expose small, msgpuck libraries symbols from tarantool executable and ship corresponsing header files? - How to ensure that exposed API / ABI is stable: one may use old headers to compile, but symbols from newer executable at runtime. - Of course, we should test tarantool changes against external modules. But it is not general ABI compatibility verification: some cases may not be covered by a module test, there may be closed-source modules. - How existing ABI compatibility checkers are? Say, [2]. - Looks promising: at least the description suggests that the case above would be catched ('renamed fields'). - We should define rules how to change public API structures and functions. Existing of such checklists makes life easier. - Many points should be here, but I'll highlight one that comes into my mind (just to don't forgot about it): we possibly will need to use padding at end of public structures to have ability to extend it. Or explicitly state that a structure is not known at build time, so it may not be used in arrays or allocated on a stack. If there is no need to provide direct access to first N fields (say, due to performance matters), we can just make it opaque. - Can we just ship small / msgpuck header files and expose its symbols from tarantool? Or we need a separate public API layer? - The former would obligate us to keep those libraries ABI compatible. - The latter don't: this way the library should only be used as static one. - How about performance? Whether building a module with a library (like small or msgpuck) directly (not using of tarantool's one) may give better performance because of using inline functions and macros? - Can we make bundling a library into a module safe using symbol renaming (say, some macro magic)? - For particular case: using of fiber()->gc: can we expose some reduced API from tarantool and be happy? [2]: https://lvc.github.io/abi-compliance-checker/ Why I started the discussion? ----------------------------- I want to implement Lua API for key_def as an external module, which should be based on a public C API (which in turn should be extended for this matter). The built-in key_def Lua module uses fiber->gc region; region functions are part of the small library. Considering version mismatch problems we already met in the past I would prefer to expose small library symbols from tarantool executable and use them in the module. I found that just exposing relevant symbols does not shield us from ABI breakage problems, so the questions above should be resolved (sooner or later). Mea culpa --------- Well, I should google for 'how to write abi compatible libraries', read some articles and I guess most of my questions will gone. I wrote the letter above just to formalize things for myself, but than found that it may be used as the base for further discussions. Forward ABI compatibility guidelines ------------------------------------ This sections is added later, so it may contradict with something written above. Excerpts of useful info from different sources. - https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html - symbol versioning - when exactly it is needed? what a problem it solves? - policy: don't change anything, only add - separation of interface and implementation - how about macroses, which wraps sizeof() / alignof() calls? - testing - `make check-abi` - It just check all symbols using sizeof(), alignof() and so. I guess also check list of symbols and each structure field. - Files (from gcc tarball): - libstdc++-v3/testsuite/Makefile.in - libstdc++-v3/testsuite/util/testsuite_abi_check.cc - libstdc++-v3/testsuite/util/testsuite_abi.{h,cc} - libstdc++-v3/libsupc++/cxxabi.h - <...> - `make check-c++` just runs the C standard library test suite. The idea of ABI compatibility check is to run a testsuite from one version against another one. - http://abicheck.sourceforge.net/ - It is linked from the page. Why? Is it used in GCC? Is it related to `make check-abi`? Is it just recommendation? - It just verifies a list of symbols used by an executable file against private / unstable lists. Not ready-to-use compare ABI vs ABI tool. Traversed over several documents and, in brief, the best description is KDE project guidelines (it is often linked from other good sources): https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B https://community.kde.org/Policies/Binary_Compatibility_Examples Those sources are (looked briefly): https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html https://www.akkadia.org/drepper/dsohowto.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1976.html http://syrcose.ispras.ru/2009/files/02_paper.pdf https://accu.org/content/conf2015/JonathanWakely-What%20Is%20An%20ABI%20And%20Why%20Is%20It%20So%20Complicated.pdf What I need to think: is it okay to use padding for a structure instead of d-pointer? How much padding is okay for performance matters? Is it ever okay to have non-opaque structures (we have no ones now in module.h)? Future updates -------------- Now I investigated the area a bit and want to share certain recommendations: - How to expose a non-opacue structure to keep it ABI compatible over different tarantool versions (padding and so on). - How to write a Lua/C module that able to use a feature from a new tarantool version, but work with reduced functionality on an old tarantool version (using dlsym()). I'll do when time will permit. WBR, Alexander Turenko. From imeevma at tarantool.org Sun Sep 27 18:18:00 2020 From: imeevma at tarantool.org (Mergen Imeev) Date: Sun, 27 Sep 2020 18:18:00 +0300 Subject: [Tarantool-discussions] SQL built-in functions position Message-ID: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> Hi all. I have a question that I would like to discuss. The question is about SQL built-in functions. At the moment these functions are partially described in _func and partially in src/box/sql/func.c. I received two completely different suggestions from my reviewers on what to do with these functions: 1) Move definitions completely to _func. Remove definitions from func.c. 2) Move definitions completely to func.c. Remove definitions from _func. In the first case, users will be able to see the function definitions. Also, in the future, we may allow these functions to be called from Lua (although not sure if this is necessary). The main idea is 'all functions have the same interface'. In the second case, the implementation is simpler, and we can more easily implement some features, such as "virtual" functions. For users, the definition can only be seen in the documentation. The main idea is 'SQL built-in functions are part of SQL'. Which of these approaches do you think is more beneficial to us? From pgulutzan at ocelot.ca Sun Sep 27 23:56:22 2020 From: pgulutzan at ocelot.ca (Peter Gulutzan) Date: Sun, 27 Sep 2020 14:56:22 -0600 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> Message-ID: <9a788a90-f558-fc6c-1d28-2813e8b721f8@ocelot.ca> Hi, On 2020-09-27 9:18 a.m., Mergen Imeev wrote: > Hi all. I have a question that I would like to discuss. > > The question is about SQL built-in functions. At the moment these functions are > partially described in _func and partially in src/box/sql/func.c. I received two > completely different suggestions from my reviewers on what to do with these > functions: > 1) Move definitions completely to _func. Remove definitions from func.c. > 2) Move definitions completely to func.c. Remove definitions from _func. > > In the first case, users will be able to see the function definitions. Also, in > the future, we may allow these functions to be called from Lua (although not > sure if this is necessary). The main idea is 'all functions have the same > interface'. > > In the second case, the implementation is simpler, and we can more easily > implement some features, such as "virtual" functions. For users, the definition > can only be seen in the documentation. The main idea is 'SQL built-in functions > are part of SQL'. > > Which of these approaches do you think is more beneficial to us? > I hope you will say _func. The current built-in functions are harmless, except perhaps for RANDOMBLOB with a huge value. However, in future maybe there will be built-in functions that should require privileges. In that case, I believe, they will have to be in _func (and someday in _vfunc) so that grant() will work for them. I have tried to redirect the UPPER() function in order to violate security, thus: " tarantool> function UPPER(x) return x end --- ... tarantool> box.schema.func.create('UPPER') --- - error: Function 'UPPER' already exists ... tarantool> box.schema.func.drop('UPPER') --- - error: 'Can''t drop function 1: function is SQL built-in' ... " This is good behaviour and I think it works because UPPER() is in _func. I did not document in the manual's SQL section that built-in functions will be in _func, so removing them is not a regression from documented behaviour. Also I acknowledge that they don't exist in MySQL/MariaDB information_schema.routines. But I think users benefit from being able to see them. Peter Gulutzan From korablev at tarantool.org Mon Sep 28 21:19:13 2020 From: korablev at tarantool.org (Nikita Pettik) Date: Mon, 28 Sep 2020 18:19:13 +0000 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> Message-ID: <20200928181913.GD14909@tarantool.org> On 27 Sep 18:18, Mergen Imeev wrote: > Hi all. I have a question that I would like to discuss. > > The question is about SQL built-in functions. At the moment these functions > are > partially described in _func and partially in src/box/sql/func.c. I received > two > completely different suggestions from my reviewers on what to do with these > functions: > 1) Move definitions completely to _func. Remove definitions from func.c. That's my proposal. It makes name collisions check simple, provides unified interface to invoke built-in and non-built-in functions, allows to grant and verify priveleges in the same way and so forth. Built-ins are already declaraed in _func, so reverting this thing would result in another one unnecessary schema change and upgrade (so I doubt that implementation would be somehow 'simpler'). Finally part of functions can turn out to be really usefull in Lua someday such as date()/time(). So to me the choice is kind of obvious.. > 2) Move definitions completely to func.c. Remove definitions from _func. > > In the first case, users will be able to see the function definitions. Also, > in > the future, we may allow these functions to be called from Lua (although not > sure if this is necessary). The main idea is 'all functions have the same > interface'. > > In the second case, the implementation is simpler, and we can more easily > implement some features, such as "virtual" functions. For users, the > definition > can only be seen in the documentation. The main idea is 'SQL built-in > functions > are part of SQL'. > > Which of these approaches do you think is more beneficial to us? > From v.shpilevoy at tarantool.org Mon Sep 28 23:07:41 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Mon, 28 Sep 2020 22:07:41 +0200 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> Message-ID: <208ccd32-fb16-c2dc-3af7-2a8cb437160e@tarantool.org> Hi! On 27.09.2020 17:18, Mergen Imeev wrote: > Hi all. I have a question that I would like to discuss. > > The question is about SQL built-in functions. At the moment these functions are > partially described in _func and partially in src/box/sql/func.c. I received two > completely different suggestions from my reviewers on what to do with these > functions: > 1) Move definitions completely to _func. Remove definitions from func.c. > 2) Move definitions completely to func.c. Remove definitions from _func. > > In the first case, users will be able to see the function definitions. Also, in > the future, we may allow these functions to be called from Lua (although not > sure if this is necessary). The main idea is 'all functions have the same > interface'. > > In the second case, the implementation is simpler, and we can more easily > implement some features, such as "virtual" functions. For users, the definition > can only be seen in the documentation. The main idea is 'SQL built-in functions > are part of SQL'. > > Which of these approaches do you think is more beneficial to us? I am the one who proposed the second option. I proposed it because of several reasons. ================================================== ## Reason 1 The implementation is ugly by design. Since SQL functions are strictly typed, and some of them may take more than one type, we were forced to implement some kind of function overload by mangling function names. We are not trying to implement C++ here, it goes against the _func basic schema. To workaround that there were added a new ugly option: is_overloaded. Overloaded means that there is still one function but declared twice, just to check types. For example, LENGTH and LENGTH_VARBINARY, TRIM, TRIM_VARBINARY, and so on. That leads to an issue, that we have not existing functions in _func. For example: tarantool> box.space._func.index.name:select({'LENGTH_VARBINARY'}) --- - - [68, 1, 'LENGTH_VARBINARY', 1, 'SQL_BUILTIN', '', 'function', ['varbinary'], 'integer', 'none', 'none', true, false, true, ['SQL'], {'is_overloaded': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] ... tarantool> box.execute("SELECT LENGTH_VARBINARY('abc')") --- - null - Function 'LENGTH_VARBINARY' does not exist ... Doesn't this look bad? That is -1 point to the argument about 'it is good to have function visible in _func'. They are visible, but it does not mean anything. ==================== ## Reason 2 SQL has vararg functions - the ones which take unknown number of arguments. That led to addition of one another ugly option: has_vararg. When it is true, all arguments after last declared argument are forced to have the same type as the last argument. This looks wrong from all possible sides. Firstly, if a function has variable argument count, why the hell should they all be of the same type? It is a silly restriction motivated entirely by SQL specifics like SUM() taking all arguments of the same type, or LEAST(), or GREATEST() and so on. That is very-SQL thing. I can't imagine when such a restriction may be needed in other supported languages. Secondly, like with the option 'is_overloaded' it complicates output of _func and makes it harder for a user to create a function, when he sees so many irrelevant options, needed for SQL only. Just take a look at the _func output in the end of this email to see how bad is it [1]. Thirdly, it simply does not work for functions which are really vararg, and not some kind of aggregates, ironically. For example, PRINTF(). To workaround the workaround-option has_vararg such functions simply turn off their type checks by making the last argument SCALAR. Well, this looks really bad. It would become a bit simpler if we would split that option in two - has_vararg to make an equivalent of ... in C with values of any type, and is_aggregate to check all values have the same type. But still ugly. ==================== ## Reason 3 SQL built-in functions are a part of the language. When we store them separately, we risk to get SQL implementation different from the built-in SQL functions schema. For example, we will add a new built-in function, and a user will upgrade, but won't be able to use it until he upgrades the schema, and we will need to support that in SQL implementation - that some built-in functions actually may not exist. ==================== ## Reason 4 Some of the functions are supposed to be used for aggregated values, and this makes their implementation not reusable in other languages. That in turn makes their presence in the common _func space, used for all functions, irrelevant. I am talking about SUM(), COUNT(), AVG(), probably there are more. ================================================================================ Now talking of the points I received about how good would it be to have these functions in _func. ==================== ## Built-in functions require privileges? - No. Cite from Peter's email: The current built-in functions are harmless, except perhaps for RANDOMBLOB with a huge value. However, in future maybe there will be built-in functions that should require privileges. In that case, I believe, they will have to be in _func (and someday in _vfunc) so that grant() will work for them. Cite from Nikita's email: allows to grant and verify priveleges in the same way and so forth What are examples of such built-in functions? Built-in functions are a part of the language. Restricting access to SQL built-ins is like restricting access to 'next()', 'string.upper()', 'os.time()' and other functions in Lua. Or restricting usage of 'operator+' in C++. They are a part of the language. If something needs restrictions, it does not seem to be a part of the language. If a function is sandboxed, so it does not affect other users, I don't see why would it need restrictions. All built-ins are sandboxed. ==================== ## Built-in functions prevent duplicates in _func? - No. Cite from Nikita's email: It makes name collisions check simple This is a matter of a few code lines to add that check to _func triggers to check new function name in all languages, if necessary. Or we could just make the functions hash do the checking. So we won't even need to involve SQL-specific code. That is by the way questionable. I would think more if we want to forbid definition of functions clashing with built-ins. That would be a 'legal' overload, if we would look into _func first, and then into built-ins, but I don't really know about that. Didn't think of it properly yet. ==================== ## Storage in _func does not change _func schema and documentation? - No. Cite from Peter's email: I did not document in the manual's SQL section that built-in functions will be in _func, so removing them is not a regression from documented behaviour. It is good that their presence in _func is not documented. Because even if it would, and we would go for _func extension, we would change _func schema and options anyway, because of the new options 'is_overloaded' and 'has_vararg', and new fake functions with _VARBINARY suffix. Cite from Nikita's email: Built-ins are already declaraed in _func, so reverting this thing would result in another one unnecessary schema change and upgrade (so I doubt that implementation would be somehow 'simpler') This is also wrong. Both versions change _func. But the version with extending _func makes it bigger, adds new ugly SQL-specific options, and adds not existing functions with fake names. In the version about functions-as-part-of-language _func is cleared from these functions, and nothing more happens with the schema. So the patch will be simpler for sure, even though may get bigger due to more removals. Besides, their removal from _func would also allow to get rid of the crutch with language 'SQL_BUILTIN' in _func. It will be present in C, but won't exist in _func. This is not a language. Languages are SQL and Lua. SQL_BUILTIN was a crutch from the beginning, to make the functions work anyhow. ==================== ## Users benefit from seeing SQL-specific functions in _func? - No. Look at [1]. The output format is hardly readable. Not only because of the new options (partially because of them), but also because 1) some functions here don't really exist - LENGTH_VARBINARY, POSITION_VARBINARY, TRIM_VARBINARY, SUBSTR_VARBINARY, 2) because of lots of other fields of _func, which make the output super hard to read and understand what is what, even now. _func is a dead end if someone wants to see function definitions. For that it would be better to introduce a new pretty printer, somewhere in box.func maybe. ==================== ## Storage in _func unifies functions? - _func output is unified, yes. Other things - no. That is the only unification. Code still is going to separate them. By introduction of the new _func options you just move the complexity from a private place related to SQL only to the public space, related to the whole _func and other languages. ==================== ## Reuse SQL functions in Lua and other languages? - No. Cite from Nikita's email: Finally part of functions can turn out to be really usefull in Lua someday such as date()/time() It is not a secret, that all the languages we support already have date/time functions. In Lua these are 'os.date', 'os.time', 'fiber.clock', 'fiber.time' and more. In C these are all the standard C functions such as 'time()', 'gettimeofdat()', 'clock_gettime()', 'clock()', and more. Also lots of functions for time formatting, but I don't remember their names. So what exactly are the SQL built-in functions so much needed in Lua and C? Looking at the unreadable _func output below, I can't imagine why somebody need any of these functions out of SQL. ================================================================================ ## References: [1] Space _func select. - [2, 1, 'TRIM', 1, 'SQL_BUILTIN', '', 'function', ['string', 'unsigned', 'string'], 'string', 'none', 'none', true, false, true, ['SQL'], {'has_overload': true, 'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [3, 1, 'TYPEOF', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [4, 1, 'PRINTF', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'string', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [5, 1, 'UNICODE', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [6, 1, 'CHAR', 1, 'SQL_BUILTIN', '', 'function', ['unsigned'], 'string', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [7, 1, 'HEX', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [8, 1, 'VERSION', 1, 'SQL_BUILTIN', '', 'function', [], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [9, 1, 'QUOTE', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [10, 1, 'REPLACE', 1, 'SQL_BUILTIN', '', 'function', ['string', 'string', 'string'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [11, 1, 'SUBSTR', 1, 'SQL_BUILTIN', '', 'function', ['string', 'integer', 'integer'], 'string', 'none', 'none', true, false, true, ['SQL'], {'has_overload': true, 'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [12, 1, 'GROUP_CONCAT', 1, 'SQL_BUILTIN', '', 'function', ['scalar', 'scalar'], 'string', 'group', 'none', false, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [13, 1, 'JULIANDAY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [14, 1, 'DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [15, 1, 'TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [16, 1, 'DATETIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [17, 1, 'STRFTIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [18, 1, 'CURRENT_TIME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [19, 1, 'CURRENT_TIMESTAMP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [20, 1, 'CURRENT_DATE', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [21, 1, 'LENGTH', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'integer', 'none', 'none', true, false, true, ['SQL'], {'has_overload': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [22, 1, 'POSITION', 1, 'SQL_BUILTIN', '', 'function', ['string', 'string'], 'integer', 'none', 'none', true, false, true, ['SQL'], {'has_overload': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [23, 1, 'ROUND', 1, 'SQL_BUILTIN', '', 'function', ['double', 'unsigned'], 'double', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [24, 1, 'UPPER', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [25, 1, 'LOWER', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [26, 1, 'IFNULL', 1, 'SQL_BUILTIN', '', 'function', ['scalar', 'scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [27, 1, 'RANDOM', 1, 'SQL_BUILTIN', '', 'function', [], 'integer', 'none', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [28, 1, 'CEIL', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [29, 1, 'CEILING', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [30, 1, 'CHARACTER_LENGTH', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'integer', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [31, 1, 'CHAR_LENGTH', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'integer', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [32, 1, 'FLOOR', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [33, 1, 'MOD', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [34, 1, 'OCTET_LENGTH', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [35, 1, 'ROW_COUNT', 1, 'SQL_BUILTIN', '', 'function', [], 'integer', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [36, 1, 'COUNT', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'integer', 'group', 'none', false, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [37, 1, 'LIKE', 1, 'SQL_BUILTIN', '', 'function', ['string', 'string', 'string'], 'boolean', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [38, 1, 'ABS', 1, 'SQL_BUILTIN', '', 'function', ['number'], 'number', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [39, 1, 'EXP', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [40, 1, 'LN', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [41, 1, 'POWER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [42, 1, 'SQRT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [43, 1, 'SUM', 1, 'SQL_BUILTIN', '', 'function', ['number'], 'number', 'group', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [44, 1, 'TOTAL', 1, 'SQL_BUILTIN', '', 'function', ['number'], 'number', 'group', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [45, 1, 'AVG', 1, 'SQL_BUILTIN', '', 'function', ['number'], 'number', 'group', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [46, 1, 'RANDOMBLOB', 1, 'SQL_BUILTIN', '', 'function', ['unsigned'], 'varbinary', 'none', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [47, 1, 'NULLIF', 1, 'SQL_BUILTIN', '', 'function', ['scalar', 'scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [48, 1, 'ZEROBLOB', 1, 'SQL_BUILTIN', '', 'function', ['unsigned'], 'varbinary', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [49, 1, 'MIN', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'group', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [50, 1, 'MAX', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'group', 'none', false, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [51, 1, 'COALESCE', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [52, 1, 'EVERY', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [53, 1, 'EXISTS', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [54, 1, 'EXTRACT', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [55, 1, 'SOME', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [56, 1, 'GREATER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [57, 1, 'LESSER', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [58, 1, 'SOUNDEX', 1, 'SQL_BUILTIN', '', 'function', ['string'], 'string', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [59, 1, 'LIKELIHOOD', 1, 'SQL_BUILTIN', '', 'function', ['scalar', 'double'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [60, 1, 'LIKELY', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [61, 1, 'UNLIKELY', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [62, 1, '_sql_stat_get', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [63, 1, '_sql_stat_push', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [64, 1, '_sql_stat_init', 1, 'SQL_BUILTIN', '', 'function', [], 'any', 'none', 'none', false, false, true, [], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [65, 1, 'LUA', 1, 'LUA', 'function(code) return assert(loadstring(code))() end', 'function', ['string'], 'any', 'none', 'none', false, false, true, ['LUA', 'SQL'], {}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [66, 1, 'GREATEST', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [67, 1, 'LEAST', 1, 'SQL_BUILTIN', '', 'function', ['scalar'], 'scalar', 'none', 'none', true, false, true, ['SQL'], {'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [68, 1, 'LENGTH_VARBINARY', 1, 'SQL_BUILTIN', '', 'function', ['varbinary'], 'integer', 'none', 'none', true, false, true, ['SQL'], {'is_overloaded': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [69, 1, 'POSITION_VARBINARY', 1, 'SQL_BUILTIN', '', 'function', ['varbinary', 'varbinary'], 'integer', 'none', 'none', true, false, true, ['SQL'], {'is_overloaded': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [70, 1, 'TRIM_VARBINARY', 1, 'SQL_BUILTIN', '', 'function', ['varbinary', 'unsigned', 'varbinary'], 'string', 'none', 'none', true, false, true, ['SQL'], {'is_overloaded': true, 'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] - [71, 1, 'SUBSTR_VARBINARY', 1, 'SQL_BUILTIN', '', 'function', ['varbinary', 'integer', 'integer'], 'string', 'none', 'none', true, false, true, ['SQL'], {'is_overloaded': true, 'has_vararg': true}, '', '2020-08-14 16:27:52', '2020-08-14 16:27:52'] ... From v.shpilevoy at tarantool.org Mon Sep 28 23:07:49 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Mon, 28 Sep 2020 22:07:49 +0200 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <9a788a90-f558-fc6c-1d28-2813e8b721f8@ocelot.ca> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> <9a788a90-f558-fc6c-1d28-2813e8b721f8@ocelot.ca> Message-ID: <57513bb5-3a4d-0c3c-720e-7e78634ecfe1@tarantool.org> Hi! See my response in another email with 4 big reasons why storage of SQL-specific functions in _func is a bad idea. Also see responses on your comments in separate sections. I leave references below. > The current built-in functions are harmless, except perhaps for RANDOMBLOB with a huge value. > However, in future maybe there will be built-in functions that should require privileges. > In that case, I believe, they will have to be in _func (and someday in _vfunc) > so that grant() will work for them. See "## Built-in functions require privileges?". > I have tried to redirect the UPPER() function in order to violate security, thus: > " > tarantool> function UPPER(x) return x end > --- > ... > tarantool> box.schema.func.create('UPPER') > --- > - error: Function 'UPPER' already exists > ... > tarantool> box.schema.func.drop('UPPER') > --- > - error: 'Can''t drop function 1: function is SQL built-in' > ... > " > This is good behaviour and I think it works because UPPER() is in _func. See "## Built-in functions prevent duplicates in _func?". > I did not document in the manual's SQL section that built-in functions will > be in _func, so removing them is not a regression from documented behaviour. See "## Storage in _func does not change _func schema and documentation?". > But I think users benefit from being able to see them. See "## Users benefit from seeing SQL-specific functions in _func?". From v.shpilevoy at tarantool.org Mon Sep 28 23:07:50 2020 From: v.shpilevoy at tarantool.org (Vladislav Shpilevoy) Date: Mon, 28 Sep 2020 22:07:50 +0200 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <20200928181913.GD14909@tarantool.org> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> <20200928181913.GD14909@tarantool.org> Message-ID: <85c30425-ae7f-b485-4be5-dcad0b1c1cb6@tarantool.org> Hi! See my response in another email with 4 big reasons why storage of SQL-specific functions in _func is a bad idea. Also see responses on your comments in separate sections. I leave references below. On 28.09.2020 20:19, Nikita Pettik wrote: > On 27 Sep 18:18, Mergen Imeev wrote: >> Hi all. I have a question that I would like to discuss. >> >> The question is about SQL built-in functions. At the moment these functions >> are >> partially described in _func and partially in src/box/sql/func.c. I received >> two >> completely different suggestions from my reviewers on what to do with these >> functions: >> 1) Move definitions completely to _func. Remove definitions from func.c. > > That's my proposal. It makes name collisions check simple, See "## Built-in functions prevent duplicates in _func?". > provides unified interface to invoke built-in and non-built-in functions, See "## Storage in _func unifies functions?". > allows to grant and verify priveleges in the same way and so forth. See "## Built-in functions require privileges?". > Built-ins are already declaraed in _func, so reverting this thing would result> in another one unnecessary schema change and upgrade (so I doubt that implementation would > be somehow 'simpler'). See "## Storage in _func does not change _func schema and documentation?". > Finally part of functions can turn out to be really > usefull in Lua someday such as date()/time(). See "## Reuse SQL functions in Lua and other languages?". > So to me the choice is kind > of obvious.. It still is not to me. From pgulutzan at ocelot.ca Tue Sep 29 22:22:20 2020 From: pgulutzan at ocelot.ca (Peter Gulutzan) Date: Tue, 29 Sep 2020 13:22:20 -0600 Subject: [Tarantool-discussions] SQL built-in functions position In-Reply-To: <57513bb5-3a4d-0c3c-720e-7e78634ecfe1@tarantool.org> References: <66362762-8791-bea3-745f-afc1e3eaa199@tarantool.org> <9a788a90-f558-fc6c-1d28-2813e8b721f8@ocelot.ca> <57513bb5-3a4d-0c3c-720e-7e78634ecfe1@tarantool.org> Message-ID: Hi, On 2020-09-28 2:07 p.m., Vladislav Shpilevoy wrote: > Hi! > > See my response in another email with 4 big reasons why > storage of SQL-specific functions in _func is a bad idea. > > Also see responses on your comments in separate sections. > I leave references below. I cannot argue against your arguments about implementation, and I acknowledged that using _func is not necessary, but I extract a few things that you said that seem a bit odd to me. > Firstly, if a function has variable argument count, why the hell should they all > be of the same type? It is a silly restriction motivated entirely by SQL > specifics like SUM() taking all arguments of the same type, or LEAST(), or GREATEST() > and so on. That is very-SQL thing. I can't imagine when such a restriction may > be needed in other supported languages. I can say CREATE TABLE t (s1 SCALAR PRIMARY KEY); INSERT INTO t VALUES (1), (CAST(2 AS UNSIGNED)), (3e3); SELECT SUM(s1) FROM t; That is, data types can differ provided that they are numbers. I see that as a restriction that "makes sense". > For example, PRINTF(). I acknowledge that PRINTF() is ugly but it is different from other functions. It is not a "very-SQL thing". It happened to be in SQLite. https://sqlite.org/printf.html So I believe that its flaws do not prove something about other functions. > Some of the functions are supposed to be used for aggregated values ... When I wrote, I was only thinking about built-in scalar functions. AVG COUNT GROUP_CONCAT MAX MIN SUM TOTAL are in _func, but I thought they could be excluded from SELECT requests. (Apparently I was wrong, as I'll explain below.) > If a function is sandboxed, so it does not affect other users, I don't see why > would it need restrictions. All built-ins are sandboxed. Not all, I mentioned an exception (in a private issue). And I did mention that "RANDOMBLOB with a huge value" might not be harmless. But in my email I was saying that "in future maybe there will be built-in functions that should require privileges". You asked for an example, but: if I come up with any example that doesn't work in a sandbox, would you not reply that it doesn't need to be built in? > ## Users benefit from seeing SQL-specific functions in _func? - No. True, but not fair. A programmer who writes a client program can write something that accesses _func and displays something that users can comprehend. And information_schema.routines will probably be something that reads _func and displays as an SQL table. > So what exactly are the SQL built-in functions so much needed in Lua > and C? Looking at the unreadable _func output below, I can't imagine > why somebody need any of these functions out of SQL. You are looking at the date/time functions, which, again, are from SQLite. My memory is vague but I understood K. Osipov did not want them documented until we were sure we wanted users to use them. I had a quick look at what is in _func in version 2.5 ... tarantool>SELECT "name", "language", "is_sandboxed", "aggregate" >FROM "_func" >ORDER BY "name"; OK 67 rows selected (0.0 seconds) +----------------------+-------------+--------------+-----------+ | name???????????????? | language??? | is_sandboxed | aggregate | +----------------------+-------------+--------------+-----------+ | ABS????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | AVG????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | CEIL???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | CEILING????????????? | SQL_BUILTIN | FALSE??????? | none????? | | CHAR???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | CHARACTER_LENGTH???? | SQL_BUILTIN | FALSE??????? | none????? | | CHAR_LENGTH????????? | SQL_BUILTIN | FALSE??????? | none????? | | COALESCE???????????? | SQL_BUILTIN | FALSE??????? | none????? | | COUNT??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | CURRENT_DATE???????? | SQL_BUILTIN | FALSE??????? | none????? | | CURRENT_TIME???????? | SQL_BUILTIN | FALSE??????? | none????? | | CURRENT_TIMESTAMP??? | SQL_BUILTIN | FALSE??????? | none????? | | DATE???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | DATETIME???????????? | SQL_BUILTIN | FALSE??????? | none????? | | EVERY??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | EXISTS?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | EXP????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | EXTRACT????????????? | SQL_BUILTIN | FALSE??????? | none????? | | FLOOR??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | GREATER????????????? | SQL_BUILTIN | FALSE??????? | none????? | | GREATEST???????????? | SQL_BUILTIN | FALSE??????? | none????? | | GROUP_CONCAT???????? | SQL_BUILTIN | FALSE??????? | none????? | | HEX????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | IFNULL?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | JULIANDAY??????????? | SQL_BUILTIN | FALSE??????? | none????? | | LEAST??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LENGTH?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LESSER?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LIKE???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LIKELIHOOD?????????? | SQL_BUILTIN | FALSE??????? | none????? | | LIKELY?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LN?????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LOWER??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | LUA????????????????? | LUA???????? | FALSE??????? | none????? | | MAX????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | MIN????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | MOD????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | NULLIF?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | OCTET_LENGTH???????? | SQL_BUILTIN | FALSE??????? | none????? | | POSITION???????????? | SQL_BUILTIN | FALSE??????? | none????? | | POWER??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | PRINTF?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | QUOTE??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | RANDOM?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | RANDOMBLOB?????????? | SQL_BUILTIN | FALSE??????? | none????? | | REPLACE????????????? | SQL_BUILTIN | FALSE??????? | none????? | | ROUND??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | ROW_COUNT??????????? | SQL_BUILTIN | FALSE??????? | none????? | | SOME???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | SOUNDEX????????????? | SQL_BUILTIN | FALSE??????? | none????? | | SQRT???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | STRFTIME???????????? | SQL_BUILTIN | FALSE??????? | none????? | | SUBSTR?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | SUM????????????????? | SQL_BUILTIN | FALSE??????? | none????? | | TIME???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | TOTAL??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | TRIM???????????????? | SQL_BUILTIN | FALSE??????? | none????? | | TYPEOF?????????????? | SQL_BUILTIN | FALSE??????? | none????? | | UNICODE????????????? | SQL_BUILTIN | FALSE??????? | none????? | | UNLIKELY???????????? | SQL_BUILTIN | FALSE??????? | none????? | | UPPER??????????????? | SQL_BUILTIN | FALSE??????? | none????? | | VERSION????????????? | SQL_BUILTIN | FALSE??????? | none????? | | ZEROBLOB???????????? | SQL_BUILTIN | FALSE??????? | none????? | | _sql_stat_get??????? | SQL_BUILTIN | FALSE??????? | none????? | | _sql_stat_init?????? | SQL_BUILTIN | FALSE??????? | none????? | | _sql_stat_push?????? | SQL_BUILTIN | FALSE??????? | none????? | | box.schema.user.info | LUA???????? | FALSE??????? | none????? | +----------------------+-------------+--------------+-----------+ Well, this doesn't look entirely right, and that I guess means that you are right -- at this moment. I do not understand why is_sandboxed is always false. I do not understand why aggregate is none for aggregate functions. I have to acknowledge that many of these functions are undocumented deliberately and we might not want users to seem them, think that we will always support them, and add them to applications. (Include: CEIL/CEILING CURRENT_DATE CURRENT_TIME CURRENT_TIMESTAMP DATE DATETIME EVERY EXP EXTRACT FLOOR GREATER JULIANDAY LESSER LN MOD POWER SOME STRFTIME TIME _sql_stat_get sql_stat_init sql_stat_push box.schema.user.info.) However, that only means that at this moment the information is bad. It does not mean that the information should not exist. I admit that you made points that I hadn't realized, but stubbornly still stick with a belief in _func. Peter Gulutzan