[Tarantool-patches] [PATCH v18 0/2] box: implement cmod Lua module

[Tarantool-patches] [PATCH v18 0/2] box: implement cmod Lua module

[Cyrill Gorcunov via Tarantool-patches]

Please take a look once time permit.

v1-v3 are development ones and not sent.

v5 (by vlad):
 - drop exists, list methods: they are redundant
 - rename cfunc to cbox
 - when create a function make it callable Lua object
 - initialize cbox out of modules
 - fix error in passing module name for reloading
 - make api been cbox.func.[create|drop] and
   cbox.module.reload
 - fix test for OSX sake

v6 (by vlad):
 - move module handling into module_cache file.
v7:
 - development
v8:
 - use rbtree for function instance storage, since
   i don't like the idea of unexpected rehashing of
   values in case of massive number of functions
   allocated
 - use reference counter and free function instance
   if only load/unload are coupled
 - keep a pointer to the function inside Lua object
   so we don't need to lookup on every function call.
   this force us to implement __gc method
 - use new API and update docs
v9:
 - development
v10:
 - use hashes for function names lookup
 - simply function loads counting
 - use luaL_register_module and luaL_register_type for
   easier methods registering
 - carry functions as userdata object
v11:
 - development
v12:
 - switch to new API as been discussed in
   https://lists.tarantool.org/tarantool-patches/e186c454-6765-4776-6433-f3f791ff4c27@tarantool.org/
v13:
 - development
v14:
 - switch to refs to carry module usage
 - drop func_name structure renaming
 - carry two hashes for backward compatibility with
   functions created via box.schema.func help
 - complete rework of cmod and most parts of
   module_cache
 - account for file statistics to invalidate
   module cache
 - new API for cmod, no more :reload, the :load
   procedure uses cache invalidation
 - update test cases
 - still there is no GC test since I didn't
   manage to deal with it
v15:
 - report module state cached/orphan
 - update test cases
 - do not prevent functions lookup in orphan modules
 - there was an idea to use box.shema.func cache as on
   top of cmod's one, but this doesn't work because in case
   if module doesnt exist in any caches we would put it into
   into cmod's one as well but there wont be a module on
   cmod level which would clean it up later (which makes
   code a way more comple if we choose to track state
   of modules).
v16:
 - internal
v17:
 - drop idea of unifying box.schema.func and cmod functions
   cache, it brings more problems than solves due to too
   different context of execution;
 - make cmod self consistent, which shrink patch series
   size ~1/5 in compare with previous attempts;
 - improve tests to account internal states of modules
   and functions (tt_dev key in reports).
v18:
 - implement pass-through cache for modules loading, for
   this sake 'struct module' uses cmod internally;
 - improve tests to cover sole cmod case and a mixture
   of box.schema.func and cmod to make sure the caches
   are not corrupted.

branch gorcunov/gh-4642-func-ro-18
issue https://github.com/tarantool/tarantool/issues/4642

Cyrill Gorcunov (2):
  box/cmod: implement cmod Lua module
  test: box/cfunc -- add cmod test

 src/box/CMakeLists.txt  |   1 +
 src/box/func.c          | 237 ++++------
 src/box/func.h          |   7 +-
 src/box/lua/cmod.c      | 925 ++++++++++++++++++++++++++++++++++++++++
 src/box/lua/cmod.h      | 123 ++++++
 src/box/lua/init.c      |   2 +
 test/box/CMakeLists.txt |   4 +
 test/box/cfunc1.c       |  58 +++
 test/box/cfunc2.c       | 137 ++++++
 test/box/cfunc3.c       |  25 ++
 test/box/cfunc4.c       |  28 ++
 test/box/cmod.result    | 527 +++++++++++++++++++++++
 test/box/cmod.test.lua  | 200 +++++++++
 test/box/suite.ini      |   2 +-
 14 files changed, 2116 insertions(+), 160 deletions(-)
 create mode 100644 src/box/lua/cmod.c
 create mode 100644 src/box/lua/cmod.h
 create mode 100644 test/box/cfunc1.c
 create mode 100644 test/box/cfunc2.c
 create mode 100644 test/box/cfunc3.c
 create mode 100644 test/box/cfunc4.c
 create mode 100644 test/box/cmod.result
 create mode 100644 test/box/cmod.test.lua


base-commit: 4299daf7df12a67748e28969446c8e8a7530bce1
-- 
2.29.2

[Tarantool-patches] [PATCH v18 1/2] box/cmod: implement cmod Lua module

[Cyrill Gorcunov via Tarantool-patches]

Currently to run "C" function from some external module one
have to register it first in "_func" system space. This is
a problem if node is in read-only mode (replica).

Still people would like to have a way to run such functions
even in ro mode. For this sake we implement "cmod" lua module.

Unlike `box.schema.func` interface the cmod does not defer module
loading procedure until first call of a function. Instead a module
is loaded immediately and if some error happens (say shared
library is corrupted or not found) it pops up early.

The need of use stored C procedures implies that application is
running under serious loads most likely there is modular structure
present on Lua level (ie same shared library is loaded in different
sources) thus we cache the loaded library and reuse it on next
load attempts. To verify that cached library is up to day we
test for file attributes (device, inode, size, modification time)
on every load attempt.

Since both box.schema.func and cmod are using caching to minimize
module loading procedure the pass-through caching scheme is
implemented:

 - cmod always carries own hash table for loaded modules;
 - box.schema.func does snoop into cmod hash table when attempt
   to load a new module, if module is present in cmod hash then
   it simply referenced from there and added into own hash table;
   in case if module is not present then it loaded from the scratch
   and put into both hashes;
 - the module_reload action in box.schema.func does not affect
   cmod hash but only own hash updated, the reference in cmod
   table is dropped.

Closes #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

@TarantoolBot document
Title: cmod module

Overview
========

`cmod` module provides a way to create, delete and execute
`C` procedures from shared libraries. Unlike `box.schema.func`
methods the functions created with `cmod` help are not persistent
and live purely in memory. Once a node get turned off they are
vanished. An initial purpose for them is to execute them on
nodes which are running in read-only mode.

Module functions
================

`require('cmod').load(path) -> obj | error`
-------------------------------------------

Loads a module from `path` and return an object instance
associate with the module, otherwise an error is thrown.

The `path` should not end up with shared library extension
(such as `.so`), only a file name shall be there.

Possible errors:

- IllegalParams: module path is either not supplied
  or not a string.
- SystemError: unable to open a module due to a system error.
- ClientError: a module does not exist.
- OutOfMemory: unable to allocate a module.

Example:

``` Lua
-- Without error handling
m = require('cmod').load('path/to/library)

-- With error handling
m, err = pcall(require('cmod').load, 'path/to/library')
if err ~= nil then
    print(err)
end
```

`module:unload() -> true | error`
---------------------------------

Unloads a module. Returns `true` on success, otherwise an error
is thrown. Once the module is unloaded one can't load new
functions from this module instance.

Possible errors:

- IllegalParams: a module is not supplied.
- IllegalParams: a module is already unloaded.

Example:

``` Lua
m = require('cmod').load('path/to/library')
--
-- do something with module
--
m:unload()
```

If there are functions from this module referenced somewhere
in other places of Lua code they still can be executed because
the module continue sitting in memory until the last reference
to it is closed.

If the module become a target to the Lua's garbage collector
then unload is called implicitly.

module:load(name) -> obj | error`
---------------------------------

Loads a new function with name `name` from the previously
loaded `module` and return a callable object instance
associated with the function. On failure an error is thrown.

Possible errors:
 - IllegalParams: function name is either not supplied
   or not a string.
 - IllegalParams: attempt to load a function but module
   has been unloaded already.
 - ClientError: no such function in the module.
 - OutOfMemory: unable to allocate a function.

Example:

``` Lua
-- Load a module if not been loaded yet.
m = require('cmod').load('path/to/library')
-- Load a function with the `foo` name from the module `m`.
func = m:load('foo')
```

In case if there is no need for further loading of other
functions from the same module then the module might be
unloaded immediately.

``` Lua
m = require('cmod').load('path/to/library')
func = m:load('foo')
m:unload()
```

`function:unload() -> true | error`
-----------------------------------

Unloads a function. Returns `true` on success, otherwise
an error is thrown.

Possible errors:
 - IllegalParams: function name is either not supplied
   or not a string.
 - IllegalParams: the function already unloaded.

Example:

``` Lua
m = require('cmod').load('path/to/library')
func = m:load('foo')
--
-- do something with function and cleanup then
--
func:unload()
m:unload()
```

If the function become a target to the Lua's garbage collector
then unload is called implicitly.

Executing a loaded function
===========================

Once function is loaded it can be executed as an ordinary Lua call.
Lets consider the following example. We have a `C` function which
takes two numbers and returns their sum.

``` C
int
cfunc_sum(box_function_ctx_t *ctx, const char *args, const char *args_end)
{
	uint32_t arg_count = mp_decode_array(&args);
	if (arg_count != 2) {
		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
				     "invalid argument count");
	}
	uint64_t a = mp_decode_uint(&args);
	uint64_t b = mp_decode_uint(&args);

	char res[16];
	char *end = mp_encode_uint(res, a + b);
	box_return_mp(ctx, res, end);
	return 0;
}
```

The name of the function is `cfunc_sum` and the function is built into
`cfunc.so` shared library.

First we should load it as

``` Lua
m = require('cmod').load('cfunc')
cfunc_sum = m:load('cfunc_sum')
```

Once successfully loaded we can execute it. Lets call the
`cfunc_sum` with wrong number of arguments

``` Lua
cfunc_sum()
 | ---
 | - error: invalid argument count
```

We will see the `"invalid argument count"` message in output.
The error message has been set by the `box_error_set` in `C`
code above.

On success the sum of arguments will be printed out.

``` Lua
cfunc_sum(1, 2)
 | ---
 | - 3
```

The functions may return multiple results. For example a trivial
echo function which prints arguments passed in.

``` Lua
cfunc_echo(1,2,3)
 | ---
 | - 1
 | - 2
 | - 3
```

Module and function caches
==========================

Loading a module is relatively slow procedure because operating
system needs to read the library, resolve its symbols and etc.
Thus to speedup this procedure if the module is loaded for a first
time we put it into an internal cache. If module is sitting in
the cache already and new request to load comes in -- we simply
reuse a previous copy. In case if module is updated on a storage
device then on new load attempt we detect that file attributes
(such as device number, inode, size, modification time) get changed
and reload module from the scratch. Note that newly loaded module
does not intersect with previously loaded modules, the continue
operating with code previously read from cache.

Thus if there is a need to update a module then all module instances
should be unloaded (together with functions) and loaded again.

Similar caching technique applied to functions -- only first function
allocation cause symbol resolving, next ones are simply obtained from
a function cache.
---
 src/box/CMakeLists.txt |   1 +
 src/box/func.c         | 237 ++++-------
 src/box/func.h         |   7 +-
 src/box/lua/cmod.c     | 925 +++++++++++++++++++++++++++++++++++++++++
 src/box/lua/cmod.h     | 123 ++++++
 src/box/lua/init.c     |   2 +
 6 files changed, 1136 insertions(+), 159 deletions(-)
 create mode 100644 src/box/lua/cmod.c
 create mode 100644 src/box/lua/cmod.h

diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 19203f770..fa4253c5d 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -195,6 +195,7 @@ add_library(box STATIC
     lua/call.c
     lua/cfg.cc
     lua/console.c
+    lua/cmod.c
     lua/serialize_lua.c
     lua/tuple.c
     lua/slab.c
diff --git a/src/box/func.c b/src/box/func.c
index 9909cee45..f77cf00f8 100644
--- a/src/box/func.c
+++ b/src/box/func.c
@@ -34,6 +34,7 @@
 #include "assoc.h"
 #include "lua/utils.h"
 #include "lua/call.h"
+#include "lua/cmod.h"
 #include "error.h"
 #include "errinj.h"
 #include "diag.h"
@@ -95,77 +96,6 @@ func_split_name(const char *str, struct func_name *name)
 	}
 }
 
-/**
- * Arguments for luaT_module_find used by lua_cpcall()
- */
-struct module_find_ctx {
-	const char *package;
-	const char *package_end;
-	char *path;
-	size_t path_len;
-};
-
-/**
- * A cpcall() helper for module_find()
- */
-static int
-luaT_module_find(lua_State *L)
-{
-	struct module_find_ctx *ctx = (struct module_find_ctx *)
-		lua_topointer(L, 1);
-
-	/*
-	 * Call package.searchpath(name, package.cpath) and use
-	 * the path to the function in dlopen().
-	 */
-	lua_getglobal(L, "package");
-
-	lua_getfield(L, -1, "search");
-
-	/* Argument of search: name */
-	lua_pushlstring(L, ctx->package, ctx->package_end - ctx->package);
-
-	lua_call(L, 1, 1);
-	if (lua_isnil(L, -1))
-		return luaL_error(L, "module not found");
-	/* Convert path to absolute */
-	char resolved[PATH_MAX];
-	if (realpath(lua_tostring(L, -1), resolved) == NULL) {
-		diag_set(SystemError, "realpath");
-		return luaT_error(L);
-	}
-
-	snprintf(ctx->path, ctx->path_len, "%s", resolved);
-	return 0;
-}
-
-/**
- * Find path to module using Lua's package.cpath
- * @param package package name
- * @param package_end a pointer to the last byte in @a package + 1
- * @param[out] path path to shared library
- * @param path_len size of @a path buffer
- * @retval 0 on success
- * @retval -1 on error, diag is set
- */
-static int
-module_find(const char *package, const char *package_end, char *path,
-	    size_t path_len)
-{
-	struct module_find_ctx ctx = { package, package_end, path, path_len };
-	lua_State *L = tarantool_L;
-	int top = lua_gettop(L);
-	if (luaT_cpcall(L, luaT_module_find, &ctx) != 0) {
-		int package_len = (int) (package_end - package);
-		diag_set(ClientError, ER_LOAD_MODULE, package_len, package,
-			 lua_tostring(L, -1));
-		lua_settop(L, top);
-		return -1;
-	}
-	assert(top == lua_gettop(L)); /* cpcall discard results */
-	return 0;
-}
-
 static struct mh_strnptr_t *modules = NULL;
 
 static void
@@ -214,11 +144,14 @@ module_cache_find(const char *name, const char *name_end)
 static inline int
 module_cache_put(struct module *module)
 {
-	size_t package_len = strlen(module->package);
-	uint32_t name_hash = mh_strn_hash(module->package, package_len);
+	const char *package = module->cmod->package;
+	size_t package_len = module->cmod->package_len;
 	const struct mh_strnptr_node_t strnode = {
-		module->package, package_len, name_hash, module};
-
+		.str	= package,
+		.len	= package_len,
+		.hash	= mh_strn_hash(package, package_len),
+		.val	= module,
+	};
 	if (mh_strnptr_put(modules, &strnode, NULL, NULL) == mh_end(modules)) {
 		diag_set(OutOfMemory, sizeof(strnode), "malloc", "modules");
 		return -1;
@@ -238,102 +171,55 @@ module_cache_del(const char *name, const char *name_end)
 	mh_strnptr_del(modules, i, NULL);
 }
 
-/*
- * Load a dso.
- * Create a new symlink based on temporary directory and try to
- * load via this symink to load a dso twice for cases of a function
- * reload.
+/**
+ * Allocate a new module instance.
  */
 static struct module *
-module_load(const char *package, const char *package_end)
+module_new(struct cmod *cmod)
 {
-	char path[PATH_MAX];
-	if (module_find(package, package_end, path, sizeof(path)) != 0)
-		return NULL;
-
-	int package_len = package_end - package;
-	struct module *module = (struct module *)
-		malloc(sizeof(*module) + package_len + 1);
+	struct module *module = malloc(sizeof(*module));
 	if (module == NULL) {
-		diag_set(OutOfMemory, sizeof(struct module) + package_len + 1,
+		diag_set(OutOfMemory, sizeof(struct module),
 			 "malloc", "struct module");
 		return NULL;
 	}
-	memcpy(module->package, package, package_len);
-	module->package[package_len] = 0;
+
 	rlist_create(&module->funcs);
 	module->calls = 0;
+	module->cmod = cmod;
 
-	const char *tmpdir = getenv("TMPDIR");
-	if (tmpdir == NULL)
-		tmpdir = "/tmp";
-	char dir_name[PATH_MAX];
-	int rc = snprintf(dir_name, sizeof(dir_name), "%s/tntXXXXXX", tmpdir);
-	if (rc < 0 || (size_t) rc >= sizeof(dir_name)) {
-		diag_set(SystemError, "failed to generate path to tmp dir");
-		goto error;
-	}
+	return module;
+}
 
-	if (mkdtemp(dir_name) == NULL) {
-		diag_set(SystemError, "failed to create unique dir name: %s",
-			 dir_name);
-		goto error;
-	}
-	char load_name[PATH_MAX];
-	rc = snprintf(load_name, sizeof(load_name), "%s/%.*s." TARANTOOL_LIBEXT,
-		      dir_name, package_len, package);
-	if (rc < 0 || (size_t) rc >= sizeof(dir_name)) {
-		diag_set(SystemError, "failed to generate path to DSO");
-		goto error;
-	}
+/**
+ * Load a new DSO.
+ */
+static struct module *
+module_load(const char *package, const char *package_end)
+{
+	char path[PATH_MAX];
+	int package_len = package_end - package;
 
-	struct stat st;
-	if (stat(path, &st) < 0) {
-		diag_set(SystemError, "failed to stat() module %s", path);
-		goto error;
+	if (cmod_find_package(package, package_len,
+			      path, sizeof(path)) != 0) {
+		return NULL;
 	}
 
-	int source_fd = open(path, O_RDONLY);
-	if (source_fd < 0) {
-		diag_set(SystemError, "failed to open module %s file for" \
-			 " reading", path);
-		goto error;
-	}
-	int dest_fd = open(load_name, O_WRONLY|O_CREAT|O_TRUNC,
-			   st.st_mode & 0777);
-	if (dest_fd < 0) {
-		diag_set(SystemError, "failed to open file %s for writing ",
-			 load_name);
-		close(source_fd);
-		goto error;
-	}
+	struct cmod *cmod = cmod_new(package, package_len, path);
+	if (cmod == NULL)
+		return NULL;
 
-	off_t ret = eio_sendfile_sync(dest_fd, source_fd, 0, st.st_size);
-	close(source_fd);
-	close(dest_fd);
-	if (ret != st.st_size) {
-		diag_set(SystemError, "failed to copy DSO %s to %s",
-			 path, load_name);
-		goto error;
+	struct module *module = module_new(cmod);
+	if (module == NULL) {
+		cmod_unref(cmod);
+		return NULL;
 	}
 
-	module->handle = dlopen(load_name, RTLD_NOW | RTLD_LOCAL);
-	if (unlink(load_name) != 0)
-		say_warn("failed to unlink dso link %s", load_name);
-	if (rmdir(dir_name) != 0)
-		say_warn("failed to delete temporary dir %s", dir_name);
-	if (module->handle == NULL) {
-		diag_set(ClientError, ER_LOAD_MODULE, package_len,
-			  package, dlerror());
-		goto error;
-	}
 	struct errinj *e = errinj(ERRINJ_DYN_MODULE_COUNT, ERRINJ_INT);
 	if (e != NULL)
 		++e->iparam;
+
 	return module;
-error:
-	free(module);
-	return NULL;
 }
 
 static void
@@ -342,7 +228,7 @@ module_delete(struct module *module)
 	struct errinj *e = errinj(ERRINJ_DYN_MODULE_COUNT, ERRINJ_INT);
 	if (e != NULL)
 		--e->iparam;
-	dlclose(module->handle);
+	cmod_unref(module->cmod);
 	TRASH(module);
 	free(module);
 }
@@ -363,7 +249,8 @@ module_gc(struct module *module)
 static box_function_f
 module_sym(struct module *module, const char *name)
 {
-	box_function_f f = (box_function_f)dlsym(module->handle, name);
+	void *handle = module->cmod->handle;
+	box_function_f f = (box_function_f)dlsym(handle, name);
 	if (f == NULL) {
 		diag_set(ClientError, ER_LOAD_FUNCTION, name, dlerror());
 		return NULL;
@@ -529,19 +416,59 @@ func_c_load(struct func_c *func)
 	func_split_name(func->base.def->name, &name);
 
 	struct module *cached, *module;
+	struct cmod *cmod;
+
 	cached = module_cache_find(name.package, name.package_end);
-	if (cached == NULL) {
+	if (cached != NULL) {
+		module = cached;
+		goto resolve_sym;
+	}
+
+	size_t len = name.package_end - name.package;
+	cmod = cmod_cache_find(name.package, len);
+	if (cmod == NULL) {
+		/*
+		 * The module is not present in both
+		 * box.schema.func cache and in cmod
+		 * cache. Thus load it from from the
+		 * scratch and put into cmod cache
+		 * as well.
+		 */
 		module = module_load(name.package, name.package_end);
 		if (module == NULL)
 			return -1;
-		if (module_cache_put(module)) {
+		if (cmod_cache_put(module->cmod) != 0) {
 			module_delete(module);
 			return -1;
 		}
+
+		/*
+		 * Fresh cmod instance is bound to
+		 * the module and get unref upon
+		 * module unload.
+		 */
+		cmod = NULL;
 	} else {
-		module = cached;
+		/*
+		 * Someone already has loaded this
+		 * shared library via cmod interface,
+		 * thus simply increase the reference
+		 * (and don't forget to unref later).
+		 */
+		module = module_new(cmod);
+		if (module == NULL)
+			return -1;
+		cmod_ref(cmod);
+	}
+
+	if (module_cache_put(module)) {
+		if (cmod != NULL)
+			cmod_unref(cmod);
+		module_delete(module);
+		return -1;
 	}
 
+resolve_sym:
 	func->func = module_sym(module, name.sym);
 	if (func->func == NULL) {
 		if (cached == NULL) {
diff --git a/src/box/func.h b/src/box/func.h
index 581e468cb..ac9772dfe 100644
--- a/src/box/func.h
+++ b/src/box/func.h
@@ -43,19 +43,18 @@ extern "C" {
 #endif /* defined(__cplusplus) */
 
 struct func;
+struct cmod;
 
 /**
  * Dynamic shared module.
  */
 struct module {
-	/** Module dlhandle. */
-	void *handle;
+	/** Low level cmod module. */
+	struct cmod *cmod;
 	/** List of imported functions. */
 	struct rlist funcs;
 	/** Count of active calls. */
 	size_t calls;
-	/** Module's package name. */
-	char package[0];
 };
 
 /** Virtual method table for func object. */
diff --git a/src/box/lua/cmod.c b/src/box/lua/cmod.c
new file mode 100644
index 000000000..6b383d8fe
--- /dev/null
+++ b/src/box/lua/cmod.c
@@ -0,0 +1,925 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <dlfcn.h>
+#include <fcntl.h>
+#include <lua.h>
+
+#include "box/error.h"
+#include "box/port.h"
+#include "box/func_def.h"
+
+#include "tt_static.h"
+
+#include "assoc.h"
+#include "cmod.h"
+#include "fiber.h"
+
+#include "lua/utils.h"
+#include "libeio/eio.h"
+
+/**
+ * Function descriptor.
+ */
+struct cmod_func {
+	/** Module function belongs to. */
+	struct cmod *cmod;
+	/** Address to execute on call. */
+	box_function_f addr;
+	/** Number of references. */
+	int64_t refs;
+	/** Length of functon name in @a key. */
+	size_t sym_len;
+	/** Length of @a key. */
+	size_t len;
+	/** Function hash key. */
+	char key[0];
+};
+
+/** Module name to cmod hash. */
+static struct mh_strnptr_t *cmod_hash = NULL;
+
+/** Function name to cmod_func hash. */
+static struct mh_strnptr_t *cmod_func_hash = NULL;
+
+/** A type to find a module from an object. */
+static const char *uname_cmod = "tt_uname_cmod";
+
+/** A type to find a function from an object. */
+static const char *uname_func = "tt_uname_cmod_func";
+
+/** Module unique IDs. */
+static int64_t cmod_ids = 1;
+
+/** Get data associated with an object. */
+static void *
+get_udata(struct lua_State *L, const char *uname)
+{
+	void **pptr = luaL_testudata(L, 1, uname);
+	return pptr != NULL ? *pptr : NULL;
+}
+
+/** Set data to a new value. */
+static void
+set_udata(struct lua_State *L, const char *uname, void *ptr)
+{
+	void **pptr = luaL_testudata(L, 1, uname);
+	assert(pptr != NULL);
+	*pptr = ptr;
+}
+
+/** Setup a new data and associate it with an object. */
+static void
+new_udata(struct lua_State *L, const char *uname, void *ptr)
+{
+	*(void **)lua_newuserdata(L, sizeof(void *)) = ptr;
+	luaL_getmetatable(L, uname);
+	lua_setmetatable(L, -2);
+}
+
+/**
+ * Helpers for string based hash manipulations.
+ */
+static void *
+hash_find(struct mh_strnptr_t *h, const char *str, size_t len)
+{
+	mh_int_t e = mh_strnptr_find_inp(h, str, len);
+	if (e == mh_end(h))
+		return NULL;
+	return mh_strnptr_node(h, e)->val;
+}
+
+static void
+hash_update(struct mh_strnptr_t *h, const char *src_str, size_t len,
+	    const char *new_str, void *new_val)
+{
+	mh_int_t e = mh_strnptr_find_inp(h, src_str, len);
+	if (e == mh_end(h))
+		panic("cmod: failed to update hash");
+	mh_strnptr_node(h, e)->str = new_str;
+	mh_strnptr_node(h, e)->val = new_val;
+}
+
+static int
+hash_add(struct mh_strnptr_t *h, const char *str, size_t len, void *val)
+{
+	const struct mh_strnptr_node_t nd = {
+		.str	= str,
+		.len	= len,
+		.hash	= mh_strn_hash(str, len),
+		.val	= val,
+	};
+	if (mh_strnptr_put(h, &nd, NULL, NULL) == mh_end(h)) {
+		diag_set(OutOfMemory, sizeof(nd), "malloc",
+			 "cmod: hash node");
+		return -1;
+	}
+	return 0;
+}
+
+static void
+hash_del_kv(struct mh_strnptr_t *h, const char *str,
+	    size_t len, void *val)
+{
+	mh_int_t e = mh_strnptr_find_inp(h, str, len);
+	if (e != mh_end(h)) {
+		void *v = mh_strnptr_node(h, e)->val;
+		if (v == val)
+			mh_strnptr_del(h, e, NULL);
+	}
+}
+
+static void
+hash_del(struct mh_strnptr_t *h, const char *str, size_t len)
+{
+	mh_int_t e = mh_strnptr_find_inp(h, str, len);
+	if (e != mh_end(h))
+		mh_strnptr_del(h, e, NULL);
+}
+
+/** Arguments for lpackage_search. */
+struct find_ctx {
+	const char *package;
+	size_t package_len;
+	char *path;
+	size_t path_len;
+};
+
+/** A cpcall() helper for lfind_package(). */
+static int
+lpackage_search(lua_State *L)
+{
+	struct find_ctx *ctx = (void *)lua_topointer(L, 1);
+
+	lua_getglobal(L, "package");
+	lua_getfield(L, -1, "search");
+	lua_pushlstring(L, ctx->package, ctx->package_len);
+
+	lua_call(L, 1, 1);
+	if (lua_isnil(L, -1))
+		return luaL_error(L, "cmod: module not found");
+
+	char resolved[PATH_MAX];
+	if (realpath(lua_tostring(L, -1), resolved) == NULL) {
+		diag_set(SystemError, "cmod: realpath");
+		return luaT_error(L);
+	}
+
+	/*
+	 * No need for result being trimmed test, it
+	 * is guaranteed by realpath call.
+	 */
+	snprintf(ctx->path, ctx->path_len, "%s", resolved);
+	return 0;
+}
+
+int
+cmod_find_package(const char *package, size_t package_len,
+		  char *path, size_t path_len)
+{
+	struct find_ctx ctx = {
+		.package	= package,
+		.package_len	= package_len,
+		.path		= path,
+		.path_len	= path_len,
+	};
+
+	lua_State *L = tarantool_L;
+	int top = lua_gettop(L);
+	if (luaT_cpcall(L, lpackage_search, &ctx) != 0) {
+		diag_set(ClientError, ER_LOAD_MODULE, ctx.package_len,
+			 ctx.package, lua_tostring(L, -1));
+		lua_settop(L, top);
+		return -1;
+	}
+	assert(top == lua_gettop(L));
+	return 0;
+}
+
+/** Increase reference to cmod. */
+void
+cmod_ref(struct cmod *m)
+{
+	assert(m->refs >= 0);
+	++m->refs;
+}
+
+/** Decrease reference to cmod and free it if last one. */
+void
+cmod_unref(struct cmod *m)
+{
+	assert(m->refs > 0);
+	if (--m->refs == 0) {
+		hash_del_kv(cmod_hash, m->package, m->package_len, m);
+		dlclose(m->handle);
+		TRASH(m);
+		free(m);
+	}
+}
+
+struct cmod *
+cmod_cache_find(const char *package, size_t package_len)
+{
+	return hash_find(cmod_hash, package, package_len);
+}
+
+int
+cmod_cache_put(struct cmod *m)
+{
+	return hash_add(cmod_hash, m->package, m->package_len, m);
+}
+
+/** Fill cmod attributes from stat. */
+static void
+cmod_attr_fill(struct cmod_attr *attr, struct stat *st)
+{
+	attr->st_dev = st->st_dev;
+	attr->st_ino = st->st_ino;
+	attr->st_size = st->st_size;
+#ifdef TARGET_OS_DARWIN
+	attr->st_mtimespec = st->st_mtimespec;
+#else
+	attr->st_mtim = st->st_mtim;
+#endif
+}
+
+struct cmod *
+cmod_new(const char *package, size_t package_len, const char *source_path)
+{
+	size_t size = sizeof(struct cmod) + package_len + 1;
+	struct cmod *m = malloc(size);
+	if (m == NULL) {
+		diag_set(OutOfMemory, size, "malloc", "cmod");
+		return NULL;
+	}
+
+	m->package_len = package_len;
+	m->refs = 0;
+
+	memcpy(m->package, package, package_len);
+	m->package[package_len] = 0;
+
+	const char *tmpdir = getenv("TMPDIR");
+	if (tmpdir == NULL)
+		tmpdir = "/tmp";
+
+	char dir_name[PATH_MAX];
+	int rc = snprintf(dir_name, sizeof(dir_name),
+			  "%s/tntXXXXXX", tmpdir);
+	if (rc < 0 || (size_t)rc >= sizeof(dir_name)) {
+		diag_set(SystemError, "failed to generate path to tmp dir");
+		goto error;
+	}
+
+	if (mkdtemp(dir_name) == NULL) {
+		diag_set(SystemError, "failed to create unique dir name: %s",
+			 dir_name);
+		goto error;
+	}
+
+	char load_name[PATH_MAX];
+	rc = snprintf(load_name, sizeof(load_name),
+		      "%s/%.*s." TARANTOOL_LIBEXT,
+		      dir_name, (int)package_len, package);
+	if (rc < 0 || (size_t)rc >= sizeof(dir_name)) {
+		diag_set(SystemError, "failed to generate path to DSO");
+		goto error;
+	}
+
+	struct stat st;
+	if (stat(source_path, &st) < 0) {
+		diag_set(SystemError, "failed to stat() module: %s",
+			 source_path);
+		goto error;
+	}
+	cmod_attr_fill(&m->attr, &st);
+
+	int source_fd = open(source_path, O_RDONLY);
+	if (source_fd < 0) {
+		diag_set(SystemError, "failed to open module %s "
+			 "file for reading", source_path);
+		goto error;
+	}
+	int dest_fd = open(load_name, O_WRONLY | O_CREAT | O_TRUNC,
+			   st.st_mode & 0777);
+	if (dest_fd < 0) {
+		diag_set(SystemError, "failed to open file %s "
+			 "for writing ", load_name);
+		close(source_fd);
+		goto error;
+	}
+
+	off_t ret = eio_sendfile_sync(dest_fd, source_fd, 0, st.st_size);
+	close(source_fd);
+	close(dest_fd);
+	if (ret != st.st_size) {
+		diag_set(SystemError, "failed to copy DSO %s to %s",
+			 source_path, load_name);
+		goto error;
+	}
+
+	m->handle = dlopen(load_name, RTLD_NOW | RTLD_LOCAL);
+	if (unlink(load_name) != 0)
+		say_warn("failed to unlink dso link: %s", load_name);
+	if (rmdir(dir_name) != 0)
+		say_warn("failed to delete temporary dir: %s", dir_name);
+	if (m->handle == NULL) {
+		diag_set(ClientError, ER_LOAD_MODULE, package_len,
+			  package, dlerror());
+		goto error;
+	}
+
+	m->id = cmod_ids++;
+	cmod_ref(m);
+	return m;
+
+error:
+	free(m);
+	return NULL;
+}
+
+/**
+ * Load a module.
+ *
+ * This function takes a module path from the caller
+ * stack @a L and returns cached module instance or
+ * creates a new module object.
+ *
+ * Possible errors:
+ *
+ * - IllegalParams: module path is either not supplied
+ *   or not a string.
+ * - SystemError: unable to open a module due to a system error.
+ * - ClientError: a module does not exist.
+ * - OutOfMemory: unable to allocate a module.
+ *
+ * @returns module object on success or throws an error.
+ */
+static int
+lcmod_load(struct lua_State *L)
+{
+	const char msg_noname[] = "Expects cmod.load(\'name\') "
+		"but no name passed";
+
+	if (lua_gettop(L) != 1 || !lua_isstring(L, 1)) {
+		diag_set(IllegalParams, msg_noname);
+		return luaT_error(L);
+	}
+
+	size_t name_len;
+	const char *name = lua_tolstring(L, 1, &name_len);
+
+	if (name_len < 1) {
+		diag_set(IllegalParams, msg_noname);
+		return luaT_error(L);
+	}
+
+	char path[PATH_MAX];
+	if (cmod_find_package(name, name_len, path, sizeof(path)) != 0)
+		return luaT_error(L);
+
+	struct cmod *m = hash_find(cmod_hash, name, name_len);
+	if (m != NULL) {
+		struct cmod_attr attr;
+		struct stat st;
+		if (stat(path, &st) != 0) {
+			diag_set(SystemError, "failed to stat() %s",
+				 path);
+			return luaT_error(L);
+		}
+
+		/*
+		 * In case of cache hit we may reuse existing
+		 * module which speedup load procedure.
+		 */
+		cmod_attr_fill(&attr, &st);
+		if (memcmp(&attr, &m->attr, sizeof(attr)) == 0) {
+			cmod_ref(m);
+			new_udata(L, uname_cmod, m);
+			return 1;
+		}
+
+		/*
+		 * Module has been updated on a storage device,
+		 * so load a new instance and update the cache,
+		 * old entry get evicted but continue residing
+		 * in memory, fully functional, until last
+		 * function is unloaded.
+		 */
+		m = cmod_new(name, name_len, path);
+		if (m == NULL)
+			return luaT_error(L);
+
+		hash_update(cmod_hash, name, name_len, m->package, m);
+		/*
+		 * This is transparent procedure so notify a user
+		 * that new module is read otherwise it won't be
+		 * possible to figure out what is going on.
+		 */
+		say_info("cmod: attr change, reload: %s", name);
+	} else {
+		m = cmod_new(name, name_len, path);
+		if (m == NULL)
+			return luaT_error(L);
+		if (hash_add(cmod_hash, m->package, name_len, m) != 0) {
+			/* Never been in hash: safe for hash_del_kv */
+			cmod_unref(m);
+			return luaT_error(L);
+		}
+	}
+
+	new_udata(L, uname_cmod, m);
+	return 1;
+}
+
+/**
+ * Unload a module.
+ *
+ * Take a module object from the caller stack @a L and unload it.
+ *
+ * Possible errors:
+ *
+ * - IllegalParams: module is not supplied.
+ * - IllegalParams: the module is unloaded.
+ *
+ * @returns true on success or throwns an error.
+ */
+static int
+lcmod_unload(struct lua_State *L)
+{
+	if (lua_gettop(L) != 1) {
+		diag_set(IllegalParams, "Expects module:unload()");
+		return luaT_error(L);
+	}
+
+	struct cmod *m = get_udata(L, uname_cmod);
+	if (m == NULL) {
+		diag_set(IllegalParams, "The module is unloaded");
+		return luaT_error(L);
+	}
+
+	set_udata(L, uname_cmod, NULL);
+	cmod_unref(m);
+	lua_pushboolean(L, true);
+	return 1;
+}
+
+/** Handle __index request for a module object. */
+static int
+lcmod_index(struct lua_State *L)
+{
+	lua_getmetatable(L, 1);
+	lua_pushvalue(L, 2);
+	lua_rawget(L, -2);
+	if (!lua_isnil(L, -1))
+		return 1;
+
+	struct cmod *m = get_udata(L, uname_cmod);
+	if (m == NULL) {
+		lua_pushnil(L);
+		return 1;
+	}
+
+	const char *key = lua_tostring(L, 2);
+	if (key == NULL || lua_type(L, 2) != LUA_TSTRING) {
+		diag_set(IllegalParams,
+			 "Bad params, use __index(obj, <string>)");
+		return luaT_error(L);
+	}
+
+	if (strcmp(key, "path") == 0) {
+		lua_pushstring(L, m->package);
+		return 1;
+	}
+
+	/*
+	 * Internal keys for debug only, not API.
+	 */
+	if (strncmp(key, "tt_dev.", 7) == 0) {
+		const char *subkey = &key[7];
+		if (strcmp(subkey, "refs") == 0) {
+			lua_pushnumber(L, m->refs);
+			return 1;
+		} else if (strcmp(subkey, "id") == 0) {
+			lua_pushnumber(L, m->id);
+			return 1;
+		}
+	}
+	return 0;
+}
+
+/** Module representation for REPL (console). */
+static int
+lcmod_serialize(struct lua_State *L)
+{
+	struct cmod *m = get_udata(L, uname_cmod);
+	if (m == NULL) {
+		lua_pushnil(L);
+		return 1;
+	}
+
+	lua_createtable(L, 0, 1);
+	lua_pushstring(L, m->package);
+	lua_setfield(L, -2, "path");
+	return 1;
+}
+
+/** Collect a module. */
+static int
+lcmod_gc(struct lua_State *L)
+{
+	struct cmod *m = get_udata(L, uname_cmod);
+	if (m != NULL) {
+		set_udata(L, uname_cmod, NULL);
+		cmod_unref(m);
+	}
+	return 0;
+}
+
+/** Increase reference to a function. */
+static void
+cmod_func_ref(struct cmod_func *cf)
+{
+	assert(cf->refs >= 0);
+	++cf->refs;
+}
+
+/** Free function memory. */
+static void
+cmod_func_delete(struct cmod_func *cf)
+{
+	TRASH(cf);
+	free(cf);
+}
+
+/** Unreference a function and free if last one. */
+static void
+cmod_func_unref(struct cmod_func *cf)
+{
+	assert(cf->refs > 0);
+	if (--cf->refs == 0) {
+		cmod_unref(cf->cmod);
+		hash_del(cmod_func_hash, cf->key, cf->len);
+		cmod_func_delete(cf);
+	}
+}
+
+/** Function name from a hash key. */
+static char *
+cmod_func_name(struct cmod_func *cf)
+{
+	return &cf->key[cf->len - cf->sym_len];
+}
+
+/**
+ * Allocate a new function instance and resolve its address.
+ *
+ * @param m a module the function should be loaded from.
+ * @param key function hash key, ie "1.module.foo".
+ * @param len length of @a key.
+ * @param sym_len function symbol name length, ie 3 for "foo".
+ *
+ * @returns function instance on success, NULL otherwise (diag is set).
+ */
+static struct cmod_func *
+cmod_func_new(struct cmod *m, const char *key, size_t len, size_t sym_len)
+{
+	size_t size = sizeof(struct cmod_func) + len + 1;
+	struct cmod_func *cf = malloc(size);
+	if (cf == NULL) {
+		diag_set(OutOfMemory, size, "malloc", "cf");
+		return NULL;
+	}
+
+	cf->cmod = m;
+	cf->len = len;
+	cf->sym_len = sym_len;
+	cf->refs = 0;
+
+	memcpy(cf->key, key, len);
+	cf->key[len] = '\0';
+
+	cf->addr = dlsym(m->handle, cmod_func_name(cf));
+	if (cf->addr == NULL) {
+		diag_set(ClientError, ER_LOAD_FUNCTION,
+			 cmod_func_name(cf), dlerror());
+		cmod_func_delete(cf);
+		return NULL;
+	}
+
+	if (hash_add(cmod_func_hash, cf->key, cf->len, cf) != 0) {
+		cmod_func_delete(cf);
+		return NULL;
+	}
+
+	/*
+	 * Each new function depends on module presence.
+	 * Module will reside even if been unload
+	 * explicitly after the function creation.
+	 */
+	cmod_ref(cf->cmod);
+	cmod_func_ref(cf);
+	return cf;
+}
+
+/**
+ * Load a function.
+ *
+ * This function takes a function name from the caller
+ * stack @a L and either returns a cached function or
+ * creates a new function object.
+ *
+ * Possible errors:
+ *
+ * - IllegalParams: function name is either not supplied
+ *   or not a string.
+ * - SystemError: unable to open a module due to a system error.
+ * - ClientError: a module does not exist.
+ * - OutOfMemory: unable to allocate a module.
+ *
+ * @returns module object on success or throws an error.
+ */
+static int
+lcmod_load_func(struct lua_State *L)
+{
+	const char *method = "function = module:load";
+	const char fmt_noname[] = "Expects %s(\'name\') but no name passed";
+
+	if (lua_gettop(L) != 2 || !lua_isstring(L, 2)) {
+		diag_set(IllegalParams, fmt_noname, method);
+		return luaT_error(L);
+	}
+
+	struct cmod *m = get_udata(L, uname_cmod);
+	if (m == NULL) {
+		const char *fmt =
+			"Expects %s(\'name\') but not module object passed";
+		diag_set(IllegalParams, fmt, method);
+		return luaT_error(L);
+	}
+
+	size_t sym_len;
+	const char *sym = lua_tolstring(L, 2, &sym_len);
+
+	if (sym_len < 1) {
+		diag_set(IllegalParams, fmt_noname, method);
+		return luaT_error(L);
+	}
+
+	/*
+	 * Functions are bound to a module symbols, thus
+	 * since the hash is global it should be unique
+	 * per module. The symbol (function name) is the
+	 * last part of the hash key.
+	 */
+	const char *key = tt_sprintf("%lld.%s.%s", (long long)m->id,
+				     m->package, sym);
+	size_t len = strlen(key);
+
+	struct cmod_func *cf = hash_find(cmod_func_hash, key, len);
+	if (cf == NULL) {
+		cf = cmod_func_new(m, key, len, sym_len);
+		if (cf == NULL)
+			return luaT_error(L);
+	} else {
+		cmod_func_ref(cf);
+	}
+
+	new_udata(L, uname_func, cf);
+	return 1;
+}
+
+/**
+ * Unload a function.
+ *
+ * Take a function object from the caller stack @a L and unload it.
+ *
+ * Possible errors:
+ *
+ * - IllegalParams: the function is not supplied.
+ * - IllegalParams: the function already unloaded.
+ *
+ * @returns true on success or throwns an error.
+ */
+static int
+lfunc_unload(struct lua_State *L)
+{
+	if (lua_gettop(L) != 1) {
+		diag_set(IllegalParams, "Expects function:unload()");
+		return luaT_error(L);
+	}
+
+	struct cmod_func *cf = get_udata(L, uname_func);
+	if (cf == NULL) {
+		diag_set(IllegalParams, "The function is unloaded");
+		return luaT_error(L);
+	}
+
+	set_udata(L, uname_func, NULL);
+	cmod_func_unref(cf);
+
+	lua_pushboolean(L, true);
+	return 1;
+}
+
+/** Handle __index request for a function object. */
+static int
+lfunc_index(struct lua_State *L)
+{
+	lua_getmetatable(L, 1);
+	lua_pushvalue(L, 2);
+	lua_rawget(L, -2);
+	if (!lua_isnil(L, -1))
+		return 1;
+
+	struct cmod_func *cf = get_udata(L, uname_func);
+	if (cf == NULL) {
+		lua_pushnil(L);
+		return 1;
+	}
+
+	const char *key = lua_tostring(L, 2);
+	if (key == NULL || lua_type(L, 2) != LUA_TSTRING) {
+		diag_set(IllegalParams,
+			 "Bad params, use __index(obj, <string>)");
+		return luaT_error(L);
+	}
+
+	if (strcmp(key, "name") == 0) {
+		lua_pushstring(L, cmod_func_name(cf));
+		return 1;
+	}
+
+	/*
+	 * Internal keys for debug only, not API.
+	 */
+	if (strncmp(key, "tt_dev.", 7) == 0) {
+		const char *subkey = &key[7];
+		if (strcmp(subkey, "refs") == 0) {
+			lua_pushnumber(L, cf->refs);
+			return 1;
+		} else if (strcmp(subkey, "key") == 0) {
+			lua_pushstring(L, cf->key);
+			return 1;
+		} else if (strcmp(subkey, "cmod.id") == 0) {
+			lua_pushnumber(L, cf->cmod->id);
+			return 1;
+		} else if (strcmp(subkey, "cmod.refs") == 0) {
+			lua_pushnumber(L, cf->cmod->refs);
+			return 1;
+		}
+	}
+	return 0;
+}
+
+/** Function representation for REPL (console). */
+static int
+lfunc_serialize(struct lua_State *L)
+{
+	struct cmod_func *cf = get_udata(L, uname_func);
+	if (cf == NULL) {
+		lua_pushnil(L);
+		return 1;
+	}
+
+	lua_createtable(L, 0, 1);
+	lua_pushstring(L, cmod_func_name(cf));
+	lua_setfield(L, -2, "name");
+	return 1;
+}
+
+/** Collect a function. */
+static int
+lfunc_gc(struct lua_State *L)
+{
+	struct cmod_func *cf = get_udata(L, uname_func);
+	if (cf != NULL) {
+		set_udata(L, uname_func, NULL);
+		cmod_func_unref(cf);
+	}
+	return 0;
+}
+
+/** Execute a function. */
+int
+do_lfunc_call(struct cmod_func *cf, struct port *args, struct port *ret)
+{
+	struct region *region = &fiber()->gc;
+	size_t region_svp = region_used(region);
+
+	uint32_t data_sz;
+	const char *data = port_get_msgpack(args, &data_sz);
+	if (data == NULL)
+		return -1;
+
+	port_c_create(ret);
+	box_function_ctx_t ctx = {
+		.port = ret,
+	};
+
+	/*
+	 * Unlike box.schema.func cmod doesn't support
+	 * module run-time reloading and while a function
+	 * is alive a module can't disappear. Still it is
+	 * unclear if there some existing users are running
+	 * code which already tries to unload a function inside
+	 * this call execution, thus make sure the module won't
+	 * disapper inbetween.
+	 */
+	cmod_ref(cf->cmod);
+	int rc = cf->addr(&ctx, data, data + data_sz);
+	cmod_unref(cf->cmod);
+	region_truncate(region, region_svp);
+
+	if (rc != 0) {
+		if (diag_last_error(&fiber()->diag) == NULL)
+			diag_set(ClientError, ER_PROC_C, "unknown error");
+		port_destroy(ret);
+		return -1;
+	}
+
+	return rc;
+}
+
+/** Call a function by its name from the Lua code. */
+static int
+lfunc_call(struct lua_State *L)
+{
+	struct cmod_func *cf = get_udata(L, uname_func);
+	if (cf == NULL) {
+		diag_set(IllegalParams, "The function is unloaded");
+		return luaT_error(L);
+	}
+
+	lua_State *args_L = luaT_newthread(tarantool_L);
+	if (args_L == NULL)
+		return luaT_error(L);
+
+	int coro_ref = luaL_ref(tarantool_L, LUA_REGISTRYINDEX);
+	lua_xmove(L, args_L, lua_gettop(L) - 1);
+
+	struct port args;
+	port_lua_create(&args, args_L);
+	((struct port_lua *)&args)->ref = coro_ref;
+
+	struct port ret;
+
+	if (do_lfunc_call(cf, &args, &ret) != 0) {
+		port_destroy(&args);
+		return luaT_error(L);
+	}
+
+	int top = lua_gettop(L);
+	port_dump_lua(&ret, L, true);
+	int cnt = lua_gettop(L) - top;
+
+	port_destroy(&ret);
+	port_destroy(&args);
+
+	return cnt;
+}
+
+/** Initialize cmod. */
+void
+box_lua_cmod_init(struct lua_State *L)
+{
+	cmod_func_hash = mh_strnptr_new();
+	if (cmod_func_hash == NULL)
+		panic("cmod: Can't allocate func hash table");
+	cmod_hash = mh_strnptr_new();
+	if (cmod_hash == NULL)
+		panic("cmod: Can't allocate cmod hash table");
+
+	(void)L;
+	static const struct luaL_Reg top_methods[] = {
+		{ "load",		lcmod_load		},
+		{ NULL, NULL },
+	};
+	luaL_register_module(L, "cmod", top_methods);
+	lua_pop(L, 1);
+
+	static const struct luaL_Reg lcmod_methods[] = {
+		{ "unload",		lcmod_unload		},
+		{ "load",		lcmod_load_func		},
+		{ "__index",		lcmod_index		},
+		{ "__serialize",	lcmod_serialize		},
+		{ "__gc",		lcmod_gc		},
+		{ NULL, NULL },
+	};
+	luaL_register_type(L, uname_cmod, lcmod_methods);
+
+	static const struct luaL_Reg lfunc_methods[] = {
+		{ "unload",		lfunc_unload		},
+		{ "__index",		lfunc_index		},
+		{ "__serialize",	lfunc_serialize		},
+		{ "__gc",		lfunc_gc		},
+		{ "__call",		lfunc_call		},
+		{ NULL, NULL },
+	};
+	luaL_register_type(L, uname_func, lfunc_methods);
+}
diff --git a/src/box/lua/cmod.h b/src/box/lua/cmod.h
new file mode 100644
index 000000000..224779a04
--- /dev/null
+++ b/src/box/lua/cmod.h
@@ -0,0 +1,123 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#pragma once
+
+#include <sys/types.h>
+#include <sys/stat.h>
+
+#include "trivia/config.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+struct lua_State;
+
+/**
+ * Attributes for cmod cache invalidation.
+ */
+struct cmod_attr {
+#ifdef TARGET_OS_DARWIN
+	struct timespec st_mtimespec;
+#else
+	struct timespec st_mtim;
+#endif
+	dev_t st_dev;
+	ino_t st_ino;
+	off_t st_size;
+};
+
+/**
+ * Shared library module.
+ */
+struct cmod {
+	/** Module dlhandle. */
+	void *handle;
+	/** Module ID. */
+	int64_t id;
+	/** Number of references. */
+	int64_t refs;
+	/** File attributes. */
+	struct cmod_attr attr;
+	/** Length of @a package. */
+	size_t package_len;
+	/** Path to the module. */
+	char package[0];
+};
+
+/** Increase reference to cmod. */
+void
+cmod_ref(struct cmod *m);
+
+/** Decrease reference to cmod and free it if last one. */
+void
+cmod_unref(struct cmod *m);
+
+/**
+ * Lookup for cmod entry in cache.
+ *
+ * @param package package name.
+ * @param package_len length of @a package.
+ *
+ * @returns module pointer if found, NULL otherwise.
+ */
+struct cmod *
+cmod_cache_find(const char *package, size_t package_len);
+
+/**
+ * Put new cmod entry to cache.
+ *
+ * @param cmod entry to put.
+ *
+ * @returns 0 on success, -1 otherwise (diag is set).
+ */
+int
+cmod_cache_put(struct cmod *m);
+
+/**
+ * Allocate and load a new C module instance.
+ *
+ * Allocates a new C module instance, copies shared library
+ * to a safe place, loads it and remove then leaving DSO purely
+ * in memory. This is done because libc doesn't detect file
+ * updates properly. The module get cached by putting it into
+ * the modules hash.
+ *
+ * @param package package name.
+ * @param package_len length of @a package.
+ * @param source_path path to the shared library.
+ *
+ * @returns module pointer on succes, NULL otherwise, diag is set.
+ */
+struct cmod *
+cmod_new(const char *package, size_t package_len, const char *source_path);
+
+/**
+ * Find package in Lua's "package.cpath".
+ *
+ * @param package package name.
+ * @param package_len length of @package.
+ * @param path resolved path.
+ * @param path_len length of @a path.
+ *
+ * @return 0 on success, -1 otherwise (diag is set).
+ */
+int
+cmod_find_package(const char *package, size_t package_len,
+		  char *path, size_t path_len);
+
+/**
+ * Initialize cmod Lua module.
+ *
+ * @param L Lua state where to register the cmod module.
+ */
+void
+box_lua_cmod_init(struct lua_State *L);
+
+#if defined(__cplusplus)
+}
+#endif /* defined(__plusplus) */
diff --git a/src/box/lua/init.c b/src/box/lua/init.c
index fbcdfb20b..bad2b7ca9 100644
--- a/src/box/lua/init.c
+++ b/src/box/lua/init.c
@@ -60,6 +60,7 @@
 #include "box/lua/cfg.h"
 #include "box/lua/xlog.h"
 #include "box/lua/console.h"
+#include "box/lua/cmod.h"
 #include "box/lua/tuple.h"
 #include "box/lua/execute.h"
 #include "box/lua/key_def.h"
@@ -465,6 +466,7 @@ box_lua_init(struct lua_State *L)
 	box_lua_tuple_init(L);
 	box_lua_call_init(L);
 	box_lua_cfg_init(L);
+	box_lua_cmod_init(L);
 	box_lua_slab_init(L);
 	box_lua_index_init(L);
 	box_lua_space_init(L);
-- 
2.29.2

[Tarantool-patches] [PATCH v18 2/2] test: box/cfunc -- add cmod test

[Cyrill Gorcunov via Tarantool-patches]

Note that the test is disabled in suite.ini for a while
because we need to update test-run first like

 | diff --git a/pretest_clean.lua b/pretest_clean.lua
 | index 9b5ac9d..b0280c4 100644
 | --- a/pretest_clean.lua
 | +++ b/pretest_clean.lua
 | @@ -272,6 +272,7 @@ local function clean()
 |          package = true,
 |          pickle = true,
 |          popen = true,
 | +        cmod = true,
 |          pwd = true,
 |          socket = true,
 |          strict = true,

Part-of #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
---
 test/box/CMakeLists.txt |   4 +
 test/box/cfunc1.c       |  58 +++++
 test/box/cfunc2.c       | 137 +++++++++++
 test/box/cfunc3.c       |  25 ++
 test/box/cfunc4.c       |  28 +++
 test/box/cmod.result    | 527 ++++++++++++++++++++++++++++++++++++++++
 test/box/cmod.test.lua  | 200 +++++++++++++++
 test/box/suite.ini      |   2 +-
 8 files changed, 980 insertions(+), 1 deletion(-)
 create mode 100644 test/box/cfunc1.c
 create mode 100644 test/box/cfunc2.c
 create mode 100644 test/box/cfunc3.c
 create mode 100644 test/box/cfunc4.c
 create mode 100644 test/box/cmod.result
 create mode 100644 test/box/cmod.test.lua

diff --git a/test/box/CMakeLists.txt b/test/box/CMakeLists.txt
index 06bfbbe9d..939353981 100644
--- a/test/box/CMakeLists.txt
+++ b/test/box/CMakeLists.txt
@@ -3,3 +3,7 @@ build_module(function1 function1.c)
 build_module(reload1 reload1.c)
 build_module(reload2 reload2.c)
 build_module(tuple_bench tuple_bench.c)
+build_module(cfunc1 cfunc1.c)
+build_module(cfunc2 cfunc2.c)
+build_module(cfunc3 cfunc3.c)
+build_module(cfunc4 cfunc4.c)
diff --git a/test/box/cfunc1.c b/test/box/cfunc1.c
new file mode 100644
index 000000000..f6829372a
--- /dev/null
+++ b/test/box/cfunc1.c
@@ -0,0 +1,58 @@
+#include <stdio.h>
+#include <stdbool.h>
+#include <msgpuck.h>
+
+#include "module.h"
+
+/*
+ * Before the reload functions are just declared
+ * and simply exit with zero.
+ *
+ * After the module reload we should provide real
+ * functionality.
+ */
+
+int
+cfunc_nop(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
+
+int
+cfunc_fetch_seq_evens(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
+
+int
+cfunc_multireturn(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
+
+int
+cfunc_args(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
+
+int
+cfunc_sum(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
diff --git a/test/box/cfunc2.c b/test/box/cfunc2.c
new file mode 100644
index 000000000..8c583e993
--- /dev/null
+++ b/test/box/cfunc2.c
@@ -0,0 +1,137 @@
+#include <stdio.h>
+#include <stdbool.h>
+#include <msgpuck.h>
+
+#include "module.h"
+
+/*
+ * Just make sure we've been called.
+ */
+int
+cfunc_nop(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	(void)ctx;
+	(void)args;
+	(void)args_end;
+	return 0;
+}
+
+/*
+ * Fetch first N even numbers (just to make sure the order of
+ * arguments is not screwed).
+ */
+int
+cfunc_fetch_seq_evens(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	int arg_count = mp_decode_array(&args);
+	if (arg_count != 1) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid argument count");
+	}
+	int field_count = mp_decode_array(&args);
+	if (field_count < 1) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid array size");
+	}
+
+	/*
+	 * We expect even numbers sequence here. The idea is
+	 * to test invalid data an issue an error from inside
+	 * of C function.
+	 */
+	for (int i = 1; i <= field_count; i++) {
+		int val = mp_decode_uint(&args);
+		int needed = 2 * i;
+		if (val != needed) {
+			char res[128];
+			snprintf(res, sizeof(res), "%s %d != %d",
+				 "invalid argument", val, needed);
+			return box_error_set(__FILE__, __LINE__,
+					     ER_PROC_C, "%s", res);
+		}
+	}
+
+	return 0;
+}
+
+/*
+ * Return one element array twice.
+ */
+int
+cfunc_multireturn(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	char tuple_buf[512];
+	char *d = tuple_buf;
+	d = mp_encode_array(d, 1);
+	d = mp_encode_uint(d, 1);
+	assert(d <= tuple_buf + sizeof(tuple_buf));
+
+	box_tuple_format_t *fmt = box_tuple_format_default();
+	box_tuple_t *tuple_a = box_tuple_new(fmt, tuple_buf, d);
+	if (tuple_a == NULL)
+		return -1;
+	int rc = box_return_tuple(ctx, tuple_a);
+	if (rc == 0)
+		return box_return_tuple(ctx, tuple_a);
+	return rc;
+}
+
+/*
+ * Encode int + string pair back.
+ */
+int
+cfunc_args(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	uint32_t arg_count = mp_decode_array(&args);
+	if (arg_count != 2) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid argument count");
+	}
+
+	if (mp_typeof(*args) != MP_UINT) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "tuple field must be uint");
+	}
+	uint32_t num = mp_decode_uint(&args);
+
+	if (mp_typeof(*args) != MP_STR) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "tuple field must be string");
+	}
+	const char *str = args;
+	uint32_t len = mp_decode_strl(&str);
+
+	char tuple_buf[512];
+	char *d = tuple_buf;
+	d = mp_encode_array(d, 2);
+	d = mp_encode_uint(d, num);
+	d = mp_encode_str(d, str, len);
+	assert(d <= tuple_buf + sizeof(tuple_buf));
+
+	box_tuple_format_t *fmt = box_tuple_format_default();
+	box_tuple_t *tuple = box_tuple_new(fmt, tuple_buf, d);
+	if (tuple == NULL)
+		return -1;
+
+	return box_return_tuple(ctx, tuple);
+}
+
+/*
+ * Sum two integers.
+ */
+int
+cfunc_sum(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	uint32_t arg_count = mp_decode_array(&args);
+	if (arg_count != 2) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid argument count");
+	}
+	uint64_t a = mp_decode_uint(&args);
+	uint64_t b = mp_decode_uint(&args);
+
+	char res[16];
+	char *end = mp_encode_uint(res, a + b);
+	box_return_mp(ctx, res, end);
+	return 0;
+}
diff --git a/test/box/cfunc3.c b/test/box/cfunc3.c
new file mode 100644
index 000000000..668790fbf
--- /dev/null
+++ b/test/box/cfunc3.c
@@ -0,0 +1,25 @@
+#include <stdio.h>
+#include <stdbool.h>
+#include <msgpuck.h>
+
+#include "module.h"
+
+/*
+ * Sum two integers.
+ */
+int
+cfunc_add(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	uint32_t arg_count = mp_decode_array(&args);
+	if (arg_count != 2) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid argument count");
+	}
+	uint64_t a = mp_decode_uint(&args);
+	uint64_t b = mp_decode_uint(&args);
+
+	char res[16];
+	char *end = mp_encode_uint(res, a + b);
+	box_return_mp(ctx, res, end);
+	return 0;
+}
diff --git a/test/box/cfunc4.c b/test/box/cfunc4.c
new file mode 100644
index 000000000..cc079b335
--- /dev/null
+++ b/test/box/cfunc4.c
@@ -0,0 +1,28 @@
+#include <stdio.h>
+#include <stdbool.h>
+#include <msgpuck.h>
+
+#include "module.h"
+
+/*
+ * Sum two integers and add a constant,
+ * so that result will be different after
+ * the reload.
+ */
+int
+cfunc_add(box_function_ctx_t *ctx, const char *args, const char *args_end)
+{
+	const uint32_t delta = 10;
+	uint32_t arg_count = mp_decode_array(&args);
+	if (arg_count != 2) {
+		return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s",
+				     "invalid argument count");
+	}
+	uint64_t a = mp_decode_uint(&args);
+	uint64_t b = mp_decode_uint(&args);
+
+	char res[16];
+	char *end = mp_encode_uint(res, a + b + delta);
+	box_return_mp(ctx, res, end);
+	return 0;
+}
diff --git a/test/box/cmod.result b/test/box/cmod.result
new file mode 100644
index 000000000..257e60d2c
--- /dev/null
+++ b/test/box/cmod.result
@@ -0,0 +1,527 @@
+-- test-run result file version 2
+--
+-- gh-4642: New cmod module to be able to
+-- run C stored functions on read only nodes
+-- without requirement to register them with
+-- box.schema.func help.
+--
+build_path = os.getenv("BUILDDIR")
+ | ---
+ | ...
+package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath
+ | ---
+ | ...
+
+cmod = require('cmod')
+ | ---
+ | ...
+fio = require('fio')
+ | ---
+ | ...
+
+ext = (jit.os == "OSX" and "dylib" or "so")
+ | ---
+ | ...
+
+cfunc_path = fio.pathjoin(build_path, "test/box/cfunc.") .. ext
+ | ---
+ | ...
+cfunc1_path = fio.pathjoin(build_path, "test/box/cfunc1.") .. ext
+ | ---
+ | ...
+cfunc2_path = fio.pathjoin(build_path, "test/box/cfunc2.") .. ext
+ | ---
+ | ...
+cfunc3_path = fio.pathjoin(build_path, "test/box/cfunc3.") .. ext
+ | ---
+ | ...
+cfunc4_path = fio.pathjoin(build_path, "test/box/cfunc4.") .. ext
+ | ---
+ | ...
+
+_ = pcall(fio.unlink(cfunc_path))
+ | ---
+ | ...
+fio.symlink(cfunc1_path, cfunc_path)
+ | ---
+ | - true
+ | ...
+
+_, err = pcall(cmod.load, 'non-such-module')
+ | ---
+ | ...
+assert(err ~= nil)
+ | ---
+ | - true
+ | ...
+
+-- All functions are sitting in cfunc.so.
+old_module = cmod.load('cfunc')
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 1)
+ | ---
+ | - true
+ | ...
+old_module_copy = cmod.load('cfunc')
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 2)
+ | ---
+ | - true
+ | ...
+assert(old_module_copy['tt_dev.refs'] == 2)
+ | ---
+ | - true
+ | ...
+old_module_copy:unload()
+ | ---
+ | - true
+ | ...
+assert(old_module['tt_dev.refs'] == 1)
+ | ---
+ | - true
+ | ...
+old_cfunc_nop = old_module:load('cfunc_nop')
+ | ---
+ | ...
+old_cfunc_fetch_seq_evens = old_module:load('cfunc_fetch_seq_evens')
+ | ---
+ | ...
+old_cfunc_multireturn = old_module:load('cfunc_multireturn')
+ | ---
+ | ...
+old_cfunc_args = old_module:load('cfunc_args')
+ | ---
+ | ...
+old_cfunc_sum = old_module:load('cfunc_sum')
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 6)
+ | ---
+ | - true
+ | ...
+
+-- Test for error on nonexisting function.
+_, err = pcall(old_module.load, old_module, 'no-such-func')
+ | ---
+ | ...
+assert(err ~= nil)
+ | ---
+ | - true
+ | ...
+
+-- Make sure they all are callable.
+old_cfunc_nop()
+ | ---
+ | ...
+old_cfunc_fetch_seq_evens()
+ | ---
+ | ...
+old_cfunc_multireturn()
+ | ---
+ | ...
+old_cfunc_args()
+ | ---
+ | ...
+old_cfunc_sum()
+ | ---
+ | ...
+
+-- Unload the module but keep old functions alive, so
+-- they keep reference to NOP module internally
+-- and still callable.
+old_module:unload()
+ | ---
+ | - true
+ | ...
+-- Test refs via function name.
+assert(old_cfunc_nop['tt_dev.cmod.refs'] == 5)
+ | ---
+ | - true
+ | ...
+old_cfunc_nop()
+ | ---
+ | ...
+old_cfunc_fetch_seq_evens()
+ | ---
+ | ...
+old_cfunc_multireturn()
+ | ---
+ | ...
+old_cfunc_args()
+ | ---
+ | ...
+old_cfunc_sum()
+ | ---
+ | ...
+
+-- The module is unloaded I should not be able
+-- to load new shared library.
+old_module:load('cfunc')
+ | ---
+ | - error: Expects function = module:load('name') but not module object passed
+ | ...
+-- Neither I should be able to unload module twise.
+old_module:unload()
+ | ---
+ | - error: The module is unloaded
+ | ...
+
+-- Clean old functions.
+old_cfunc_nop:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_fetch_seq_evens:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_multireturn:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_args:unload()
+ | ---
+ | - true
+ | ...
+assert(old_cfunc_sum['tt_dev.cmod.refs'] == 1)
+ | ---
+ | - true
+ | ...
+old_cfunc_sum:unload()
+ | ---
+ | - true
+ | ...
+
+-- And reload old module again.
+old_module = cmod.load('cfunc')
+ | ---
+ | ...
+old_module_id = old_module['tt_dev.id']
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 1)
+ | ---
+ | - true
+ | ...
+
+-- Overwrite module with new contents.
+_ = pcall(fio.unlink(cfunc_path))
+ | ---
+ | ...
+fio.symlink(cfunc2_path, cfunc_path)
+ | ---
+ | - true
+ | ...
+
+-- Load new module, cache should be updated.
+new_module = cmod.load('cfunc')
+ | ---
+ | ...
+new_module_id = new_module['tt_dev.id']
+ | ---
+ | ...
+
+-- Old and new module keep one reference with
+-- different IDs.
+assert(old_module['tt_dev.refs'] == 1)
+ | ---
+ | - true
+ | ...
+assert(old_module['tt_dev.refs'] == new_module['tt_dev.refs'])
+ | ---
+ | - true
+ | ...
+assert(old_module_id ~= new_module_id)
+ | ---
+ | - true
+ | ...
+
+-- All functions from old module should be loadable.
+old_cfunc_nop = old_module:load('cfunc_nop')
+ | ---
+ | ...
+old_cfunc_fetch_seq_evens = old_module:load('cfunc_fetch_seq_evens')
+ | ---
+ | ...
+old_cfunc_multireturn = old_module:load('cfunc_multireturn')
+ | ---
+ | ...
+old_cfunc_args = old_module:load('cfunc_args')
+ | ---
+ | ...
+old_cfunc_sum = old_module:load('cfunc_sum')
+ | ---
+ | ...
+assert(old_cfunc_nop['tt_dev.cmod.id'] == old_module_id)
+ | ---
+ | - true
+ | ...
+assert(old_cfunc_fetch_seq_evens['tt_dev.cmod.id'] == old_module_id)
+ | ---
+ | - true
+ | ...
+assert(old_cfunc_multireturn['tt_dev.cmod.id'] == old_module_id)
+ | ---
+ | - true
+ | ...
+assert(old_cfunc_args['tt_dev.cmod.id'] == old_module_id)
+ | ---
+ | - true
+ | ...
+assert(old_cfunc_sum['tt_dev.cmod.id'] == old_module_id)
+ | ---
+ | - true
+ | ...
+assert(old_module['tt_dev.refs'] == 6)
+ | ---
+ | - true
+ | ...
+
+-- Lookup for updated symbols.
+new_cfunc_nop = new_module:load('cfunc_nop')
+ | ---
+ | ...
+new_cfunc_fetch_seq_evens = new_module:load('cfunc_fetch_seq_evens')
+ | ---
+ | ...
+new_cfunc_multireturn = new_module:load('cfunc_multireturn')
+ | ---
+ | ...
+new_cfunc_args = new_module:load('cfunc_args')
+ | ---
+ | ...
+new_cfunc_sum = new_module:load('cfunc_sum')
+ | ---
+ | ...
+assert(new_cfunc_nop['tt_dev.cmod.id'] == new_module_id)
+ | ---
+ | - true
+ | ...
+assert(new_cfunc_fetch_seq_evens['tt_dev.cmod.id'] == new_module_id)
+ | ---
+ | - true
+ | ...
+assert(new_cfunc_multireturn['tt_dev.cmod.id'] == new_module_id)
+ | ---
+ | - true
+ | ...
+assert(new_cfunc_args['tt_dev.cmod.id'] == new_module_id)
+ | ---
+ | - true
+ | ...
+assert(new_cfunc_sum['tt_dev.cmod.id'] == new_module_id)
+ | ---
+ | - true
+ | ...
+assert(new_module['tt_dev.refs'] == 6)
+ | ---
+ | - true
+ | ...
+
+-- Call old functions.
+old_cfunc_nop()
+ | ---
+ | ...
+old_cfunc_fetch_seq_evens()
+ | ---
+ | ...
+old_cfunc_multireturn()
+ | ---
+ | ...
+old_cfunc_args()
+ | ---
+ | ...
+old_cfunc_sum()
+ | ---
+ | ...
+
+-- Call new functions.
+new_cfunc_nop()
+ | ---
+ | ...
+new_cfunc_multireturn()
+ | ---
+ | - [1]
+ | - [1]
+ | ...
+new_cfunc_fetch_seq_evens({2,4,6})
+ | ---
+ | ...
+new_cfunc_fetch_seq_evens({1,2,3})  -- error, odd numbers sequence
+ | ---
+ | - error: invalid argument 1 != 2
+ | ...
+new_cfunc_args(1, "hello")
+ | ---
+ | - [1, 'hello']
+ | ...
+new_cfunc_sum(1) -- error, one arg passed
+ | ---
+ | - error: invalid argument count
+ | ...
+new_cfunc_sum(1,2)
+ | ---
+ | - 3
+ | ...
+
+-- Cleanup old module's functions.
+old_cfunc_nop:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_fetch_seq_evens:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_multireturn:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_args:unload()
+ | ---
+ | - true
+ | ...
+old_cfunc_sum:unload()
+ | ---
+ | - true
+ | ...
+old_module:unload()
+ | ---
+ | - true
+ | ...
+
+-- Cleanup new module data.
+new_cfunc_nop:unload()
+ | ---
+ | - true
+ | ...
+new_cfunc_multireturn:unload()
+ | ---
+ | - true
+ | ...
+new_cfunc_fetch_seq_evens:unload()
+ | ---
+ | - true
+ | ...
+new_cfunc_args:unload()
+ | ---
+ | - true
+ | ...
+new_cfunc_sum:unload()
+ | ---
+ | - true
+ | ...
+new_module:unload()
+ | ---
+ | - true
+ | ...
+
+-- Cleanup the generated symlink.
+_ = pcall(fio.unlink(cfunc_path))
+ | ---
+ | ...
+
+-- Test double hashing: create function
+-- in box.schema.fun so that it should
+-- appear in cmod hash.
+fio.symlink(cfunc3_path, cfunc_path)
+ | ---
+ | - true
+ | ...
+box.schema.func.create('cfunc.cfunc_add', {language = "C"})
+ | ---
+ | ...
+box.schema.user.grant('guest', 'execute', 'function', 'cfunc.cfunc_add')
+ | ---
+ | ...
+box.func['cfunc.cfunc_add']:call({1,2})
+ | ---
+ | - 3
+ | ...
+
+old_module = cmod.load('cfunc')
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 2) -- one for cmod + box instance
+ | ---
+ | - true
+ | ...
+old_func = old_module:load('cfunc_add')
+ | ---
+ | ...
+assert(old_module['tt_dev.refs'] == 3) -- plus function instance
+ | ---
+ | - true
+ | ...
+old_func(1,2)
+ | ---
+ | - 3
+ | ...
+
+-- Now update on disk and reload the module.
+_ = pcall(fio.unlink(cfunc_path))
+ | ---
+ | ...
+fio.symlink(cfunc4_path, cfunc_path)
+ | ---
+ | - true
+ | ...
+
+box.schema.func.reload("cfunc")
+ | ---
+ | ...
+box.func['cfunc.cfunc_add']:call({1,2})
+ | ---
+ | - 13
+ | ...
+
+-- The cmod instance should carry own
+-- references to the module and old
+-- function. And reloading must not
+-- affect old functions.
+assert(old_module['tt_dev.refs'] == 2)
+ | ---
+ | - true
+ | ...
+old_func(1,2)
+ | ---
+ | - 3
+ | ...
+old_func:unload()
+ | ---
+ | - true
+ | ...
+old_module:unload()
+ | ---
+ | - true
+ | ...
+
+-- Same time the reload should not
+-- affect cmod cache.
+new_module = cmod.load('cfunc')
+ | ---
+ | ...
+assert(new_module['tt_dev.refs'] == 1)
+ | ---
+ | - true
+ | ...
+
+-- Box function should carry own module.
+box.func['cfunc.cfunc_add']:call({1,2})
+ | ---
+ | - 13
+ | ...
+
+-- Cleanup.
+_ = pcall(fio.unlink(cfunc_path))
+ | ---
+ | ...
+new_module:unload()
+ | ---
+ | - true
+ | ...
+box.schema.func.drop('cfunc.cfunc_add')
+ | ---
+ | ...
diff --git a/test/box/cmod.test.lua b/test/box/cmod.test.lua
new file mode 100644
index 000000000..e22c84534
--- /dev/null
+++ b/test/box/cmod.test.lua
@@ -0,0 +1,200 @@
+--
+-- gh-4642: New cmod module to be able to
+-- run C stored functions on read only nodes
+-- without requirement to register them with
+-- box.schema.func help.
+--
+build_path = os.getenv("BUILDDIR")
+package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath
+
+cmod = require('cmod')
+fio = require('fio')
+
+ext = (jit.os == "OSX" and "dylib" or "so")
+
+cfunc_path = fio.pathjoin(build_path, "test/box/cfunc.") .. ext
+cfunc1_path = fio.pathjoin(build_path, "test/box/cfunc1.") .. ext
+cfunc2_path = fio.pathjoin(build_path, "test/box/cfunc2.") .. ext
+cfunc3_path = fio.pathjoin(build_path, "test/box/cfunc3.") .. ext
+cfunc4_path = fio.pathjoin(build_path, "test/box/cfunc4.") .. ext
+
+_ = pcall(fio.unlink(cfunc_path))
+fio.symlink(cfunc1_path, cfunc_path)
+
+_, err = pcall(cmod.load, 'non-such-module')
+assert(err ~= nil)
+
+-- All functions are sitting in cfunc.so.
+old_module = cmod.load('cfunc')
+assert(old_module['tt_dev.refs'] == 1)
+old_module_copy = cmod.load('cfunc')
+assert(old_module['tt_dev.refs'] == 2)
+assert(old_module_copy['tt_dev.refs'] == 2)
+old_module_copy:unload()
+assert(old_module['tt_dev.refs'] == 1)
+old_cfunc_nop = old_module:load('cfunc_nop')
+old_cfunc_fetch_seq_evens = old_module:load('cfunc_fetch_seq_evens')
+old_cfunc_multireturn = old_module:load('cfunc_multireturn')
+old_cfunc_args = old_module:load('cfunc_args')
+old_cfunc_sum = old_module:load('cfunc_sum')
+assert(old_module['tt_dev.refs'] == 6)
+
+-- Test for error on nonexisting function.
+_, err = pcall(old_module.load, old_module, 'no-such-func')
+assert(err ~= nil)
+
+-- Make sure they all are callable.
+old_cfunc_nop()
+old_cfunc_fetch_seq_evens()
+old_cfunc_multireturn()
+old_cfunc_args()
+old_cfunc_sum()
+
+-- Unload the module but keep old functions alive, so
+-- they keep reference to NOP module internally
+-- and still callable.
+old_module:unload()
+-- Test refs via function name.
+assert(old_cfunc_nop['tt_dev.cmod.refs'] == 5)
+old_cfunc_nop()
+old_cfunc_fetch_seq_evens()
+old_cfunc_multireturn()
+old_cfunc_args()
+old_cfunc_sum()
+
+-- The module is unloaded I should not be able
+-- to load new shared library.
+old_module:load('cfunc')
+-- Neither I should be able to unload module twise.
+old_module:unload()
+
+-- Clean old functions.
+old_cfunc_nop:unload()
+old_cfunc_fetch_seq_evens:unload()
+old_cfunc_multireturn:unload()
+old_cfunc_args:unload()
+assert(old_cfunc_sum['tt_dev.cmod.refs'] == 1)
+old_cfunc_sum:unload()
+
+-- And reload old module again.
+old_module = cmod.load('cfunc')
+old_module_id = old_module['tt_dev.id']
+assert(old_module['tt_dev.refs'] == 1)
+
+-- Overwrite module with new contents.
+_ = pcall(fio.unlink(cfunc_path))
+fio.symlink(cfunc2_path, cfunc_path)
+
+-- Load new module, cache should be updated.
+new_module = cmod.load('cfunc')
+new_module_id = new_module['tt_dev.id']
+
+-- Old and new module keep one reference with
+-- different IDs.
+assert(old_module['tt_dev.refs'] == 1)
+assert(old_module['tt_dev.refs'] == new_module['tt_dev.refs'])
+assert(old_module_id ~= new_module_id)
+
+-- All functions from old module should be loadable.
+old_cfunc_nop = old_module:load('cfunc_nop')
+old_cfunc_fetch_seq_evens = old_module:load('cfunc_fetch_seq_evens')
+old_cfunc_multireturn = old_module:load('cfunc_multireturn')
+old_cfunc_args = old_module:load('cfunc_args')
+old_cfunc_sum = old_module:load('cfunc_sum')
+assert(old_cfunc_nop['tt_dev.cmod.id'] == old_module_id)
+assert(old_cfunc_fetch_seq_evens['tt_dev.cmod.id'] == old_module_id)
+assert(old_cfunc_multireturn['tt_dev.cmod.id'] == old_module_id)
+assert(old_cfunc_args['tt_dev.cmod.id'] == old_module_id)
+assert(old_cfunc_sum['tt_dev.cmod.id'] == old_module_id)
+assert(old_module['tt_dev.refs'] == 6)
+
+-- Lookup for updated symbols.
+new_cfunc_nop = new_module:load('cfunc_nop')
+new_cfunc_fetch_seq_evens = new_module:load('cfunc_fetch_seq_evens')
+new_cfunc_multireturn = new_module:load('cfunc_multireturn')
+new_cfunc_args = new_module:load('cfunc_args')
+new_cfunc_sum = new_module:load('cfunc_sum')
+assert(new_cfunc_nop['tt_dev.cmod.id'] == new_module_id)
+assert(new_cfunc_fetch_seq_evens['tt_dev.cmod.id'] == new_module_id)
+assert(new_cfunc_multireturn['tt_dev.cmod.id'] == new_module_id)
+assert(new_cfunc_args['tt_dev.cmod.id'] == new_module_id)
+assert(new_cfunc_sum['tt_dev.cmod.id'] == new_module_id)
+assert(new_module['tt_dev.refs'] == 6)
+
+-- Call old functions.
+old_cfunc_nop()
+old_cfunc_fetch_seq_evens()
+old_cfunc_multireturn()
+old_cfunc_args()
+old_cfunc_sum()
+
+-- Call new functions.
+new_cfunc_nop()
+new_cfunc_multireturn()
+new_cfunc_fetch_seq_evens({2,4,6})
+new_cfunc_fetch_seq_evens({1,2,3})  -- error, odd numbers sequence
+new_cfunc_args(1, "hello")
+new_cfunc_sum(1) -- error, one arg passed
+new_cfunc_sum(1,2)
+
+-- Cleanup old module's functions.
+old_cfunc_nop:unload()
+old_cfunc_fetch_seq_evens:unload()
+old_cfunc_multireturn:unload()
+old_cfunc_args:unload()
+old_cfunc_sum:unload()
+old_module:unload()
+
+-- Cleanup new module data.
+new_cfunc_nop:unload()
+new_cfunc_multireturn:unload()
+new_cfunc_fetch_seq_evens:unload()
+new_cfunc_args:unload()
+new_cfunc_sum:unload()
+new_module:unload()
+
+-- Cleanup the generated symlink.
+_ = pcall(fio.unlink(cfunc_path))
+
+-- Test double hashing: create function
+-- in box.schema.fun so that it should
+-- appear in cmod hash.
+fio.symlink(cfunc3_path, cfunc_path)
+box.schema.func.create('cfunc.cfunc_add', {language = "C"})
+box.schema.user.grant('guest', 'execute', 'function', 'cfunc.cfunc_add')
+box.func['cfunc.cfunc_add']:call({1,2})
+
+old_module = cmod.load('cfunc')
+assert(old_module['tt_dev.refs'] == 2) -- one for cmod + box instance
+old_func = old_module:load('cfunc_add')
+assert(old_module['tt_dev.refs'] == 3) -- plus function instance
+old_func(1,2)
+
+-- Now update on disk and reload the module.
+_ = pcall(fio.unlink(cfunc_path))
+fio.symlink(cfunc4_path, cfunc_path)
+
+box.schema.func.reload("cfunc")
+box.func['cfunc.cfunc_add']:call({1,2})
+
+-- The cmod instance should carry own
+-- references to the module and old
+-- function. And reloading must not
+-- affect old functions.
+assert(old_module['tt_dev.refs'] == 2)
+old_func(1,2)
+old_func:unload()
+old_module:unload()
+
+-- Same time the reload should not
+-- affect cmod cache.
+new_module = cmod.load('cfunc')
+assert(new_module['tt_dev.refs'] == 1)
+
+-- Box function should carry own module.
+box.func['cfunc.cfunc_add']:call({1,2})
+
+-- Cleanup.
+_ = pcall(fio.unlink(cfunc_path))
+new_module:unload()
+box.schema.func.drop('cfunc.cfunc_add')
diff --git a/test/box/suite.ini b/test/box/suite.ini
index e700d0b9e..fc16c5951 100644
--- a/test/box/suite.ini
+++ b/test/box/suite.ini
@@ -2,7 +2,7 @@
 core = tarantool
 description = Database tests
 script = box.lua
-disabled = rtree_errinj.test.lua tuple_bench.test.lua
+disabled = rtree_errinj.test.lua tuple_bench.test.lua cmod.test.lua
 long_run = huge_field_map_long.test.lua
 config = engine.cfg
 release_disabled = errinj.test.lua errinj_index.test.lua rtree_errinj.test.lua upsert_errinj.test.lua iproto_stress.test.lua gh-4648-func-load-unload.test.lua
-- 
2.29.2

[Tarantool-patches] [PATCH 0/4] cmod: refactor box/func

[Cyrill Gorcunov via Tarantool-patches]

Add more use of cmod code in box.schema.func related
functionality. This allows us to shrink code a bit.

Note the series is on top of

 > [PATCH v18 0/2] box: implement cmod Lua module
 > https://lists.tarantool.org/tarantool-patches/20210217123945.396396-1-gorcunov@gmail.com/

thus not on plain master branch code. They are sitting
in branch gorcunov/gh-4642-func-ro-18-unify

Cyrill Gorcunov (4):
  box/func: drop redundant module argument
  box/func: update module cache in one pass
  cmod: export cmod_call helper
  box/func: simplify func_c_call

 src/box/call.c     | 11 +++-------
 src/box/func.c     | 55 ++++++++++++++++++----------------------------
 src/box/func.h     |  3 +--
 src/box/lua/cmod.c | 24 ++++++++------------
 src/box/lua/cmod.h | 15 +++++++++++++
 5 files changed, 49 insertions(+), 59 deletions(-)


base-commit: 99b13539c8534460bf99e5a90c8865a35413dbbd
-- 
2.29.2

[Tarantool-patches] [PATCH 1/4] box/func: drop redundant module argument

[Cyrill Gorcunov via Tarantool-patches]

This argument is only needed if there is
no such module present. Just setup error
inside module_reload function.

Part-of #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
---
 src/box/call.c | 11 +++--------
 src/box/func.c |  8 +++-----
 src/box/func.h |  3 +--
 3 files changed, 7 insertions(+), 15 deletions(-)

diff --git a/src/box/call.c b/src/box/call.c
index 9c291260e..bffd17c5a 100644
--- a/src/box/call.c
+++ b/src/box/call.c
@@ -128,14 +128,9 @@ box_module_reload(const char *name)
 				 user->def->name);
 		return -1;
 	}
-	struct module *module = NULL;
-	if (module_reload(name, name + strlen(name), &module) == 0) {
-		if (module != NULL)
-			return 0;
-		else
-			diag_set(ClientError, ER_NO_SUCH_MODULE, name);
-	}
-	return -1;
+	if (module_reload(name, name + strlen(name)) != 0)
+		return -1;
+	return 0;
 }
 
 int
diff --git a/src/box/func.c b/src/box/func.c
index f77cf00f8..0d552652d 100644
--- a/src/box/func.c
+++ b/src/box/func.c
@@ -259,13 +259,12 @@ module_sym(struct module *module, const char *name)
 }
 
 int
-module_reload(const char *package, const char *package_end, struct module **module)
+module_reload(const char *package, const char *package_end)
 {
 	struct module *old_module = module_cache_find(package, package_end);
 	if (old_module == NULL) {
-		/* Module wasn't loaded - do nothing. */
-		*module = NULL;
-		return 0;
+		diag_set(ClientError, ER_NO_SUCH_MODULE, package);
+		return -1;
 	}
 
 	struct module *new_module = module_load(package, package_end);
@@ -286,7 +285,6 @@ module_reload(const char *package, const char *package_end, struct module **modu
 	if (module_cache_put(new_module) != 0)
 		goto restore;
 	module_gc(old_module);
-	*module = new_module;
 	return 0;
 restore:
 	/*
diff --git a/src/box/func.h b/src/box/func.h
index ac9772dfe..c6861df8e 100644
--- a/src/box/func.h
+++ b/src/box/func.h
@@ -112,12 +112,11 @@ func_call(struct func *func, struct port *args, struct port *ret);
  *
  * @param package name begin pointer.
  * @param package_end package_end name end pointer.
- * @param[out] module a pointer to store module object on success.
  * @retval -1 on error.
  * @retval 0 on success.
  */
 int
-module_reload(const char *package, const char *package_end, struct module **module);
+module_reload(const char *package, const char *package_end);
 
 #if defined(__cplusplus)
 } /* extern "C" */
-- 
2.29.2

[Tarantool-patches] [PATCH 2/4] box/func: update module cache in one pass

[Cyrill Gorcunov via Tarantool-patches]

When module get reloaded we can update the
cache entry directly instead of delete+add
pair.

Part-of #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
---
 src/box/func.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/src/box/func.c b/src/box/func.c
index 0d552652d..5757f7c25 100644
--- a/src/box/func.c
+++ b/src/box/func.c
@@ -159,6 +159,21 @@ module_cache_put(struct module *module)
 	return 0;
 }
 
+/**
+ * Update the module cache.
+ */
+static void
+module_cache_update(struct module *module)
+{
+	const char *package = module->cmod->package;
+	size_t package_len = module->cmod->package_len;
+	mh_int_t i = mh_strnptr_find_inp(modules, package, package_len);
+	if (i == mh_end(modules))
+		panic("failed to update module cache");
+	mh_strnptr_node(modules, i)->str = package;
+	mh_strnptr_node(modules, i)->val = module;
+}
+
 /**
  * Delete a module from the module cache
  */
@@ -281,9 +296,7 @@ module_reload(const char *package, const char *package_end)
 		func->module = new_module;
 		rlist_move(&new_module->funcs, &func->item);
 	}
-	module_cache_del(package, package_end);
-	if (module_cache_put(new_module) != 0)
-		goto restore;
+	module_cache_update(new_module);
 	module_gc(old_module);
 	return 0;
 restore:
-- 
2.29.2

[Tarantool-patches] [PATCH 3/4] cmod: export cmod_call helper

[Cyrill Gorcunov via Tarantool-patches]

This will allow us to reuse this code
in when function is executed from
box.schema.func.

Part-of #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
---
 src/box/lua/cmod.c | 24 +++++++++---------------
 src/box/lua/cmod.h | 15 +++++++++++++++
 2 files changed, 24 insertions(+), 15 deletions(-)

diff --git a/src/box/lua/cmod.c b/src/box/lua/cmod.c
index 6b383d8fe..d7d395b7d 100644
--- a/src/box/lua/cmod.c
+++ b/src/box/lua/cmod.c
@@ -12,7 +12,6 @@
 
 #include "box/error.h"
 #include "box/port.h"
-#include "box/func_def.h"
 
 #include "tt_static.h"
 
@@ -805,9 +804,9 @@ lfunc_gc(struct lua_State *L)
 	return 0;
 }
 
-/** Execute a function. */
 int
-do_lfunc_call(struct cmod_func *cf, struct port *args, struct port *ret)
+cmod_call(struct cmod *m, box_function_f func_addr,
+	  struct port *args, struct port *ret)
 {
 	struct region *region = &fiber()->gc;
 	size_t region_svp = region_used(region);
@@ -823,17 +822,12 @@ do_lfunc_call(struct cmod_func *cf, struct port *args, struct port *ret)
 	};
 
 	/*
-	 * Unlike box.schema.func cmod doesn't support
-	 * module run-time reloading and while a function
-	 * is alive a module can't disappear. Still it is
-	 * unclear if there some existing users are running
-	 * code which already tries to unload a function inside
-	 * this call execution, thus make sure the module won't
-	 * disapper inbetween.
+	 * The function may get rescheduled inside,
+	 * thus make sure the module won't disappear.
 	 */
-	cmod_ref(cf->cmod);
-	int rc = cf->addr(&ctx, data, data + data_sz);
-	cmod_unref(cf->cmod);
+	cmod_ref(m);
+	int rc = func_addr(&ctx, data, data + data_sz);
+	cmod_unref(m);
 	region_truncate(region, region_svp);
 
 	if (rc != 0) {
@@ -843,7 +837,7 @@ do_lfunc_call(struct cmod_func *cf, struct port *args, struct port *ret)
 		return -1;
 	}
 
-	return rc;
+	return 0;
 }
 
 /** Call a function by its name from the Lua code. */
@@ -869,7 +863,7 @@ lfunc_call(struct lua_State *L)
 
 	struct port ret;
 
-	if (do_lfunc_call(cf, &args, &ret) != 0) {
+	if (cmod_call(cf->cmod, cf->addr, &args, &ret) != 0) {
 		port_destroy(&args);
 		return luaT_error(L);
 	}
diff --git a/src/box/lua/cmod.h b/src/box/lua/cmod.h
index 224779a04..c34cc156a 100644
--- a/src/box/lua/cmod.h
+++ b/src/box/lua/cmod.h
@@ -9,6 +9,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 
+#include "box/func_def.h"
 #include "trivia/config.h"
 
 #if defined(__cplusplus)
@@ -110,6 +111,20 @@ int
 cmod_find_package(const char *package, size_t package_len,
 		  char *path, size_t path_len);
 
+/**
+ * Execute a function.
+ *
+ * @param m the module function sits in.
+ * @param func_addr function address to call.
+ * @args args incoming arguments.
+ * @ret rets execution results.
+ *
+ * @return 0 on success, -1 otherwise(diag is set).
+ */
+int
+cmod_call(struct cmod *m, box_function_f func_addr,
+	  struct port *args, struct port *ret);
+
 /**
  * Initialize cmod Lua module.
  *
-- 
2.29.2

[Tarantool-patches] [PATCH 4/4] box/func: simplify func_c_call

[Cyrill Gorcunov via Tarantool-patches]

We can reuse cmod_call to eliminate code
duplication. Moreover since cmod_call keeps
a reference to cmod instance while executing
a call, we no longer need module::calls counter
(the module won't be unloaded unti execution is
complete).

Part-of #4642

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>
---
 src/box/func.c | 28 ++--------------------------
 1 file changed, 2 insertions(+), 26 deletions(-)

diff --git a/src/box/func.c b/src/box/func.c
index 5757f7c25..942c7c3c1 100644
--- a/src/box/func.c
+++ b/src/box/func.c
@@ -200,7 +200,6 @@ module_new(struct cmod *cmod)
 	}
 
 	rlist_create(&module->funcs);
-	module->calls = 0;
 	module->cmod = cmod;
 
 	return module;
@@ -254,7 +253,7 @@ module_delete(struct module *module)
 static void
 module_gc(struct module *module)
 {
-	if (rlist_empty(&module->funcs) && module->calls == 0)
+	if (rlist_empty(&module->funcs))
 		module_delete(module);
 }
 
@@ -513,32 +512,9 @@ func_c_call(struct func *base, struct port *args, struct port *ret)
 			return -1;
 	}
 
-	struct region *region = &fiber()->gc;
-	size_t region_svp = region_used(region);
-	uint32_t data_sz;
-	const char *data = port_get_msgpack(args, &data_sz);
-	if (data == NULL)
-		return -1;
-
-	port_c_create(ret);
-	box_function_ctx_t ctx = { ret };
-
-	/* Module can be changed after function reload. */
 	struct module *module = func->module;
-	assert(module != NULL);
-	++module->calls;
-	int rc = func->func(&ctx, data, data + data_sz);
-	--module->calls;
+	int rc = cmod_call(module->cmod, func->func, args, ret);
 	module_gc(module);
-	region_truncate(region, region_svp);
-	if (rc != 0) {
-		if (diag_last_error(&fiber()->diag) == NULL) {
-			/* Stored procedure forget to set diag  */
-			diag_set(ClientError, ER_PROC_C, "unknown error");
-		}
-		port_destroy(ret);
-		return -1;
-	}
 	return rc;
 }
 
-- 
2.29.2