[Tarantool-patches] [PATCH v4 5/6] box/func: implement cfunc Lua module

[Cyrill Gorcunov <gorcunov@gmail.com>]

Currently to run "C" function from some external module
one have to register it first in "_func" system space. This
is a problem if node is in read-only mode (replica).

Still people would like to have a way to run such functions
even in ro mode. For this sake we implement "cfunc" lua
module.

Fixes #4692

Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

@TarantoolBot document
Title: cfunc module
---
.. _cfunc-module:

-------------------------------------------------------------------------------
                                   Module `cfunc`
-------------------------------------------------------------------------------

.. module:: cfunc

===============================================================================
                                   Overview
===============================================================================

``cfunc`` module provides a way to create and execute external C functions without
registering them in system ``_func`` space. These functions are not persistent and
dissapear once instance goes offline. Also they are not replicated and can be
executed even if node is in read-only mode.

===============================================================================
                                    Index
===============================================================================

Below is a list of all ``cfunc`` functions and handle methods.

.. container:: table

    .. rst-class:: left-align-column-1
    .. rst-class:: left-align-column-2

    +-----------------------------+-----------------------------+
    | Name                        | Use                         |
    +=============================+=============================+
    | :ref:`cfunc.create()        | Create a new function       |
    | <cfunc-create>`             |                             |
    +-----------------------------+-----------------------------+
    | :ref:`cfunc.drop()          | Drop a function             |
    | <cfunc-drop>`               |                             |
    +-----------------------------+-----------------------------+
    | :ref:`cfunc.exists()        | Test if a function exists   |
    | <cfunc-exists>`             |                             |
    +-----------------------------+-----------------------------+
    | :ref:`cfunc.call()          | Execute a function          |
    | <cfunc-call>`               |                             |
    +-----------------------------+-----------------------------+
    | :ref:`cfunc.reload()        | Reload a module             |
    | <cfunc-reload>`             |                             |
    +-----------------------------+-----------------------------+
    | :ref:`cfunc.list()          | List functions              |
    | <cfunc-list>`               |                             |
    +-----------------------------+-----------------------------+

.. _cfunc-create:

.. function:: create(name)

    Create a new function.

    :param string name: a function name to create
    :return: (if success) ``nil``

    Possible raised errors are:

    * IllegalParams: incorrect type or value of a parameter
    * IllegalParams: a function is already exist
    * ClientError: function name is not valid identifier
    * OutOfMemory: no free memory available

    **Example**

    .. code-block:: lua

        require('cfunc').create('easy')

    Here we create a ``C`` function with name ``easy`` from
    ``easy.so`` shared library.

.. _cfunc-drop:

.. function:: drop(name)

    Drop previously created function.

    :param string name: function name to drop
    :return: (if success) ``nil``

    Possible raised errors are:

    * IllegalParams: incorrect type or value of a parameter
    * IllegalParams: the function does not exist

    **Example**

    .. code-block:: lua

        require('cfunc').drop('easy')

.. _cfunc-exists:

.. function:: exists(name)

    Check if a function exists.

    :param string name: function name to test
    :return: (if exists) ``true``
             (if does not exist) ``false``

    Possible raised errors are:

    * IllegalParams: incorrect type or value of a parameter

    **Example**

    .. code-block:: lua

        require('cfunc').exists('easy')

.. _cfunc-call:

.. function:: call(name[, {arguments}])

    Execute a function.

    :param string name: function name to run
    :param table arguments: table of arguments
    :return: (if success) ``nil``

    Possible raised errors are:

    * IllegalParams: incorrect type or value of a parameter
    * IllegalParams: the function is not created

    **Example**

    .. code-block:: lua

        require('cfunc').call('easy')

.. _cfunc-reload:

.. function:: reload(name)

    Reload a module.

    :param string name: modlule name
    :return: (if success) ``nil``

    Possible raised errors are:

    * IllegalParams: incorrect type or value of a parameter
    * IllegalParams: no such module registered
    * ClientError: module has not been loaded or not found

    **Example**

    .. code-block:: lua

        require('cfunc').reload('easy')

.. _cfunc-list:

.. function:: list()

    List created functions.

    **Example**

    .. code-block:: lua

        require('cfunc').list()
        ---
        - - easy
        ...
---
 src/box/CMakeLists.txt |   1 +
 src/box/func.c         |  10 ++
 src/box/lua/cfunc.c    | 362 +++++++++++++++++++++++++++++++++++++++++
 src/box/lua/cfunc.h    |  55 +++++++
 src/box/lua/init.c     |   2 +
 5 files changed, 430 insertions(+)
 create mode 100644 src/box/lua/cfunc.c
 create mode 100644 src/box/lua/cfunc.h

diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 8b2e704cf..3d283618a 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -194,6 +194,7 @@ add_library(box STATIC
     ${sql_sources}
     ${lua_sources}
     lua/init.c
+    lua/cfunc.c
     lua/call.c
     lua/cfg.cc
     lua/console.c
diff --git a/src/box/func.c b/src/box/func.c
index ba98f0f9e..032b6062c 100644
--- a/src/box/func.c
+++ b/src/box/func.c
@@ -34,6 +34,7 @@
 #include "assoc.h"
 #include "lua/utils.h"
 #include "lua/call.h"
+#include "lua/cfunc.h"
 #include "error.h"
 #include "errinj.h"
 #include "diag.h"
@@ -152,6 +153,14 @@ module_init(void)
 			  "modules hash table");
 		return -1;
 	}
+	/*
+	 * cfunc depends on module engine,
+	 * so initialize them together.
+	 */
+	if (cfunc_init() != 0) {
+		module_free();
+		return -1;
+	}
 	return 0;
 }
 
@@ -166,6 +175,7 @@ module_free(void)
 		module_gc(module);
 	}
 	mh_strnptr_delete(modules);
+	cfunc_free();
 }
 
 /**
diff --git a/src/box/lua/cfunc.c b/src/box/lua/cfunc.c
new file mode 100644
index 000000000..d57ed7d1b
--- /dev/null
+++ b/src/box/lua/cfunc.c
@@ -0,0 +1,362 @@
+/*
+ * Copyright 2010-2020, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include <string.h>
+
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+
+#include "assoc.h"
+#include "lua/utils.h"
+#include "trivia/util.h"
+#include "box/func.h"
+#include "box/call.h"
+#include "box/port.h"
+#include "box/identifier.h"
+#include "box/session.h"
+#include "box/user.h"
+#include "say.h"
+#include "diag.h"
+#include "tt_static.h"
+
+#include "box/lua/cfunc.h"
+#include "box/lua/call.h"
+
+static struct mh_strnptr_t *cfunc_hash;
+static unsigned int nr_cfunc = 0;
+
+struct cfunc {
+	struct module_sym	mod_sym;
+	size_t			name_len;
+	char			name[0];
+};
+
+struct cfunc *
+cfunc_lookup(const char *name, size_t len)
+{
+	if (name != NULL && len > 0) {
+		mh_int_t e = mh_strnptr_find_inp(cfunc_hash, name, len);
+		if (e != mh_end(cfunc_hash))
+			return mh_strnptr_node(cfunc_hash, e)->val;
+	}
+	return NULL;
+}
+
+static int
+cfunc_add(struct cfunc *cfunc)
+{
+	const struct mh_strnptr_node_t nd = {
+		.str	= cfunc->name,
+		.len	= cfunc->name_len,
+		.hash	= mh_strn_hash(cfunc->name, cfunc->name_len),
+		.val	= cfunc,
+	};
+
+	mh_int_t h = mh_strnptr_put(cfunc_hash, &nd, NULL, NULL);
+	if (h == mh_end(cfunc_hash)) {
+		diag_set(OutOfMemory, sizeof(nd), "cfunc_hash_add", "h");
+		return -1;
+	}
+	return 0;
+}
+
+static void
+luaT_param_type_error(struct lua_State *L, int idx, const char *func_name,
+		      const char *param, const char *exp)
+{
+	const char *typename = idx == 0 ?
+		"<unknown>" : lua_typename(L, lua_type(L, idx));
+	static const char *fmt =
+		"%s: wrong parameter \"%s\": expected %s, got %s";
+	diag_set(IllegalParams, fmt, func_name, param, exp, typename);
+}
+
+static int
+lbox_cfunc_create(struct lua_State *L)
+{
+	static const char method[] = "cfunc.create";
+	struct cfunc *cfunc = NULL;
+
+	if (lua_gettop(L) != 1) {
+		static const char *fmt =
+			"%s: expects %s(\'name\')";
+		diag_set(IllegalParams, fmt, method, method);
+		goto out;
+	}
+
+	if (lua_type(L, 1) != LUA_TSTRING) {
+		luaT_param_type_error(L, 1, method,
+				      lua_tostring(L, 1),
+				      "function name");
+		goto out;
+	}
+
+	size_t name_len;
+	const char *name = lua_tolstring(L, 1, &name_len);
+
+	if (identifier_check(name, name_len) != 0)
+		goto out;
+
+	if (cfunc_lookup(name, name_len) != NULL) {
+		const char *fmt = tnt_errcode_desc(ER_FUNCTION_EXISTS);
+		diag_set(IllegalParams, fmt, name);
+		goto out;
+	}
+
+	cfunc = malloc(sizeof(*cfunc) + name_len + 1);
+	if (cfunc == NULL) {
+		diag_set(OutOfMemory, sizeof(*cfunc), "malloc", "cfunc");
+		goto out;
+	}
+
+	cfunc->mod_sym.addr	= NULL;
+	cfunc->mod_sym.module	= NULL;
+	cfunc->mod_sym.name	= cfunc->name;
+	cfunc->name_len		= name_len;
+
+	memcpy(cfunc->name, name, name_len);
+	cfunc->name[name_len] = '\0';
+
+	if (cfunc_add(cfunc) != 0)
+		goto out;
+
+	nr_cfunc++;
+	return 0;
+
+out:
+	free(cfunc);
+	return luaT_error(L);
+}
+
+static int
+lbox_cfunc_drop(struct lua_State *L)
+{
+	static const char method[] = "cfunc.drop";
+	const char *name = NULL;
+
+	if (lua_gettop(L) != 1) {
+		static const char *fmt =
+			"%s: expects %s(\'name\')";
+		diag_set(IllegalParams, fmt, method, method);
+		return luaT_error(L);
+	}
+
+	if (lua_type(L, 1) != LUA_TSTRING) {
+		luaT_param_type_error(L, 1, method,
+				      lua_tostring(L, 1),
+				      "function name or id");
+		return luaT_error(L);
+	}
+
+	size_t name_len;
+	name = lua_tolstring(L, 1, &name_len);
+
+	mh_int_t e = mh_strnptr_find_inp(cfunc_hash, name, name_len);
+	if (e == mh_end(cfunc_hash)) {
+		const char *fmt = tnt_errcode_desc(ER_NO_SUCH_FUNCTION);
+		diag_set(IllegalParams, fmt, name);
+		return luaT_error(L);
+	}
+
+	struct cfunc *cfunc = mh_strnptr_node(cfunc_hash, e)->val;
+	mh_strnptr_del(cfunc_hash, e, NULL);
+
+	nr_cfunc--;
+	free(cfunc);
+
+	return 0;
+}
+
+static int
+lbox_cfunc_exists(struct lua_State *L)
+{
+	static const char method[] = "cfunc.exists";
+	if (lua_gettop(L) != 1) {
+		static const char *fmt =
+			"%s: expects %s(\'name\') but no name passed";
+		diag_set(IllegalParams, fmt, method, method);
+		return luaT_error(L);
+	}
+
+	size_t name_len;
+	const char *name = lua_tolstring(L, 1, &name_len);
+
+	struct cfunc *cfunc = cfunc_lookup(name, name_len);
+	lua_pushboolean(L, cfunc != NULL ? true : false);
+
+	return 1;
+}
+
+static int
+lbox_cfunc_reload(struct lua_State *L)
+{
+	static const char method[] = "cfunc.reload";
+	if (lua_gettop(L) != 1 || !lua_isstring(L, 1)) {
+		static const char *fmt =
+			"%s: expects %s(\'name\') but no name passed";
+		diag_set(IllegalParams, fmt, method, method);
+		return luaT_error(L);
+	}
+
+	size_t name_len;
+	const char *name = lua_tolstring(L, 1, &name_len);
+
+	/*
+	 * Since we use module engine do not allow to
+	 * access arbitrary names only the ones we've
+	 * really created.
+	 */
+	struct cfunc *cfunc = cfunc_lookup(name, name_len);
+	if (cfunc == NULL) {
+		const char *fmt = tnt_errcode_desc(ER_NO_SUCH_FUNCTION);
+		diag_set(IllegalParams, fmt, name);
+		return luaT_error(L);
+	}
+
+	struct module *module = NULL;
+	struct func_name n;
+
+	func_split_name(name, &n);
+	if (module_reload(n.package, n.package_end, &module) == 0) {
+		if (module != NULL)
+			return 0;
+		diag_set(ClientError, ER_NO_SUCH_MODULE, name);
+	}
+
+	return luaT_error(L);
+}
+
+static int
+lbox_cfunc_list(struct lua_State *L)
+{
+	if (nr_cfunc == 0)
+		return 0;
+
+	lua_createtable(L, nr_cfunc, 0);
+
+	int nr = 1;
+	mh_int_t i;
+	mh_foreach(cfunc_hash, i) {
+		struct cfunc *c = mh_strnptr_node(cfunc_hash, i)->val;
+		lua_pushstring(L, c->name);
+		lua_rawseti(L, -2, nr++);
+	}
+
+	return 1;
+}
+
+static int
+lbox_cfunc_call(struct lua_State *L)
+{
+	static const char method[] = "cfunc.call";
+	if (lua_gettop(L) < 1 || !lua_isstring(L, 1)) {
+		static const char *fmt =
+			"%s: expects %s(\'name\')";
+		diag_set(IllegalParams, fmt, method, method);
+	}
+
+	size_t name_len;
+	const char *name = lua_tolstring(L, 1, &name_len);
+
+	struct cfunc *cfunc = cfunc_lookup(name, name_len);
+	if (cfunc == NULL) {
+		const char *fmt = tnt_errcode_desc(ER_NO_SUCH_FUNCTION);
+		diag_set(IllegalParams, fmt, name);
+		return luaT_error(L);
+	}
+
+	lua_State *args_L = luaT_newthread(tarantool_L);
+	if (args_L == NULL)
+		return luaT_error(L);
+
+	int coro_ref = luaL_ref(tarantool_L, LUA_REGISTRYINDEX);
+	lua_xmove(L, args_L, lua_gettop(L) - 1);
+
+	struct port args;
+	port_lua_create(&args, args_L);
+	((struct port_lua *)&args)->ref = coro_ref;
+
+	struct port ret;
+	if (module_sym_call(&cfunc->mod_sym,  &args, &ret) != 0) {
+		port_destroy(&args);
+		return luaT_error(L);
+	}
+
+	int top = lua_gettop(L);
+	port_dump_lua(&ret, L, true);
+	int cnt = lua_gettop(L) - top;
+
+	port_destroy(&ret);
+	port_destroy(&args);
+	return cnt;
+}
+
+int
+cfunc_init(void)
+{
+	cfunc_hash = mh_strnptr_new();
+	if (cfunc_hash == NULL) {
+		diag_set(OutOfMemory, sizeof(*cfunc_hash), "malloc",
+			  "cfunc hash table");
+		return -1;
+	}
+	return 0;
+}
+
+void
+cfunc_free(void)
+{
+	while (mh_size(cfunc_hash) > 0) {
+		mh_int_t e = mh_first(cfunc_hash);
+		struct cfunc *c = mh_strnptr_node(cfunc_hash, e)->val;
+		module_sym_unload(&c->mod_sym);
+		mh_strnptr_del(cfunc_hash, e, NULL);
+	}
+	mh_strnptr_delete(cfunc_hash);
+	cfunc_hash = NULL;
+}
+
+void
+box_lua_cfunc_init(struct lua_State *L)
+{
+	static const struct luaL_Reg cfunclib[] = {
+		{ "create",	lbox_cfunc_create },
+		{ "drop",	lbox_cfunc_drop },
+		{ "exists",	lbox_cfunc_exists },
+		{ "call",	lbox_cfunc_call },
+		{ "reload",	lbox_cfunc_reload },
+		{ "list",	lbox_cfunc_list },
+		{ }
+	};
+
+	luaL_register_module(L, "cfunc", cfunclib);
+	lua_pop(L, 1);
+}
diff --git a/src/box/lua/cfunc.h b/src/box/lua/cfunc.h
new file mode 100644
index 000000000..51cec28b8
--- /dev/null
+++ b/src/box/lua/cfunc.h
@@ -0,0 +1,55 @@
+#ifndef INCLUDES_TARANTOOL_MOD_BOX_LUA_CFUNC_H
+#define INCLUDES_TARANTOOL_MOD_BOX_LUA_CFUNC_H
+/*
+ * Copyright 2010-2020, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <stdint.h>
+
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+struct lua_State;
+
+void
+box_lua_cfunc_init(struct lua_State *L);
+
+int
+cfunc_init(void);
+
+void
+cfunc_free(void);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
+#endif /* INCLUDES_TARANTOOL_MOD_BOX_LUA_CFUNC_H */
diff --git a/src/box/lua/init.c b/src/box/lua/init.c
index d0316ef86..972ab9d8c 100644
--- a/src/box/lua/init.c
+++ b/src/box/lua/init.c
@@ -48,6 +48,7 @@
 #include "box/lua/error.h"
 #include "box/lua/tuple.h"
 #include "box/lua/call.h"
+#include "box/lua/cfunc.h"
 #include "box/lua/slab.h"
 #include "box/lua/index.h"
 #include "box/lua/space.h"
@@ -465,6 +466,7 @@ box_lua_init(struct lua_State *L)
 	box_lua_error_init(L);
 	box_lua_tuple_init(L);
 	box_lua_call_init(L);
+	box_lua_cfunc_init(L);
 	box_lua_cfg_init(L);
 	box_lua_slab_init(L);
 	box_lua_index_init(L);
-- 
2.26.2