From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 4A9ED64677; Fri, 5 Feb 2021 22:00:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4A9ED64677 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1612551612; bh=X6RCkGxpSjfQoqyu+kTSesO9SXctGqrbXCeuOPiOuNY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=aBkEYFHwc84t62J7hLXxQtwTzGQBdixEP0Ei4TVqqh5Zl/+KFRWD7TSlhTQsGn3q5 SS1siWO1+2VmYRaTCWV54cgI+NhAStOeL9zH3Z/gezTKnm0Sm6T7g4lfsNbQrF9LYx CAlWGfswz68BkFdXmmhFNmjj42AteBh5pVXXzmzI= Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 0E5E864678 for ; Fri, 5 Feb 2021 21:56:53 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0E5E864678 Received: by mail-lj1-f177.google.com with SMTP id r23so7109634ljh.1 for ; Fri, 05 Feb 2021 10:56:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bxSsmW0OmQSCTofuvi6CL00RbRR7t8tdMh2CVT65ElI=; b=uQa57RMvqHqrcP7rWmuSNrpE5fthgVH2hm37L3aqEb/PYp434pbsNLk21N/1KKVWux TPSJPVGXGZXv4d0/zoCTAbgEZ4XGdfrQ2X3sZeRzvRm6PbBa0ETwwR7N4ktJ3zxR145s kDqksxTkMQTwGlUOdZhzPOAHRG+pqRn5DnOuemsxBhbkVcp3yeML5YO/AWN+l0RSibTL 1GZi9Q7rsPEBqnnfQYRIaifu/VTHXFtxDe+8V+zn5XJQ5rCC6mrrVyrdyRexQ4UjsymO Bu6TTg0IWxxJeHYxg7gTYQ2nwPhajYrsAZfDP2sD/YzX+qhRNAbxXhlDcI8yU3s3ZARe FiqA== X-Gm-Message-State: AOAM530ZglA4RbHoRV2pf66wRfYlIY7lIgPVCOoSvz4p4A40McSdvGfh 6sr6hSH1UlsALtWzC2UvW40= X-Google-Smtp-Source: ABdhPJyYa0U84GiaM+53EyiFjgJhQp7bOT4F5CC02UntxalOywXbslihgZcv+PHg9IwwwuDgK/8tag== X-Received: by 2002:a2e:b0fb:: with SMTP id h27mr3354594ljl.183.1612551412515; Fri, 05 Feb 2021 10:56:52 -0800 (PST) Received: from grain.localdomain ([5.18.103.226]) by smtp.gmail.com with ESMTPSA id p21sm1067928lfu.227.2021.02.05.10.56.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Feb 2021 10:56:51 -0800 (PST) Received: by grain.localdomain (Postfix, from userid 1000) id B5B1756029D; Fri, 5 Feb 2021 21:54:37 +0300 (MSK) To: tml Date: Fri, 5 Feb 2021 21:54:36 +0300 Message-Id: <20210205185436.638894-12-gorcunov@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210205185436.638894-1-gorcunov@gmail.com> References: <20210205185436.638894-1-gorcunov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH v15 11/11] test: box/cfunc -- add cmod test X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Cyrill Gorcunov via Tarantool-patches Reply-To: Cyrill Gorcunov Cc: Mons Anderson , Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Note that the test is disabled for a while since we need to update test-run first like | diff --git a/pretest_clean.lua b/pretest_clean.lua | index 9b5ac9d..b0280c4 100644 | --- a/pretest_clean.lua | +++ b/pretest_clean.lua | @@ -272,6 +272,7 @@ local function clean() | package = true, | pickle = true, | popen = true, | + cmod = true, | pwd = true, | socket = true, | strict = true, ie need to enable cmod module. Part-of #4642 Signed-off-by: Cyrill Gorcunov --- test/box/CMakeLists.txt | 2 + test/box/cfunc1.c | 58 +++++++ test/box/cfunc2.c | 137 ++++++++++++++++ test/box/cmod.result | 336 ++++++++++++++++++++++++++++++++++++++++ test/box/cmod.test.lua | 130 ++++++++++++++++ test/box/suite.ini | 2 +- 6 files changed, 664 insertions(+), 1 deletion(-) create mode 100644 test/box/cfunc1.c create mode 100644 test/box/cfunc2.c create mode 100644 test/box/cmod.result create mode 100644 test/box/cmod.test.lua diff --git a/test/box/CMakeLists.txt b/test/box/CMakeLists.txt index 06bfbbe9d..2afbeadc3 100644 --- a/test/box/CMakeLists.txt +++ b/test/box/CMakeLists.txt @@ -3,3 +3,5 @@ build_module(function1 function1.c) build_module(reload1 reload1.c) build_module(reload2 reload2.c) build_module(tuple_bench tuple_bench.c) +build_module(cfunc1 cfunc1.c) +build_module(cfunc2 cfunc2.c) diff --git a/test/box/cfunc1.c b/test/box/cfunc1.c new file mode 100644 index 000000000..8f6d3990c --- /dev/null +++ b/test/box/cfunc1.c @@ -0,0 +1,58 @@ +#include +#include +#include + +#include "module.h" + +/* + * Before the reload functions are just declared + * and simply exit with zero. + * + * After the module reload we should provide real + * functionality. + */ + +int +cfunc_nop(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} + +int +cfunc_fetch_evens(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} + +int +cfunc_multireturn(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} + +int +cfunc_args(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} + +int +cfunc_sum(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} diff --git a/test/box/cfunc2.c b/test/box/cfunc2.c new file mode 100644 index 000000000..a4a871359 --- /dev/null +++ b/test/box/cfunc2.c @@ -0,0 +1,137 @@ +#include +#include +#include + +#include "module.h" + +/* + * Just make sure we've been called. + */ +int +cfunc_nop(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + (void)ctx; + (void)args; + (void)args_end; + return 0; +} + +/* + * Fetch first N even numbers (just to make sure the order of + * arguments is not screwed). + */ +int +cfunc_fetch_evens(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + int arg_count = mp_decode_array(&args); + if (arg_count != 1) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "invalid argument count"); + } + int field_count = mp_decode_array(&args); + if (field_count < 1) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "invalid array size"); + } + + /* + * We expect even numbers sequence here. The idea is + * to test invalid data an issue an error from inside + * of C function. + */ + for (int i = 1; i <= field_count; i++) { + int val = mp_decode_uint(&args); + int needed = 2 * i; + if (val != needed) { + char res[128]; + snprintf(res, sizeof(res), "%s %d != %d", + "invalid argument", val, needed); + return box_error_set(__FILE__, __LINE__, + ER_PROC_C, "%s", res); + } + } + + return 0; +} + +/* + * Return one element array twice. + */ +int +cfunc_multireturn(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + char tuple_buf[512]; + char *d = tuple_buf; + d = mp_encode_array(d, 1); + d = mp_encode_uint(d, 1); + assert(d <= tuple_buf + sizeof(tuple_buf)); + + box_tuple_format_t *fmt = box_tuple_format_default(); + box_tuple_t *tuple_a = box_tuple_new(fmt, tuple_buf, d); + if (tuple_a == NULL) + return -1; + int rc = box_return_tuple(ctx, tuple_a); + if (rc == 0) + return box_return_tuple(ctx, tuple_a); + return rc; +} + +/* + * Encode int + string pair back. + */ +int +cfunc_args(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + uint32_t arg_count = mp_decode_array(&args); + if (arg_count != 2) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "invalid argument count"); + } + + if (mp_typeof(*args) != MP_UINT) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "tuple field must be uint"); + } + uint32_t num = mp_decode_uint(&args); + + if (mp_typeof(*args) != MP_STR) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "tuple field must be string"); + } + const char *str = args; + uint32_t len = mp_decode_strl(&str); + + char tuple_buf[512]; + char *d = tuple_buf; + d = mp_encode_array(d, 2); + d = mp_encode_uint(d, num); + d = mp_encode_str(d, str, len); + assert(d <= tuple_buf + sizeof(tuple_buf)); + + box_tuple_format_t *fmt = box_tuple_format_default(); + box_tuple_t *tuple = box_tuple_new(fmt, tuple_buf, d); + if (tuple == NULL) + return -1; + + return box_return_tuple(ctx, tuple); +} + +/* + * Sum two integers. + */ +int +cfunc_sum(box_function_ctx_t *ctx, const char *args, const char *args_end) +{ + uint32_t arg_count = mp_decode_array(&args); + if (arg_count != 2) { + return box_error_set(__FILE__, __LINE__, ER_PROC_C, "%s", + "invalid argument count"); + } + uint64_t a = mp_decode_uint(&args); + uint64_t b = mp_decode_uint(&args); + + char res[16]; + char *end = mp_encode_uint(res, a + b); + box_return_mp(ctx, res, end); + return 0; +} diff --git a/test/box/cmod.result b/test/box/cmod.result new file mode 100644 index 000000000..72d2f1d7d --- /dev/null +++ b/test/box/cmod.result @@ -0,0 +1,336 @@ +-- test-run result file version 2 +-- +-- gh-4642: New cmod module to be able to +-- run C stored functions on read only nodes +-- without requirement to register them with +-- box.schema.func help. +-- +build_path = os.getenv("BUILDDIR") + | --- + | ... +package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath + | --- + | ... + +cmod = require('cmod') + | --- + | ... +fio = require('fio') + | --- + | ... + +ext = (jit.os == "OSX" and "dylib" or "so") + | --- + | ... + +cfunc_path = fio.pathjoin(build_path, "test/box/cfunc.") .. ext + | --- + | ... +cfunc1_path = fio.pathjoin(build_path, "test/box/cfunc1.") .. ext + | --- + | ... +cfunc2_path = fio.pathjoin(build_path, "test/box/cfunc2.") .. ext + | --- + | ... + +_ = pcall(fio.unlink(cfunc_path)) + | --- + | ... +fio.symlink(cfunc1_path, cfunc_path) + | --- + | - true + | ... + +_, err = pcall(cmod.load, 'non-such-module') + | --- + | ... +assert(err ~= nil) + | --- + | - true + | ... + +-- They all are sitting in cfunc.so. The "nop" module +-- contains functions which are simply return nothing. +old_module = cmod.load('cfunc') + | --- + | ... +old_cfunc_nop = old_module:load('cfunc_nop') + | --- + | ... +old_cfunc_fetch_evens = old_module:load('cfunc_fetch_evens') + | --- + | ... +old_cfunc_multireturn = old_module:load('cfunc_multireturn') + | --- + | ... +old_cfunc_args = old_module:load('cfunc_args') + | --- + | ... +old_cfunc_sum = old_module:load('cfunc_sum') + | --- + | ... +-- Test for error on nonexisting function. +_, err = pcall(old_module.load, old_module, 'no-such-func') + | --- + | ... +assert(err ~= nil) + | --- + | - true + | ... + +-- Make sure they all are callable. +old_cfunc_nop() + | --- + | - true + | ... +old_cfunc_fetch_evens() + | --- + | - true + | ... +old_cfunc_multireturn() + | --- + | - true + | ... +old_cfunc_args() + | --- + | - true + | ... +old_cfunc_sum() + | --- + | - true + | ... + +-- Unload the module but keep old functions alive, so +-- they keep reference to NOP module internally +-- and still callable. +old_module:unload() + | --- + | - true + | ... +old_cfunc_nop() + | --- + | - true + | ... +old_cfunc_fetch_evens() + | --- + | - true + | ... +old_cfunc_multireturn() + | --- + | - true + | ... +old_cfunc_args() + | --- + | - true + | ... +old_cfunc_sum() + | --- + | - true + | ... + +-- The module is unloaded I should not be able +-- to load new shared library. +old_module:load('cfunc') + | --- + | - error: Expects function = module:load('name') but not module object passed + | ... +-- Neither I should be able to unload module twise. +old_module:unload() + | --- + | - error: The module is already unloaded + | ... + +-- Clean old functions. +old_cfunc_nop:unload() + | --- + | - true + | ... +old_cfunc_fetch_evens:unload() + | --- + | - true + | ... +old_cfunc_multireturn:unload() + | --- + | - true + | ... +old_cfunc_args:unload() + | --- + | - true + | ... +old_cfunc_sum:unload() + | --- + | - true + | ... + +-- And reload old module again. +old_module = cmod.load('cfunc') + | --- + | ... +assert(old_module.state == "cached") + | --- + | - true + | ... +old_cfunc_nop = old_module:load('cfunc_nop') + | --- + | ... + +-- Overwrite module with new contents. +_ = pcall(fio.unlink(cfunc_path)) + | --- + | ... +fio.symlink(cfunc2_path, cfunc_path) + | --- + | - true + | ... + +-- Now try to load a function from old module, +-- we should get chache invalidation but all +-- old functions should be callable. +new_module = cmod.load('cfunc') + | --- + | ... +assert(old_module.state == "orphan") + | --- + | - true + | ... + +-- Still all functions from old module should +-- be callable. +old_cfunc_nop = old_module:load('cfunc_nop') + | --- + | ... +old_cfunc_fetch_evens = old_module:load('cfunc_fetch_evens') + | --- + | ... +old_cfunc_multireturn = old_module:load('cfunc_multireturn') + | --- + | ... +old_cfunc_args = old_module:load('cfunc_args') + | --- + | ... +old_cfunc_sum = old_module:load('cfunc_sum') + | --- + | ... + +-- New module is loaded, lets lookup for updated symbols. +new_cfunc_nop = new_module:load('cfunc_nop') + | --- + | ... +new_cfunc_fetch_evens = new_module:load('cfunc_fetch_evens') + | --- + | ... +new_cfunc_multireturn = new_module:load('cfunc_multireturn') + | --- + | ... +new_cfunc_args = new_module:load('cfunc_args') + | --- + | ... +new_cfunc_sum = new_module:load('cfunc_sum') + | --- + | ... + +-- Call old functions. +old_cfunc_nop() + | --- + | - true + | ... +old_cfunc_fetch_evens() + | --- + | - true + | ... +old_cfunc_multireturn() + | --- + | - true + | ... +old_cfunc_args() + | --- + | - true + | ... +old_cfunc_sum() + | --- + | - true + | ... + +-- And newly loaded ones. +new_cfunc_nop() + | --- + | - true + | ... +new_cfunc_multireturn() + | --- + | - true + | ... +new_cfunc_fetch_evens({2,4,6}) + | --- + | - true + | ... +new_cfunc_fetch_evens({1,2,3}) -- error + | --- + | - true + | ... +new_cfunc_args(1, "hello") + | --- + | - true + | ... +new_cfunc_sum(1) -- error + | --- + | - true + | ... +new_cfunc_sum(1,2) + | --- + | - true + | ... + +-- Cleanup old module's functions. +old_cfunc_nop:unload() + | --- + | - true + | ... +old_cfunc_fetch_evens:unload() + | --- + | - true + | ... +old_cfunc_multireturn:unload() + | --- + | - true + | ... +old_cfunc_args:unload() + | --- + | - true + | ... +old_cfunc_sum:unload() + | --- + | - true + | ... + +-- Cleanup new module data. +new_cfunc_nop:unload() + | --- + | - true + | ... +new_cfunc_multireturn:unload() + | --- + | - true + | ... +new_cfunc_fetch_evens:unload() + | --- + | - true + | ... +new_cfunc_args:unload() + | --- + | - true + | ... +new_cfunc_sum:unload() + | --- + | - true + | ... +new_module:unload() + | --- + | - true + | ... + +-- +-- Cleanup the generated symlink +_ = pcall(fio.unlink(cfunc_path)) + | --- + | ... diff --git a/test/box/cmod.test.lua b/test/box/cmod.test.lua new file mode 100644 index 000000000..4d8291651 --- /dev/null +++ b/test/box/cmod.test.lua @@ -0,0 +1,130 @@ +-- +-- gh-4642: New cmod module to be able to +-- run C stored functions on read only nodes +-- without requirement to register them with +-- box.schema.func help. +-- +build_path = os.getenv("BUILDDIR") +package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath + +cmod = require('cmod') +fio = require('fio') + +ext = (jit.os == "OSX" and "dylib" or "so") + +cfunc_path = fio.pathjoin(build_path, "test/box/cfunc.") .. ext +cfunc1_path = fio.pathjoin(build_path, "test/box/cfunc1.") .. ext +cfunc2_path = fio.pathjoin(build_path, "test/box/cfunc2.") .. ext + +_ = pcall(fio.unlink(cfunc_path)) +fio.symlink(cfunc1_path, cfunc_path) + +_, err = pcall(cmod.load, 'non-such-module') +assert(err ~= nil) + +-- They all are sitting in cfunc.so. The "nop" module +-- contains functions which are simply return nothing. +old_module = cmod.load('cfunc') +old_cfunc_nop = old_module:load('cfunc_nop') +old_cfunc_fetch_evens = old_module:load('cfunc_fetch_evens') +old_cfunc_multireturn = old_module:load('cfunc_multireturn') +old_cfunc_args = old_module:load('cfunc_args') +old_cfunc_sum = old_module:load('cfunc_sum') +-- Test for error on nonexisting function. +_, err = pcall(old_module.load, old_module, 'no-such-func') +assert(err ~= nil) + +-- Make sure they all are callable. +old_cfunc_nop() +old_cfunc_fetch_evens() +old_cfunc_multireturn() +old_cfunc_args() +old_cfunc_sum() + +-- Unload the module but keep old functions alive, so +-- they keep reference to NOP module internally +-- and still callable. +old_module:unload() +old_cfunc_nop() +old_cfunc_fetch_evens() +old_cfunc_multireturn() +old_cfunc_args() +old_cfunc_sum() + +-- The module is unloaded I should not be able +-- to load new shared library. +old_module:load('cfunc') +-- Neither I should be able to unload module twise. +old_module:unload() + +-- Clean old functions. +old_cfunc_nop:unload() +old_cfunc_fetch_evens:unload() +old_cfunc_multireturn:unload() +old_cfunc_args:unload() +old_cfunc_sum:unload() + +-- And reload old module again. +old_module = cmod.load('cfunc') +assert(old_module.state == "cached") +old_cfunc_nop = old_module:load('cfunc_nop') + +-- Overwrite module with new contents. +_ = pcall(fio.unlink(cfunc_path)) +fio.symlink(cfunc2_path, cfunc_path) + +-- Now try to load a function from old module, +-- we should get chache invalidation but all +-- old functions should be callable. +new_module = cmod.load('cfunc') +assert(old_module.state == "orphan") + +-- Still all functions from old module should +-- be callable. +old_cfunc_nop = old_module:load('cfunc_nop') +old_cfunc_fetch_evens = old_module:load('cfunc_fetch_evens') +old_cfunc_multireturn = old_module:load('cfunc_multireturn') +old_cfunc_args = old_module:load('cfunc_args') +old_cfunc_sum = old_module:load('cfunc_sum') + +-- New module is loaded, lets lookup for updated symbols. +new_cfunc_nop = new_module:load('cfunc_nop') +new_cfunc_fetch_evens = new_module:load('cfunc_fetch_evens') +new_cfunc_multireturn = new_module:load('cfunc_multireturn') +new_cfunc_args = new_module:load('cfunc_args') +new_cfunc_sum = new_module:load('cfunc_sum') + +-- Call old functions. +old_cfunc_nop() +old_cfunc_fetch_evens() +old_cfunc_multireturn() +old_cfunc_args() +old_cfunc_sum() + +-- And newly loaded ones. +new_cfunc_nop() +new_cfunc_multireturn() +new_cfunc_fetch_evens({2,4,6}) +new_cfunc_fetch_evens({1,2,3}) -- error +new_cfunc_args(1, "hello") +new_cfunc_sum(1) -- error +new_cfunc_sum(1,2) + +-- Cleanup old module's functions. +old_cfunc_nop:unload() +old_cfunc_fetch_evens:unload() +old_cfunc_multireturn:unload() +old_cfunc_args:unload() +old_cfunc_sum:unload() + +-- Cleanup new module data. +new_cfunc_nop:unload() +new_cfunc_multireturn:unload() +new_cfunc_fetch_evens:unload() +new_cfunc_args:unload() +new_cfunc_sum:unload() +new_module:unload() + +-- +-- Cleanup the generated symlink +_ = pcall(fio.unlink(cfunc_path)) diff --git a/test/box/suite.ini b/test/box/suite.ini index e700d0b9e..fc16c5951 100644 --- a/test/box/suite.ini +++ b/test/box/suite.ini @@ -2,7 +2,7 @@ core = tarantool description = Database tests script = box.lua -disabled = rtree_errinj.test.lua tuple_bench.test.lua +disabled = rtree_errinj.test.lua tuple_bench.test.lua cmod.test.lua long_run = huge_field_map_long.test.lua config = engine.cfg release_disabled = errinj.test.lua errinj_index.test.lua rtree_errinj.test.lua upsert_errinj.test.lua iproto_stress.test.lua gh-4648-func-load-unload.test.lua -- 2.29.2