From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id CCEC56EC5D; Mon, 5 Apr 2021 13:53:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org CCEC56EC5D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617619992; bh=cgSn42Ibx08VUPOq8ZmXsVgeKyc6WpWLxUQECCFo42E=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ar9vKHwm8eokJNx9uPX7nA848EXdYUAOO7jlSDtuusRJ0AWOvQZCieGoCneHNbVAu 6NR2gfHn90yqEmgNw5LxhKU0pxjIidqg/uVP5L4+ib0o2IUVa3Eq9FjPG/ygp+Rg4H Qc+MbvjyeGQRsUHfEBhOwCupjNQHfV7UTEr5N3Yo= Received: from smtp60.i.mail.ru (smtp60.i.mail.ru [217.69.128.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 08C096EC5D for ; Mon, 5 Apr 2021 13:53:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 08C096EC5D Received: by smtp60.i.mail.ru with esmtpa (envelope-from ) id 1lTMqz-0001pg-Eh; Mon, 05 Apr 2021 13:53:10 +0300 To: Cyrill Gorcunov , tml References: <20210402123420.885834-1-gorcunov@gmail.com> <20210402123420.885834-4-gorcunov@gmail.com> Message-ID: Date: Mon, 5 Apr 2021 13:53:07 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <20210402123420.885834-4-gorcunov@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9ED7173E37F4E3294CA3588DDE0233B0D17711AF1EA2D7DB9182A05F53808504029A9377D26D14BD7CFB3959E839DD7797C39E09EE31BBE2089C104B27A46F5E5 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE79488F21A45FBCE3EEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063726CA83C7ABDB938E8638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C16EE06F5A270FE6AE8ED69CACD587D2E0F8CF20746FF021DA471835C12D1D9774AD6D5ED66289B5259CC434672EE6371117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658359CC434672EE6371117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CF7A9652BCE453B95DBA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B6DE73887A47B4E64C76E601842F6C81A1F004C906525384307823802FF610243DF43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A3920E30A4C9C8E338DAA65EE29973F47D2F43847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A5AA22A34047DA23980A0693D1C55D4C893F2822550E0B8FF3D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D345CB6DE26F16546678AD091E58CF8A5CE9B9B06610367296759AEEF4D409C9C68E550BA44D6E41F321D7E09C32AA3244C0A2153A7ED0DE9B04AE92B372D0E1A29C86C126E7119A0FE927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojM00ve/f+0olFWfgMoMjUKQ== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F8F56650DFA0649F332C704EFABB041820E0DCC56E243D36AB424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v20 3/7] box/func: fix modules functions restore X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for the patch! Please find 3 minor comments below. 02.04.2021 15:34, Cyrill Gorcunov пишет: > In commit 96938fafb an ability to hot reload of modules > has been introduced. When module is been reloaded his typo: either 'is being reloaded' or 'has been reloaded'. > functions are resolved to new symbols but if something > went wrong it is supposed to restore old symbols from > the old module. Actually current code restores only > one function and may crash if there a bunch of functions > to restore. Lets fix it. > > Part-of #4642 > > Signed-off-by: Cyrill Gorcunov > --- > changelogs/unreleased/fix-module-reload.md | 4 + > src/box/func.c | 13 ++- > test/box/CMakeLists.txt | 2 + > test/box/func_restore.result | 103 +++++++++++++++++++++ > test/box/func_restore.test.lua | 43 +++++++++ > test/box/func_restore1.c | 34 +++++++ > test/box/func_restore2.c | 28 ++++++ > 7 files changed, 220 insertions(+), 7 deletions(-) > create mode 100644 changelogs/unreleased/fix-module-reload.md > create mode 100644 test/box/func_restore.result > create mode 100644 test/box/func_restore.test.lua > create mode 100644 test/box/func_restore1.c > create mode 100644 test/box/func_restore2.c > > diff --git a/changelogs/unreleased/fix-module-reload.md b/changelogs/unreleased/fix-module-reload.md > new file mode 100644 > index 000000000..7e189617f > --- /dev/null > +++ b/changelogs/unreleased/fix-module-reload.md > @@ -0,0 +1,4 @@ > +## bugfix/core > + > +* Fix module reloading procedure which may crash in case if > + new module is corrupted (gh-4642). > diff --git a/src/box/func.c b/src/box/func.c > index 233696a4f..1cd7073de 100644 > --- a/src/box/func.c > +++ b/src/box/func.c > @@ -387,13 +387,14 @@ module_reload(const char *package, const char *package_end) > > struct func_c *func, *tmp_func; > rlist_foreach_entry_safe(func, &old_module->funcs, item, tmp_func) { > + /* Move immediately for restore sake. */ > + rlist_move(&new_module->funcs, &func->item); > struct func_name name; > func_split_name(func->base.def->name, &name); > func->func = module_sym(new_module, name.sym); > if (func->func == NULL) > goto restore; > func->module = new_module; > - rlist_move(&new_module->funcs, &func->item); > } > module_cache_del(package, package_end); > if (module_cache_put(new_module) != 0) > @@ -402,10 +403,10 @@ module_reload(const char *package, const char *package_end) > return 0; > restore: > /* > - * Some old-dso func can't be load from new module, restore old > - * functions. > + * Some old functions are not found int the new module, > + * thus restore all migrated functions back to original. typo: in the new module. > */ > - do { > + rlist_foreach_entry_safe(func, &new_module->funcs, item, tmp_func) { > struct func_name name; > func_split_name(func->base.def->name, &name); > func->func = module_sym(old_module, name.sym); > @@ -419,9 +420,7 @@ module_reload(const char *package, const char *package_end) > } > func->module = old_module; > rlist_move(&old_module->funcs, &func->item); > - } while (func != rlist_first_entry(&old_module->funcs, > - struct func_c, item)); > - assert(rlist_empty(&new_module->funcs)); Let's keep this assert. > + } > module_delete(new_module); > return -1; > } > diff --git a/test/box/CMakeLists.txt b/test/box/CMakeLists.txt > index 06bfbbe9d..944831af2 100644 > --- a/test/box/CMakeLists.txt > +++ b/test/box/CMakeLists.txt > @@ -2,4 +2,6 @@ include_directories(${MSGPUCK_INCLUDE_DIRS}) > build_module(function1 function1.c) > build_module(reload1 reload1.c) > build_module(reload2 reload2.c) > +build_module(func_restore1 func_restore1.c) > +build_module(func_restore2 func_restore2.c) > build_module(tuple_bench tuple_bench.c) > diff --git a/test/box/func_restore.result b/test/box/func_restore.result > new file mode 100644 > index 000000000..e679f4eb8 > --- /dev/null > +++ b/test/box/func_restore.result > @@ -0,0 +1,103 @@ > +-- test-run result file version 2 > +-- > +-- Test that function can be restored to the > +-- former values when new module can't be > +-- loaded for some reason (say there some > +-- missing functions). > +-- > +build_path = os.getenv("BUILDDIR") > + | --- > + | ... > +package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath > + | --- > + | ... > + > +fio = require('fio') > + | --- > + | ... > + > +ext = (jit.os == "OSX" and "dylib" or "so") > + | --- > + | ... > + > +path_func_restore = fio.pathjoin(build_path, "test/box/func_restore.") .. ext > + | --- > + | ... > +path_func_good = fio.pathjoin(build_path, "test/box/func_restore1.") .. ext > + | --- > + | ... > +path_func_bad = fio.pathjoin(build_path, "test/box/func_restore2.") .. ext > + | --- > + | ... > + > +_ = pcall(fio.unlink(path_func_restore)) > + | --- > + | ... > +fio.symlink(path_func_good, path_func_restore) > + | --- > + | - true > + | ... > + > +box.schema.func.create('func_restore.echo_1', {language = "C"}) > + | --- > + | ... > +box.schema.func.create('func_restore.echo_2', {language = "C"}) > + | --- > + | ... > +box.schema.func.create('func_restore.echo_3', {language = "C"}) > + | --- > + | ... > + > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_3') > + | --- > + | ... > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_2') > + | --- > + | ... > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_1') > + | --- > + | ... > + > +-- Order *does* matter since we bind functions in > +-- a list where entries are added to the top. > +box.func['func_restore.echo_3']:call() > + | --- > + | - 3 > + | ... > +box.func['func_restore.echo_2']:call() > + | --- > + | - 2 > + | ... > +box.func['func_restore.echo_1']:call() > + | --- > + | - 1 > + | ... > + > +_ = pcall(fio.unlink(path_func_restore)) > + | --- > + | ... > +fio.symlink(path_func_bad, path_func_restore) > + | --- > + | - true > + | ... > + > +ok, _ = pcall(box.schema.func.reload, "func_restore") > + | --- > + | ... > +assert(not ok) > + | --- > + | - true > + | ... > + > +box.func['func_restore.echo_1']:call() > + | --- > + | - 1 > + | ... > +box.func['func_restore.echo_2']:call() > + | --- > + | - 2 > + | ... > +box.func['func_restore.echo_3']:call() > + | --- > + | - 3 > + | ... > diff --git a/test/box/func_restore.test.lua b/test/box/func_restore.test.lua > new file mode 100644 > index 000000000..8a38a6074 > --- /dev/null > +++ b/test/box/func_restore.test.lua > @@ -0,0 +1,43 @@ > +-- > +-- Test that function can be restored to the > +-- former values when new module can't be > +-- loaded for some reason (say there some > +-- missing functions). > +-- > +build_path = os.getenv("BUILDDIR") > +package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath > + > +fio = require('fio') > + > +ext = (jit.os == "OSX" and "dylib" or "so") > + > +path_func_restore = fio.pathjoin(build_path, "test/box/func_restore.") .. ext > +path_func_good = fio.pathjoin(build_path, "test/box/func_restore1.") .. ext > +path_func_bad = fio.pathjoin(build_path, "test/box/func_restore2.") .. ext > + > +_ = pcall(fio.unlink(path_func_restore)) > +fio.symlink(path_func_good, path_func_restore) > + > +box.schema.func.create('func_restore.echo_1', {language = "C"}) > +box.schema.func.create('func_restore.echo_2', {language = "C"}) > +box.schema.func.create('func_restore.echo_3', {language = "C"}) > + > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_3') > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_2') > +box.schema.user.grant('guest', 'execute', 'function', 'func_restore.echo_1') > + > +-- Order *does* matter since we bind functions in > +-- a list where entries are added to the top. > +box.func['func_restore.echo_3']:call() > +box.func['func_restore.echo_2']:call() > +box.func['func_restore.echo_1']:call() > + > +_ = pcall(fio.unlink(path_func_restore)) > +fio.symlink(path_func_bad, path_func_restore) > + > +ok, _ = pcall(box.schema.func.reload, "func_restore") > +assert(not ok) > + > +box.func['func_restore.echo_1']:call() > +box.func['func_restore.echo_2']:call() > +box.func['func_restore.echo_3']:call() > diff --git a/test/box/func_restore1.c b/test/box/func_restore1.c > new file mode 100644 > index 000000000..ffd69fd2b > --- /dev/null > +++ b/test/box/func_restore1.c > @@ -0,0 +1,34 @@ > +#include > +#include > +#include > + > +#include "module.h" > + > +static int > +echo_num(box_function_ctx_t *ctx, const char *args, > + const char *args_end, unsigned int num) > +{ > + char res[16]; > + char *end = mp_encode_uint(res, num); > + box_return_mp(ctx, res, end); > + return 0; > +} > + > + > +int > +echo_1(box_function_ctx_t *ctx, const char *args, const char *args_end) > +{ > + return echo_num(ctx, args, args_end, 1); > +} > + > +int > +echo_2(box_function_ctx_t *ctx, const char *args, const char *args_end) > +{ > + return echo_num(ctx, args, args_end, 2); > +} > + > +int > +echo_3(box_function_ctx_t *ctx, const char *args, const char *args_end) > +{ > + return echo_num(ctx, args, args_end, 3); > +} > diff --git a/test/box/func_restore2.c b/test/box/func_restore2.c > new file mode 100644 > index 000000000..0d77e78b2 > --- /dev/null > +++ b/test/box/func_restore2.c > @@ -0,0 +1,28 @@ > +#include > +#include > +#include > + > +#include "module.h" > + > +static int > +echo_num(box_function_ctx_t *ctx, const char *args, > + const char *args_end, unsigned int num) > +{ > + char res[16]; > + char *end = mp_encode_uint(res, num); > + box_return_mp(ctx, res, end); > + return 0; > +} > + > + > +int > +echo_1(box_function_ctx_t *ctx, const char *args, const char *args_end) > +{ > + return echo_num(ctx, args, args_end, 1); > +} > + > +int > +echo_2(box_function_ctx_t *ctx, const char *args, const char *args_end) > +{ > + return echo_num(ctx, args, args_end, 2); > +} -- Serge Petrenko