From: Cyrill Gorcunov via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: tml <tarantool-patches@dev.tarantool.org> Cc: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Subject: [Tarantool-patches] [PATCH v20 0/7] box: implement box.lib Lua module Date: Fri, 2 Apr 2021 15:34:13 +0300 [thread overview] Message-ID: <20210402123420.885834-1-gorcunov@gmail.com> (raw) Please take a look once time permit. v1-v3 are development ones and not sent. v5 (by vlad): - drop exists, list methods: they are redundant - rename cfunc to cbox - when create a function make it callable Lua object - initialize cbox out of modules - fix error in passing module name for reloading - make api been cbox.func.[create|drop] and cbox.module.reload - fix test for OSX sake v6 (by vlad): - move module handling into module_cache file. v7: - development v8: - use rbtree for function instance storage, since i don't like the idea of unexpected rehashing of values in case of massive number of functions allocated - use reference counter and free function instance if only load/unload are coupled - keep a pointer to the function inside Lua object so we don't need to lookup on every function call. this force us to implement __gc method - use new API and update docs v9: - development v10: - use hashes for function names lookup - simply function loads counting - use luaL_register_module and luaL_register_type for easier methods registering - carry functions as userdata object v11: - development v12: - switch to new API as been discussed in https://lists.tarantool.org/tarantool-patches/e186c454-6765-4776-6433-f3f791ff4c27@tarantool.org/ v13: - development v14: - switch to refs to carry module usage - drop func_name structure renaming - carry two hashes for backward compatibility with functions created via box.schema.func help - complete rework of cmod and most parts of module_cache - account for file statistics to invalidate module cache - new API for cmod, no more :reload, the :load procedure uses cache invalidation - update test cases - still there is no GC test since I didn't manage to deal with it v15: - report module state cached/orphan - update test cases - do not prevent functions lookup in orphan modules - there was an idea to use box.shema.func cache as on top of cmod's one, but this doesn't work because in case if module doesnt exist in any caches we would put it into into cmod's one as well but there wont be a module on cmod level which would clean it up later (which makes code a way more comple if we choose to track state of modules). v16: - internal v17: - drop idea of unifying box.schema.func and cmod functions cache, it brings more problems than solves due to too different context of execution; - make cmod self consistent, which shrink patch series size ~1/5 in compare with previous attempts; - improve tests to account internal states of modules and functions (tt_dev key in reports). v18: - implement pass-through cache for modules loading, for this sake 'struct module' uses cmod internally; - improve tests to cover sole cmod case and a mixture of box.schema.func and cmod to make sure the caches are not corrupted. v19: - move module handling into separate subsystem; - switch box.schema.func and cmod to use this shared code to eliminate code duplication; - update tests. v20: - fix potential nil dereference in schema_init while I'm in the code; - during working on the series found a bug in module recovery, fixed it in the series because my further work depends on it; - main module which allows to create C functions now named as box.lib; - structures renaming: - box.schema.func uses struct schema_module to carry underlied struct module; - box.lib uses struct module directly and struct box_module_func to carry cache of functions. - name unification: - schema_module variables are named as `mod`; - box.schema.func api prefixed as `schema_module_x`; - debug properties for module counting prefixed as `debug_x`; - the test remains disabled because - I run it locally all the time - better to have it in repo and enable then since patching test-run can take weeks, it is slow procedure Cyrill Gorcunov (7): box/schema: make sure hashes are created box/func: module_reload -- drop redundant argument box/func: fix modules functions restore box/module_cache: introduce modules subsystem box/schema.func: switch to new module api box: implement box.lib module test: add box.lib test changelogs/unreleased/fix-module-reload.md | 4 + src/box/CMakeLists.txt | 2 + src/box/box.cc | 4 +- src/box/call.c | 9 +- src/box/func.c | 545 ++++++++----------- src/box/func.h | 28 +- src/box/func_def.h | 14 - src/box/lua/box_lib.c | 590 +++++++++++++++++++++ src/box/lua/box_lib.h | 25 + src/box/lua/init.c | 2 + src/box/module_cache.c | 474 +++++++++++++++++ src/box/module_cache.h | 208 ++++++++ src/box/schema.cc | 7 + src/main.cc | 3 + test/box/CMakeLists.txt | 6 + test/box/cfunc1.c | 58 ++ test/box/cfunc2.c | 137 +++++ test/box/cfunc3.c | 25 + test/box/cfunc4.c | 28 + test/box/func_restore.result | 103 ++++ test/box/func_restore.test.lua | 43 ++ test/box/func_restore1.c | 34 ++ test/box/func_restore2.c | 28 + test/box/gh-4648-func-load-unload.result | 7 +- test/box/gh-4648-func-load-unload.test.lua | 7 +- test/box/lib.result | 530 ++++++++++++++++++ test/box/lib.test.lua | 203 +++++++ test/box/misc.result | 1 + test/box/suite.ini | 2 +- 29 files changed, 2757 insertions(+), 370 deletions(-) create mode 100644 changelogs/unreleased/fix-module-reload.md create mode 100644 src/box/lua/box_lib.c create mode 100644 src/box/lua/box_lib.h create mode 100644 src/box/module_cache.c create mode 100644 src/box/module_cache.h create mode 100644 test/box/cfunc1.c create mode 100644 test/box/cfunc2.c create mode 100644 test/box/cfunc3.c create mode 100644 test/box/cfunc4.c create mode 100644 test/box/func_restore.result create mode 100644 test/box/func_restore.test.lua create mode 100644 test/box/func_restore1.c create mode 100644 test/box/func_restore2.c create mode 100644 test/box/lib.result create mode 100644 test/box/lib.test.lua base-commit: 22e2e4eaad7796f7b0142cc7e603f9fe895f6642 -- 2.30.2
next reply other threads:[~2021-04-02 12:34 UTC|newest] Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-02 12:34 Cyrill Gorcunov via Tarantool-patches [this message] 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 1/7] box/schema: make sure hashes are created Cyrill Gorcunov via Tarantool-patches 2021-04-05 9:28 ` Serge Petrenko via Tarantool-patches 2021-04-05 9:50 ` Cyrill Gorcunov via Tarantool-patches 2021-04-05 10:13 ` Serge Petrenko via Tarantool-patches 2021-04-05 15:45 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-06 7:44 ` Cyrill Gorcunov via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 2/7] box/func: module_reload -- drop redundant argument Cyrill Gorcunov via Tarantool-patches 2021-04-05 10:23 ` Serge Petrenko via Tarantool-patches 2021-04-05 10:26 ` Serge Petrenko via Tarantool-patches 2021-04-05 10:31 ` Cyrill Gorcunov via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 3/7] box/func: fix modules functions restore Cyrill Gorcunov via Tarantool-patches 2021-04-05 10:53 ` Serge Petrenko via Tarantool-patches 2021-04-05 11:26 ` Cyrill Gorcunov via Tarantool-patches 2021-04-05 11:42 ` Serge Petrenko via Tarantool-patches 2021-04-05 15:47 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-06 8:38 ` Cyrill Gorcunov via Tarantool-patches 2021-04-06 20:02 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-06 20:42 ` Cyrill Gorcunov via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 4/7] box/module_cache: introduce modules subsystem Cyrill Gorcunov via Tarantool-patches 2021-04-05 12:34 ` Serge Petrenko via Tarantool-patches 2021-04-05 15:52 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-06 14:33 ` Cyrill Gorcunov via Tarantool-patches 2021-04-06 20:09 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-06 22:05 ` Cyrill Gorcunov via Tarantool-patches 2021-04-06 23:43 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-07 7:03 ` Cyrill Gorcunov via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 5/7] box/schema.func: switch to new module api Cyrill Gorcunov via Tarantool-patches 2021-04-05 15:56 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 6/7] box: implement box.lib module Cyrill Gorcunov via Tarantool-patches 2021-04-05 16:04 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-07 16:59 ` Cyrill Gorcunov via Tarantool-patches 2021-04-07 20:22 ` Cyrill Gorcunov via Tarantool-patches 2021-04-07 20:28 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-07 20:37 ` Cyrill Gorcunov via Tarantool-patches 2021-04-07 20:45 ` Cyrill Gorcunov via Tarantool-patches 2021-04-07 21:04 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-02 12:34 ` [Tarantool-patches] [PATCH v20 7/7] test: add box.lib test Cyrill Gorcunov via Tarantool-patches 2021-04-05 16:04 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-05 15:45 ` [Tarantool-patches] [PATCH v20 0/7] box: implement box.lib Lua module Vladislav Shpilevoy via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210402123420.885834-1-gorcunov@gmail.com \ --to=tarantool-patches@dev.tarantool.org \ --cc=gorcunov@gmail.com \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v20 0/7] box: implement box.lib Lua module' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox