From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 825336EC40; Thu, 12 Aug 2021 20:07:37 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 825336EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628788057; bh=CIgg0E010/YURsu0sqv3761swtXN0zkuMQYHkk+5Ncg=; h=To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=qp5eHpolMzE6H2cwedhjygCthamXExiE1ValJ3NOtNBfR35KQB/ng3VnRpKKseJEy XHPbBZ+PydOX556ARf3wx+fKCRQs0vsoLDURUkppXKiMFMVjaZqIhxMwhw0Hre3EDD ieF3yCqPL9q5uyTyujKAunAzHYdwrrVcOH+vXcjQ= Received: from smtp46.i.mail.ru (smtp46.i.mail.ru [94.100.177.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 96F586EC40 for ; Thu, 12 Aug 2021 20:07:36 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 96F586EC40 Received: by smtp46.i.mail.ru with esmtpa (envelope-from ) id 1mEEB5-0003DR-H3; Thu, 12 Aug 2021 20:07:36 +0300 To: tarantool-patches@dev.tarantool.org Cc: v.shpilevoy@tarantool.org, yaroslav.dynnikov@tarantool.org Date: Thu, 12 Aug 2021 20:07:35 +0300 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210811074938.7ttf4sog77you5ml@esperanza> References: <20210811074938.7ttf4sog77you5ml@esperanza> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9D5AC6413C25DCF08CC98B8FCC5CD86F3182A05F538085040AF73D52EB798D5D1D8F5E5D9E9DA765AD77588DA80C814C9BD85B9CC454A1BE0 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE76C0A440987CA342DC2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE737694D551A905FE2EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F2ED74CFFDAB1E542D72460A9C55444E97CC7F00164DA146DAFE8445B8C89999728AA50765F790063783E00425F71A4181389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC82FFDA4F57982C5F4F6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C2F20CBB68F4D2F81302FCEF25BFAB3454AD6D5ED66289B52698AB9A7B718F8C46E0066C2D8992A16725E5C173C3A84C3D264F704C83CBBC7BA3038C0950A5D36B5C8C57E37DE458B0BC6067A898B09E46D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE7E1BCFB2C0BE3F189731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B50589A32824256D1A91FF9707671F0B3C8B X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C69415AB31670C86C944872997B69EC11734149960026BEBB9C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF4387DEC30D31F6BC699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34C75F910DBB8BE898CB0BB1B97EE7FC00E77BF4F7F657911A88248539AD827152D4608FDEFD01B83F1D7E09C32AA3244CA3B671788FE4EA832100CB60F16D02B96C24832127668422927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojKW4rnL99YhIHwEZv3b9Grw== X-Mailru-Sender: EAC3DA4C40F0802FC8DB1BBF04B7D9F23F414C7A8B75FB2412DF59AB5C0AFC145CEB263B7F1DB82FF0798C444D2DF93A342CD0BA774DB6A9B195EA2AC6D0D44E902A426E4A0C259E9437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v2] net.box: allow to store user-defined fields in future object X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladimir Davydov via Tarantool-patches Reply-To: Vladimir Davydov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Before commit 954194a1ca5c ("net.box: rewrite request implementation in C"), net.box future was a plain Lua table so that the caller could attach extra information to it. Now it isn't true anymore - a future is a userdata object, and it doesn't have indexing methods. For backward compatibility, let's add __index and __newindex fields and store user-defined fields in an mhash, which is created lazily on the first __newindex invocation. __index falls back on the metatable methods if a field isn't found in the mhash. Follow-up #6241 Closes #6306 --- Changes in v2: - When a key is deleted from the hash (by setting its value to nil), check that it's actually present in the hash. v1: https://lists.tarantool.org/pipermail/tarantool-patches/2021-August/025346.html src/box/lua/net_box.c | 93 ++++++++++++++ test/box/net.box_fiber-async_gh-3107.result | 117 +++++++++++++++++- test/box/net.box_fiber-async_gh-3107.test.lua | 41 +++++- 3 files changed, 243 insertions(+), 8 deletions(-) diff --git a/src/box/lua/net_box.c b/src/box/lua/net_box.c index 06e574cdf746..6ab0f71cdbe9 100644 --- a/src/box/lua/net_box.c +++ b/src/box/lua/net_box.c @@ -134,6 +134,14 @@ struct netbox_request { * the response hasn't been received yet. */ struct error *error; + /** + * User-defined fields: field name -> Lua object reference. + * We allow the user to attach extra information to a future object, + * e.g. a reference to a connection or the invoked method name/args. + * All the information is stored in this table, which is created + * lazily, on the first __newindex invocation. + */ + struct mh_strnptr_t *index; }; static const char netbox_registry_typename[] = "net.box.registry"; @@ -166,6 +174,15 @@ netbox_request_destroy(struct netbox_request *request) luaL_unref(tarantool_L, LUA_REGISTRYINDEX, request->result_ref); if (request->error != NULL) error_unref(request->error); + if (request->index != NULL) { + struct mh_strnptr_t *h = request->index; + mh_int_t k; + mh_foreach(h, k) { + int ref = (intptr_t)mh_strnptr_node(h, k)->val; + luaL_unref(tarantool_L, LUA_REGISTRYINDEX, ref); + } + mh_strnptr_delete(h); + } } /** @@ -1358,6 +1375,79 @@ luaT_netbox_request_gc(struct lua_State *L) return 0; } +static int +luaT_netbox_request_index(struct lua_State *L) +{ + struct netbox_request *request = luaT_check_netbox_request(L, 1); + size_t field_name_len; + const char *field_name = lua_tolstring(L, 2, &field_name_len); + if (field_name == NULL) + return luaL_error(L, "invalid index"); + if (request->index != NULL) { + mh_int_t k = mh_strnptr_find_inp(request->index, field_name, + field_name_len); + if (k != mh_end(request->index)) { + int field_value_ref = (intptr_t)mh_strnptr_node( + request->index, k)->val; + lua_rawgeti(L, LUA_REGISTRYINDEX, field_value_ref); + return 1; + } + } + /* Fall back on metatable methods. */ + return luaL_getmetafield(L, 1, field_name) != LUA_TNIL ? 1 : 0; +} + +static int +luaT_netbox_request_newindex(struct lua_State *L) +{ + struct netbox_request *request = luaT_check_netbox_request(L, 1); + struct mh_strnptr_t *h = request->index; + size_t field_name_len; + const char *field_name = lua_tolstring(L, 2, &field_name_len); + if (field_name == NULL) + return luaL_error(L, "invalid index"); + int field_value_ref = luaL_ref(L, LUA_REGISTRYINDEX); + if (field_value_ref == LUA_REFNIL) { + /* The field is set to nil. Delete it from the index. */ + if (h == NULL) + return 0; + mh_int_t k = mh_strnptr_find_inp(h, field_name, + field_name_len); + if (k != mh_end(h)) { + int ref = (intptr_t)mh_strnptr_node(h, k)->val; + luaL_unref(L, LUA_REGISTRYINDEX, ref); + mh_strnptr_del(h, k, NULL); + } + return 0; + } + if (h == NULL) { + /* Lazily create the index on the first invocation. */ + h = request->index = mh_strnptr_new(); + if (h == NULL) { + luaL_unref(L, LUA_REGISTRYINDEX, field_value_ref); + return luaL_error(L, "out of memory"); + } + } + /* Insert a reference to the new value into the index. */ + struct mh_strnptr_node_t node = { + .str = field_name, + .len = field_name_len, + .hash = mh_strn_hash(field_name, field_name_len), + .val = (void *)(intptr_t)field_value_ref, + }; + struct mh_strnptr_node_t old_node = { NULL, 0, 0, NULL }; + struct mh_strnptr_node_t *p_old_node = &old_node; + if (mh_strnptr_put(h, &node, &p_old_node, + NULL) == mh_end(h)) { + luaL_unref(L, LUA_REGISTRYINDEX, field_value_ref); + return luaL_error(L, "out of memory"); + } + /* Drop the reference to the overwritten value. */ + if (p_old_node != NULL) + luaL_unref(L, LUA_REGISTRYINDEX, (intptr_t)old_node.val); + return 0; +} + /** * Returns true if the response was received for the given request. */ @@ -1602,6 +1692,7 @@ netbox_make_request(struct lua_State *L, int idx, fiber_cond_create(&request->cond); request->result_ref = LUA_NOREF; request->error = NULL; + request->index = NULL; if (netbox_request_register(request, registry) != 0) { netbox_request_destroy(request); luaT_error(L); @@ -1988,6 +2079,8 @@ luaopen_net_box(struct lua_State *L) static const struct luaL_Reg netbox_request_meta[] = { { "__gc", luaT_netbox_request_gc }, + { "__index", luaT_netbox_request_index }, + { "__newindex", luaT_netbox_request_newindex }, { "is_ready", luaT_netbox_request_is_ready }, { "result", luaT_netbox_request_result }, { "wait_result", luaT_netbox_request_wait_result }, diff --git a/test/box/net.box_fiber-async_gh-3107.result b/test/box/net.box_fiber-async_gh-3107.result index aaaca351a579..1c48a115e6b5 100644 --- a/test/box/net.box_fiber-async_gh-3107.result +++ b/test/box/net.box_fiber-async_gh-3107.result @@ -10,10 +10,7 @@ net = require('net.box') cond = nil --- ... -box.schema.func.create('long_function') ---- -... -box.schema.user.grant('guest', 'execute', 'function', 'long_function') +box.schema.user.grant('guest', 'execute', 'universe') --- ... function long_function(...) cond = fiber.cond() cond:wait() return ... end @@ -104,7 +101,117 @@ err:find('Usage') ~= nil --- - true ... -box.schema.func.drop('long_function') +-- Storing user data in future object. +future = c:eval('return 123', {}, {is_async = true}) +--- +... +future.abc -- nil +--- +- null +... +future.abc = nil +--- +... +future.abc -- nil +--- +- null +... +future.abc = 'abc' +--- +... +future.abc -- abc +--- +- abc +... +future.abc = nil +--- +... +future.abc -- nil +--- +- null +... +future.abc = nil +--- +... +future.abc -- nil +--- +- null +... +future[{}] -- error +--- +- error: '[string "return future[{}] -- error "]:1: invalid index' +... +future[{}] = 'abc' -- error +--- +- error: '[string "future[{}] = ''abc'' -- error "]:1: invalid index' +... +future:wait_result() -- 123 +--- +- [123] +... +-- Garbage collection of stored user data. +future = c:eval('return 123', {}, {is_async = true}) +--- +... +future.data1 = {1} +--- +... +future.data2 = {2} +--- +... +future.data3 = {3} +--- +... +gc = setmetatable({}, {__mode = 'v'}) +--- +... +gc.data1 = future.data1 +--- +... +gc.data2 = future.data2 +--- +... +gc.data3 = future.data3 +--- +... +future.data1 = nil +--- +... +_ = collectgarbage('collect') +--- +... +gc.data1 == nil +--- +- true +... +future.data2 = 123 +--- +... +_ = collectgarbage('collect') +--- +... +gc.data2 == nil +--- +- true +... +future:wait_result() -- 123 +--- +- [123] +... +future = nil +--- +... +_ = collectgarbage('collect') +--- +... +_ = collectgarbage('collect') +--- +... +gc.data3 == nil +--- +- true +... +box.schema.user.revoke('guest', 'execute', 'universe') --- ... c:close() diff --git a/test/box/net.box_fiber-async_gh-3107.test.lua b/test/box/net.box_fiber-async_gh-3107.test.lua index d23f368cbce4..e08bad056cba 100644 --- a/test/box/net.box_fiber-async_gh-3107.test.lua +++ b/test/box/net.box_fiber-async_gh-3107.test.lua @@ -5,8 +5,7 @@ net = require('net.box') -- gh-3107: fiber-async netbox. -- cond = nil -box.schema.func.create('long_function') -box.schema.user.grant('guest', 'execute', 'function', 'long_function') +box.schema.user.grant('guest', 'execute', 'universe') function long_function(...) cond = fiber.cond() cond:wait() return ... end function finalize_long() while not cond do fiber.sleep(0.01) end cond:signal() cond = nil end s = box.schema.create_space('test') @@ -36,7 +35,43 @@ err:find('Usage') ~= nil _, err = pcall(future.wait_result, future, '100') err:find('Usage') ~= nil -box.schema.func.drop('long_function') +-- Storing user data in future object. +future = c:eval('return 123', {}, {is_async = true}) +future.abc -- nil +future.abc = nil +future.abc -- nil +future.abc = 'abc' +future.abc -- abc +future.abc = nil +future.abc -- nil +future.abc = nil +future.abc -- nil +future[{}] -- error +future[{}] = 'abc' -- error +future:wait_result() -- 123 + +-- Garbage collection of stored user data. +future = c:eval('return 123', {}, {is_async = true}) +future.data1 = {1} +future.data2 = {2} +future.data3 = {3} +gc = setmetatable({}, {__mode = 'v'}) +gc.data1 = future.data1 +gc.data2 = future.data2 +gc.data3 = future.data3 +future.data1 = nil +_ = collectgarbage('collect') +gc.data1 == nil +future.data2 = 123 +_ = collectgarbage('collect') +gc.data2 == nil +future:wait_result() -- 123 +future = nil +_ = collectgarbage('collect') +_ = collectgarbage('collect') +gc.data3 == nil + +box.schema.user.revoke('guest', 'execute', 'universe') c:close() s:drop() -- 2.25.1