From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 624B26EC40; Tue, 10 Aug 2021 19:53:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 624B26EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628614436; bh=PEoyt+7UXHfh4pKocuoHIFUjMWmmHKPWdv+yYlBU5GQ=; h=To:Cc:Date:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=ZOBgeGZlpmd0vC4AOQ35Xv8TbWZ/vgNmg/frihL8uIalNgc3nbxgiPF0paVY9DwTi esDZ9Qy0czCAAtWid0fx9aRAEAtuU86QdEd7cWwDTnPr1KdC/wCiQFoAfxCx6a5hbc Q5Yzrc01eo6jxq/G4KYMr66LXQjgsRO/A9RSK3IU= Received: from smtpng2.i.mail.ru (smtpng2.i.mail.ru [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 1CCF16EC40 for ; Tue, 10 Aug 2021 19:53:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1CCF16EC40 Received: by smtpng2.m.smailru.net with esmtpa (envelope-from ) id 1mDV0k-0001Yy-A6; Tue, 10 Aug 2021 19:53:54 +0300 To: v.shpilevoy@tarantool.org Cc: tarantool-patches@dev.tarantool.org Date: Tue, 10 Aug 2021 19:53:54 +0300 Message-Id: X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD906AB4890CDABF0C5CB76CEE71D3E4007182A05F5380850407625CBFB5FD23A4DD85FEC39E90494AC2E85799E2C13EBA91507E59D6747C75E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE721D130CF676D2164EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637C59BB75D821356298638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D87B150E1384D63E5E4E56F3325632D8DB117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18C26CFBAC0749D213D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B613439FA09F3DCB32089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C6B668463F4F89F867736B992BF35AC582CA083B0022A4F029C2B6934AE262D3EE7EAB7254005DCED42BE5BF2787C37971E0A4E2319210D9B64D260DF9561598F01A9E91200F654B0B684FE8DC1FE10A18E8E86DC7131B365E7726E8460B7C23C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34E69981C39E7B068A948A51CF8FC9DA607B49326C7024A493CE4D8675C085DC2598F36824B9732D191D7E09C32AA3244CFC8FF1070E92095FF0CA439938E4BA78CE0B41342B755BCDFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj6qlzQV0oSZOMAeZJsNWH1w== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D4CE95102C245D71FFA12A149DABD7CB4274CEFED1673C562683ABF942079399BFB559BB5D741EB966A65DFF43FF7BE03240331F90058701C67EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH] net.box: allow to store user-defined fields in future object X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladimir Davydov via Tarantool-patches Reply-To: Vladimir Davydov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Before commit 954194a1ca5c ("net.box: rewrite request implementation in C"), net.box future was a plain Lua table so that the caller could attach extra information to it. Now it isn't true anymore - a future is a userdata object, and it doesn't have indexing methods. For backward compatibility, let's add __index and __newindex fields and store user-defined fields in an mhash, which is created lazily on the first __newindex invocation. __index falls back on the metatable methods if a field isn't found in the mhash. Follow-up #6241 Closes #6306 --- https://github.com/tarantool/tarantool/tree/vdavydov/netbox-allow-to-store-user-data-in-future https://github.com/tarantool/tarantool/issues/6241 https://github.com/tarantool/tarantool/issues/6306 As a result of this patch, async net.box.call performance degrades by about 2-3% because of an extra C function call executed per each future member access: | KRPS (PROC TIME) | KRPS (WALL TIME) before | 276 +- 9 | 448 +- 22 after | 270 +- 8 | 434 +- 21 Still, it's better than wrapping a future object into a table, which degrades performance by 10%. src/box/lua/net_box.c | 91 ++++++++++++++++ test/box/net.box_fiber-async_gh-3107.result | 103 +++++++++++++++++- test/box/net.box_fiber-async_gh-3107.test.lua | 37 ++++++- 3 files changed, 223 insertions(+), 8 deletions(-) diff --git a/src/box/lua/net_box.c b/src/box/lua/net_box.c index 06e574cdf746..162ff6c82fb0 100644 --- a/src/box/lua/net_box.c +++ b/src/box/lua/net_box.c @@ -134,6 +134,14 @@ struct netbox_request { * the response hasn't been received yet. */ struct error *error; + /** + * User-defined fields: field name -> Lua object reference. + * We allow the user to attach extra information to a future object, + * e.g. a reference to a connection or the invoked method name/args. + * All the information is stored in this table, which is created + * lazily, on the first __newindex invocation. + */ + struct mh_strnptr_t *index; }; static const char netbox_registry_typename[] = "net.box.registry"; @@ -166,6 +174,15 @@ netbox_request_destroy(struct netbox_request *request) luaL_unref(tarantool_L, LUA_REGISTRYINDEX, request->result_ref); if (request->error != NULL) error_unref(request->error); + if (request->index != NULL) { + struct mh_strnptr_t *h = request->index; + mh_int_t k; + mh_foreach(h, k) { + int ref = (intptr_t)mh_strnptr_node(h, k)->val; + luaL_unref(tarantool_L, LUA_REGISTRYINDEX, ref); + } + mh_strnptr_delete(h); + } } /** @@ -1358,6 +1375,77 @@ luaT_netbox_request_gc(struct lua_State *L) return 0; } +static int +luaT_netbox_request_index(struct lua_State *L) +{ + struct netbox_request *request = luaT_check_netbox_request(L, 1); + size_t field_name_len; + const char *field_name = lua_tolstring(L, 2, &field_name_len); + if (field_name == NULL) + return luaL_error(L, "invalid index"); + if (request->index != NULL) { + mh_int_t k = mh_strnptr_find_inp(request->index, field_name, + field_name_len); + if (k != mh_end(request->index)) { + int field_value_ref = (intptr_t)mh_strnptr_node( + request->index, k)->val; + lua_rawgeti(L, LUA_REGISTRYINDEX, field_value_ref); + return 1; + } + } + /* Fall back on metatable methods. */ + return luaL_getmetafield(L, 1, field_name) != LUA_TNIL ? 1 : 0; +} + +static int +luaT_netbox_request_newindex(struct lua_State *L) +{ + struct netbox_request *request = luaT_check_netbox_request(L, 1); + struct mh_strnptr_t *h = request->index; + size_t field_name_len; + const char *field_name = lua_tolstring(L, 2, &field_name_len); + if (field_name == NULL) + return luaL_error(L, "invalid index"); + int field_value_ref = luaL_ref(L, LUA_REGISTRYINDEX); + if (field_value_ref == LUA_REFNIL) { + /* The field is set to nil. Delete it from the index. */ + if (h == NULL) + return 0; + mh_int_t k = mh_strnptr_find_inp(h, field_name, + field_name_len); + int ref = (intptr_t)mh_strnptr_node(h, k)->val; + luaL_unref(L, LUA_REGISTRYINDEX, ref); + mh_strnptr_del(h, k, NULL); + return 0; + } + if (h == NULL) { + /* Lazily create the index on the first invocation. */ + h = request->index = mh_strnptr_new(); + if (h == NULL) { + luaL_unref(L, LUA_REGISTRYINDEX, field_value_ref); + return luaL_error(L, "out of memory"); + } + } + /* Insert a reference to the new value into the index. */ + struct mh_strnptr_node_t node = { + .str = field_name, + .len = field_name_len, + .hash = mh_strn_hash(field_name, field_name_len), + .val = (void *)(intptr_t)field_value_ref, + }; + struct mh_strnptr_node_t old_node = { NULL, 0, 0, NULL }; + struct mh_strnptr_node_t *p_old_node = &old_node; + if (mh_strnptr_put(h, &node, &p_old_node, + NULL) == mh_end(h)) { + luaL_unref(L, LUA_REGISTRYINDEX, field_value_ref); + return luaL_error(L, "out of memory"); + } + /* Drop the reference to the overwritten value. */ + if (p_old_node != NULL) + luaL_unref(L, LUA_REGISTRYINDEX, (intptr_t)old_node.val); + return 0; +} + /** * Returns true if the response was received for the given request. */ @@ -1602,6 +1690,7 @@ netbox_make_request(struct lua_State *L, int idx, fiber_cond_create(&request->cond); request->result_ref = LUA_NOREF; request->error = NULL; + request->index = NULL; if (netbox_request_register(request, registry) != 0) { netbox_request_destroy(request); luaT_error(L); @@ -1988,6 +2077,8 @@ luaopen_net_box(struct lua_State *L) static const struct luaL_Reg netbox_request_meta[] = { { "__gc", luaT_netbox_request_gc }, + { "__index", luaT_netbox_request_index }, + { "__newindex", luaT_netbox_request_newindex }, { "is_ready", luaT_netbox_request_is_ready }, { "result", luaT_netbox_request_result }, { "wait_result", luaT_netbox_request_wait_result }, diff --git a/test/box/net.box_fiber-async_gh-3107.result b/test/box/net.box_fiber-async_gh-3107.result index aaaca351a579..e44db726c8cb 100644 --- a/test/box/net.box_fiber-async_gh-3107.result +++ b/test/box/net.box_fiber-async_gh-3107.result @@ -10,10 +10,7 @@ net = require('net.box') cond = nil --- ... -box.schema.func.create('long_function') ---- -... -box.schema.user.grant('guest', 'execute', 'function', 'long_function') +box.schema.user.grant('guest', 'execute', 'universe') --- ... function long_function(...) cond = fiber.cond() cond:wait() return ... end @@ -104,7 +101,103 @@ err:find('Usage') ~= nil --- - true ... -box.schema.func.drop('long_function') +-- Storing user data in future object. +future = c:eval('return 123', {}, {is_async = true}) +--- +... +future.abc -- nil +--- +- null +... +future.abc = 'abc' +--- +... +future.abc -- abc +--- +- abc +... +future.abc = nil +--- +... +future.abc -- nil +--- +- null +... +future[{}] -- error +--- +- error: '[string "return future[{}] -- error "]:1: invalid index' +... +future[{}] = 'abc' -- error +--- +- error: '[string "future[{}] = ''abc'' -- error "]:1: invalid index' +... +future:wait_result() -- 123 +--- +- [123] +... +-- Garbage collection of stored user data. +future = c:eval('return 123', {}, {is_async = true}) +--- +... +future.data1 = {1} +--- +... +future.data2 = {2} +--- +... +future.data3 = {3} +--- +... +gc = setmetatable({}, {__mode = 'v'}) +--- +... +gc.data1 = future.data1 +--- +... +gc.data2 = future.data2 +--- +... +gc.data3 = future.data3 +--- +... +future.data1 = nil +--- +... +_ = collectgarbage('collect') +--- +... +gc.data1 == nil +--- +- true +... +future.data2 = 123 +--- +... +_ = collectgarbage('collect') +--- +... +gc.data2 == nil +--- +- true +... +future:wait_result() -- 123 +--- +- [123] +... +future = nil +--- +... +_ = collectgarbage('collect') +--- +... +_ = collectgarbage('collect') +--- +... +gc.data3 == nil +--- +- true +... +box.schema.user.revoke('guest', 'execute', 'universe') --- ... c:close() diff --git a/test/box/net.box_fiber-async_gh-3107.test.lua b/test/box/net.box_fiber-async_gh-3107.test.lua index d23f368cbce4..5571aeb935e0 100644 --- a/test/box/net.box_fiber-async_gh-3107.test.lua +++ b/test/box/net.box_fiber-async_gh-3107.test.lua @@ -5,8 +5,7 @@ net = require('net.box') -- gh-3107: fiber-async netbox. -- cond = nil -box.schema.func.create('long_function') -box.schema.user.grant('guest', 'execute', 'function', 'long_function') +box.schema.user.grant('guest', 'execute', 'universe') function long_function(...) cond = fiber.cond() cond:wait() return ... end function finalize_long() while not cond do fiber.sleep(0.01) end cond:signal() cond = nil end s = box.schema.create_space('test') @@ -36,7 +35,39 @@ err:find('Usage') ~= nil _, err = pcall(future.wait_result, future, '100') err:find('Usage') ~= nil -box.schema.func.drop('long_function') +-- Storing user data in future object. +future = c:eval('return 123', {}, {is_async = true}) +future.abc -- nil +future.abc = 'abc' +future.abc -- abc +future.abc = nil +future.abc -- nil +future[{}] -- error +future[{}] = 'abc' -- error +future:wait_result() -- 123 + +-- Garbage collection of stored user data. +future = c:eval('return 123', {}, {is_async = true}) +future.data1 = {1} +future.data2 = {2} +future.data3 = {3} +gc = setmetatable({}, {__mode = 'v'}) +gc.data1 = future.data1 +gc.data2 = future.data2 +gc.data3 = future.data3 +future.data1 = nil +_ = collectgarbage('collect') +gc.data1 == nil +future.data2 = 123 +_ = collectgarbage('collect') +gc.data2 == nil +future:wait_result() -- 123 +future = nil +_ = collectgarbage('collect') +_ = collectgarbage('collect') +gc.data3 == nil + +box.schema.user.revoke('guest', 'execute', 'universe') c:close() s:drop() -- 2.25.1