From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 1811F71814; Tue, 23 Feb 2021 03:19:02 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1811F71814 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1614039542; bh=CAmMxP43Uuox1w7KJ9Zh0+6CySD1BdnvQ8FqnOCNpkk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=zth+h6J9YxueR8PahYtt0a/WCsnkkKvK1OyItHnDn8wT69zB6rv15tVZsnS8RgPrC LOlZ9YYxmPw+yhXG0/0hwRWEq83ZlUUTGRG7ox+6PQt53fqCFuwHWedOjYeDcwNsXb zPikdH3AXlpWnHk5t8zEFJqLkTQuhH/8qj5SJtqE= Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 7356D71865 for ; Tue, 23 Feb 2021 03:15:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7356D71865 Received: by smtpng2.m.smailru.net with esmtpa (envelope-from ) id 1lELMo-0003CR-Ng; Tue, 23 Feb 2021 03:15:55 +0300 To: tarantool-patches@dev.tarantool.org, olegrok@tarantool.org, yaroslav.dynnikov@tarantool.org Date: Tue, 23 Feb 2021 01:15:42 +0100 Message-Id: <92da76963b5bd26fa824ae152c7ed2dda7526320.1614039039.git.v.shpilevoy@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD975C3EC174F56692254B0AABE1FB071B2BA6557555153D6A0182A05F53808504085159528EA8B95C5C0A56E96C2C2572E59516E431BBEDF7F3C35ECE90CF681E7 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7AED985C8E545F588EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063716A4A39B750036BB8638F802B75D45FF5571747095F342E8C7A0BC55FA0FE5FC425FEB8730D38403E044403CB9B78DD0D932B531CC1D96BB389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0A29E2F051442AF778941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6B0E9FD5D4288160ECC7F00164DA146DA6F5DAA56C3B73B23C77107234E2CFBA567F23339F89546C55F5C1EE8F4F765FC08F9A42B2210255C75ECD9A6C639B01BBD4B6F7A4D31EC0BC0CAF46E325F83A522CA9DD8327EE4930A3850AC1BE2E73579C543ECCDAE434EC4224003CC836476C0CAF46E325F83A50BF2EBBBDD9D6B0F347543BADC64E7283B503F486389A921A5CC5B56E945C8DA X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C2115B036FF9F6D25D14B0BACE849054F2D12E94C7FAD4E0F9C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF1638054B7D09EC08699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340FE9920E7E3E5C62DD0407475290D9FC9E0988BFFF54F256ECA96A93CC8E36ED9611710720934C291D7E09C32AA3244CFF2EDB06FD0E3257B8E4A71001D674477C0C08F7987826B9FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj2drqE2xHc+GxitdJt99fVw== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822052FEBE03A13C16DC67AFA0F215AA64B3841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH vshard 04/11] registry: module for circular deps resolution X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Registry is a way to resolve cyclic dependencies which normally can exist between files of the same module/library. It is a global table hidden in _G with a long unlikely anywhere used name. Files, which want to expose their API to the other files, which in turn can't require the formers directly, should put their API to the registry. The files use the registry to get API of the other files. They don't require() and use the latter directly. At runtime, when all require() are done, the registry is full, and all the files see API of each other. Such circular dependency will exist between new files implementing map-reduce engine as a set of relatively independent submodules of the storage. In particular there will be storage_ref and storage_sched. Both require a few functions from the main storage file, and will use API of each other. Having the modules accessed via registry adds at lest +1 indexing operation at runtime when need to get a function from there. But sometimes it can be cached similar to how bucket count cache works in the main storage file. Main purpose is not to increase size of the main storage file again. It wouldn't fix the circular deps anyway, and would make it much harder to follow the code. Part of #147 --- vshard/CMakeLists.txt | 3 +- vshard/registry.lua | 67 +++++++++++++++++++++++++++++++++++++++++ vshard/storage/init.lua | 5 ++- 3 files changed, 73 insertions(+), 2 deletions(-) create mode 100644 vshard/registry.lua diff --git a/vshard/CMakeLists.txt b/vshard/CMakeLists.txt index 78a3f07..2a15df5 100644 --- a/vshard/CMakeLists.txt +++ b/vshard/CMakeLists.txt @@ -7,4 +7,5 @@ add_subdirectory(router) # Install module install(FILES cfg.lua error.lua consts.lua hash.lua init.lua replicaset.lua - util.lua lua_gc.lua rlist.lua heap.lua DESTINATION ${TARANTOOL_INSTALL_LUADIR}/vshard) + util.lua lua_gc.lua rlist.lua heap.lua registry.lua + DESTINATION ${TARANTOOL_INSTALL_LUADIR}/vshard) diff --git a/vshard/registry.lua b/vshard/registry.lua new file mode 100644 index 0000000..9583add --- /dev/null +++ b/vshard/registry.lua @@ -0,0 +1,67 @@ +-- +-- Registry is a way to resolve cyclic dependencies which normally can exist +-- between files of the same module/library. +-- +-- Files, which want to expose their API to the other files, which in turn can't +-- require the formers directly, should put their API to the registry. +-- +-- The files should use the registry to get API of the other files. They don't +-- require() and use the latter directly if there is a known loop dependency +-- between them. +-- +-- At runtime, when all require() are done, the registry is full, and all the +-- files see API of each other. +-- +-- Having the modules accessed via the registry adds at lest +1 indexing +-- operation at runtime when need to get a function from there. But sometimes it +-- can be cached to reduce the effect in perf-sensitive code. For example, like +-- this: +-- +-- local lreg = require('vshard.registry') +-- +-- local storage_func +-- +-- local function storage_func_no_cache(...) +-- storage_func = lreg.storage.func +-- return storage_func(...) +-- end +-- +-- storage_func = storage_func_no_cache +-- +-- The code will always call storage_func(), but will load it from the registry +-- only on first invocation. +-- +-- However in case reload is important, it is not possible - the original +-- function object in the registry may change. In such situation still makes +-- sense to cache at least 'lreg.storage' to save 1 indexing operation. +-- +-- local lreg = require('vshard.registry') +-- +-- local lstorage +-- +-- local function storage_func_cache(...) +-- return lstorage.storage_func(...) +-- end +-- +-- local function storage_func_no_cache(...) +-- lstorage = lref.storage +-- storage_func = storage_func_cache +-- return lstorage.storage_func(...) +-- end +-- +-- storage_func = storage_func_no_cache +-- +-- A harder way would be to use the first approach + add triggers on reload of +-- the cached module to update the cached function refs. If the code is +-- extremely perf-critical (which should not be Lua then). +-- + +local MODULE_INTERNALS = '__module_vshard_registry' + +local M = rawget(_G, MODULE_INTERNALS) +if not M then + M = {} + rawset(_G, MODULE_INTERNALS, M) +end + +return M diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua index 9b74bcb..b47665b 100644 --- a/vshard/storage/init.lua +++ b/vshard/storage/init.lua @@ -16,7 +16,7 @@ if rawget(_G, MODULE_INTERNALS) then 'vshard.consts', 'vshard.error', 'vshard.cfg', 'vshard.replicaset', 'vshard.util', 'vshard.storage.reload_evolution', - 'vshard.lua_gc', 'vshard.rlist' + 'vshard.lua_gc', 'vshard.rlist', 'vshard.registry', } for _, module in pairs(vshard_modules) do package.loaded[module] = nil @@ -29,6 +29,7 @@ local lcfg = require('vshard.cfg') local lreplicaset = require('vshard.replicaset') local util = require('vshard.util') local lua_gc = require('vshard.lua_gc') +local lregistry = require('vshard.registry') local reload_evolution = require('vshard.storage.reload_evolution') local bucket_ref_new @@ -2782,6 +2783,8 @@ M.schema_upgrade_handlers = schema_upgrade_handlers M.schema_version_make = schema_version_make M.schema_bootstrap = schema_init_0_1_15_0 +lregistry.storage = M + return { sync = sync, bucket_force_create = bucket_force_create, -- 2.24.3 (Apple Git-128)