From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 0C4A271814; Tue, 23 Feb 2021 03:20:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0C4A271814 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1614039658; bh=m10bIPGEjGIYKeIX/61GEkZy+Clfvhx5878VrYPxNao=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=M7YGqwHWbiqaeagZTN92a9lRWdUQBkrc+UOJpNi4PXQ0+hxNiPbXadjxyCFmHWHA8 WiC12pW5fMyrG/DGCF45H4+Q+ui2cHSbLYw4jM/xmlpOQQrE9zuUENKPAQzH0wFtbc FOzvlIHR+MrblA3za6cd7eNi0d88kzXz2zphiH48= Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id EED346BD3F for ; Tue, 23 Feb 2021 03:16:01 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org EED346BD3F Received: by smtpng2.m.smailru.net with esmtpa (envelope-from ) id 1lELMv-0003CR-2C; Tue, 23 Feb 2021 03:16:01 +0300 To: tarantool-patches@dev.tarantool.org, olegrok@tarantool.org, yaroslav.dynnikov@tarantool.org Date: Tue, 23 Feb 2021 01:15:46 +0100 Message-Id: <73d839321957f99f69ba662f5a3655d31c3e99cd.1614039039.git.v.shpilevoy@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD975C3EC174F56692242B8E6687D03D8974314021AB65B8FCC182A05F5380850403887A97B237197F0608AB18FAADF9338A530759125FD615EDFF79C8CBC9608A1 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7EED5D2FAB4CEB1EDEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637DB576DCB83B448D28638F802B75D45FF5571747095F342E8C7A0BC55FA0FE5FC425FEB8730D384037A3982A2A0EEC9B9F31CF7D55693E1E6389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C008ED818AD11EC8E48941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6AE75D666CF637F52CC7F00164DA146DA6F5DAA56C3B73B23C77107234E2CFBA567F23339F89546C55F5C1EE8F4F765FC08F9A42B2210255C75ECD9A6C639B01BBD4B6F7A4D31EC0BC0CAF46E325F83A522CA9DD8327EE4930A3850AC1BE2E73579C543ECCDAE434EC4224003CC836476C0CAF46E325F83A50BF2EBBBDD9D6B0F347543BADC64E7283B503F486389A921A5CC5B56E945C8DA X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C4C7A0BC55FA0FE5FC425FEB8730D384037A3982A2A0EEC9B91324EEAE9290E256B1881A6453793CE9C32612AADDFBE061C61BE10805914D3804EBA3D8E7E5B87ABF8C51168CD8EBDB587F3D2152687E5CDC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A246DFF68F0EE19E4C3DA30B2E2F5BD6B8D62CCCA15980F675EEFA9DCDA99298B0D888F00CF38E421D7E09C32AA3244CEA4BEA6CAC250AFAF5201AB5C68A434695A9E0DC41E9A4CFFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj2drqE2xHc+HPIpCAWy1hsA== X-Mailru-Sender: 689FA8AB762F73936BC43F508A06382275295F1FB54D83C4717E8FB40B7B95D53841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH vshard 08/11] storage: introduce bucket_are_all_rw() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" In the future map-reduce code it will be needed to be able to check if all buckets on the storage are in writable state. If they are - any request can do anything with all the data on the storage. Such 'all writable' state will be pinned by a new module 'storage_ref' so as map-reduce requests could execute without being afraid of the rebalancer. The patch adds a function bucket_are_all_rw() which is registered in registry.storage. The function is not trivial because tries to cache the returned value. It makes a lot of sense, because the value changes super rare and the calculation costs a lot (4 lookups in an index by a string key via FFI + each lookup returns a tuple which is +1 Lua GC object). The function is going to be used almost on each map-reduce request, so it must be fast. Part of #147 --- test/storage/storage.result | 37 +++++++++++++++++++++++++++++++++++ test/storage/storage.test.lua | 14 +++++++++++++ vshard/storage/init.lua | 37 +++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+) diff --git a/test/storage/storage.result b/test/storage/storage.result index 4730e20..2c9784a 100644 --- a/test/storage/storage.result +++ b/test/storage/storage.result @@ -808,6 +808,43 @@ assert(not ok and err.message) --- - fiber is cancelled ... +-- +-- Bucket_are_all_rw() registry function. +-- +assert(lstorage.bucket_are_all_rw()) +--- +- true +... +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = true +--- +... +-- Let it stuck in the errinj. +vshard.storage.recovery_wakeup() +--- +... +vshard.storage.bucket_force_create(10) +--- +- true +... +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.SENDING}}) +--- +- [10, 'sending'] +... +assert(not lstorage.bucket_are_all_rw()) +--- +- true +... +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.ACTIVE}}) +--- +- [10, 'active'] +... +assert(lstorage.bucket_are_all_rw()) +--- +- true +... +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = false +--- +... _ = test_run:switch("default") --- ... diff --git a/test/storage/storage.test.lua b/test/storage/storage.test.lua index 86c5e33..33f0498 100644 --- a/test/storage/storage.test.lua +++ b/test/storage/storage.test.lua @@ -241,6 +241,20 @@ f:cancel() _ = test_run:wait_cond(function() return ok or err end) assert(not ok and err.message) +-- +-- Bucket_are_all_rw() registry function. +-- +assert(lstorage.bucket_are_all_rw()) +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = true +-- Let it stuck in the errinj. +vshard.storage.recovery_wakeup() +vshard.storage.bucket_force_create(10) +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.SENDING}}) +assert(not lstorage.bucket_are_all_rw()) +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.ACTIVE}}) +assert(lstorage.bucket_are_all_rw()) +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = false + _ = test_run:switch("default") test_run:drop_cluster(REPLICASET_2) test_run:drop_cluster(REPLICASET_1) diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua index ffa48b6..c3ed236 100644 --- a/vshard/storage/init.lua +++ b/vshard/storage/init.lua @@ -115,6 +115,10 @@ if not M then -- Fast alternative to box.space._bucket:count(). But may be nil. Reset -- on each generation change. bucket_count_cache = nil, + -- Fast alternative to checking multiple keys presence in + -- box.space._bucket status index. But may be nil. Reset on each + -- generation change. + bucket_are_all_rw_cache = nil, -- Redirects for recently sent buckets. They are kept for a while to -- help routers to find a new location for sent and deleted buckets -- without whole cluster scan. @@ -220,12 +224,44 @@ local function bucket_count_public() return bucket_count() end +-- +-- Check if all buckets on the storage are writable. The idea is the same as +-- with bucket count - the value changes very rare, and is cached most of the +-- time. Only that its non-cached calculation is more expensive than with count. +-- +local bucket_are_all_rw + +local function bucket_are_all_rw_cache() + return M.bucket_are_all_rw_cache +end + +local function bucket_are_all_rw_not_cache() + local status_index = box.space._bucket.index.status + local status = consts.BUCKET + local res = not status_index:min(status.SENDING) and + not status_index:min(status.SENT) and + not status_index:min(status.RECEIVING) and + not status_index:min(status.GARBAGE) + + M.bucket_are_all_rw_cache = res + bucket_are_all_rw = bucket_are_all_rw_cache + return res +end + +bucket_are_all_rw = bucket_are_all_rw_not_cache + +local function bucket_are_all_rw_public() + return bucket_are_all_rw() +end + -- -- Trigger for on replace into _bucket to update its generation. -- local function bucket_generation_increment() bucket_count = bucket_count_not_cache + bucket_are_all_rw = bucket_are_all_rw_not_cache M.bucket_count_cache = nil + M.bucket_are_all_rw_cache = nil M.bucket_generation = M.bucket_generation + 1 M.bucket_generation_cond:broadcast() end @@ -2788,6 +2824,7 @@ M.schema_upgrade_handlers = schema_upgrade_handlers M.schema_version_make = schema_version_make M.schema_bootstrap = schema_init_0_1_15_0 +M.bucket_are_all_rw = bucket_are_all_rw_public M.bucket_generation_wait = bucket_generation_wait lregistry.storage = M -- 2.24.3 (Apple Git-128)