From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id A1B0370202; Wed, 24 Feb 2021 13:30:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A1B0370202 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1614162650; bh=9MjxweNTmueXlrFXFcdmgZyaM69dZCCL5YoLzlBnB74=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=H7i+1cIYxi6NywVeLa73JN05hkE7H1NLtcrKql546lL+Ne3dXaBAYYKVlf7Ag1qMR dBYXms1VFk2WN/wxXFNX0MA9y0WvyqEstmjX5m63YmFdFV5/VzjZAo2gh6+TYRaPwn JsvmgBZK+LUW9HXuHwBHnZNvkfZh/Qp6cY/RP4xs= Received: from smtp46.i.mail.ru (smtp46.i.mail.ru [94.100.177.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id D2E6A7185E for ; Wed, 24 Feb 2021 13:27:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D2E6A7185E Received: by smtp46.i.mail.ru with esmtpa (envelope-from ) id 1lErOd-0004jc-RP; Wed, 24 Feb 2021 13:27:56 +0300 To: Vladislav Shpilevoy , tarantool-patches@dev.tarantool.org, yaroslav.dynnikov@tarantool.org References: <73d839321957f99f69ba662f5a3655d31c3e99cd.1614039039.git.v.shpilevoy@tarantool.org> Message-ID: <67990e86-75a2-89c6-4363-2f62c1e46fb3@tarantool.org> Date: Wed, 24 Feb 2021 13:27:55 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <73d839321957f99f69ba662f5a3655d31c3e99cd.1614039039.git.v.shpilevoy@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD975C3EC174F56692243410BA6471F0166336C1783AA96243D182A05F538085040466594ED6B30796CA2094152E9CADC1F51328B315D57579D1CEB1F7372A7B832 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7922D113DFDC6D5A3EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063795AFAF91F541EBCE8638F802B75D45FF5571747095F342E8C7A0BC55FA0FE5FC37516A87862161414B4503C15590019ADA07E0815E982F3E389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0DEC8C2C8BCD2534D8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B64AAE2D1698E8717BCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C22495E9FA80E707FC84476E601842F6C81A12EF20D2F80756B5F7E9C4E3C761E06A776E601842F6C81A127C277FBC8AE2E8B0E6D796F508274C4D81D268191BDAD3D698AB9A7B718F8C442539A7722CA490C13377AFFFEAFD26923F8577A6DFFEA7CB59C7783CC88FA9693EC92FD9297F6715571747095F342E857739F23D657EF2BD5E8D9A59859A8B655EC1579764E9EAF089D37D7C0E48F6C5571747095F342E857739F23D657EF2B6825BDBE14D8E7028C9DFF55498CEFB0BD9CCCA9EDD067B1EDA766A37F9254B7 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A24A6D60772A99906F8E1CD14B953EB46DCD8220F05D94508A355D89D7DBCDD132 X-C1DE0DAB: 0D63561A33F958A58A67AB5958DCE743D186865DBAD163A7523AF3F6409E32D8D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75B7BFB303F1C7DB4D8E8E86DC7131B365E7726E8460B7C23C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34EC58E35A639A2471483220FD4996D800413A7D3F9A95F96AA1267C18FA63E880D900D34DF347BAA41D7E09C32AA3244C06860D3DD0D8F57A68C22FC0BE68864351E887DA02A9F7BFFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojyK6JYJ15DtLNKU2IHjz70g== X-Mailru-Sender: 583F1D7ACE8F49BD9317CE1922F30C7E17DA322882E9F3E479C4D397B88637EB988CC5A86461B1B123E75C7104EB1B885DEE61814008E47C7013064206BFB89F93956FB04BA385BE9437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH vshard 08/11] storage: introduce bucket_are_all_rw() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Oleg Babin via Tarantool-patches Reply-To: Oleg Babin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thanks for your patch. Seems here I should return to one of my previous e-mail. Maybe it's reasonable to cache all bucket stats? On 23.02.2021 03:15, Vladislav Shpilevoy wrote: > In the future map-reduce code it will be needed to be able to > check if all buckets on the storage are in writable state. If they > are - any request can do anything with all the data on the > storage. > > Such 'all writable' state will be pinned by a new module > 'storage_ref' so as map-reduce requests could execute without > being afraid of the rebalancer. > > The patch adds a function bucket_are_all_rw() which is registered > in registry.storage. > > The function is not trivial because tries to cache the returned > value. It makes a lot of sense, because the value changes super > rare and the calculation costs a lot (4 lookups in an index by a > string key via FFI + each lookup returns a tuple which is +1 Lua > GC object). > > The function is going to be used almost on each map-reduce > request, so it must be fast. > > Part of #147 > --- > test/storage/storage.result | 37 +++++++++++++++++++++++++++++++++++ > test/storage/storage.test.lua | 14 +++++++++++++ > vshard/storage/init.lua | 37 +++++++++++++++++++++++++++++++++++ > 3 files changed, 88 insertions(+) > > diff --git a/test/storage/storage.result b/test/storage/storage.result > index 4730e20..2c9784a 100644 > --- a/test/storage/storage.result > +++ b/test/storage/storage.result > @@ -808,6 +808,43 @@ assert(not ok and err.message) > --- > - fiber is cancelled > ... > +-- > +-- Bucket_are_all_rw() registry function. > +-- > +assert(lstorage.bucket_are_all_rw()) > +--- > +- true > +... > +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = true > +--- > +... > +-- Let it stuck in the errinj. > +vshard.storage.recovery_wakeup() > +--- > +... > +vshard.storage.bucket_force_create(10) > +--- > +- true > +... > +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.SENDING}}) > +--- > +- [10, 'sending'] > +... > +assert(not lstorage.bucket_are_all_rw()) > +--- > +- true > +... > +box.space._bucket:update(10, {{'=', 2, vshard.consts.BUCKET.ACTIVE}}) > +--- > +- [10, 'active'] > +... > +assert(lstorage.bucket_are_all_rw()) > +--- > +- true > +... > +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = false > +--- > +... > _ = test_run:switch("default") --- ... diff --git a/test/storage/storage.test.lua > b/test/storage/storage.test.lua index 86c5e33..33f0498 100644 --- > a/test/storage/storage.test.lua +++ b/test/storage/storage.test.lua @@ > -241,6 +241,20 @@ f:cancel() _ = test_run:wait_cond(function() return > ok or err end) assert(not ok and err.message) +-- +-- > Bucket_are_all_rw() registry function. +-- > +assert(lstorage.bucket_are_all_rw()) > +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = true +-- Let it > stuck in the errinj. +vshard.storage.recovery_wakeup() > +vshard.storage.bucket_force_create(10) +box.space._bucket:update(10, > {{'=', 2, vshard.consts.BUCKET.SENDING}}) +assert(not > lstorage.bucket_are_all_rw()) +box.space._bucket:update(10, {{'=', 2, > vshard.consts.BUCKET.ACTIVE}}) +assert(lstorage.bucket_are_all_rw()) > +vshard.storage.internal.errinj.ERRINJ_NO_RECOVERY = false + _ = > test_run:switch("default") > test_run:drop_cluster(REPLICASET_2) > test_run:drop_cluster(REPLICASET_1) > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index ffa48b6..c3ed236 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -115,6 +115,10 @@ if not M then > -- Fast alternative to box.space._bucket:count(). But may be nil. Reset > -- on each generation change. > bucket_count_cache = nil, > + -- Fast alternative to checking multiple keys presence in > + -- box.space._bucket status index. But may be nil. Reset on each > + -- generation change. > + bucket_are_all_rw_cache = nil, > -- Redirects for recently sent buckets. They are kept for a while to > -- help routers to find a new location for sent and deleted buckets > -- without whole cluster scan. > @@ -220,12 +224,44 @@ local function bucket_count_public() > return bucket_count() > end > > +-- > +-- Check if all buckets on the storage are writable. The idea is the same as > +-- with bucket count - the value changes very rare, and is cached most of the > +-- time. Only that its non-cached calculation is more expensive than with count. > +-- > +local bucket_are_all_rw > + > +local function bucket_are_all_rw_cache() > + return M.bucket_are_all_rw_cache > +end > + > +local function bucket_are_all_rw_not_cache() > + local status_index = box.space._bucket.index.status > + local status = consts.BUCKET > + local res = not status_index:min(status.SENDING) and > + not status_index:min(status.SENT) and > + not status_index:min(status.RECEIVING) and > + not status_index:min(status.GARBAGE) > + > + M.bucket_are_all_rw_cache = res > + bucket_are_all_rw = bucket_are_all_rw_cache > + return res > +end > + > +bucket_are_all_rw = bucket_are_all_rw_not_cache > + > +local function bucket_are_all_rw_public() > + return bucket_are_all_rw() > +end > + > -- > -- Trigger for on replace into _bucket to update its generation. > -- > local function bucket_generation_increment() > bucket_count = bucket_count_not_cache > + bucket_are_all_rw = bucket_are_all_rw_not_cache > M.bucket_count_cache = nil > + M.bucket_are_all_rw_cache = nil > M.bucket_generation = M.bucket_generation + 1 > M.bucket_generation_cond:broadcast() > end > @@ -2788,6 +2824,7 @@ M.schema_upgrade_handlers = schema_upgrade_handlers > M.schema_version_make = schema_version_make > M.schema_bootstrap = schema_init_0_1_15_0 > > +M.bucket_are_all_rw = bucket_are_all_rw_public > M.bucket_generation_wait = bucket_generation_wait > lregistry.storage = M >