Tarantool development patches archive
 help / color / mirror / Atom feed
From: Oleg Babin via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>,
	tarantool-patches@dev.tarantool.org,
	yaroslav.dynnikov@tarantool.org
Subject: Re: [Tarantool-patches] [PATCH vshard 07/11] storage: introduce bucket_generation_wait()
Date: Wed, 24 Feb 2021 13:27:51 +0300	[thread overview]
Message-ID: <ebd4971e-567a-8c96-3db4-e68f74a090c4@tarantool.org> (raw)
In-Reply-To: <e687e462271f8ef2624a8833d69e9d883510a627.1614039039.git.v.shpilevoy@tarantool.org>

Thanks for your patch. LGTM.

On 23.02.2021 03:15, Vladislav Shpilevoy wrote:
> In the future map-reduce code it will be needed to be able to wait
> until all buckets on the storage enter writable state. If they are
> not writable, the code should wait efficiently, without polling.
>
> The patch adds a function bucket_generation_wait() which is
> registered in registry.storage.
>
> It helps to wait until state of any bucket is changed. The caller
> code, if wants to wait for all buckets to enter writable state,
> should wait on the generation and re-check the requested condition
> until it matches or timeout happens.
>
> Part of #147
> ---
>   test/storage/storage.result   | 86 +++++++++++++++++++++++++++++++++++
>   test/storage/storage.test.lua | 36 +++++++++++++++
>   vshard/storage/init.lua       |  6 +++
>   3 files changed, 128 insertions(+)
>
> diff --git a/test/storage/storage.result b/test/storage/storage.result
> index edb45be..4730e20 100644
> --- a/test/storage/storage.result
> +++ b/test/storage/storage.result
> @@ -722,6 +722,92 @@ assert(vshard.storage.buckets_count() == 10)
>   ---
>   - true
>   ...
> +--
> +-- Bucket_generation_wait() registry function.
> +--
> +lstorage = require('vshard.registry').storage
> +---
> +...
> +ok, err = lstorage.bucket_generation_wait(-1)
> +---
> +...
> +assert(not ok and err.message)
> +---
> +- Timeout exceeded
> +...
> +ok, err = lstorage.bucket_generation_wait(0)
> +---
> +...
> +assert(not ok and err.message)
> +---
> +- Timeout exceeded
> +...
> +small_timeout = 0.000001
> +---
> +...
> +ok, err = lstorage.bucket_generation_wait(small_timeout)
> +---
> +...
> +assert(not ok and err.message)
> +---
> +- Timeout exceeded
> +...
> +ok, err = nil
> +---
> +...
> +big_timeout = 1000000
> +---
> +...
> +_ = fiber.create(function()                                                     \
> +    ok, err = lstorage.bucket_generation_wait(big_timeout)                      \
> +end)
> +---
> +...
> +fiber.sleep(small_timeout)
> +---
> +...
> +assert(not ok and not err)
> +---
> +- true
> +...
> +vshard.storage.bucket_force_drop(10)
> +---
> +- true
> +...
> +test_run:wait_cond(function() return ok or err end)
> +---
> +- true
> +...
> +assert(ok)
> +---
> +- true
> +...
> +-- Cancel should interrupt the waiting.
> +ok, err = nil
> +---
> +...
> +f = fiber.create(function()                                                     \
> +    ok, err = lstorage.bucket_generation_wait(big_timeout)                      \
> +end)
> +---
> +...
> +fiber.sleep(small_timeout)
> +---
> +...
> +assert(not ok and not err)
> +---
> +- true
> +...
> +f:cancel()
> +---
> +...
> +_ = test_run:wait_cond(function() return ok or err end)
> +---
> +...
> +assert(not ok and err.message)
> +---
> +- fiber is cancelled
> +...
>   _ = test_run:switch("default") --- ... diff --git a/test/storage/storage.test.lua 
> b/test/storage/storage.test.lua index db014ef..86c5e33 100644 --- 
> a/test/storage/storage.test.lua +++ b/test/storage/storage.test.lua @@ 
> -205,6 +205,42 @@ assert(vshard.storage.buckets_count() == 5) 
> vshard.storage.bucket_force_create(6, 5) 
> assert(vshard.storage.buckets_count() == 10) +-- +-- 
> Bucket_generation_wait() registry function. +-- +lstorage = 
> require('vshard.registry').storage +ok, err = 
> lstorage.bucket_generation_wait(-1) +assert(not ok and err.message) + 
> +ok, err = lstorage.bucket_generation_wait(0) +assert(not ok and 
> err.message) + +small_timeout = 0.000001 +ok, err = 
> lstorage.bucket_generation_wait(small_timeout) +assert(not ok and 
> err.message) + +ok, err = nil +big_timeout = 1000000 +_ = 
> fiber.create(function() \ + ok, err = 
> lstorage.bucket_generation_wait(big_timeout) \ +end) 
> +fiber.sleep(small_timeout) +assert(not ok and not err) 
> +vshard.storage.bucket_force_drop(10) +test_run:wait_cond(function() 
> return ok or err end) +assert(ok) + +-- Cancel should interrupt the 
> waiting. +ok, err = nil +f = fiber.create(function() \ + ok, err = 
> lstorage.bucket_generation_wait(big_timeout) \ +end) 
> +fiber.sleep(small_timeout) +assert(not ok and not err) +f:cancel() +_ 
> = test_run:wait_cond(function() return ok or err end) +assert(not ok 
> and err.message) + _ = test_run:switch("default")
>   test_run:drop_cluster(REPLICASET_2)
>   test_run:drop_cluster(REPLICASET_1)
> diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua
> index b47665b..ffa48b6 100644
> --- a/vshard/storage/init.lua
> +++ b/vshard/storage/init.lua
> @@ -31,6 +31,7 @@ local util = require('vshard.util')
>   local lua_gc = require('vshard.lua_gc')
>   local lregistry = require('vshard.registry')
>   local reload_evolution = require('vshard.storage.reload_evolution')
> +local fiber_cond_wait = util.fiber_cond_wait
>   local bucket_ref_new
>   
>   local M = rawget(_G, MODULE_INTERNALS)
> @@ -229,6 +230,10 @@ local function bucket_generation_increment()
>       M.bucket_generation_cond:broadcast()
>   end
>   
> +local function bucket_generation_wait(timeout)
> +    return fiber_cond_wait(M.bucket_generation_cond, timeout)
> +end
> +
>   --
>   -- Check if this replicaset is locked. It means be invisible for
>   -- the rebalancer.
> @@ -2783,6 +2788,7 @@ M.schema_upgrade_handlers = schema_upgrade_handlers
>   M.schema_version_make = schema_version_make
>   M.schema_bootstrap = schema_init_0_1_15_0
>   
> +M.bucket_generation_wait = bucket_generation_wait
>   lregistry.storage = M
>   
>   return {

  reply	other threads:[~2021-02-24 10:30 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-23  0:15 [Tarantool-patches] [PATCH vshard 00/11] VShard Map-Reduce, part 2: Ref, Sched, Map Vladislav Shpilevoy via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 01/11] error: introduce vshard.error.timeout() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-24 21:46     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-25 12:42       ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 10/11] sched: introduce vshard.storage.sched module Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:28   ` Oleg Babin via Tarantool-patches
2021-02-24 21:50     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-04 21:02   ` Oleg Babin via Tarantool-patches
2021-03-05 22:06     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-09  8:03       ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 11/11] router: introduce map_callrw() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:28   ` Oleg Babin via Tarantool-patches
2021-02-24 22:04     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-25 12:43       ` Oleg Babin via Tarantool-patches
2021-02-26 23:58         ` Vladislav Shpilevoy via Tarantool-patches
2021-03-01 10:58           ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 02/11] storage: add helper for local functions invocation Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 03/11] storage: cache bucket count Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-24 21:47     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-25 12:42       ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 04/11] registry: module for circular deps resolution Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 05/11] util: introduce safe fiber_cond_wait() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-24 21:48     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-25 12:42       ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 06/11] util: introduce fiber_is_self_canceled() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 07/11] storage: introduce bucket_generation_wait() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches [this message]
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 08/11] storage: introduce bucket_are_all_rw() Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:27   ` Oleg Babin via Tarantool-patches
2021-02-24 21:48     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-23  0:15 ` [Tarantool-patches] [PATCH vshard 09/11] ref: introduce vshard.storage.ref module Vladislav Shpilevoy via Tarantool-patches
2021-02-24 10:28   ` Oleg Babin via Tarantool-patches
2021-02-24 21:49     ` Vladislav Shpilevoy via Tarantool-patches
2021-02-25 12:42       ` Oleg Babin via Tarantool-patches
2021-03-04 21:22   ` Oleg Babin via Tarantool-patches
2021-03-05 22:06     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-09  8:03       ` Oleg Babin via Tarantool-patches
2021-03-21 18:49   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-12 23:13 ` [Tarantool-patches] [PATCH vshard 00/11] VShard Map-Reduce, part 2: Ref, Sched, Map Vladislav Shpilevoy via Tarantool-patches
2021-03-15  7:05   ` Oleg Babin via Tarantool-patches
2021-03-28 18:17 ` Vladislav Shpilevoy via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ebd4971e-567a-8c96-3db4-e68f74a090c4@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=olegrok@tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --cc=yaroslav.dynnikov@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH vshard 07/11] storage: introduce bucket_generation_wait()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox