Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladislav Shpilevoy via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: tarantool-patches@dev.tarantool.org, olegrok@tarantool.org
Subject: [Tarantool-patches] [PATCH vshard 2/5] storage: auto enable/disable
Date: Fri, 17 Dec 2021 01:25:28 +0100	[thread overview]
Message-ID: <9d767a02cabc032ff4ad478b4a51c0a254276569.1639700518.git.v.shpilevoy@tarantool.org> (raw)
In-Reply-To: <cover.1639700518.git.v.shpilevoy@tarantool.org>

While vshard.storage.cfg() is not done, accessing vshard functions
is not safe. It will fail with low level errors like
'access denied' or 'no such function'.

However there can be even worse cases. The user can have universe
access rights. And vshard can be already in global namespace after
require(). So vshard.storage functions are already visible.

The previous patch fixed only the case when function access was
restricted properly. And still fixed it just partially.

New problems are:

- box.cfg{} is already called, but the instance is still
  'loading'. Then data is not fully recovered yet. Accessing is
  not safe from the data consistency perspective.

- vshard.storage.cfg() is not started, or is not finished yet. In
  the end it might be doing something on what the public functions
  depend.

This patch addresses these issues. Now all non-trivial
vshard.storage functions are disabled until vshard.storage.cfg()
is finished and the instance is fully recovered.

They raise an error with a special code. Returning it via
'nil, err' pair wouldn't work. Because firstly, some functions
return a boolean value and are not documented as ever failing.
People would miss this new error.

Second reason - vshard.storage.call() needs to signal the remote
caller that the storage is disabled and it was found before the
user's function was called. If it would be done via 'nil, err',
then the user's function could emulate the storage being disabled.
Or even worse, it could make some changes and then get that error
accidentally by going to another storage remotely which would be
disabled. Hence it is not allowed. Too easy to break something.

It was an option to change vshard.storage.call() signature to
return 'true, retvals...' when user's function was called and
'false, err' when it wasn't, but that would break backward
compatibility. Supporting it only for new routers does not seem
possible.

The patch also drops 'memtx_memory' setting from the config
because an attempt to apply it after calling box.cfg() (for
example, via boot_like_vshard()) raises an error - default memory
is bigger than this setting. It messed the new tests.

Part of #298
Closes #123
---
 example/localcfg.lua          |   1 -
 test/storage/storage.result   | 101 +++++++++++++++++++
 test/storage/storage.test.lua |  43 ++++++++
 test/unit/garbage.result      |  17 ++--
 test/unit/garbage.test.lua    |  15 +--
 test/unit/rebalancer.result   |   2 +-
 test/unit/rebalancer.test.lua |   2 +-
 vshard/error.lua              |   5 +
 vshard/storage/init.lua       | 179 ++++++++++++++++++++++++++--------
 9 files changed, 305 insertions(+), 60 deletions(-)

diff --git a/example/localcfg.lua b/example/localcfg.lua
index 71f9d6f..9235d45 100644
--- a/example/localcfg.lua
+++ b/example/localcfg.lua
@@ -1,5 +1,4 @@
 return {
-    memtx_memory = 100 * 1024 * 1024,
     sharding = {
         ['cbf06940-0790-498b-948d-042b62cf3d29'] = { -- replicaset #1
             replicas = {
diff --git a/test/storage/storage.result b/test/storage/storage.result
index af48a13..e83b34f 100644
--- a/test/storage/storage.result
+++ b/test/storage/storage.result
@@ -951,6 +951,107 @@ vshard.storage._call('info')
 ---
 - is_master: false
 ...
+--
+-- gh-123, gh-298: storage auto-enable/disable depending on instance state.
+--
+_ = test_run:cmd('stop server storage_1_a')
+---
+...
+_ = test_run:cmd('start server storage_1_a with wait=False, '..                 \
+                 'args="boot_before_cfg"')
+---
+...
+_ = test_run:switch('storage_1_a')
+---
+...
+-- Leaving box.cfg() not called won't work because at 1.10 test-run somewhy
+-- raises an error when try to start an instance without box.cfg(). It can only
+-- be emulated.
+old_info = box.info
+---
+...
+box.info = setmetatable({}, {__index = function() error('not configured') end})
+---
+...
+assert(not pcall(function() return box.info.status end))
+---
+- true
+...
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+---
+...
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+---
+- true
+...
+assert(err.message:match('box seem not to be configured') ~= nil)
+---
+- true
+...
+box.info = old_info
+---
+...
+-- Disabled until box is loaded.
+vshard.storage.internal.errinj.ERRINJ_CFG_DELAY = true
+---
+...
+f = fiber.create(vshard.storage.cfg, cfg, instance_uuid)
+---
+...
+f:set_joinable(true)
+---
+...
+old_info = box.info
+---
+...
+box.info = {status = 'loading'}
+---
+...
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+---
+...
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+---
+- true
+...
+assert(err.message:match('instance status is "loading"') ~= nil)
+---
+- true
+...
+box.info = old_info
+---
+...
+-- Disabled until storage is configured.
+test_run:wait_cond(function() return box.info.status == 'running' end)
+---
+- true
+...
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+---
+...
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+---
+- true
+...
+assert(err.message:match('storage is not configured') ~= nil)
+---
+- true
+...
+vshard.storage.internal.errinj.ERRINJ_CFG_DELAY = false
+---
+...
+f:join()
+---
+- true
+...
+-- Enabled when all criteria are finally satisfied.
+ok, res = vshard.storage.call(1, 'read', 'echo', {100})
+---
+...
+assert(ok and res == 100)
+---
+- true
+...
 _ = test_run:switch("default")
 ---
 ...
diff --git a/test/storage/storage.test.lua b/test/storage/storage.test.lua
index 99ef2c4..ff39f2f 100644
--- a/test/storage/storage.test.lua
+++ b/test/storage/storage.test.lua
@@ -299,6 +299,49 @@ vshard.storage._call('info')
 _ = test_run:switch('storage_1_b')
 vshard.storage._call('info')
 
+--
+-- gh-123, gh-298: storage auto-enable/disable depending on instance state.
+--
+_ = test_run:cmd('stop server storage_1_a')
+_ = test_run:cmd('start server storage_1_a with wait=False, '..                 \
+                 'args="boot_before_cfg"')
+_ = test_run:switch('storage_1_a')
+-- Leaving box.cfg() not called won't work because at 1.10 test-run somewhy
+-- raises an error when try to start an instance without box.cfg(). It can only
+-- be emulated.
+old_info = box.info
+box.info = setmetatable({}, {__index = function() error('not configured') end})
+assert(not pcall(function() return box.info.status end))
+
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+assert(err.message:match('box seem not to be configured') ~= nil)
+box.info = old_info
+
+-- Disabled until box is loaded.
+vshard.storage.internal.errinj.ERRINJ_CFG_DELAY = true
+f = fiber.create(vshard.storage.cfg, cfg, instance_uuid)
+f:set_joinable(true)
+
+old_info = box.info
+box.info = {status = 'loading'}
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+assert(err.message:match('instance status is "loading"') ~= nil)
+box.info = old_info
+
+-- Disabled until storage is configured.
+test_run:wait_cond(function() return box.info.status == 'running' end)
+ok, err = pcall(vshard.storage.call, 1, 'read', 'echo', {100})
+assert(not ok and err.code == vshard.error.code.STORAGE_IS_DISABLED)
+assert(err.message:match('storage is not configured') ~= nil)
+vshard.storage.internal.errinj.ERRINJ_CFG_DELAY = false
+f:join()
+
+-- Enabled when all criteria are finally satisfied.
+ok, res = vshard.storage.call(1, 'read', 'echo', {100})
+assert(ok and res == 100)
+
 _ = test_run:switch("default")
 test_run:drop_cluster(REPLICASET_2)
 test_run:drop_cluster(REPLICASET_1)
diff --git a/test/unit/garbage.result b/test/unit/garbage.result
index a530496..c4b0fc3 100644
--- a/test/unit/garbage.result
+++ b/test/unit/garbage.result
@@ -16,7 +16,7 @@ test_run:cmd("setopt delimiter ';'")
 ...
 function show_sharded_spaces()
     local result = {}
-    for k, space in pairs(vshard.storage.sharded_spaces()) do
+    for k, space in pairs(vshard.storage._sharded_spaces()) do
         table.insert(result, space.name)
     end
     table.sort(result)
@@ -489,7 +489,10 @@ f:cancel()
 util = require('util')
 ---
 ...
-util.check_error(vshard.storage.bucket_delete_garbage)
+delete_garbage = vshard.storage._bucket_delete_garbage
+---
+...
+util.check_error(delete_garbage)
 ---
 - 'Usage: bucket_delete_garbage(bucket_id, opts)'
 ...
@@ -506,7 +509,7 @@ s:replace{6, 4}
 ---
 - [6, 4]
 ...
-vshard.storage.bucket_delete_garbage(4)
+delete_garbage(4)
 ---
 ...
 s:select{}
@@ -526,7 +529,7 @@ s:replace{6, 4}
 ---
 - [6, 4]
 ...
-vshard.storage.bucket_delete_garbage(4)
+delete_garbage(4)
 ---
 ...
 s:select{}
@@ -547,16 +550,16 @@ s:replace{6, 4}
 ---
 - [6, 4]
 ...
-util.check_error(vshard.storage.bucket_delete_garbage, 4)
+util.check_error(delete_garbage, 4)
 ---
 - Can not delete not garbage bucket. Use "{force=true}" to ignore this attention
 ...
-util.check_error(vshard.storage.bucket_delete_garbage, 4, 10000)
+util.check_error(delete_garbage, 4, 10000)
 ---
 - 'Usage: bucket_delete_garbage(bucket_id, opts)'
 ...
 -- 'Force' option ignores this error.
-vshard.storage.bucket_delete_garbage(4, {force = true})
+delete_garbage(4, {force = true})
 ---
 ...
 s:select{}
diff --git a/test/unit/garbage.test.lua b/test/unit/garbage.test.lua
index 250afb0..1416f58 100644
--- a/test/unit/garbage.test.lua
+++ b/test/unit/garbage.test.lua
@@ -6,7 +6,7 @@ engine = test_run:get_cfg('engine')
 test_run:cmd("setopt delimiter ';'")
 function show_sharded_spaces()
     local result = {}
-    for k, space in pairs(vshard.storage.sharded_spaces()) do
+    for k, space in pairs(vshard.storage._sharded_spaces()) do
         table.insert(result, space.name)
     end
     table.sort(result)
@@ -197,30 +197,31 @@ f:cancel()
 --
 util = require('util')
 
-util.check_error(vshard.storage.bucket_delete_garbage)
+delete_garbage = vshard.storage._bucket_delete_garbage
+util.check_error(delete_garbage)
 
 -- Delete an existing garbage bucket.
 _bucket:replace{4, vshard.consts.BUCKET.SENT}
 s:replace{5, 4}
 s:replace{6, 4}
-vshard.storage.bucket_delete_garbage(4)
+delete_garbage(4)
 s:select{}
 
 -- Delete a not existing garbage bucket.
 _ = _bucket:delete{4}
 s:replace{5, 4}
 s:replace{6, 4}
-vshard.storage.bucket_delete_garbage(4)
+delete_garbage(4)
 s:select{}
 
 -- Fail to delete a not garbage bucket.
 _bucket:replace{4, vshard.consts.BUCKET.ACTIVE}
 s:replace{5, 4}
 s:replace{6, 4}
-util.check_error(vshard.storage.bucket_delete_garbage, 4)
-util.check_error(vshard.storage.bucket_delete_garbage, 4, 10000)
+util.check_error(delete_garbage, 4)
+util.check_error(delete_garbage, 4, 10000)
 -- 'Force' option ignores this error.
-vshard.storage.bucket_delete_garbage(4, {force = true})
+delete_garbage(4, {force = true})
 s:select{}
 
 --
diff --git a/test/unit/rebalancer.result b/test/unit/rebalancer.result
index 19aa480..a931d63 100644
--- a/test/unit/rebalancer.result
+++ b/test/unit/rebalancer.result
@@ -287,7 +287,7 @@ build_routes(replicasets)
 --
 -- Test rebalancer local state.
 --
-get_state = vshard.storage.rebalancer_request_state
+get_state = vshard.storage._rebalancer_request_state
 ---
 ...
 _bucket = box.schema.create_space('_bucket')
diff --git a/test/unit/rebalancer.test.lua b/test/unit/rebalancer.test.lua
index 8087d42..1cdbbcb 100644
--- a/test/unit/rebalancer.test.lua
+++ b/test/unit/rebalancer.test.lua
@@ -75,7 +75,7 @@ build_routes(replicasets)
 --
 -- Test rebalancer local state.
 --
-get_state = vshard.storage.rebalancer_request_state
+get_state = vshard.storage._rebalancer_request_state
 _bucket = box.schema.create_space('_bucket')
 pk = _bucket:create_index('pk')
 status = _bucket:create_index('status', {parts = {{2, 'string'}}, unique = false})
diff --git a/vshard/error.lua b/vshard/error.lua
index 0ce208a..2b97eae 100644
--- a/vshard/error.lua
+++ b/vshard/error.lua
@@ -165,6 +165,11 @@ local error_message_template = {
               'Last error was %s',
         args = {'replicaset_uuid', 'error'}
     },
+    [33] = {
+        name = 'STORAGE_IS_DISABLED',
+        msg = 'Storage is disabled: %s',
+        args = {'reason'}
+    },
 }
 
 --
diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua
index 3158f3c..d3c4e2a 100644
--- a/vshard/storage/init.lua
+++ b/vshard/storage/init.lua
@@ -134,6 +134,16 @@ if not M then
         -- help routers find a new location for sent and deleted buckets without
         -- whole cluster scan.
         route_map = {},
+        -- Flag whether vshard.storage.cfg() is finished.
+        is_configured = false,
+        -- Flag whether box.info.status is acceptable. For instance, 'loading'
+        -- is not.
+        is_loaded = false,
+        -- Reference to the function-proxy to most of the public functions. It
+        -- allows to avoid 'if's in each function by adding expensive
+        -- conditional checks in one rarely used version of the wrapper and no
+        -- checks into the other almost always used wrapper.
+        api_call_cache = nil,
 
         ------------------- Garbage collection -------------------
         -- Fiber to remove garbage buckets data.
@@ -670,19 +680,19 @@ local function this_is_master()
            M.this_replica == M.this_replicaset.master
 end
 
-local function on_master_disable(...)
-    M._on_master_disable(...)
+local function on_master_disable(new_func, old_func)
+    M._on_master_disable(new_func, old_func)
     -- If a trigger is set after storage.cfg(), then notify an
     -- user, that the current instance is not master.
-    if select('#', ...) == 1 and not this_is_master() then
+    if old_func == nil and not this_is_master() then
         M._on_master_disable:run()
     end
 end
 
-local function on_master_enable(...)
-    M._on_master_enable(...)
+local function on_master_enable(new_func, old_func)
+    M._on_master_enable(new_func, old_func)
     -- Same as above, but notify, that the instance is master.
-    if select('#', ...) == 1 and this_is_master() then
+    if old_func == nil and this_is_master() then
         M._on_master_enable:run()
     end
 end
@@ -1363,6 +1373,13 @@ local function find_sharded_spaces()
     return spaces
 end
 
+--
+-- Public wrapper for sharded spaces list getter.
+--
+local function storage_sharded_spaces()
+    return table.deepcopy(find_sharded_spaces())
+end
+
 if M.errinj.ERRINJ_RELOAD then
     error('Error injection: reload')
 end
@@ -2764,6 +2781,7 @@ local function storage_cfg(cfg, this_replica_uuid, is_reload)
         M.rebalancer_fiber = nil
     end
     lua_gc.set_state(M.collect_lua_garbage, consts.COLLECT_LUA_GARBAGE_INTERVAL)
+    M.is_configured = true
     -- Destroy connections, not used in a new configuration.
     collectgarbage()
 end
@@ -2915,6 +2933,58 @@ local function storage_info()
     return state
 end
 
+--------------------------------------------------------------------------------
+-- Public API protection
+--------------------------------------------------------------------------------
+
+--
+-- Arguments are listed explicitly instead of '...' because the latter does not
+-- jit.
+--
+local function storage_api_call_safe(func, arg1, arg2, arg3, arg4)
+    return func(arg1, arg2, arg3, arg4)
+end
+
+--
+-- Unsafe proxy is loaded with protections. But it is used rarely and only in
+-- the beginning of instance's lifetime.
+--
+local function storage_api_call_unsafe(func, arg1, arg2, arg3, arg4)
+    -- box.info is quite expensive. Avoid calling it again when the instance
+    -- is finally loaded.
+    if not M.is_loaded then
+        -- box.info raises an error until box.cfg() is started.
+        local ok, status = pcall(function()
+            return box.info.status
+        end)
+        if not ok then
+            local msg = 'box seem not to be configured'
+            return error(lerror.vshard(lerror.code.STORAGE_IS_DISABLED, msg))
+        end
+        -- 'Orphan' is allowed because even if a replica is an orphan, it still
+        -- could be up to date. Just not all other replicas are connected.
+        if status ~= 'running' and status ~= 'orphan' then
+            local msg = ('instance status is "%s"'):format(status)
+            return error(lerror.vshard(lerror.code.STORAGE_IS_DISABLED, msg))
+        end
+        M.is_loaded = true
+    end
+    if not M.is_configured then
+        local msg = 'storage is not configured'
+        return error(lerror.vshard(lerror.code.STORAGE_IS_DISABLED, msg))
+    end
+    M.api_call_cache = storage_api_call_safe
+    return func(arg1, arg2, arg3, arg4)
+end
+
+M.api_call_cache = storage_api_call_unsafe
+
+local function storage_make_api(func)
+    return function(arg1, arg2, arg3, arg4)
+        return M.api_call_cache(func, arg1, arg2, arg3, arg4)
+    end
+end
+
 --------------------------------------------------------------------------------
 -- Module definition
 --------------------------------------------------------------------------------
@@ -3021,44 +3091,67 @@ M.bucket_are_all_rw = bucket_are_all_rw_public
 M.bucket_generation_wait = bucket_generation_wait
 lregistry.storage = M
 
+--
+-- Not all methods are public here. Private methods should not be exposed if
+-- possible. At least not without notable difference in naming.
+--
 return {
-    sync = sync,
-    bucket_force_create = bucket_force_create,
-    bucket_force_drop = bucket_force_drop,
-    bucket_collect = bucket_collect,
-    bucket_recv = bucket_recv,
-    bucket_send = bucket_send,
-    bucket_stat = bucket_stat,
-    bucket_pin = bucket_pin,
-    bucket_unpin = bucket_unpin,
-    bucket_ref = bucket_ref,
-    bucket_unref = bucket_unref,
-    bucket_refro = bucket_refro,
-    bucket_refrw = bucket_refrw,
-    bucket_unrefro = bucket_unrefro,
-    bucket_unrefrw = bucket_unrefrw,
-    bucket_delete_garbage = bucket_delete_garbage,
-    garbage_collector_wakeup = garbage_collector_wakeup,
-    rebalancer_wakeup = rebalancer_wakeup,
-    rebalancer_apply_routes = rebalancer_apply_routes,
-    rebalancer_disable = rebalancer_disable,
-    rebalancer_enable = rebalancer_enable,
-    is_locked = is_this_replicaset_locked,
-    rebalancing_is_in_progress = rebalancing_is_in_progress,
-    recovery_wakeup = recovery_wakeup,
-    call = storage_call,
-    _call = service_call,
+    --
+    -- Bucket methods.
+    --
+    bucket_force_create = storage_make_api(bucket_force_create),
+    bucket_force_drop = storage_make_api(bucket_force_drop),
+    bucket_collect = storage_make_api(bucket_collect),
+    bucket_recv = storage_make_api(bucket_recv),
+    bucket_send = storage_make_api(bucket_send),
+    bucket_stat = storage_make_api(bucket_stat),
+    bucket_pin = storage_make_api(bucket_pin),
+    bucket_unpin = storage_make_api(bucket_unpin),
+    bucket_ref = storage_make_api(bucket_ref),
+    bucket_unref = storage_make_api(bucket_unref),
+    bucket_refro = storage_make_api(bucket_refro),
+    bucket_refrw = storage_make_api(bucket_refrw),
+    bucket_unrefro = storage_make_api(bucket_unrefro),
+    bucket_unrefrw = storage_make_api(bucket_unrefrw),
+    bucket_delete_garbage = storage_make_api(bucket_delete_garbage),
+    _bucket_delete_garbage = bucket_delete_garbage,
+    buckets_info = storage_make_api(storage_buckets_info),
+    buckets_count = storage_make_api(bucket_count_public),
+    buckets_discovery = storage_make_api(buckets_discovery),
+    --
+    -- Garbage collector.
+    --
+    garbage_collector_wakeup = storage_make_api(garbage_collector_wakeup),
+    --
+    -- Rebalancer.
+    --
+    rebalancer_wakeup = storage_make_api(rebalancer_wakeup),
+    rebalancer_apply_routes = storage_make_api(rebalancer_apply_routes),
+    rebalancer_disable = storage_make_api(rebalancer_disable),
+    rebalancer_enable = storage_make_api(rebalancer_enable),
+    rebalancing_is_in_progress = storage_make_api(rebalancing_is_in_progress),
+    rebalancer_request_state = storage_make_api(rebalancer_request_state),
+    _rebalancer_request_state = rebalancer_request_state,
+    --
+    -- Recovery.
+    --
+    recovery_wakeup = storage_make_api(recovery_wakeup),
+    --
+    -- Instance info.
+    --
+    is_locked = storage_make_api(is_this_replicaset_locked),
+    info = storage_make_api(storage_info),
+    sharded_spaces = storage_make_api(storage_sharded_spaces),
+    _sharded_spaces = storage_sharded_spaces,
+    module_version = function() return M.module_version end,
+    --
+    -- Miscellaneous.
+    --
+    call = storage_make_api(storage_call),
+    _call = storage_make_api(service_call),
+    sync = storage_make_api(sync),
     cfg = function(cfg, uuid) return storage_cfg(cfg, uuid, false) end,
-    info = storage_info,
-    buckets_info = storage_buckets_info,
-    buckets_count = bucket_count_public,
-    buckets_discovery = buckets_discovery,
-    rebalancer_request_state = rebalancer_request_state,
+    on_master_enable = storage_make_api(on_master_enable),
+    on_master_disable = storage_make_api(on_master_disable),
     internal = M,
-    on_master_enable = on_master_enable,
-    on_master_disable = on_master_disable,
-    sharded_spaces = function()
-        return table.deepcopy(find_sharded_spaces())
-    end,
-    module_version = function() return M.module_version end,
 }
-- 
2.24.3 (Apple Git-128)


  parent reply	other threads:[~2021-12-17  0:26 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-17  0:25 [Tarantool-patches] [PATCH vshard 0/5] Router backoff, storage disable Vladislav Shpilevoy via Tarantool-patches
2021-12-17  0:25 ` [Tarantool-patches] [PATCH vshard 1/5] router: backoff on some box errors Vladislav Shpilevoy via Tarantool-patches
2021-12-17 11:09   ` Oleg Babin via Tarantool-patches
2021-12-17 23:10     ` Vladislav Shpilevoy via Tarantool-patches
2021-12-18 13:57       ` Oleg Babin via Tarantool-patches
2021-12-17  0:25 ` Vladislav Shpilevoy via Tarantool-patches [this message]
2021-12-17 11:09   ` [Tarantool-patches] [PATCH vshard 2/5] storage: auto enable/disable Oleg Babin via Tarantool-patches
2021-12-17 23:10     ` Vladislav Shpilevoy via Tarantool-patches
2021-12-18 13:58       ` Oleg Babin via Tarantool-patches
2021-12-17  0:25 ` [Tarantool-patches] [PATCH vshard 3/5] storage: manual enable/disable Vladislav Shpilevoy via Tarantool-patches
2021-12-17 11:09   ` Oleg Babin via Tarantool-patches
2021-12-17  0:25 ` [Tarantool-patches] [PATCH vshard 4/5] error: introduce from_string Vladislav Shpilevoy via Tarantool-patches
2021-12-17 11:09   ` Oleg Babin via Tarantool-patches
2021-12-17 23:10     ` Vladislav Shpilevoy via Tarantool-patches
2021-12-17  0:25 ` [Tarantool-patches] [PATCH vshard 5/5] router: backoff on storage being disabled Vladislav Shpilevoy via Tarantool-patches
2021-12-17 11:09   ` Oleg Babin via Tarantool-patches
2021-12-18 13:58 ` [Tarantool-patches] [PATCH vshard 0/5] Router backoff, storage disable Oleg Babin via Tarantool-patches
2021-12-20 23:52 ` Vladislav Shpilevoy via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9d767a02cabc032ff4ad478b4a51c0a254276569.1639700518.git.v.shpilevoy@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=olegrok@tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH vshard 2/5] storage: auto enable/disable' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox