From: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> To: tarantool-patches@freelists.org Cc: georgy@tarantool.org, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Subject: [tarantool-patches] [PATCH vshard 2/7] rebalancer: remember the currently sending bucket id Date: Wed, 28 Mar 2018 00:24:09 +0300 [thread overview] Message-ID: <d4e30949676341c9b9755651843e40d1a99b95f6.1522185711.git.v.shpilevoy@tarantool.org> (raw) In-Reply-To: <cover.1522185711.git.v.shpilevoy@tarantool.org> In-Reply-To: <cover.1522185711.git.v.shpilevoy@tarantool.org> If is needed for recovery - if a bucket is RECEIVING on one replicaset, and SENDING on another, but on the source replicaset it is known, that the rebalancer does not send it now, then this bucket must be recovered. Its partially received data must be deleted on a destination replicaset, and its state must be ACTIVE on a source replicaset. --- vshard/storage/init.lua | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua index 405585d..6cbeb4b 100644 --- a/vshard/storage/init.lua +++ b/vshard/storage/init.lua @@ -82,6 +82,13 @@ if not M then -- Maximal bucket count that can be received by a single -- replicaset simultaneously. rebalancer_max_receiving = 0, + -- Identifier of a bucket that rebalancer is sending now, + -- or else 0. If a bucket has state SENDING, but its id is + -- not stored here, it means, that its sending was + -- interrupted, for example by restart of an instance, and + -- a destination replicaset must drop already received + -- data. + rebalancer_sending_bucket = 0, } end @@ -1109,15 +1116,17 @@ local function rebalancer_apply_routes_f(routes) -- guarantee that an event loop does not contain events -- between this fiber and its creator. M.rebalancer_applier_fiber = lfiber.self() + M.rebalancer_sending_bucket = 0 local active_buckets = _status:select{consts.BUCKET.ACTIVE} local i = 1 for dst_uuid, bucket_count in pairs(routes) do assert(i + bucket_count - 1 <= #active_buckets) - log.info('Send %d buckets to %s', bucket_count, - M.replicasets[dst_uuid]) + log.info('Send %d buckets to %s', bucket_count, M.replicasets[dst_uuid]) for j = i, i + bucket_count - 1 do - local status, ret = pcall(bucket_send, active_buckets[j].id, - dst_uuid) + local bucket_id = active_buckets[j].id + M.rebalancer_sending_bucket = bucket_id + local status, ret = pcall(bucket_send, bucket_id, dst_uuid) + M.rebalancer_sending_bucket = 0 if not status or ret ~= true then if not status then log.error('Error during rebalancer routes applying: %s', -- 2.14.3 (Apple Git-98)
next prev parent reply other threads:[~2018-03-27 21:24 UTC|newest] Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-03-27 21:24 [tarantool-patches] [PATCH vshard 0/7] Replicaset lock and bucket pin Vladislav Shpilevoy 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 1/7] rebalancer: allow to lock a replicaset from rebalancing Vladislav Shpilevoy 2018-03-27 21:24 ` Vladislav Shpilevoy [this message] 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 3/7] storage: rework recovery Vladislav Shpilevoy 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 4/7] storage: wrap bucket status checks into functions Vladislav Shpilevoy 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 5/7] rebalancer: introduce pinned bucket concept into rebalancer algo Vladislav Shpilevoy 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 6/7] storage: open public API to pin/unpin buckets Vladislav Shpilevoy 2018-03-27 21:24 ` [tarantool-patches] [PATCH vshard 7/7] rfc: add RFC for replicaset lock and bucket pin Vladislav Shpilevoy 2018-03-30 4:15 ` [tarantool-patches] Re: [PATCH vshard 0/7] Replicaset " Georgy Kirichenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=d4e30949676341c9b9755651843e40d1a99b95f6.1522185711.git.v.shpilevoy@tarantool.org \ --to=v.shpilevoy@tarantool.org \ --cc=georgy@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='Re: [tarantool-patches] [PATCH vshard 2/7] rebalancer: remember the currently sending bucket id' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox