From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 6A0CA299D6 for ; Fri, 30 Mar 2018 00:15:31 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cyCbjWxXsZhb for ; Fri, 30 Mar 2018 00:15:31 -0400 (EDT) Received: from smtp32.i.mail.ru (smtp32.i.mail.ru [94.100.177.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id C9C2A292F8 for ; Fri, 30 Mar 2018 00:15:30 -0400 (EDT) From: Georgy Kirichenko Subject: [tarantool-patches] Re: [PATCH vshard 0/7] Replicaset lock and bucket pin Date: Fri, 30 Mar 2018 07:15:24 +0300 Message-ID: <2086653.5umClPjUaL@home.lan> In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3252914.epYSD3vsTd"; micalg="pgp-sha256"; protocol="application/pgp-signature" Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-subscribe: List-owner: List-post: List-archive: To: tarantool-patches@freelists.org Cc: Vladislav Shpilevoy --nextPart3252914.epYSD3vsTd Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Seems to be Ok On Wednesday, March 28, 2018 12:24:07 AM MSK Vladislav Shpilevoy wrote: > Branch: http://github.com/tarantool/tarantool/tree/gh-71-bucket-pin-lock > Issue: https://github.com/tarantool/vshard/issues/71 > > Replicaset lock makes it invisible for the rebalancer - a locked > replicaset can neither receive new buckets nor send its own. > Bucket pin blocks this concrete bucket sending - it will stay on a > replicaset to which pinned, until it is unpinned. Pinning all > replicaset buckets is not the same as replicaset locking - even if > you pin all buckets, the non-locked replicaset still can receive > new buckets. > > Replicaset lock allows, for example, to separate a replicaset for > testsing from production replicasets. Or to preserve some > application metadata, that must not be sharded for a while. Bucket > pin allows the same, but in the smaller scope. > > Difference between replicaset lock and all buckets pinning is > motivated by ability to button-up an entire replicaset. > > Mostly locked and pinned buckets affect the rebalancing algorithm, > which must ignore locked replicasets, and take pinned buckets into > account, attempting to reach the best possible balance. It is not > a trivial task, because a user can pin to a replicaset so many > buckets, that a perfect balance is unreachable. > > Vladislav Shpilevoy (7): > rebalancer: allow to lock a replicaset from rebalancing > rebalancer: remember the currently sending bucket id > storage: rework recovery > storage: wrap bucket status checks into functions > rebalancer: introduce pinned bucket concept into rebalancer algo > storage: open public API to pin/unpin buckets > rfc: add RFC for replicaset lock and bucket pin > > docs/RFC/replicaset_lock_and_bucket.md | 84 ++++ > test/rebalancer/box_1_a.lua | 2 +- > test/rebalancer/rebalancer_lock_and_pin.result | 503 > ++++++++++++++++++++++ test/rebalancer/rebalancer_lock_and_pin.test.lua | > 224 ++++++++++ test/rebalancer/restart_during_rebalancing.result | 4 + > test/storage/recovery.result | 85 +++- > test/storage/recovery.test.lua | 38 +- > test/storage/storage.result | 7 + > test/unit/rebalancer.result | 333 +++++++++++++- > test/unit/rebalancer.test.lua | 79 ++++ > vshard/cfg.lua | 3 +- > vshard/consts.lua | 1 + > vshard/error.lua | 12 +- > vshard/replicaset.lua | 105 +++-- > vshard/router/init.lua | 46 +- > vshard/storage/init.lua | 418 +++++++++++------- > 16 files changed, 1713 insertions(+), 231 deletions(-) > create mode 100644 docs/RFC/replicaset_lock_and_bucket.md > create mode 100644 test/rebalancer/rebalancer_lock_and_pin.result > create mode 100644 test/rebalancer/rebalancer_lock_and_pin.test.lua --nextPart3252914.epYSD3vsTd Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEBJFDbU76LsBbgHBsvKOmCX79zb4FAlq9udwACgkQvKOmCX79 zb6hnAf/RilCka/G9ufxM1VARGIelEYE0FyckQDjin0lWH9CFafK/zMYrNVnsE5E L1QjiWm87T+mGtIzYgplE8mWjO0F11rCsyxE3qxgJlkBlwhbapMv46whd8DMfD/W kVdtxzoDhwVmFtqcI9luHCjX2WQqnekpzlugyTWX6CJfEW/1lN6Af16LfjIlX3Pc hlCucfwpf/fTPAIu1yNIlPvf8Rzf/g7TjHp9U0ypK5lFOlq3qNQytqbt+pUmYMMe /TuKzb1gJSMFza71B6HyMijfB2KOM6nBQRplywpBiS/Y/m5qz6vx9RblYukO1D8W 4l/fxC1r4UnbarodYPtJEuXtlU9Qow== =fedG -----END PGP SIGNATURE----- --nextPart3252914.epYSD3vsTd--