[Tarantool-patches] [PATCH 0/9] VShard Map-Reduce, part 1, preparations
Vladislav Shpilevoy
v.shpilevoy at tarantool.org
Wed Feb 10 02:46:06 MSK 2021
The patchset makes some preparations for the incoming consistent map-reduce
feature. Mostly it reworks bucket GC and recovery so as they wouldn't block
map-reduce requests for too long time.
The last commit is a first big part related directly to map-reduce. It
introduces binary heap data structure implementation, which is going to be a
core storage for map-reduce artifacts.
Some commits were done alongside while working on the code. These are rlist
extraction, and fix of a bug in bucket_recv. I did them in first version of the
patchset as necessary, and then after some reworks they were not used anymore.
But I still want to merge them. One fixes a potential bug, other improves code
readability.
Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-147-map-reduce-part1
Issue: https://github.com/tarantool/tarantool/issues/147
Vladislav Shpilevoy (9):
rlist: move rlist to a new module
Use fiber.clock() instead of .time() everywhere
test: introduce a helper to wait for bucket GC
storage: bucket_recv() should check rs lock
util: introduce yielding table functions
cfg: introduce 'deprecated option' feature
gc: introduce reactive garbage collector
recovery: introduce reactive recovery
util: introduce binary heap data structure
test/failover/failover.result | 4 +-
test/failover/failover.test.lua | 4 +-
test/lua_libs/storage_template.lua | 9 +
test/misc/reconfigure.result | 10 -
test/misc/reconfigure.test.lua | 3 -
test/rebalancer/bucket_ref.result | 19 +-
test/rebalancer/bucket_ref.test.lua | 8 +-
test/rebalancer/errinj.result | 20 +-
test/rebalancer/errinj.test.lua | 12 +-
test/rebalancer/rebalancer.result | 5 +-
test/rebalancer/rebalancer.test.lua | 3 +-
.../rebalancer/rebalancer_lock_and_pin.result | 14 +
.../rebalancer_lock_and_pin.test.lua | 4 +
test/rebalancer/receiving_bucket.result | 10 +-
test/rebalancer/receiving_bucket.test.lua | 3 +-
test/reload_evolution/storage.result | 7 +-
test/reload_evolution/storage.test.lua | 3 +-
test/router/reroute_wrong_bucket.result | 8 +-
test/router/reroute_wrong_bucket.test.lua | 4 +-
test/router/router.result | 22 +-
test/router/router.test.lua | 13 +-
test/storage/recovery.result | 11 +-
test/storage/recovery.test.lua | 5 +
test/storage/recovery_errinj.result | 16 +-
test/storage/recovery_errinj.test.lua | 9 +-
test/storage/storage.result | 10 +-
test/storage/storage.test.lua | 1 +
test/unit-tap/heap.test.lua | 310 ++++++++++++++++
test/unit-tap/suite.ini | 4 +
test/unit/config.result | 35 +-
test/unit/config.test.lua | 16 +-
test/unit/garbage.result | 106 +++---
test/unit/garbage.test.lua | 47 ++-
test/unit/garbage_errinj.result | 223 ------------
test/unit/garbage_errinj.test.lua | 73 ----
test/unit/rebalancer.result | 99 -----
test/unit/rebalancer.test.lua | 27 --
test/unit/rlist.result | 114 ++++++
test/unit/rlist.test.lua | 33 ++
test/unit/util.result | 113 ++++++
test/unit/util.test.lua | 49 +++
vshard/cfg.lua | 10 +-
vshard/consts.lua | 7 +-
vshard/heap.lua | 226 ++++++++++++
vshard/replicaset.lua | 13 +-
vshard/rlist.lua | 53 +++
vshard/router/init.lua | 16 +-
vshard/storage/init.lua | 342 +++++++++---------
vshard/storage/reload_evolution.lua | 8 +
vshard/util.lua | 40 ++
50 files changed, 1368 insertions(+), 833 deletions(-)
create mode 100755 test/unit-tap/heap.test.lua
create mode 100644 test/unit-tap/suite.ini
delete mode 100644 test/unit/garbage_errinj.result
delete mode 100644 test/unit/garbage_errinj.test.lua
create mode 100644 test/unit/rlist.result
create mode 100644 test/unit/rlist.test.lua
create mode 100644 vshard/heap.lua
create mode 100644 vshard/rlist.lua
--
2.24.3 (Apple Git-128)
More information about the Tarantool-patches
mailing list