From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id B1FD86F873; Fri, 28 Jan 2022 08:47:24 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B1FD86F873 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1643348844; bh=dP6fUaVFTHKMyCpKaf6o8GOVYSD8R4S2XGpr3y4dkvg=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=n9U+q9TdP53Ox34LQbdM7Vz7beiXwscXA2kndv6EqWM1KX78Cdi3fdiMLyYQqjNvj zs1LXi2zCrxPKNvN08mbAv4gpx+LJIsHCjYMuSmConrxcUTHwdfu1Hkb7qOJ7DJP5+ 0BFf+3jBXbBfkuS6Bttrqq4Ea8oHb5Fk1T5c33c4= Received: from smtp55.i.mail.ru (smtp55.i.mail.ru [217.69.128.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 051B76F873 for ; Fri, 28 Jan 2022 08:47:24 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 051B76F873 Received: by smtp55.i.mail.ru with esmtpa (envelope-from ) id 1nDK6V-0005uF-6m; Fri, 28 Jan 2022 08:47:23 +0300 Message-ID: <640e1bef-d7a1-4e8e-ccd4-bde0d24d2904@tarantool.org> Date: Fri, 28 Jan 2022 08:47:22 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 To: Vladislav Shpilevoy , tarantool-patches@dev.tarantool.org References: <4027b571e5f7b9edbf9e153f127e2d1172cc2647.1643329358.git.v.shpilevoy@tarantool.org> In-Reply-To: <4027b571e5f7b9edbf9e153f127e2d1172cc2647.1643329358.git.v.shpilevoy@tarantool.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9DB8349C4A7292191E0ED1315537C79F89497FA079FF96BFA1313CFAB8367EF908E2BE116634AD74D93FD68D4DD6AA8CD39BB7347A6A9D734B2B42A5DC8E2320028D9CF984B353DE0 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B264C8851FD8E810EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006372094AD700861FA748638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D84F449D72CEF8AD8751AA37D743FE736F117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8BF1175FABE1C0F9B6A471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FCC18E17B53F8A116A3AA81AA40904B5D9CF19DD082D7633A078D18283394535A93AA81AA40904B5D98AA50765F7900637875A4EB61F868389D81D268191BDAD3D698AB9A7B718F8C4D1B931868CE1C5781A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89F29C2079CDE5AC98057739F23D657EF2BB5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A505D0DB1F769F22DD653EA455271B5ADB8DA3033812251AAFD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA755279A685C79E1982410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3468964757141B82E0150BC40D0F6B2A28303DDC9C0F1E7ED75B2D81A2C4ED20DE8DBF096DB43F61F71D7E09C32AA3244CE7B2A54F785E59ADCEDF546D6DBA84293E8609A02908F271729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojqLK0kLh8sGeXQEPv/9ppvQ== X-Mailru-Sender: 583F1D7ACE8F49BD508BD8DBBD09B14DBBE1EF2777531D73E8A0026A593788A96D80FFBE066E7F2C80EE221D05932256AD9BA6614E257C8ED9E51C16F2486AFBE342CF4F05FB7E8CB0DAF586E7D11B3E67EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH vshard 1/2] Drop Tarantool < 1.10.1 support X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Oleg Babin via Tarantool-patches Reply-To: Oleg Babin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thanks for your patch. LGTM. On 28.01.2022 03:23, Vladislav Shpilevoy wrote: > The reasons are: > > - Rebalancer won't work. It uses fiber.join(). > > - Bucket discovery on router is extra expensive - it downloads the > entire _bucket from each storage. That can make the latter and > the router unresponsive for notable time if bucket count is big > enough. > > - There are other minor hacks which complicate development. For > instance, box.error.new() couldn't be used. > > - Tests don't pass. Thus can't tell what else is broken. Makes no > sense to support it if it can't be even tested properly > > The reason why it is done now is because the code will get more > complicated soon and will use more new features which even 1.10 > won't support. This patch should reduce the level of version > dependency complications before they get even worse. > > Closes #256 > Closes #267 > > @TarantoolBot document > Title: VShard now supports only Tarantool >= 1.10.1 > > It used to support 1.9 too, but now it is dropped. To use VShard >> 0.1.19 a user needs Tarantool >= 1.10.1. > --- > vshard/cfg.lua | 5 ---- > vshard/error.lua | 9 ++---- > vshard/router/init.lua | 65 ++--------------------------------------- > vshard/storage/init.lua | 21 ++----------- > vshard/util.lua | 4 +++ > 5 files changed, 13 insertions(+), 91 deletions(-) > > diff --git a/vshard/cfg.lua b/vshard/cfg.lua > index a059d44..1903967 100644 > --- a/vshard/cfg.lua > +++ b/vshard/cfg.lua > @@ -2,7 +2,6 @@ > > local log = require('log') > local luri = require('uri') > -local lutil = require('vshard.util') > local consts = require('vshard.consts') > > local function check_uri(uri) > @@ -22,10 +21,6 @@ local function check_replica_master(master, ctx) > end > > local function check_replicaset_master(master, ctx) > - -- Limit the version due to extensive usage of netbox is_async feature. > - if not lutil.version_is_at_least(1, 10, 1) then > - error('Only versions >= 1.10.1 support master discovery') > - end > if master ~= 'auto' then > error('Only "auto" master is supported') > end > diff --git a/vshard/error.lua b/vshard/error.lua > index c673827..d36803a 100644 > --- a/vshard/error.lua > +++ b/vshard/error.lua > @@ -235,8 +235,7 @@ local function make_error(e) > -- box.error, return unpacked > return box_error(e) > elseif type(e) == 'string' then > - local ok, err = pcall(box.error, box.error.PROC_LUA, e) > - return box_error(err) > + return box_error(box.error.new(box.error.PROC_LUA, e)) > elseif type(e) == 'table' then > return setmetatable(e, {__tostring = json.encode}) > else > @@ -271,12 +270,10 @@ local function make_alert(code, ...) > end > > -- > --- Create a timeout error object. Box.error.new() can't be used because is > --- present only since 1.10. > +-- Create a timeout error object. > -- > local function make_timeout() > - local _, err = pcall(box.error, box.error.TIMEOUT) > - return make_error(err) > + return make_error(box.error.new(box.error.TIMEOUT)) > end > > return { > diff --git a/vshard/router/init.lua b/vshard/router/init.lua > index 5a94a2f..ebeaac9 100644 > --- a/vshard/router/init.lua > +++ b/vshard/router/init.lua > @@ -249,15 +249,10 @@ local function discovery_handle_buckets(router, replicaset, buckets) > end > end > > -local discovery_f > - > -if util.version_is_at_least(1, 10, 0) then > -- > --- >= 1.10 version of discovery fiber does parallel discovery of > --- all replicasets at once. It uses is_async feature of netbox > --- for that. > +-- Bucket discovery main loop. > -- > -discovery_f = function(router) > +local function discovery_f(router) > local module_version = M.module_version > assert(router.discovery_mode == 'on' or router.discovery_mode == 'once') > local iterators = {} > @@ -394,47 +389,6 @@ discovery_f = function(router) > end > end > > --- Version >= 1.10. > -else > --- Version < 1.10. > - > --- > --- Background fiber to perform discovery. It periodically scans > --- replicasets one by one and updates route_map. > --- > -discovery_f = function(router) > - local module_version = M.module_version > - while module_version == M.module_version do > - while not next(router.replicasets) do > - lfiber.sleep(consts.DISCOVERY_IDLE_INTERVAL) > - end > - local old_replicasets = router.replicasets > - for rs_uuid, replicaset in pairs(router.replicasets) do > - local active_buckets, err = > - replicaset:callro('vshard.storage.buckets_discovery', {}, > - {timeout = 2}) > - while M.errinj.ERRINJ_LONG_DISCOVERY do > - M.errinj.ERRINJ_LONG_DISCOVERY = 'waiting' > - lfiber.sleep(0.01) > - end > - -- Renew replicasets object captured by the for loop > - -- in case of reconfigure and reload events. > - if router.replicasets ~= old_replicasets then > - break > - end > - if not active_buckets then > - log.error('Error during discovery %s: %s', replicaset, err) > - else > - discovery_handle_buckets(router, replicaset, active_buckets) > - end > - lfiber.sleep(consts.DISCOVERY_IDLE_INTERVAL) > - end > - end > -end > - > --- Version < 1.10. > -end > - > -- > -- Immediately wakeup discovery fiber if exists. > -- > @@ -758,9 +712,6 @@ local function router_call(router, bucket_id, opts, ...) > ...) > end > > -local router_map_callrw > - > -if util.version_is_at_least(1, 10, 0) then > -- > -- Consistent Map-Reduce. The given function is called on all masters in the > -- cluster with a guarantee that in case of success it was executed with all > @@ -800,7 +751,7 @@ if util.version_is_at_least(1, 10, 0) then > -- about concrete replicaset. For example, not all buckets were found even > -- though all replicasets were scanned. > -- > -router_map_callrw = function(router, func, args, opts) > +local function router_map_callrw(router, func, args, opts) > local replicasets = router.replicasets > local timeout = opts and opts.timeout or consts.CALL_TIMEOUT_MIN > local deadline = fiber_clock() + timeout > @@ -919,16 +870,6 @@ router_map_callrw = function(router, func, args, opts) > return nil, err, err_uuid > end > > --- Version >= 1.10. > -else > --- Version < 1.10. > - > -router_map_callrw = function() > - error('Supported for Tarantool >= 1.10') > -end > - > -end > - > -- > -- Get replicaset object by bucket identifier. > -- @param bucket_id Bucket identifier. > diff --git a/vshard/storage/init.lua b/vshard/storage/init.lua > index 3eaad9c..78103cf 100644 > --- a/vshard/storage/init.lua > +++ b/vshard/storage/init.lua > @@ -1288,11 +1288,8 @@ local function bucket_recv_xc(bucket_id, from, data, opts) > local space_name, space_data = row[1], row[2] > local space = box.space[space_name] > if space == nil then > - -- Tarantool doesn't provide API to create box.error > - -- objects before 1.10. > - local _, boxerror = pcall(box.error, box.error.NO_SUCH_SPACE, > - space_name) > - return nil, lerror.box(boxerror) > + local err = box.error.new(box.error.NO_SUCH_SPACE, space_name) > + return nil, lerror.box(err) > end > box.begin() > for _, tuple in ipairs(space_data) do > @@ -2209,13 +2206,6 @@ end > -- > local function rebalancer_worker_f(worker_id, dispenser, quit_cond) > lfiber.name(string.format('vshard.rebalancer_worker_%d', worker_id)) > - if not util.version_is_at_least(1, 10, 0) then > - -- Return control to the caller immediately to allow it > - -- to finish preparations. In 1.9 a caller couldn't create > - -- a fiber without switching to it. > - lfiber.yield() > - end > - > local _status = box.space._bucket.index.status > local opts = {timeout = consts.REBALANCER_CHUNK_TIMEOUT} > local active_key = {consts.BUCKET.ACTIVE} > @@ -2299,12 +2289,7 @@ local function rebalancer_apply_routes_f(routes) > local quit_cond = lfiber.cond() > local workers = table.new(worker_count, 0) > for i = 1, worker_count do > - local f > - if util.version_is_at_least(1, 10, 0) then > - f = lfiber.new(rebalancer_worker_f, i, dispenser, quit_cond) > - else > - f = lfiber.create(rebalancer_worker_f, i, dispenser, quit_cond) > - end > + local f = lfiber.new(rebalancer_worker_f, i, dispenser, quit_cond) > f:set_joinable(true) > workers[i] = f > end > diff --git a/vshard/util.lua b/vshard/util.lua > index afa658b..366fdea 100644 > --- a/vshard/util.lua > +++ b/vshard/util.lua > @@ -154,6 +154,10 @@ local function version_is_at_least(major_need, middle_need, minor_need) > return minor >= minor_need > end > > +if not version_is_at_least(1, 10, 1) then > + error("VShard supports only Tarantool >= 1.10.1") > +end > + > -- > -- Copy @a src table. Fiber yields every @a interval keys copied. Does not give > -- any guarantees on what is the result when the source table is changed during