From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id A27C16EC40; Fri, 13 Aug 2021 14:40:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A27C16EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628854852; bh=EgH7evec3Dslz71kfXJCa3uW6cmCyvScqNyUgnwbILc=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=y5CPuhLovN0HBemncn9Q/oB8LFVwkIBkRtlA0hcCBbGoAYKd4nvImh4uQM8oQGLjK HXSH0O35F1jj+DiDrwvR0sAyEjD/9LZyO4aflVduunmDVT1RV74yEFiLevirAwXfzQ zojONxzIdpE8KxkFYoUTwaKow0OfSv+gxI+F9gzk= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id A8F2A6EC40 for ; Fri, 13 Aug 2021 14:40:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A8F2A6EC40 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1mEVYQ-0006CK-LB; Fri, 13 Aug 2021 14:40:51 +0300 Date: Fri, 13 Aug 2021 14:40:49 +0300 To: Serge Petrenko Message-ID: <20210813114049.3qjkgzwperinvz3y@esperanza> References: <5d4f821905e21466ca25542f70c393c1057637fe.1628810253.git.sergepetrenko@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5d4f821905e21466ca25542f70c393c1057637fe.1628810253.git.sergepetrenko@tarantool.org> X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD906AB4890CDABF0C5CB76CEE71D3E4007182A05F53808504050D48FAD8EFCC373AD4ED6B3C0A7CCCB7A2E622CBA51A681CDAB1D9A66DC992A X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE77BF46084C0059042EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637188F5654332B449D8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D82792F4783F2D7E5EDBC9F146A499D4AD117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC60CDF180582EB8FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735201E561CDFBCA1751FE5D25F19253116ADD2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B613439FA09F3DCB32089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: 0D63561A33F958A59FB09800613739515432F8317F4DDECD0137A595D7B7C653D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA752DA3D96DA0CEF5C48E8E86DC7131B365E7726E8460B7C23C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3455AC8BF8E3153BA0823792531A9B8C94C762EA4F284D6A18221EE21FE9BB2386C41E9C67B27A23741D7E09C32AA3244CA5C88C4497AD634A6665E0EB6BC2E338795D98D676DD64D0927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj0dLV0c3jbkwb1mvsynKsew== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5DF46EBDAE0C6361B339BD208360D3CC47274CEFED1673C562683ABF942079399BFB559BB5D741EB966A65DFF43FF7BE03240331F90058701C67EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 2/2] box: allow upgrading from version 1.6 X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladimir Davydov via Tarantool-patches Reply-To: Vladimir Davydov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" On Fri, Aug 13, 2021 at 02:30:32AM +0300, Serge Petrenko wrote: > Direct upgrade support from pre-1.7.5 versions was removed in commit > 7d3b80e78206c479cab75b4893629cfa1932252e > (Forbid upgrade from Tarantool < 1.7.5 and refactor upgrade.lua) > The reason for that was the mandatory space format checks introduced > back then. With these space format checks, old schema couldn't be > recovered on new Tarantool versions, because newer versions had > different system space formats. So old schema couldn't be upgraded > because it couldn't even be recovered. > > Actually this was rather inconvenient. One had to perform an extra > upgrade step when upgrading from, say, 1.6 to 2.x: instead of > performing a direct upgrade one had to do 1.6 -> 1.10 -> 2.x upgrade > which takes twice the time. > > Make it possible to boot from snapshots coming from Tarantool version > 1.6.8 and above. > > In order to do so, introduce before_replace triggers on system spaces, > which work during snapshot/xlog recovery. The triggers will set tuple > formats to the ones supported by current Tarantool (2.x). This way the > recovered data will have the correct format for a usual schema upgrade. > > Also add upgrade_to_1_7_5() handler, which finishes transformation of > old schema to 1.7.5. The handler is fired together with other > box.schema.upgrade() handlers, so there's no user-visible behaviour > change. > > Side note: it would be great to use the same technique to allow booting > from pre-1.6.8 snapshots. Unfortunately, this is not possible. > > Current triggers don't break the order of schema upgrades, so 1.7.1 > upgrades come before 1.7.2 and 1.7.5. This is because all the upgrades > in these versions are replacing existing tuples and not inserting new > ones, so the upgrades may be handled by the before_replace triggers. > > Upgrade to 1.6.8 requires inserting new tuples: creating sysviews, like > _vspace, _vuser and so on. This can't be done from the before_replace > triggers, so we would have to run triggers for 1.7.x first which would > allow Tarantool to recover the snapshot, and then run an upgrade handler for > 1.6.8. This looks really messy. > > Closes #5894 > --- > src/box/lua/load_cfg.lua | 14 + > src/box/lua/upgrade.lua | 276 +++++++++++- > test/xlog/gh-5894-pre-1.7.7-upgrade.result | 400 ++++++++++++++++++ > test/xlog/gh-5894-pre-1.7.7-upgrade.test.lua | 77 ++++ > .../1.6.8/gh-5894-pre-1.7.7-upgrade/fill.lua | 1 + > .../1.7.1/gh-5894-pre-1.7.7-upgrade/fill.lua | 1 + > .../1.7.2/gh-5894-pre-1.7.7-upgrade/fill.lua | 1 + > .../1.7.5/gh-5894-pre-1.7.7-upgrade/fill.lua | 1 + > test/xlog/upgrade/fill.lua | 4 + > 9 files changed, 773 insertions(+), 2 deletions(-) > create mode 100644 test/xlog/gh-5894-pre-1.7.7-upgrade.result > create mode 100644 test/xlog/gh-5894-pre-1.7.7-upgrade.test.lua > create mode 120000 test/xlog/upgrade/1.6.8/gh-5894-pre-1.7.7-upgrade/fill.lua > create mode 120000 test/xlog/upgrade/1.7.1/gh-5894-pre-1.7.7-upgrade/fill.lua > create mode 120000 test/xlog/upgrade/1.7.2/gh-5894-pre-1.7.7-upgrade/fill.lua > create mode 120000 test/xlog/upgrade/1.7.5/gh-5894-pre-1.7.7-upgrade/fill.lua LGTM