Re: [Tarantool-patches] [PATCH v2 2/2] box: allow upgrading from version 1.6

[Vladimir Davydov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

On Fri, Aug 13, 2021 at 02:30:32AM +0300, Serge Petrenko wrote:
> Direct upgrade support from pre-1.7.5 versions was removed in commit
> 7d3b80e78206c479cab75b4893629cfa1932252e
> (Forbid upgrade from Tarantool < 1.7.5 and refactor upgrade.lua)
> The reason for that was the mandatory space format checks introduced
> back then. With these space format checks, old schema couldn't be
> recovered on new Tarantool versions, because newer versions had
> different system space formats. So old schema couldn't be upgraded
> because it couldn't even be recovered.
> 
> Actually this was rather inconvenient. One had to perform an extra
> upgrade step when upgrading from, say, 1.6 to 2.x: instead of
> performing a direct upgrade one had to do 1.6 -> 1.10 -> 2.x upgrade
> which takes twice the time.
> 
> Make it possible to boot from snapshots coming from Tarantool version
> 1.6.8 and above.
> 
> In order to do so, introduce before_replace triggers on system spaces,
> which work during snapshot/xlog recovery. The triggers will set tuple
> formats to the ones supported by current Tarantool (2.x). This way the
> recovered data will have the correct format for a usual schema upgrade.
> 
> Also add upgrade_to_1_7_5() handler, which finishes transformation of
> old schema to 1.7.5. The handler is fired together with other
> box.schema.upgrade() handlers, so there's no user-visible behaviour
> change.
> 
> Side note: it would be great to use the same technique to allow booting
> from pre-1.6.8 snapshots. Unfortunately, this is not possible.
> 
> Current triggers don't break the order of schema upgrades, so 1.7.1
> upgrades come before 1.7.2 and 1.7.5. This is because all the upgrades
> in these versions are replacing existing tuples and not inserting new
> ones, so the upgrades may be handled by the before_replace triggers.
> 
> Upgrade to 1.6.8 requires inserting new tuples: creating sysviews, like
> _vspace, _vuser and so on. This can't be done from the before_replace
> triggers, so we would have to run triggers for 1.7.x first which would
> allow Tarantool to recover the snapshot, and then run an upgrade handler for
> 1.6.8. This looks really messy.
> 
> Closes #5894
> ---
>  src/box/lua/load_cfg.lua                      |  14 +
>  src/box/lua/upgrade.lua                       | 276 +++++++++++-
>  test/xlog/gh-5894-pre-1.7.7-upgrade.result    | 400 ++++++++++++++++++
>  test/xlog/gh-5894-pre-1.7.7-upgrade.test.lua  |  77 ++++
>  .../1.6.8/gh-5894-pre-1.7.7-upgrade/fill.lua  |   1 +
>  .../1.7.1/gh-5894-pre-1.7.7-upgrade/fill.lua  |   1 +
>  .../1.7.2/gh-5894-pre-1.7.7-upgrade/fill.lua  |   1 +
>  .../1.7.5/gh-5894-pre-1.7.7-upgrade/fill.lua  |   1 +
>  test/xlog/upgrade/fill.lua                    |   4 +
>  9 files changed, 773 insertions(+), 2 deletions(-)
>  create mode 100644 test/xlog/gh-5894-pre-1.7.7-upgrade.result
>  create mode 100644 test/xlog/gh-5894-pre-1.7.7-upgrade.test.lua
>  create mode 120000 test/xlog/upgrade/1.6.8/gh-5894-pre-1.7.7-upgrade/fill.lua
>  create mode 120000 test/xlog/upgrade/1.7.1/gh-5894-pre-1.7.7-upgrade/fill.lua
>  create mode 120000 test/xlog/upgrade/1.7.2/gh-5894-pre-1.7.7-upgrade/fill.lua
>  create mode 120000 test/xlog/upgrade/1.7.5/gh-5894-pre-1.7.7-upgrade/fill.lua

LGTM