[Tarantool-patches] [PATCH v10] gitlab-ci: push Deb/RPM packages to S3 based repos

[Alexander Turenko]

Pushed to master, 2.3, 2.2, 2.1 (w/o Fedora 31), 1.10.

CCed Kirill.

I'll file an issue for follow up activities.

WBR, Alexander Turenko.

On Tue, Feb 04, 2020 at 08:37:50AM +0300, Alexander V. Tikhonov wrote:
> We're going to use S3 compatible storage for Deb and RPM repositories
> instead of packagecloud.io service. The main reason is that
> packagecloud.io provides a limited amount of storage, which is not
> enough for keeping all packages (w/o regular pruning of old versions).
> 
> Note: At the moment packages are still pushed to packagecloud.io from
> Travis-CI. Disabling this is out of scope of this patch.
> 
> This patch implements saving of packages on an S3 compatible storage and
> regeneration of a repository metadata.
> 
> The layout is a bit different from one we have on packagecloud.io.
> 
> packagecloud.io:
> 
>  | - 1.10
>  | - 2.1
>  | - 2.2
>  | - ...
> 
> S3 compatible storage:
> 
>  | - live
>  |   - 1.10
>  |   - 2.1
>  |   - 2.2
>  |   - ...
>  | - release
>  |   - 1.10
>  |   - 2.1
>  |   - 2.2
>  |   - ...
> 
> Both 'live' and 'release' repositories track release branches (named as
> <major>.<minor>) and master branch. The difference is that 'live' is
> updated on every push, but 'release' is only for tagged versions
> (<major>.<minor>.<patch>.0).
> 
> Packages are also built on '*-full-ci' branches, but only for testing
> purposes: they don't pushed anywhere.
> 
> The core logic is in the tools/update_repo.sh script, which implements
> the following flow:
> 
> - create metadata for new packages
> - fetch relevant metadata from the S3 storage
> - push new packages to the S3 storage
> - merge and push the updated metadata to the S3 storage
> 
> The script uses 'createrepo' for RPM repositories and 'reprepro' for Deb
> repositories.
> 
> Closes #3380
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3380-push-packages-s3-full-ci
> Issue: https://github.com/tarantool/tarantool/issues/3380