[Tarantool-patches] [PATCH v1] Add S3 package checker

[Alexander Tikhonov]

Hi Sergey, thanks for the review, I’ve added shell-checker flag and fixed found warnings.

  
>Понедельник, 23 марта 2020, 12:04 +03:00 от Sergey Bronnikov <sergeyb at tarantool.org>:
> 
>Hello,
>
>I propose to check script tools/check_package.sh with shellcheck.
>Right now it reports a number of warnings.
>
>Sergey
>
>On 10:20 Mon 23 Mar , Alexander V. Tikhonov wrote:
>> Created Dockerfile based script to check packages available at the S3
>> repository. Set this script to be run just after the package creation.
>>
>> Follows up #3380
>> ---
>>
>> Github:  https://github.com/tarantool/tarantool/tree/avtikhon/gh-3380-check-packages-s3-full-ci
>> Issue:  https://github.com/tarantool/tarantool/issues/3380
>> Gitlab:  https://gitlab.com/tarantool/tarantool/pipelines/128701075
>>
>> .gitlab.mk | 1 +
>> tools/Dockerfile.s3 | 23 +++++++++++++++++
>> tools/check_package.sh | 57 ++++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 81 insertions(+)
>> create mode 100644 tools/Dockerfile.s3
>> create mode 100755 tools/check_package.sh
>>
>> diff --git a/.gitlab.mk b/.gitlab.mk
>> index b39c5c651..5053b914f 100644
>> --- a/.gitlab.mk
>> +++ b/.gitlab.mk
>> @@ -127,6 +127,7 @@ deploy: package
>> echo ${GPG_SECRET_KEY} | base64 -d | gpg --batch --import || true
>> ./tools/update_repo.sh -o=${OS} -d=${DIST} \
>> -b="${LIVE_REPO_S3_DIR}/${BUCKET}" build
>> + ./tools/check_package.sh "" ${OS}:${DIST} ${BUCKET}
>> if git name-rev --name-only --tags --no-undefined HEAD 2>/dev/null ; then \
>> ./tools/update_repo.sh -o=${OS} -d=${DIST} \
>> -b="${RELEASE_REPO_S3_DIR}/${BUCKET}" build ; \
>> diff --git a/tools/Dockerfile.s3 b/tools/Dockerfile.s3
>> new file mode 100644
>> index 000000000..338cf6926
>> --- /dev/null
>> +++ b/tools/Dockerfile.s3
>> @@ -0,0 +1,23 @@
>> +ARG IMG
>> +FROM ${IMG}
>> +
>> +ARG PKG_TOOL
>> +RUN ${PKG_TOOL} update -y >1.log 2>&1 || ( cat 1.log && false )
>> +RUN ${PKG_TOOL} install -y curl >2.log 2>&1 || ( cat 2.log && false )
>> +
>> +RUN curl -L  https://tarantool.io/installer.sh >installer.sh 2>3.log || ( cat 3.log && false )
>> +RUN chmod a+x installer.sh
>> +RUN sed "s#/bin/bash#/bin/bash -x#g" -i installer.sh
>> +
>> +# ./installer.sh is the external tool which return status is not
>> +# expectable and can be false because of its additional features,
>> +# all that is needed for us from the script just to install Tarantool,
>> +# NOTE: found that on centos:8 script fails but on rerun passes
>> +ARG REL
>> +RUN VER=${REL} ./installer.sh >4.log 2>&1 || ( VER=${REL} ./installer.sh >>4.log 2>&1 || ( cat 4.log && false ))
>> +ARG TNT_VER
>> +RUN CVER=$(tarantool -V | head -n 1 | awk '{print $2}' | sed 's#-#.#g') && \
>> + echo "Tarantool version:" && \
>> + echo "Expected: ${TNT_VER}" && \
>> + echo "Returned: ${CVER}" && \
>> + if ! echo "${CVER}" | grep -e "^${TNT_VER}" >/dev/null ; then false ; fi
>> diff --git a/tools/check_package.sh b/tools/check_package.sh
>> new file mode 100755
>> index 000000000..b3fdcdcda
>> --- /dev/null
>> +++ b/tools/check_package.sh
>> @@ -0,0 +1,57 @@
>> +#!/bin/bash
>> +set -e
>> +
>> +ver="$1"
>> +imgs="$2"
>> +rels="$3"
>> +
>> +if [ "$ver" == "" ]; then
>> + # some OS like CentOS/Fedora use only 6 hexadecimal
>> + # digits as the abbreviated object name
>> + ver=$(git describe --abbrev=6 | sed 's#-#.#g')
>> +fi
>> +
>> +if [ "$imgs" == "" ]; then
>> + for dist in 14.04 16.04 18.04 19.04 19.10 ; do imgs="$imgs ubuntu:$dist" ; done
>> + for dist in jessie stretch buster ; do imgs="$imgs debian:$dist" ; done
>> + for dist in 6 7 8 ; do imgs="$imgs centos:$dist" ; done
>> + for dist in 28 29 30 31 ; do imgs="$imgs fedora:$dist" ; done
>> +fi
>> +
>> +imgs=$(echo $imgs | sed "s#el:#centos:#g")
>> +
>> +if [ "$rels" == "" ]; then
>> + rels="1.10 2.1 2.2 2.3 2.4"
>> +fi
>> +
>> +cwd=`dirname $0`
>> +
>> +run_docker () {
>> + VER=$1
>> + IMG=$2
>> + REL=$3
>> +
>> + PKG_TOOL=yum
>> + if [[ $IMG =~ .*ubuntu.* || $IMG =~ .*debian.* ]]; then
>> + PKG_TOOL=apt
>> + fi
>> + echo "Testing $REL Tarantool on $IMG image ..."
>> + newimg="check-s3_"`echo $IMG | sed 's#:#-#g'`"_$REL"
>> + # repository update command must always be rerun to be able
>> + # to get the new available versions of the Tarantool, so the
>> + # flag "--no-cache" must be set
>> + docker build \
>> + --network=host \
>> + --no-cache \
>> + --build-arg PKG_TOOL=$PKG_TOOL \
>> + --build-arg REL=$REL \
>> + --build-arg IMG=$IMG \
>> + --build-arg TNT_VER=$VER \
>> + -f $cwd/Dockerfile.s3 -t $newimg .
>> +}
>> +
>> +for rel in $rels ; do
>> + for img in $imgs ; do
>> + run_docker $ver $img $rel
>> + done
>> +done
>> --
>> 2.17.1
>>
>--
>sergeyb@ 
 
 
--
Alexander Tikhonov
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.tarantool.org/pipermail/tarantool-patches/attachments/20200415/cd115723/attachment.html>