From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp37.i.mail.ru (smtp37.i.mail.ru [94.100.177.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C979845C316 for ; Mon, 23 Mar 2020 12:04:25 +0300 (MSK) Date: Mon, 23 Mar 2020 12:04:23 +0300 From: Sergey Bronnikov Message-ID: <20200323090423.GA90840@pony.bronevichok.ru> References: <2fc104925872d7ab1b334f7687250d82aed13634.1584947520.git.avtikhon@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <2fc104925872d7ab1b334f7687250d82aed13634.1584947520.git.avtikhon@tarantool.org> Subject: Re: [Tarantool-patches] [PATCH v1] Add S3 package checker List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Alexander V. Tikhonov" Cc: Oleg Piskunov , tarantool-patches@dev.tarantool.org Hello, I propose to check script tools/check_package.sh with shellcheck. Right now it reports a number of warnings. Sergey On 10:20 Mon 23 Mar , Alexander V. Tikhonov wrote: > Created Dockerfile based script to check packages available at the S3 > repository. Set this script to be run just after the package creation. > > Follows up #3380 > --- > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3380-check-packages-s3-full-ci > Issue: https://github.com/tarantool/tarantool/issues/3380 > Gitlab: https://gitlab.com/tarantool/tarantool/pipelines/128701075 > > .gitlab.mk | 1 + > tools/Dockerfile.s3 | 23 +++++++++++++++++ > tools/check_package.sh | 57 ++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 81 insertions(+) > create mode 100644 tools/Dockerfile.s3 > create mode 100755 tools/check_package.sh > > diff --git a/.gitlab.mk b/.gitlab.mk > index b39c5c651..5053b914f 100644 > --- a/.gitlab.mk > +++ b/.gitlab.mk > @@ -127,6 +127,7 @@ deploy: package > echo ${GPG_SECRET_KEY} | base64 -d | gpg --batch --import || true > ./tools/update_repo.sh -o=${OS} -d=${DIST} \ > -b="${LIVE_REPO_S3_DIR}/${BUCKET}" build > + ./tools/check_package.sh "" ${OS}:${DIST} ${BUCKET} > if git name-rev --name-only --tags --no-undefined HEAD 2>/dev/null ; then \ > ./tools/update_repo.sh -o=${OS} -d=${DIST} \ > -b="${RELEASE_REPO_S3_DIR}/${BUCKET}" build ; \ > diff --git a/tools/Dockerfile.s3 b/tools/Dockerfile.s3 > new file mode 100644 > index 000000000..338cf6926 > --- /dev/null > +++ b/tools/Dockerfile.s3 > @@ -0,0 +1,23 @@ > +ARG IMG > +FROM ${IMG} > + > +ARG PKG_TOOL > +RUN ${PKG_TOOL} update -y >1.log 2>&1 || ( cat 1.log && false ) > +RUN ${PKG_TOOL} install -y curl >2.log 2>&1 || ( cat 2.log && false ) > + > +RUN curl -L https://tarantool.io/installer.sh >installer.sh 2>3.log || ( cat 3.log && false ) > +RUN chmod a+x installer.sh > +RUN sed "s#/bin/bash#/bin/bash -x#g" -i installer.sh > + > +# ./installer.sh is the external tool which return status is not > +# expectable and can be false because of its additional features, > +# all that is needed for us from the script just to install Tarantool, > +# NOTE: found that on centos:8 script fails but on rerun passes > +ARG REL > +RUN VER=${REL} ./installer.sh >4.log 2>&1 || ( VER=${REL} ./installer.sh >>4.log 2>&1 || ( cat 4.log && false )) > +ARG TNT_VER > +RUN CVER=$(tarantool -V | head -n 1 | awk '{print $2}' | sed 's#-#.#g') && \ > + echo "Tarantool version:" && \ > + echo "Expected: ${TNT_VER}" && \ > + echo "Returned: ${CVER}" && \ > + if ! echo "${CVER}" | grep -e "^${TNT_VER}" >/dev/null ; then false ; fi > diff --git a/tools/check_package.sh b/tools/check_package.sh > new file mode 100755 > index 000000000..b3fdcdcda > --- /dev/null > +++ b/tools/check_package.sh > @@ -0,0 +1,57 @@ > +#!/bin/bash > +set -e > + > +ver="$1" > +imgs="$2" > +rels="$3" > + > +if [ "$ver" == "" ]; then > + # some OS like CentOS/Fedora use only 6 hexadecimal > + # digits as the abbreviated object name > + ver=$(git describe --abbrev=6 | sed 's#-#.#g') > +fi > + > +if [ "$imgs" == "" ]; then > + for dist in 14.04 16.04 18.04 19.04 19.10 ; do imgs="$imgs ubuntu:$dist" ; done > + for dist in jessie stretch buster ; do imgs="$imgs debian:$dist" ; done > + for dist in 6 7 8 ; do imgs="$imgs centos:$dist" ; done > + for dist in 28 29 30 31 ; do imgs="$imgs fedora:$dist" ; done > +fi > + > +imgs=$(echo $imgs | sed "s#el:#centos:#g") > + > +if [ "$rels" == "" ]; then > + rels="1.10 2.1 2.2 2.3 2.4" > +fi > + > +cwd=`dirname $0` > + > +run_docker () { > + VER=$1 > + IMG=$2 > + REL=$3 > + > + PKG_TOOL=yum > + if [[ $IMG =~ .*ubuntu.* || $IMG =~ .*debian.* ]]; then > + PKG_TOOL=apt > + fi > + echo "Testing $REL Tarantool on $IMG image ..." > + newimg="check-s3_"`echo $IMG | sed 's#:#-#g'`"_$REL" > + # repository update command must always be rerun to be able > + # to get the new available versions of the Tarantool, so the > + # flag "--no-cache" must be set > + docker build \ > + --network=host \ > + --no-cache \ > + --build-arg PKG_TOOL=$PKG_TOOL \ > + --build-arg REL=$REL \ > + --build-arg IMG=$IMG \ > + --build-arg TNT_VER=$VER \ > + -f $cwd/Dockerfile.s3 -t $newimg . > +} > + > +for rel in $rels ; do > + for img in $imgs ; do > + run_docker $ver $img $rel > + done > +done > -- > 2.17.1 > -- sergeyb@