From: Alexander Turenko <alexander.turenko@tarantool.org> To: "Alexander V . Tikhonov" <avtikhon@tarantool.org> Cc: tarantool-patches@dev.tarantool.org, Alexander Turenko <alexander.turenko@tarantool.org> Subject: [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables Date: Sun, 27 Dec 2020 07:13:25 +0300 [thread overview] Message-ID: <44ba959d94166293b4a44cd466d3aef4ad18478b.1609041893.git.alexander.turenko@tarantool.org> (raw) In-Reply-To: <cover.1609041893.git.alexander.turenko@tarantool.org> We want to increase testing timeouts for GitLab CI, where we use our own runners and observe stalls and high disk pressure when several vinyl tests are run in parallel. The idea is to set variables in GitLab CI web interface and read them from test-run (see [1]). First, we need to pass the variables into inner environments. GitLab CI jobs run the testing using packpack, Docker or VirtualBox. Packpack already preserves environment variables that are listed in the PRESERVE_ENVVARS variable (see [2]). This commit passes the variables that are listed in the PRESERVE_ENVVARS variable into Docker and VirtualBox environment. So, all jobs will have given variables in the enviroment. (Also dropped unused EXTRA_ENV variable.) The next commit will update the test-run submodule with support of setting timeouts using environment variables. [1]: https://github.com/tarantool/test-run/issues/258 [2]: https://github.com/packpack/packpack/pull/135 --- .gitlab.mk | 14 +++++++++++--- .travis.mk | 22 ++++++++++++++++++++-- 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/.gitlab.mk b/.gitlab.mk index 5dbf66cef..f4fcc8fae 100644 --- a/.gitlab.mk +++ b/.gitlab.mk @@ -85,6 +85,16 @@ perf_cleanup: perf_clone_benchs_repo perf_cleanup_image # Run tests under a virtual machine # ################################# +# Transform the ${PRESERVE_ENVVARS} comma separated variables list +# to the 'key="value" key="value" <...>' string. +# +# Add PRESERVE_ENVVARS itself to the list to allow to use this +# make script again from the inner environment (if there will be +# a need). +comma := , +ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS)) +PRESERVE_ENV := $(foreach var,$(ENVVARS),$(var)="$($(var))") + vms_start: VBoxManage controlvm ${VMS_NAME} poweroff || true VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME} @@ -93,9 +103,7 @@ vms_start: vms_test_%: tar czf - ../tarantool | ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} tar xzf - ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \ - '${EXTRA_ENV} \ - cd tarantool && \ - ${TRAVIS_MAKE} $(subst vms_,,$@)'" + 'cd tarantool && ${PRESERVE_ENV} ${TRAVIS_MAKE} $(subst vms_,,$@)'" vms_shutdown: VBoxManage controlvm ${VMS_NAME} poweroff diff --git a/.travis.mk b/.travis.mk index 8396ba9e5..b835a603b 100644 --- a/.travis.mk +++ b/.travis.mk @@ -20,6 +20,16 @@ LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lei TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip" TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME) +# Transform the ${PRESERVE_ENVVARS} comma separated variables list +# to the '-e key="value" -e key="value" <...>' string. +# +# Add PRESERVE_ENVVARS itself to the list to allow to use this +# make script again from the inner environment (if there will be +# a need). +comma := , +ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS)) +DOCKER_ENV := $(foreach var,$(ENVVARS),-e $(var)="$($(var))") + all: package package: @@ -50,6 +60,7 @@ docker_%: -e APT_EXTRA_FLAGS="${APT_EXTRA_FLAGS}" \ -e CC=${CC} \ -e CXX=${CXX} \ + ${DOCKER_ENV} \ ${DOCKER_IMAGE} \ make -f .travis.mk $(subst docker_,,$@) @@ -232,8 +243,14 @@ test_static_build_cmake_linux: # ################### test_debian_docker_luacheck: - docker run -w ${OOS_SRC_PATH} -v ${PWD}:${OOS_SRC_PATH} --privileged \ - --cap-add=sys_nice --network=host -i ${DOCKER_IMAGE_TARANTOOL} \ + docker run \ + -w ${OOS_SRC_PATH} \ + -v ${PWD}:${OOS_SRC_PATH} \ + --privileged \ + --cap-add=sys_nice \ + --network=host \ + ${DOCKER_ENV} \ + -i ${DOCKER_IMAGE_TARANTOOL} \ make -f .travis.mk test_debian_luacheck test_debian_install_luacheck: @@ -264,6 +281,7 @@ test_oos_build: docker run --network=host -w ${OOS_SRC_PATH} \ --mount type=bind,source="${PWD}",target=${OOS_SRC_PATH},readonly,bind-propagation=rslave \ --tmpfs ${OOS_BUILD_PATH}:exec \ + ${DOCKER_ENV} \ -i ${DOCKER_IMAGE_TARANTOOL} \ make -f .travis.mk ${OOS_BUILD_RULE} -- 2.25.0
next prev parent reply other threads:[~2020-12-27 4:13 UTC|newest] Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-27 4:13 [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander Turenko 2020-12-27 4:13 ` Alexander Turenko [this message] 2020-12-27 4:13 ` [Tarantool-patches] [PATCH 2/2] test: update test-run (pass timeouts via env) Alexander Turenko 2020-12-27 4:37 ` [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander V. Tikhonov 2020-12-27 5:22 ` Alexander Turenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=44ba959d94166293b4a44cd466d3aef4ad18478b.1609041893.git.alexander.turenko@tarantool.org \ --to=alexander.turenko@tarantool.org \ --cc=avtikhon@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox