Tarantool development patches archive
 help / color / mirror / Atom feed
From: Alexander Turenko <alexander.turenko@tarantool.org>
To: "Alexander V . Tikhonov" <avtikhon@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org,
	Alexander Turenko <alexander.turenko@tarantool.org>
Subject: [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables
Date: Sun, 27 Dec 2020 07:13:25 +0300	[thread overview]
Message-ID: <44ba959d94166293b4a44cd466d3aef4ad18478b.1609041893.git.alexander.turenko@tarantool.org> (raw)
In-Reply-To: <cover.1609041893.git.alexander.turenko@tarantool.org>

We want to increase testing timeouts for GitLab CI, where we use our own
runners and observe stalls and high disk pressure when several vinyl
tests are run in parallel. The idea is to set variables in GitLab CI web
interface and read them from test-run (see [1]).

First, we need to pass the variables into inner environments. GitLab CI
jobs run the testing using packpack, Docker or VirtualBox.

Packpack already preserves environment variables that are listed in the
PRESERVE_ENVVARS variable (see [2]).

This commit passes the variables that are listed in the PRESERVE_ENVVARS
variable into Docker and VirtualBox environment. So, all jobs will have
given variables in the enviroment. (Also dropped unused EXTRA_ENV
variable.)

The next commit will update the test-run submodule with support of
setting timeouts using environment variables.

[1]: https://github.com/tarantool/test-run/issues/258
[2]: https://github.com/packpack/packpack/pull/135
---
 .gitlab.mk | 14 +++++++++++---
 .travis.mk | 22 ++++++++++++++++++++--
 2 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/.gitlab.mk b/.gitlab.mk
index 5dbf66cef..f4fcc8fae 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -85,6 +85,16 @@ perf_cleanup: perf_clone_benchs_repo perf_cleanup_image
 # Run tests under a virtual machine
 # #################################
 
+# Transform the ${PRESERVE_ENVVARS} comma separated variables list
+# to the 'key="value" key="value" <...>' string.
+#
+# Add PRESERVE_ENVVARS itself to the list to allow to use this
+# make script again from the inner environment (if there will be
+# a need).
+comma := ,
+ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS))
+PRESERVE_ENV := $(foreach var,$(ENVVARS),$(var)="$($(var))")
+
 vms_start:
 	VBoxManage controlvm ${VMS_NAME} poweroff || true
 	VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME}
@@ -93,9 +103,7 @@ vms_start:
 vms_test_%:
 	tar czf - ../tarantool | ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} tar xzf -
 	ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \
-		'${EXTRA_ENV} \
-		cd tarantool && \
-		${TRAVIS_MAKE} $(subst vms_,,$@)'"
+		'cd tarantool && ${PRESERVE_ENV} ${TRAVIS_MAKE} $(subst vms_,,$@)'"
 
 vms_shutdown:
 	VBoxManage controlvm ${VMS_NAME} poweroff
diff --git a/.travis.mk b/.travis.mk
index 8396ba9e5..b835a603b 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -20,6 +20,16 @@ LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lei
 TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip"
 TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME)
 
+# Transform the ${PRESERVE_ENVVARS} comma separated variables list
+# to the '-e key="value" -e key="value" <...>' string.
+#
+# Add PRESERVE_ENVVARS itself to the list to allow to use this
+# make script again from the inner environment (if there will be
+# a need).
+comma := ,
+ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS))
+DOCKER_ENV := $(foreach var,$(ENVVARS),-e $(var)="$($(var))")
+
 all: package
 
 package:
@@ -50,6 +60,7 @@ docker_%:
 		-e APT_EXTRA_FLAGS="${APT_EXTRA_FLAGS}" \
 		-e CC=${CC} \
 		-e CXX=${CXX} \
+		${DOCKER_ENV} \
 		${DOCKER_IMAGE} \
 		make -f .travis.mk $(subst docker_,,$@)
 
@@ -232,8 +243,14 @@ test_static_build_cmake_linux:
 # ###################
 
 test_debian_docker_luacheck:
-	docker run -w ${OOS_SRC_PATH} -v ${PWD}:${OOS_SRC_PATH} --privileged \
-		--cap-add=sys_nice --network=host -i ${DOCKER_IMAGE_TARANTOOL} \
+	docker run                       \
+		-w ${OOS_SRC_PATH}           \
+		-v ${PWD}:${OOS_SRC_PATH}    \
+		--privileged                 \
+		--cap-add=sys_nice           \
+		--network=host               \
+		${DOCKER_ENV}                \
+		-i ${DOCKER_IMAGE_TARANTOOL} \
 		make -f .travis.mk test_debian_luacheck
 
 test_debian_install_luacheck:
@@ -264,6 +281,7 @@ test_oos_build:
 	docker run --network=host -w ${OOS_SRC_PATH} \
 		--mount type=bind,source="${PWD}",target=${OOS_SRC_PATH},readonly,bind-propagation=rslave \
 		--tmpfs ${OOS_BUILD_PATH}:exec \
+		${DOCKER_ENV} \
 		-i ${DOCKER_IMAGE_TARANTOOL} \
 		make -f .travis.mk ${OOS_BUILD_RULE}
 
-- 
2.25.0

  reply	other threads:[~2020-12-27  4:13 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-27  4:13 [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander Turenko
2020-12-27  4:13 ` Alexander Turenko [this message]
2020-12-27  4:13 ` [Tarantool-patches] [PATCH 2/2] test: update test-run (pass timeouts via env) Alexander Turenko
2020-12-27  4:37 ` [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander V. Tikhonov
2020-12-27  5:22   ` Alexander Turenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=44ba959d94166293b4a44cd466d3aef4ad18478b.1609041893.git.alexander.turenko@tarantool.org \
    --to=alexander.turenko@tarantool.org \
    --cc=avtikhon@tarantool.org \
    --cc=tarantool-patches@dev.tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox