* [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables
2020-12-27 4:13 [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander Turenko
@ 2020-12-27 4:13 ` Alexander Turenko
2020-12-27 4:13 ` [Tarantool-patches] [PATCH 2/2] test: update test-run (pass timeouts via env) Alexander Turenko
2020-12-27 4:37 ` [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander V. Tikhonov
2 siblings, 0 replies; 5+ messages in thread
From: Alexander Turenko @ 2020-12-27 4:13 UTC (permalink / raw)
To: Alexander V . Tikhonov; +Cc: tarantool-patches, Alexander Turenko
We want to increase testing timeouts for GitLab CI, where we use our own
runners and observe stalls and high disk pressure when several vinyl
tests are run in parallel. The idea is to set variables in GitLab CI web
interface and read them from test-run (see [1]).
First, we need to pass the variables into inner environments. GitLab CI
jobs run the testing using packpack, Docker or VirtualBox.
Packpack already preserves environment variables that are listed in the
PRESERVE_ENVVARS variable (see [2]).
This commit passes the variables that are listed in the PRESERVE_ENVVARS
variable into Docker and VirtualBox environment. So, all jobs will have
given variables in the enviroment. (Also dropped unused EXTRA_ENV
variable.)
The next commit will update the test-run submodule with support of
setting timeouts using environment variables.
[1]: https://github.com/tarantool/test-run/issues/258
[2]: https://github.com/packpack/packpack/pull/135
---
.gitlab.mk | 14 +++++++++++---
.travis.mk | 22 ++++++++++++++++++++--
2 files changed, 31 insertions(+), 5 deletions(-)
diff --git a/.gitlab.mk b/.gitlab.mk
index 5dbf66cef..f4fcc8fae 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -85,6 +85,16 @@ perf_cleanup: perf_clone_benchs_repo perf_cleanup_image
# Run tests under a virtual machine
# #################################
+# Transform the ${PRESERVE_ENVVARS} comma separated variables list
+# to the 'key="value" key="value" <...>' string.
+#
+# Add PRESERVE_ENVVARS itself to the list to allow to use this
+# make script again from the inner environment (if there will be
+# a need).
+comma := ,
+ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS))
+PRESERVE_ENV := $(foreach var,$(ENVVARS),$(var)="$($(var))")
+
vms_start:
VBoxManage controlvm ${VMS_NAME} poweroff || true
VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME}
@@ -93,9 +103,7 @@ vms_start:
vms_test_%:
tar czf - ../tarantool | ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} tar xzf -
ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \
- '${EXTRA_ENV} \
- cd tarantool && \
- ${TRAVIS_MAKE} $(subst vms_,,$@)'"
+ 'cd tarantool && ${PRESERVE_ENV} ${TRAVIS_MAKE} $(subst vms_,,$@)'"
vms_shutdown:
VBoxManage controlvm ${VMS_NAME} poweroff
diff --git a/.travis.mk b/.travis.mk
index 8396ba9e5..b835a603b 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -20,6 +20,16 @@ LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lei
TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip"
TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME)
+# Transform the ${PRESERVE_ENVVARS} comma separated variables list
+# to the '-e key="value" -e key="value" <...>' string.
+#
+# Add PRESERVE_ENVVARS itself to the list to allow to use this
+# make script again from the inner environment (if there will be
+# a need).
+comma := ,
+ENVVARS := PRESERVE_ENVVARS $(subst $(comma), ,$(PRESERVE_ENVVARS))
+DOCKER_ENV := $(foreach var,$(ENVVARS),-e $(var)="$($(var))")
+
all: package
package:
@@ -50,6 +60,7 @@ docker_%:
-e APT_EXTRA_FLAGS="${APT_EXTRA_FLAGS}" \
-e CC=${CC} \
-e CXX=${CXX} \
+ ${DOCKER_ENV} \
${DOCKER_IMAGE} \
make -f .travis.mk $(subst docker_,,$@)
@@ -232,8 +243,14 @@ test_static_build_cmake_linux:
# ###################
test_debian_docker_luacheck:
- docker run -w ${OOS_SRC_PATH} -v ${PWD}:${OOS_SRC_PATH} --privileged \
- --cap-add=sys_nice --network=host -i ${DOCKER_IMAGE_TARANTOOL} \
+ docker run \
+ -w ${OOS_SRC_PATH} \
+ -v ${PWD}:${OOS_SRC_PATH} \
+ --privileged \
+ --cap-add=sys_nice \
+ --network=host \
+ ${DOCKER_ENV} \
+ -i ${DOCKER_IMAGE_TARANTOOL} \
make -f .travis.mk test_debian_luacheck
test_debian_install_luacheck:
@@ -264,6 +281,7 @@ test_oos_build:
docker run --network=host -w ${OOS_SRC_PATH} \
--mount type=bind,source="${PWD}",target=${OOS_SRC_PATH},readonly,bind-propagation=rslave \
--tmpfs ${OOS_BUILD_PATH}:exec \
+ ${DOCKER_ENV} \
-i ${DOCKER_IMAGE_TARANTOOL} \
make -f .travis.mk ${OOS_BUILD_RULE}
--
2.25.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [Tarantool-patches] [PATCH 2/2] test: update test-run (pass timeouts via env)
2020-12-27 4:13 [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander Turenko
2020-12-27 4:13 ` [Tarantool-patches] [PATCH 1/2] ci: preserve certain environment variables Alexander Turenko
@ 2020-12-27 4:13 ` Alexander Turenko
2020-12-27 4:37 ` [Tarantool-patches] [PATCH 0/2] gitlab-ci: increase testing timeouts Alexander V. Tikhonov
2 siblings, 0 replies; 5+ messages in thread
From: Alexander Turenko @ 2020-12-27 4:13 UTC (permalink / raw)
To: Alexander V . Tikhonov; +Cc: tarantool-patches, Alexander Turenko
The following variables now control timeouts (if corresponding command
line options are not passed): TEST_TIMEOUT, NO_OUTPUT_TIMEOUT,
REPLICATION_SYNC_TIMEOUT. See [1] for details.
I set the following values in the GitLab CI web interface:
| Variable | Value |
| ------------------------ | ------------------------------------------------------- |
| REPLICATION_SYNC_TIMEOUT | 300 |
| TEST_TIMEOUT | 310 |
| NO_OUTPUT_TIMEOUT | 320 |
| PRESERVE_ENVVARS | REPLICATION_SYNC_TIMEOUT,TEST_TIMEOUT,NO_OUTPUT_TIMEOUT |
See packpack change [2] and the commit 'ci: preserve certain environment
variables' regarding the PRESERVE_ENVVARS variable.
The reason, why we need to increase timeouts, comes from the following
facts:
- We use self-hosted runners to serve GitLab CI jobs. So, the machine
resources are limited.
- We run testing with high level of parallelism to speed it up.
- We have a bunch of vinyl tests, which intensively use disk.
Disk accesses may be quite long within this infrastructure and the
obvious way to workaround the problem is to increase timeouts.
In the long term we should scale resources depending on the testing
needs. We'll try to use GitHub hosted runners or, if we'll reach some
limits, will setup GitHub runners on the Mail.Ru Cloud Solutions
infrastructure.
[1]: https://github.com/tarantool/test-run/issues/258
[2]: https://github.com/packpack/packpack/pull/135
---
test-run | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test-run b/test-run
index 584e273b1..9e5fc6d0f 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit 584e273b134ffa0ee7d44419f7443f752ff4fb33
+Subproject commit 9e5fc6d0fede3b1d142a44c3ca36cfff3cb32f9f
--
2.25.0
^ permalink raw reply [flat|nested] 5+ messages in thread