* [Tarantool-patches] [PATCH v1] gitlab-ci: save failed test results files
@ 2020-07-27 13:07 Alexander V. Tikhonov
2020-07-28 11:49 ` Alexander Turenko
0 siblings, 1 reply; 2+ messages in thread
From: Alexander V. Tikhonov @ 2020-07-27 13:07 UTC (permalink / raw)
To: Kirill Yukhin, Alexander Turenko; +Cc: tarantool-patches
Added artifacts saver to all gitlab-ci jobs with testing. It
saves all found 'test/*/*.reject' files from the failed tests.
After each finished failed gitlab-ci job, user would be able to
download these files from it using gitlab-ci GUI.
Additionally added OSX_VARDIR environment variable to be able to
setup common path for artifacts and OSX shell scripts options.
Needed for #5050
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/artifacts_rejectfiles
Issue: https://github.com/tarantool/tarantool/issues/5050
.gitlab-ci.yml | 21 ++++++++++++++++++++-
.travis.mk | 4 ++--
test-run | 2 +-
3 files changed, 23 insertions(+), 4 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 580825005..48f0d0098 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -12,6 +12,7 @@ variables:
GITLAB_MAKE: "make -f .gitlab.mk"
GIT_STRATEGY: none
GIT_CLEAN_FLAGS: none
+ OSX_VARDIR: /tmp/tnt
# 1. Git checkout strategy used from:
# https://docs.gitlab.com/ee/ci/yaml/README.html#git-checkout
@@ -35,6 +36,12 @@ before_script:
# Jobs templates
+.artifacts_reject_files_template: &artifacts_reject_files_definition
+ artifacts:
+ when: always
+ paths:
+ - "test/*/*.reject"
+
.deploy_only_template: &deploy_only_definition
only:
- master
@@ -62,12 +69,14 @@ before_script:
stage: test
tags:
- docker_test
+ <<: *artifacts_reject_files_definition
.docker_test_clang8_template: &docker_test_clang8_definition
image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-buster:latest"
stage: test
tags:
- docker_test
+ <<: *artifacts_reject_files_definition
.pack_template: &pack_definition
<<: *pack_only_definition
@@ -84,6 +93,7 @@ before_script:
- deploy_test
script:
- ${GITLAB_MAKE} package
+ <<: *artifacts_reject_files_definition
.deploy_template: &deploy_definition
<<: *deploy_only_definition
@@ -100,16 +110,23 @@ before_script:
- deploy_test
script:
- ${GITLAB_MAKE} deploy
+ <<: *artifacts_reject_files_definition
.osx_template: &osx_definition
stage: test
script:
- ${GITLAB_MAKE} test_osx
+ variables:
+ VARDIR: ${OSX_VARDIR}
+ after_script:
+ - rm -rf test/var ; mkdir test/var && cp -rfp ${OSX_VARDIR}/*.reject test/var/.
+ <<: *artifacts_reject_files_definition
.vbox_template: &vbox_definition
stage: test
after_script:
- ${GITLAB_MAKE} vms_shutdown
+ <<: *artifacts_reject_files_definition
.perf_docker_test_template: &perf_docker_test_definition
<<: *perf_only_definition
@@ -193,9 +210,10 @@ osx_15_release:
osx_15_release_lto:
tags:
- osx_15
+ <<: *osx_definition
variables:
+ VARDIR: ${OSX_VARDIR}
EXTRA_ENV: 'export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;'
- <<: *osx_definition
freebsd_12_release:
<<: *vbox_definition
@@ -540,3 +558,4 @@ static_docker_build:
- deploy_test
script:
- ${GITLAB_MAKE} test_static_docker_build
+ <<: *artifacts_reject_files_definition
diff --git a/.travis.mk b/.travis.mk
index efc05cf05..fe71a4416 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -210,8 +210,8 @@ test_osx_no_deps: build_osx
launchctl limit maxproc || : ; \
ulimit -u ${MAX_PROC} || : ; \
ulimit -u ; \
- rm -rf /tmp/tnt ; \
- cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
+ rm -rf ${VARDIR} ; \
+ cd test && ./test-run.py --vardir ${VARDIR} --force $(TEST_RUN_EXTRA_PARAMS)
test_osx: deps_osx test_osx_no_deps
diff --git a/test-run b/test-run
index d92032233..0e8da635b 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit d92032233e10db8934ae5fed17fe3f3ae8293645
+Subproject commit 0e8da635b76416e2ae0704c285e2bf42279f53ae
--
2.17.1
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] gitlab-ci: save failed test results files
2020-07-27 13:07 [Tarantool-patches] [PATCH v1] gitlab-ci: save failed test results files Alexander V. Tikhonov
@ 2020-07-28 11:49 ` Alexander Turenko
0 siblings, 0 replies; 2+ messages in thread
From: Alexander Turenko @ 2020-07-28 11:49 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
> Needed for #5050
As I stated in [2] there is nothing about CI artefacts in #5050. There
is [3] about this.
> +.artifacts_reject_files_template: &artifacts_reject_files_definition
> + artifacts:
> + when: always
> + paths:
> + - "test/*/*.reject"
Are we need it when a job passes? Maybe save artefacts only for failed
jobs?
I would set `expire_in` into some large value (it is not possible to
store an artefact infinitely, gitlab docs lie), because we not always
start to investigate a flaky fail within 4 weeks. See [1] (it suggests
`expire_in: 1000 yrs`).
I also commented [2] and it seems the proposed changes will affect this
PR.
> + variables:
> + VARDIR: ${OSX_VARDIR}
> + after_script:
> + - rm -rf test/var ; mkdir test/var && cp -rfp ${OSX_VARDIR}/*.reject test/var/.
> <...>
> - rm -rf /tmp/tnt ; \
> - cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
> + rm -rf ${VARDIR} ; \
> + cd test && ./test-run.py --vardir ${VARDIR} --force $(TEST_RUN_EXTRA_PARAMS)
I don't understand what is going on here, but I hope that we can drop it
after PR #218 (see [2]) changes.
> diff --git a/test-run b/test-run
> index d92032233..0e8da635b 160000
> --- a/test-run
> +++ b/test-run
> @@ -1 +1 @@
> -Subproject commit d92032233e10db8934ae5fed17fe3f3ae8293645
> +Subproject commit 0e8da635b76416e2ae0704c285e2bf42279f53ae
I'll update the test-run submodule separately as I always do.
[1]: https://gitlab.com/gitlab-org/gitlab/-/issues/22761#note_251722017
[2]: https://github.com/tarantool/test-run/pull/218
[3]: https://github.com/tarantool/test-run/issues/90
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-07-28 11:50 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-27 13:07 [Tarantool-patches] [PATCH v1] gitlab-ci: save failed test results files Alexander V. Tikhonov
2020-07-28 11:49 ` Alexander Turenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox