* [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files
@ 2020-09-17 8:04 Alexander V. Tikhonov
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov
0 siblings, 2 replies; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw)
To: Alexander Turenko; +Cc: tarantool-patches
Added artifacts saver to all gitlab-ci jobs with testing.
Gitlab-ci artifacts saves all found *.reject files in format:
test/<suite>/*.reject
Also it saves logs files collected by test-run in artifacts
directory in format:
test/var/artifacts
After each finished failed gitlab-ci job, user would be able to
download these files from it using gitlab-ci GUI.
Closes #5050
Github: https://github.com/tarantool/tarantool/tree/avtikhon/artifacts_rejectfiles
Issue: https://github.com/tarantool/tarantool/issues/5050
Test-run issue: https://github.com/tarantool/test-run/pull/218
Test-run issue: https://github.com/tarantool/test-run/issues/90
Alexander V. Tikhonov (2):
gitlab-ci: save failed test results files
gitlab-ci: set opensuse jobs to test group
.gitlab-ci.yml | 56 ++++++++++++++++++++++++++++++++++++++++++++------
.travis.mk | 6 +++---
2 files changed, 53 insertions(+), 9 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files
2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov
@ 2020-09-17 8:04 ` Alexander V. Tikhonov
2020-09-21 2:18 ` Alexander Turenko
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov
1 sibling, 1 reply; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw)
To: Alexander Turenko; +Cc: tarantool-patches
Added artifacts saver to all gitlab-ci jobs with testing.
Gitlab-ci jobs saves its results files in the following paths:
1. base jobs for testing different features:
- test/<suite>/*.reject
- test/var/artifacts
2. OSX jobs:
- test/<suite>/*.reject
- ${OSX_VARDIR}/artifacts/*
3. pack/deploy jobs:
- build/usr/src/*/tarantool-*/test/*/*.reject
- build/usr/src/*/tarantool-*/test/var/artifacts
4. VBOX jobs (freebsd_12) on virtual host:
- ~/tarantool/test/*/*.reject
- ~/tarantool/test/var/artifacts
Due to gitlab-ci jobs have different places of testings in patch
added 'after_script' instructions which collects it to the common
place known by gitlab-ci artifactory patterns. It will provide
gitlab-ci to have the same format of artifacts paths in its packages.
Gitlab-ci artifacts saves all found *.reject files in format:
test/<suite>/*.reject
Also it saves logs files collected by test-run in artifacts
directory in format:
test/var/artifacts
After each finished failed gitlab-ci job, user would be able to
download these files from it using gitlab-ci GUI.
Additionally added OSX_VARDIR environment variable to be able to
setup common path for artifacts and OSX shell scripts options.
OSX_VARDIR: /tmp/tnt
Part of #5050
---
.gitlab-ci.yml | 48 ++++++++++++++++++++++++++++++++++++++++++++++--
.travis.mk | 6 +++---
2 files changed, 49 insertions(+), 5 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 712e248fe..5a00641d9 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -12,6 +12,7 @@ variables:
GITLAB_MAKE: "make -f .gitlab.mk"
GIT_STRATEGY: none
GIT_CLEAN_FLAGS: none
+ OSX_VARDIR: /tmp/tnt
# 1. Git checkout strategy used from:
# https://docs.gitlab.com/ee/ci/yaml/README.html#git-checkout
@@ -35,6 +36,17 @@ before_script:
# Jobs templates
+.artifacts_reject_files_template: &artifacts_reject_files_definition
+ artifacts:
+ expire_in: 1000 yrs
+ when: always
+ paths:
+ # default paths to reject files and logs
+ - "test/*/*.reject"
+ - "test/var/artifacts/*"
+ # OSX jobs paths to logs, reject files are allready in default locations
+ - "${OSX_VARDIR}/artifacts/*"
+
.deploy_only_template: &deploy_only_definition
only:
- master
@@ -65,12 +77,26 @@ before_script:
stage: test
tags:
- docker_test
+ <<: *artifacts_reject_files_definition
.docker_test_clang8_template: &docker_test_clang8_definition
image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-buster:latest"
stage: test
tags:
- docker_test
+ <<: *artifacts_reject_files_definition
+
+.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition
+ <<: *artifacts_reject_files_definition
+ after_script:
+ - >
+ for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do
+ d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ;
+ mkdir -p test/$d 2>/dev/null ;
+ cp $f test/$d/. ;
+ done ;
+ rm -rf test/var ; mkdir -p test/var ;
+ cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/.
.pack_template: &pack_definition
<<: *pack_only_definition
@@ -87,6 +113,7 @@ before_script:
- deploy_test
script:
- ${GITLAB_MAKE} package
+ <<: *pack_artifacts_reject_files_definition
.deploy_template: &deploy_definition
<<: *deploy_only_definition
@@ -103,16 +130,33 @@ before_script:
- deploy_test
script:
- ${GITLAB_MAKE} deploy
+ <<: *pack_artifacts_reject_files_definition
.osx_template: &osx_definition
stage: test
script:
- ${GITLAB_MAKE} test_osx
+ after_script:
+ # Artifacts can't be used from outside the project directory, check:
+ # https://docs.gitlab.com/ee/ci/yaml/README.html#artifactspaths
+ # "Paths are relative to the project directory ($CI_PROJECT_DIR)"
+ # "and can’t directly link outside it."
+ - rm -rf test/var ; mkdir test/var && cp -rf ${OSX_VARDIR}/artifacts test/var/.
+ <<: *artifacts_reject_files_definition
.vbox_template: &vbox_definition
stage: test
after_script:
- - ${GITLAB_MAKE} vms_shutdown
+ - >
+ for f in `ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} ls tarantool/test/*/*.reject` ; do
+ d=`dirname $f | sed 's#^tarantool/##g'` ;
+ mkdir -p $d 2>/dev/null ;
+ scp -P ${VMS_PORT} ${VMS_USER}@127.0.0.1:tarantool/$d/*.reject $d/.
+ done ;
+ rm -rf test/var ; mkdir -p test/var ;
+ scp -r -P ${VMS_PORT} ${VMS_USER}@127.0.0.1:tarantool/test/var/artifacts test/var/. ;
+ ${GITLAB_MAKE} vms_shutdown
+ <<: *artifacts_reject_files_definition
.perf_docker_test_template: &perf_docker_test_definition
<<: *perf_only_definition
@@ -567,7 +611,7 @@ static_build_cmake_linux:
- ${GITLAB_MAKE} test_static_build_cmake_linux
static_build_cmake_osx_15:
- stage: test
+ <<: *osx_definition
tags:
- osx_15
script:
diff --git a/.travis.mk b/.travis.mk
index 58d0c1596..db457bf56 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -217,11 +217,11 @@ INIT_TEST_ENV_OSX=\
launchctl limit maxproc || : ; \
ulimit -u ${MAX_PROC} || : ; \
ulimit -u ; \
- rm -rf /tmp/tnt
+ rm -rf ${OSX_VARDIR}
test_osx_no_deps: build_osx
${INIT_TEST_ENV_OSX}; \
- cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
+ cd test && ./test-run.py --vardir ${OSX_VARDIR} --force $(TEST_RUN_EXTRA_PARAMS)
test_osx: deps_osx test_osx_no_deps
@@ -240,7 +240,7 @@ test_static_build_cmake_osx: base_deps_osx
cd static-build && cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" . && \
make -j && ctest -V
${INIT_TEST_ENV_OSX}; \
- cd test && ./test-run.py --vardir /tmp/tnt \
+ cd test && ./test-run.py --vardir ${OSX_VARDIR} \
--builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \
--force $(TEST_RUN_EXTRA_PARAMS)
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group
2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov
@ 2020-09-17 8:04 ` Alexander V. Tikhonov
2020-09-25 15:50 ` Kirill Yukhin
1 sibling, 1 reply; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw)
To: Alexander Turenko; +Cc: tarantool-patches
Set opensuse jobs to test group to be sure that it will be run with
artifacts collecting and without gitlab-ci jobs extra parallization.
---
.gitlab-ci.yml | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5a00641d9..27d65ff7a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -478,13 +478,13 @@ debian_10:
DIST: 'buster'
opensuse_15_1:
- <<: *pack_definition
+ <<: *pack_test_definition
variables:
OS: 'opensuse-leap'
DIST: '15.1'
opensuse_15_2:
- <<: *pack_definition
+ <<: *pack_test_definition
variables:
OS: 'opensuse-leap'
DIST: '15.2'
@@ -587,13 +587,13 @@ debian_10_deploy:
DIST: 'buster'
opensuse_15_1_deploy:
- <<: *deploy_definition
+ <<: *deploy_test_definition
variables:
OS: 'opensuse-leap'
DIST: '15.1'
opensuse_15_2_deploy:
- <<: *deploy_definition
+ <<: *deploy_test_definition
variables:
OS: 'opensuse-leap'
DIST: '15.2'
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov
@ 2020-09-21 2:18 ` Alexander Turenko
2020-09-23 6:45 ` Alexander V. Tikhonov
0 siblings, 1 reply; 6+ messages in thread
From: Alexander Turenko @ 2020-09-21 2:18 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
> + <<: *artifacts_reject_files_definition
> +
> +.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition
> + <<: *artifacts_reject_files_definition
> + after_script:
> + - >
> + for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do
Nit: 'for f in path-with-star-globbing; do' would do almost the same and
I would suggest to use the simpler form (it also does not depend on an
external tool and does not involve forking), but this way we should
handle 'no such files' case explicitly. See example here:
https://github.com/tarantool/tarantool/commit/080beba063b05b184d7befcb15a8bcf32e7424f6
| for file in @TARANTOOL_ENABLEDDIR@/*.lua; do
| instance=`basename $file .lua`
| [ "${instance}" = "*" ] && break # skip empty directory
| <...>
| done
When the glob is not matched, it falls into the loop with the 'as is'
value (with star symbols).
We all like bourne shell for those funny pitfalls.
> + d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ;
> + mkdir -p test/$d 2>/dev/null ;
> + cp $f test/$d/. ;
> + done ;
> + rm -rf test/var ; mkdir -p test/var ;
> + cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/.
Nit: The period at the end looks unusual, but don't change anything.
The patch would be a way simpler if test-run would collect everything
that is necessary to upload within the artifacts directory.
In fact, reject files are already there:
test/var/artifacts/<worker_name>/<test_name>.result. But if you want to
organize them in an another way, I would do this on the test-run side
(somewhere within test/var/artifacts).
Related code: [1]. See, the reject file is just copied, not moved.
[1]: https://github.com/tarantool/test-run/blob/461731d71cf27faa6295bc759941f402fedc88e4/lib/test.py#L254
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files
2020-09-21 2:18 ` Alexander Turenko
@ 2020-09-23 6:45 ` Alexander V. Tikhonov
0 siblings, 0 replies; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-23 6:45 UTC (permalink / raw)
To: Alexander Turenko; +Cc: tarantool-patches
Hi Alexander, thanks for the review. I've removed collecting of results
files *.reject, which duplicates *.result files in artifactory path.
And set 'artifactory' path as root in saved artifacts packages.
On Mon, Sep 21, 2020 at 05:18:13AM +0300, Alexander Turenko wrote:
> > + <<: *artifacts_reject_files_definition
> > +
> > +.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition
> > + <<: *artifacts_reject_files_definition
> > + after_script:
> > + - >
> > + for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do
>
> Nit: 'for f in path-with-star-globbing; do' would do almost the same and
> I would suggest to use the simpler form (it also does not depend on an
> external tool and does not involve forking), but this way we should
> handle 'no such files' case explicitly. See example here:
>
> https://github.com/tarantool/tarantool/commit/080beba063b05b184d7befcb15a8bcf32e7424f6
>
> | for file in @TARANTOOL_ENABLEDDIR@/*.lua; do
> | instance=`basename $file .lua`
> | [ "${instance}" = "*" ] && break # skip empty directory
> | <...>
> | done
>
> When the glob is not matched, it falls into the loop with the 'as is'
> value (with star symbols).
>
> We all like bourne shell for those funny pitfalls.
>
> > + d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ;
> > + mkdir -p test/$d 2>/dev/null ;
> > + cp $f test/$d/. ;
> > + done ;
> > + rm -rf test/var ; mkdir -p test/var ;
> > + cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/.
>
> Nit: The period at the end looks unusual, but don't change anything.
>
> The patch would be a way simpler if test-run would collect everything
> that is necessary to upload within the artifacts directory.
>
> In fact, reject files are already there:
> test/var/artifacts/<worker_name>/<test_name>.result. But if you want to
> organize them in an another way, I would do this on the test-run side
> (somewhere within test/var/artifacts).
>
> Related code: [1]. See, the reject file is just copied, not moved.
>
> [1]: https://github.com/tarantool/test-run/blob/461731d71cf27faa6295bc759941f402fedc88e4/lib/test.py#L254
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov
@ 2020-09-25 15:50 ` Kirill Yukhin
0 siblings, 0 replies; 6+ messages in thread
From: Kirill Yukhin @ 2020-09-25 15:50 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches, Alexander Turenko
Hello,
On 17 сен 11:04, Alexander V. Tikhonov wrote:
> Set opensuse jobs to test group to be sure that it will be run with
> artifacts collecting and without gitlab-ci jobs extra parallization.
> ---
> .gitlab-ci.yml | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
I've checked your patch into 1.10, 2.4, 2.5, master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-25 15:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov
2020-09-21 2:18 ` Alexander Turenko
2020-09-23 6:45 ` Alexander V. Tikhonov
2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov
2020-09-25 15:50 ` Kirill Yukhin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox