* [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files @ 2020-09-17 8:04 Alexander V. Tikhonov 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov 0 siblings, 2 replies; 6+ messages in thread From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw) To: Alexander Turenko; +Cc: tarantool-patches Added artifacts saver to all gitlab-ci jobs with testing. Gitlab-ci artifacts saves all found *.reject files in format: test/<suite>/*.reject Also it saves logs files collected by test-run in artifacts directory in format: test/var/artifacts After each finished failed gitlab-ci job, user would be able to download these files from it using gitlab-ci GUI. Closes #5050 Github: https://github.com/tarantool/tarantool/tree/avtikhon/artifacts_rejectfiles Issue: https://github.com/tarantool/tarantool/issues/5050 Test-run issue: https://github.com/tarantool/test-run/pull/218 Test-run issue: https://github.com/tarantool/test-run/issues/90 Alexander V. Tikhonov (2): gitlab-ci: save failed test results files gitlab-ci: set opensuse jobs to test group .gitlab-ci.yml | 56 ++++++++++++++++++++++++++++++++++++++++++++------ .travis.mk | 6 +++--- 2 files changed, 53 insertions(+), 9 deletions(-) -- 2.17.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files 2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov @ 2020-09-17 8:04 ` Alexander V. Tikhonov 2020-09-21 2:18 ` Alexander Turenko 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov 1 sibling, 1 reply; 6+ messages in thread From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw) To: Alexander Turenko; +Cc: tarantool-patches Added artifacts saver to all gitlab-ci jobs with testing. Gitlab-ci jobs saves its results files in the following paths: 1. base jobs for testing different features: - test/<suite>/*.reject - test/var/artifacts 2. OSX jobs: - test/<suite>/*.reject - ${OSX_VARDIR}/artifacts/* 3. pack/deploy jobs: - build/usr/src/*/tarantool-*/test/*/*.reject - build/usr/src/*/tarantool-*/test/var/artifacts 4. VBOX jobs (freebsd_12) on virtual host: - ~/tarantool/test/*/*.reject - ~/tarantool/test/var/artifacts Due to gitlab-ci jobs have different places of testings in patch added 'after_script' instructions which collects it to the common place known by gitlab-ci artifactory patterns. It will provide gitlab-ci to have the same format of artifacts paths in its packages. Gitlab-ci artifacts saves all found *.reject files in format: test/<suite>/*.reject Also it saves logs files collected by test-run in artifacts directory in format: test/var/artifacts After each finished failed gitlab-ci job, user would be able to download these files from it using gitlab-ci GUI. Additionally added OSX_VARDIR environment variable to be able to setup common path for artifacts and OSX shell scripts options. OSX_VARDIR: /tmp/tnt Part of #5050 --- .gitlab-ci.yml | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- .travis.mk | 6 +++--- 2 files changed, 49 insertions(+), 5 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 712e248fe..5a00641d9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,6 +12,7 @@ variables: GITLAB_MAKE: "make -f .gitlab.mk" GIT_STRATEGY: none GIT_CLEAN_FLAGS: none + OSX_VARDIR: /tmp/tnt # 1. Git checkout strategy used from: # https://docs.gitlab.com/ee/ci/yaml/README.html#git-checkout @@ -35,6 +36,17 @@ before_script: # Jobs templates +.artifacts_reject_files_template: &artifacts_reject_files_definition + artifacts: + expire_in: 1000 yrs + when: always + paths: + # default paths to reject files and logs + - "test/*/*.reject" + - "test/var/artifacts/*" + # OSX jobs paths to logs, reject files are allready in default locations + - "${OSX_VARDIR}/artifacts/*" + .deploy_only_template: &deploy_only_definition only: - master @@ -65,12 +77,26 @@ before_script: stage: test tags: - docker_test + <<: *artifacts_reject_files_definition .docker_test_clang8_template: &docker_test_clang8_definition image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-buster:latest" stage: test tags: - docker_test + <<: *artifacts_reject_files_definition + +.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition + <<: *artifacts_reject_files_definition + after_script: + - > + for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do + d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ; + mkdir -p test/$d 2>/dev/null ; + cp $f test/$d/. ; + done ; + rm -rf test/var ; mkdir -p test/var ; + cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/. .pack_template: &pack_definition <<: *pack_only_definition @@ -87,6 +113,7 @@ before_script: - deploy_test script: - ${GITLAB_MAKE} package + <<: *pack_artifacts_reject_files_definition .deploy_template: &deploy_definition <<: *deploy_only_definition @@ -103,16 +130,33 @@ before_script: - deploy_test script: - ${GITLAB_MAKE} deploy + <<: *pack_artifacts_reject_files_definition .osx_template: &osx_definition stage: test script: - ${GITLAB_MAKE} test_osx + after_script: + # Artifacts can't be used from outside the project directory, check: + # https://docs.gitlab.com/ee/ci/yaml/README.html#artifactspaths + # "Paths are relative to the project directory ($CI_PROJECT_DIR)" + # "and can’t directly link outside it." + - rm -rf test/var ; mkdir test/var && cp -rf ${OSX_VARDIR}/artifacts test/var/. + <<: *artifacts_reject_files_definition .vbox_template: &vbox_definition stage: test after_script: - - ${GITLAB_MAKE} vms_shutdown + - > + for f in `ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} ls tarantool/test/*/*.reject` ; do + d=`dirname $f | sed 's#^tarantool/##g'` ; + mkdir -p $d 2>/dev/null ; + scp -P ${VMS_PORT} ${VMS_USER}@127.0.0.1:tarantool/$d/*.reject $d/. + done ; + rm -rf test/var ; mkdir -p test/var ; + scp -r -P ${VMS_PORT} ${VMS_USER}@127.0.0.1:tarantool/test/var/artifacts test/var/. ; + ${GITLAB_MAKE} vms_shutdown + <<: *artifacts_reject_files_definition .perf_docker_test_template: &perf_docker_test_definition <<: *perf_only_definition @@ -567,7 +611,7 @@ static_build_cmake_linux: - ${GITLAB_MAKE} test_static_build_cmake_linux static_build_cmake_osx_15: - stage: test + <<: *osx_definition tags: - osx_15 script: diff --git a/.travis.mk b/.travis.mk index 58d0c1596..db457bf56 100644 --- a/.travis.mk +++ b/.travis.mk @@ -217,11 +217,11 @@ INIT_TEST_ENV_OSX=\ launchctl limit maxproc || : ; \ ulimit -u ${MAX_PROC} || : ; \ ulimit -u ; \ - rm -rf /tmp/tnt + rm -rf ${OSX_VARDIR} test_osx_no_deps: build_osx ${INIT_TEST_ENV_OSX}; \ - cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) + cd test && ./test-run.py --vardir ${OSX_VARDIR} --force $(TEST_RUN_EXTRA_PARAMS) test_osx: deps_osx test_osx_no_deps @@ -240,7 +240,7 @@ test_static_build_cmake_osx: base_deps_osx cd static-build && cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" . && \ make -j && ctest -V ${INIT_TEST_ENV_OSX}; \ - cd test && ./test-run.py --vardir /tmp/tnt \ + cd test && ./test-run.py --vardir ${OSX_VARDIR} \ --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \ --force $(TEST_RUN_EXTRA_PARAMS) -- 2.17.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov @ 2020-09-21 2:18 ` Alexander Turenko 2020-09-23 6:45 ` Alexander V. Tikhonov 0 siblings, 1 reply; 6+ messages in thread From: Alexander Turenko @ 2020-09-21 2:18 UTC (permalink / raw) To: Alexander V. Tikhonov; +Cc: tarantool-patches > + <<: *artifacts_reject_files_definition > + > +.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition > + <<: *artifacts_reject_files_definition > + after_script: > + - > > + for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do Nit: 'for f in path-with-star-globbing; do' would do almost the same and I would suggest to use the simpler form (it also does not depend on an external tool and does not involve forking), but this way we should handle 'no such files' case explicitly. See example here: https://github.com/tarantool/tarantool/commit/080beba063b05b184d7befcb15a8bcf32e7424f6 | for file in @TARANTOOL_ENABLEDDIR@/*.lua; do | instance=`basename $file .lua` | [ "${instance}" = "*" ] && break # skip empty directory | <...> | done When the glob is not matched, it falls into the loop with the 'as is' value (with star symbols). We all like bourne shell for those funny pitfalls. > + d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ; > + mkdir -p test/$d 2>/dev/null ; > + cp $f test/$d/. ; > + done ; > + rm -rf test/var ; mkdir -p test/var ; > + cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/. Nit: The period at the end looks unusual, but don't change anything. The patch would be a way simpler if test-run would collect everything that is necessary to upload within the artifacts directory. In fact, reject files are already there: test/var/artifacts/<worker_name>/<test_name>.result. But if you want to organize them in an another way, I would do this on the test-run side (somewhere within test/var/artifacts). Related code: [1]. See, the reject file is just copied, not moved. [1]: https://github.com/tarantool/test-run/blob/461731d71cf27faa6295bc759941f402fedc88e4/lib/test.py#L254 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 1/2] gitlab-ci: save failed test results files 2020-09-21 2:18 ` Alexander Turenko @ 2020-09-23 6:45 ` Alexander V. Tikhonov 0 siblings, 0 replies; 6+ messages in thread From: Alexander V. Tikhonov @ 2020-09-23 6:45 UTC (permalink / raw) To: Alexander Turenko; +Cc: tarantool-patches Hi Alexander, thanks for the review. I've removed collecting of results files *.reject, which duplicates *.result files in artifactory path. And set 'artifactory' path as root in saved artifacts packages. On Mon, Sep 21, 2020 at 05:18:13AM +0300, Alexander Turenko wrote: > > + <<: *artifacts_reject_files_definition > > + > > +.pack_artifacts_reject_files_template: &pack_artifacts_reject_files_definition > > + <<: *artifacts_reject_files_definition > > + after_script: > > + - > > > + for f in `ls build/usr/src/*/tarantool-*/test/*/*.reject` ; do > > Nit: 'for f in path-with-star-globbing; do' would do almost the same and > I would suggest to use the simpler form (it also does not depend on an > external tool and does not involve forking), but this way we should > handle 'no such files' case explicitly. See example here: > > https://github.com/tarantool/tarantool/commit/080beba063b05b184d7befcb15a8bcf32e7424f6 > > | for file in @TARANTOOL_ENABLEDDIR@/*.lua; do > | instance=`basename $file .lua` > | [ "${instance}" = "*" ] && break # skip empty directory > | <...> > | done > > When the glob is not matched, it falls into the loop with the 'as is' > value (with star symbols). > > We all like bourne shell for those funny pitfalls. > > > + d=`dirname $f | sed 's#^build/usr/src/.*/tarantool-.*/test/##g'` ; > > + mkdir -p test/$d 2>/dev/null ; > > + cp $f test/$d/. ; > > + done ; > > + rm -rf test/var ; mkdir -p test/var ; > > + cp -r build/usr/src/*/tarantool-*/test/var/artifacts test/var/. > > Nit: The period at the end looks unusual, but don't change anything. > > The patch would be a way simpler if test-run would collect everything > that is necessary to upload within the artifacts directory. > > In fact, reject files are already there: > test/var/artifacts/<worker_name>/<test_name>.result. But if you want to > organize them in an another way, I would do this on the test-run side > (somewhere within test/var/artifacts). > > Related code: [1]. See, the reject file is just copied, not moved. > > [1]: https://github.com/tarantool/test-run/blob/461731d71cf27faa6295bc759941f402fedc88e4/lib/test.py#L254 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group 2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov @ 2020-09-17 8:04 ` Alexander V. Tikhonov 2020-09-25 15:50 ` Kirill Yukhin 1 sibling, 1 reply; 6+ messages in thread From: Alexander V. Tikhonov @ 2020-09-17 8:04 UTC (permalink / raw) To: Alexander Turenko; +Cc: tarantool-patches Set opensuse jobs to test group to be sure that it will be run with artifacts collecting and without gitlab-ci jobs extra parallization. --- .gitlab-ci.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5a00641d9..27d65ff7a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -478,13 +478,13 @@ debian_10: DIST: 'buster' opensuse_15_1: - <<: *pack_definition + <<: *pack_test_definition variables: OS: 'opensuse-leap' DIST: '15.1' opensuse_15_2: - <<: *pack_definition + <<: *pack_test_definition variables: OS: 'opensuse-leap' DIST: '15.2' @@ -587,13 +587,13 @@ debian_10_deploy: DIST: 'buster' opensuse_15_1_deploy: - <<: *deploy_definition + <<: *deploy_test_definition variables: OS: 'opensuse-leap' DIST: '15.1' opensuse_15_2_deploy: - <<: *deploy_definition + <<: *deploy_test_definition variables: OS: 'opensuse-leap' DIST: '15.2' -- 2.17.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov @ 2020-09-25 15:50 ` Kirill Yukhin 0 siblings, 0 replies; 6+ messages in thread From: Kirill Yukhin @ 2020-09-25 15:50 UTC (permalink / raw) To: Alexander V. Tikhonov; +Cc: tarantool-patches, Alexander Turenko Hello, On 17 сен 11:04, Alexander V. Tikhonov wrote: > Set opensuse jobs to test group to be sure that it will be run with > artifacts collecting and without gitlab-ci jobs extra parallization. > --- > .gitlab-ci.yml | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) I've checked your patch into 1.10, 2.4, 2.5, master. -- Regards, Kirill Yukhin ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-25 15:50 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-17 8:04 [Tarantool-patches] [PATCH v2 0/2] gitlab-ci: save failed test results files Alexander V. Tikhonov 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 1/2] " Alexander V. Tikhonov 2020-09-21 2:18 ` Alexander Turenko 2020-09-23 6:45 ` Alexander V. Tikhonov 2020-09-17 8:04 ` [Tarantool-patches] [PATCH v2 2/2] gitlab-ci: set opensuse jobs to test group Alexander V. Tikhonov 2020-09-25 15:50 ` Kirill Yukhin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox