* [tarantool-patches] [PATCH v2] travis-ci: set jobs not to stop on failed tests
@ 2019-05-06 6:47 avtikhon
2019-05-06 8:39 ` [tarantool-patches] " Alexander Turenko
0 siblings, 1 reply; 2+ messages in thread
From: avtikhon @ 2019-05-06 6:47 UTC (permalink / raw)
To: Alexander Turenko; +Cc: Alexander V. Tikhonov, tarantool-patches
From: "Alexander V. Tikhonov" <avtikhon@tarantool.org>
Added --force flag to travis-ci jobs not to stop on failed tests.
Due to any found failed test breaks the testing it masks the other
fails and in the following ways it's not good:
- flaky test masks real problem
- release testing needs overall result to fix it fast
- parallel testing may produce flaky test
Close: #4131
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3863-test-in-parallel-force
Issue: https://github.com/tarantool/tarantool/issues/4131
.travis.mk | 6 +++---
rpm/tarantool.spec | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/.travis.mk b/.travis.mk
index 973c6a7c7..4ac3fc11a 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -42,7 +42,7 @@ deps_ubuntu:
test_ubuntu: deps_ubuntu
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
make -j8
- cd test && /usr/bin/python test-run.py -j 1
+ cd test && /usr/bin/python test-run.py --force -j 1
deps_osx:
brew update
@@ -59,13 +59,13 @@ test_osx: deps_osx
ulimit -S -n 20480 || :
ulimit -n
make -j8
- cd test && python test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/
+ cd test && python test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/
coverage_ubuntu: deps_ubuntu
cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON
make -j8
# Enable --long tests for coverage
- cd test && /usr/bin/python test-run.py -j 1 --long
+ cd test && /usr/bin/python test-run.py --force -j 1 --long
lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp
lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \
--output-file coverage.info
diff --git a/rpm/tarantool.spec b/rpm/tarantool.spec
index c87b1667d..8bee71ed0 100644
--- a/rpm/tarantool.spec
+++ b/rpm/tarantool.spec
@@ -150,7 +150,7 @@ echo "self.skip = True" > ./test/app/socket.skipcond
# https://github.com/tarantool/tarantool/issues/1322
echo "self.skip = True" > ./test/app/digest.skipcond
# run a safe subset of the test suite
-cd test && ./test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/
+cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/
%endif
%pre
--
2.17.1
^ permalink raw reply [flat|nested] 2+ messages in thread
* [tarantool-patches] Re: [PATCH v2] travis-ci: set jobs not to stop on failed tests
2019-05-06 6:47 [tarantool-patches] [PATCH v2] travis-ci: set jobs not to stop on failed tests avtikhon
@ 2019-05-06 8:39 ` Alexander Turenko
0 siblings, 0 replies; 2+ messages in thread
From: Alexander Turenko @ 2019-05-06 8:39 UTC (permalink / raw)
To: avtikhon; +Cc: tarantool-patches
Pushed to master and 2.1.
WBR, Alexander Turenko.
On Mon, May 06, 2019 at 09:47:14AM +0300, avtikhon wrote:
> From: "Alexander V. Tikhonov" <avtikhon@tarantool.org>
>
> Added --force flag to travis-ci jobs not to stop on failed tests.
> Due to any found failed test breaks the testing it masks the other
> fails and in the following ways it's not good:
> - flaky test masks real problem
> - release testing needs overall result to fix it fast
> - parallel testing may produce flaky test
>
> Close: #4131
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3863-test-in-parallel-force
> Issue: https://github.com/tarantool/tarantool/issues/4131
>
> .travis.mk | 6 +++---
> rpm/tarantool.spec | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/.travis.mk b/.travis.mk
> index 973c6a7c7..4ac3fc11a 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -42,7 +42,7 @@ deps_ubuntu:
> test_ubuntu: deps_ubuntu
> cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
> make -j8
> - cd test && /usr/bin/python test-run.py -j 1
> + cd test && /usr/bin/python test-run.py --force -j 1
>
> deps_osx:
> brew update
> @@ -59,13 +59,13 @@ test_osx: deps_osx
> ulimit -S -n 20480 || :
> ulimit -n
> make -j8
> - cd test && python test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/
> + cd test && python test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/
>
> coverage_ubuntu: deps_ubuntu
> cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON
> make -j8
> # Enable --long tests for coverage
> - cd test && /usr/bin/python test-run.py -j 1 --long
> + cd test && /usr/bin/python test-run.py --force -j 1 --long
> lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp
> lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \
> --output-file coverage.info
> diff --git a/rpm/tarantool.spec b/rpm/tarantool.spec
> index c87b1667d..8bee71ed0 100644
> --- a/rpm/tarantool.spec
> +++ b/rpm/tarantool.spec
> @@ -150,7 +150,7 @@ echo "self.skip = True" > ./test/app/socket.skipcond
> # https://github.com/tarantool/tarantool/issues/1322
> echo "self.skip = True" > ./test/app/digest.skipcond
> # run a safe subset of the test suite
> -cd test && ./test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/
> +cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/
> %endif
>
> %pre
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2019-05-06 8:39 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-06 6:47 [tarantool-patches] [PATCH v2] travis-ci: set jobs not to stop on failed tests avtikhon
2019-05-06 8:39 ` [tarantool-patches] " Alexander Turenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox