Tarantool development patches archive
 help / color / mirror / Atom feed
From: avtikhon <avtikhon@tarantool.org>
To: Alexander Turenko <alexander.turenko@tarantool.org>
Cc: "Alexander V. Tikhonov" <avtikhon@tarantool.org>,
	tarantool-patches@freelists.org
Subject: [tarantool-patches] [PATCH v1] travis-ci: set jobs not to stop on failed tests
Date: Wed, 10 Apr 2019 16:53:09 +0300	[thread overview]
Message-ID: <6cbbaf49cd3d18481762316e4ef4979232df65cc.1554904361.git.avtikhon@tarantool.org> (raw)

From: "Alexander V. Tikhonov" <avtikhon@tarantool.org>

Added --force flag to travis-ci jobs not to stop on failed tests.
Due to any found failed test breaks the testing it masks the other
fails and in the following ways it's not good:
- flaky test masks real problem
- release testing needs overall result to fix it fast
- parallel testing may produce flaky test

Close: #4131
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3863-test-in-parallel-force
Issue: https://github.com/tarantool/tarantool/issues/4131

 .travis.mk | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.travis.mk b/.travis.mk
index 247bb2a2d..a6f43db61 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -42,7 +42,7 @@ deps_ubuntu:
 test_ubuntu: deps_ubuntu
 	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfoWError ${CMAKE_EXTRA_PARAMS}
 	make -j8
-	cd test && /usr/bin/python test-run.py -j 1
+	cd test && /usr/bin/python test-run.py --force -j 1
 
 deps_osx:
 	brew update
@@ -59,13 +59,13 @@ test_osx: deps_osx
 	ulimit -S -n 20480 || :
 	ulimit -n
 	make -j8
-	cd test && python test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/
+	cd test && python test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/
 
 coverage_ubuntu: deps_ubuntu
 	cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON
 	make -j8
 	# Enable --long tests for coverage
-	cd test && /usr/bin/python test-run.py -j 1 --long
+	cd test && /usr/bin/python test-run.py --force -j 1 --long
 	lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp
 	lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \
 		--output-file coverage.info
-- 
2.17.1

                 reply	other threads:[~2019-04-10 13:53 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6cbbaf49cd3d18481762316e4ef4979232df65cc.1554904361.git.avtikhon@tarantool.org \
    --to=avtikhon@tarantool.org \
    --cc=alexander.turenko@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [tarantool-patches] [PATCH v1] travis-ci: set jobs not to stop on failed tests' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox