From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id ECC9A2B66A for ; Mon, 6 May 2019 02:47:24 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7pTA7yYpCQoL for ; Mon, 6 May 2019 02:47:24 -0400 (EDT) Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 4E22A2B5B5 for ; Mon, 6 May 2019 02:47:24 -0400 (EDT) From: avtikhon Subject: [tarantool-patches] [PATCH v2] travis-ci: set jobs not to stop on failed tests Date: Mon, 6 May 2019 09:47:14 +0300 Message-Id: Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: Alexander Turenko Cc: "Alexander V. Tikhonov" , tarantool-patches@freelists.org From: "Alexander V. Tikhonov" Added --force flag to travis-ci jobs not to stop on failed tests. Due to any found failed test breaks the testing it masks the other fails and in the following ways it's not good: - flaky test masks real problem - release testing needs overall result to fix it fast - parallel testing may produce flaky test Close: #4131 --- Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3863-test-in-parallel-force Issue: https://github.com/tarantool/tarantool/issues/4131 .travis.mk | 6 +++--- rpm/tarantool.spec | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.travis.mk b/.travis.mk index 973c6a7c7..4ac3fc11a 100644 --- a/.travis.mk +++ b/.travis.mk @@ -42,7 +42,7 @@ deps_ubuntu: test_ubuntu: deps_ubuntu cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} make -j8 - cd test && /usr/bin/python test-run.py -j 1 + cd test && /usr/bin/python test-run.py --force -j 1 deps_osx: brew update @@ -59,13 +59,13 @@ test_osx: deps_osx ulimit -S -n 20480 || : ulimit -n make -j8 - cd test && python test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ + cd test && python test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ coverage_ubuntu: deps_ubuntu cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON make -j8 # Enable --long tests for coverage - cd test && /usr/bin/python test-run.py -j 1 --long + cd test && /usr/bin/python test-run.py --force -j 1 --long lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \ --output-file coverage.info diff --git a/rpm/tarantool.spec b/rpm/tarantool.spec index c87b1667d..8bee71ed0 100644 --- a/rpm/tarantool.spec +++ b/rpm/tarantool.spec @@ -150,7 +150,7 @@ echo "self.skip = True" > ./test/app/socket.skipcond # https://github.com/tarantool/tarantool/issues/1322 echo "self.skip = True" > ./test/app/digest.skipcond # run a safe subset of the test suite -cd test && ./test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/ +cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/ %endif %pre -- 2.17.1