From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id C630828893 for ; Mon, 6 May 2019 04:39:21 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RRCkOgNAft_L for ; Mon, 6 May 2019 04:39:21 -0400 (EDT) Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 2F16B2C2A0 for ; Mon, 6 May 2019 04:39:18 -0400 (EDT) Date: Mon, 6 May 2019 11:39:02 +0300 From: Alexander Turenko Subject: [tarantool-patches] Re: [PATCH v2] travis-ci: set jobs not to stop on failed tests Message-ID: <20190506083901.rnabgyqk3xcbylpi@tkn_work_nb> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: avtikhon Cc: tarantool-patches@freelists.org Pushed to master and 2.1. WBR, Alexander Turenko. On Mon, May 06, 2019 at 09:47:14AM +0300, avtikhon wrote: > From: "Alexander V. Tikhonov" > > Added --force flag to travis-ci jobs not to stop on failed tests. > Due to any found failed test breaks the testing it masks the other > fails and in the following ways it's not good: > - flaky test masks real problem > - release testing needs overall result to fix it fast > - parallel testing may produce flaky test > > Close: #4131 > --- > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-3863-test-in-parallel-force > Issue: https://github.com/tarantool/tarantool/issues/4131 > > .travis.mk | 6 +++--- > rpm/tarantool.spec | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/.travis.mk b/.travis.mk > index 973c6a7c7..4ac3fc11a 100644 > --- a/.travis.mk > +++ b/.travis.mk > @@ -42,7 +42,7 @@ deps_ubuntu: > test_ubuntu: deps_ubuntu > cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} > make -j8 > - cd test && /usr/bin/python test-run.py -j 1 > + cd test && /usr/bin/python test-run.py --force -j 1 > > deps_osx: > brew update > @@ -59,13 +59,13 @@ test_osx: deps_osx > ulimit -S -n 20480 || : > ulimit -n > make -j8 > - cd test && python test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ > + cd test && python test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ > > coverage_ubuntu: deps_ubuntu > cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON > make -j8 > # Enable --long tests for coverage > - cd test && /usr/bin/python test-run.py -j 1 --long > + cd test && /usr/bin/python test-run.py --force -j 1 --long > lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp > lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \ > --output-file coverage.info > diff --git a/rpm/tarantool.spec b/rpm/tarantool.spec > index c87b1667d..8bee71ed0 100644 > --- a/rpm/tarantool.spec > +++ b/rpm/tarantool.spec > @@ -150,7 +150,7 @@ echo "self.skip = True" > ./test/app/socket.skipcond > # https://github.com/tarantool/tarantool/issues/1322 > echo "self.skip = True" > ./test/app/digest.skipcond > # run a safe subset of the test suite > -cd test && ./test-run.py -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/ > +cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/ > %endif > > %pre > -- > 2.17.1 >