* [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists @ 2020-07-09 9:46 Alexander V. Tikhonov 2020-07-10 10:01 ` Kirill Yukhin 2020-07-13 12:24 ` Sergey Bronnikov 0 siblings, 2 replies; 4+ messages in thread From: Alexander V. Tikhonov @ 2020-07-09 9:46 UTC (permalink / raw) To: Kirill Yukhin, Sergey Bronnikov; +Cc: tarantool-patches Syncronized suites 'fragile' lists with actual list of flaky tests. --- Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add test/app-tap/suite.ini | 1 + test/box-tap/suite.ini | 3 ++- test/box/suite.ini | 5 ++++- test/engine/suite.ini | 3 +-- test/replication-py/suite.ini | 1 + test/replication/suite.ini | 10 ++++++---- test/sql-tap/suite.ini | 1 - test/sql/suite.ini | 3 +-- test/vinyl/suite.ini | 3 +++ 9 files changed, 19 insertions(+), 11 deletions(-) diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini index daababef5..f06f89ed5 100644 --- a/test/app-tap/suite.ini +++ b/test/app-tap/suite.ini @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua is_parallel = True pretest_clean = True use_unix_sockets_iproto = True +fragile = tarantoolctl.test.lua ; gh-5059 diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini index 07f8880b6..0f4f4a566 100644 --- a/test/box-tap/suite.ini +++ b/test/box-tap/suite.ini @@ -4,5 +4,6 @@ description = Database tests with #! using TAP is_parallel = True pretest_clean = True use_unix_sockets_iproto = True -fragile = cfg.test.lua ; gh-4344 config = suite.cfg +fragile = cfg.test.lua ; gh-4344 + key_def.test.lua ; gh-5049 diff --git a/test/box/suite.ini b/test/box/suite.ini index de8f5a70e..7f2f5d898 100644 --- a/test/box/suite.ini +++ b/test/box/suite.ini @@ -13,7 +13,8 @@ pretest_clean = True fragile = bitset.test.lua ; gh-4095 func_reload.test.lua ; gh-4425 function1.test.lua ; gh-4199 - net.box.test.lua ; gh-3851 gh-4383 + push.test.lua ; gh-4882 + tree_pk.test.lua ; gh-4882 alter_limits.test.lua ; gh-4926 misc.test.lua ; gh-4982 tuple.test.lua ; gh-4988 @@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095 sequence.test.lua ; gh-4996 on_replace.test.lua ; gh-4997 role.test.lua ; gh-4998 + net.box_reconnect_after_gh-3164.test.lua ; gh-5081 + net.box_disconnect_gh-3859.test.lua ; gh-5056 diff --git a/test/engine/suite.ini b/test/engine/suite.ini index 3ab3cf108..d170c9f3f 100644 --- a/test/engine/suite.ini +++ b/test/engine/suite.ini @@ -10,5 +10,4 @@ config = engine.cfg lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua is_parallel = True pretest_clean = True -fragile = recover_wal.test.lua ; gh-3767 - ddl.test.lua ; gh-4353 +fragile = ddl.test.lua ; gh-4353 diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini index b4caf443b..45add028c 100644 --- a/test/replication-py/suite.ini +++ b/test/replication-py/suite.ini @@ -5,3 +5,4 @@ description = tarantool/box, replication is_parallel = True fragile = init_storage.test.py ; gh-4949 conflict.test.py ; gh-4980 + cluster.test.py ; gh-5109 diff --git a/test/replication/suite.ini b/test/replication/suite.ini index 6119a264b..8d6c9799d 100644 --- a/test/replication/suite.ini +++ b/test/replication/suite.ini @@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870 long_row_timeout.test.lua ; gh-4351 misc.test.lua ; gh-4940 skip_conflict_row.test.lua ; gh-4958 - sync.test.lua ; gh-3835 gh-3877 + sync.test.lua ; gh-3835 transaction.test.lua ; gh-4312 - wal_rw_stress.test.lua ; gh-4977 - wal_off.test.lua ; gh-4355 + autobootstrap.test.lua ; gh-4533 replica_rejoin.test.lua ; gh-4985 recover_missing_xlog.test.lua ; gh-4989 - box_set_replication_stress ; gh-4992 + box_set_replication_stress.test.lua ; gh-4992 gh-4986 + gh-4605-empty-password.test.lua ; gh-5030 + anon.test.lua ; gh-5058 + status.test.lua ; gh-5110 diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini index 3c0e309ee..bb6772823 100644 --- a/test/sql-tap/suite.ini +++ b/test/sql-tap/suite.ini @@ -30,4 +30,3 @@ config = engine.cfg show_reproduce_content = False pretest_clean = True use_unix_sockets_iproto = True -fragile = selectG.test.lua ; gh-4458 diff --git a/test/sql/suite.ini b/test/sql/suite.ini index 57e78f1c8..57733deec 100644 --- a/test/sql/suite.ini +++ b/test/sql/suite.ini @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua release_disabled = errinj.test.lua view_delayed_wal.test.lua disabled = sql-statN-index-drop.test.lua pretest_clean = True -fragile = dll.test.lua ; gh-4427 - func-recreate.test.lua ; gh-4384 +fragile = func-recreate.test.lua ; gh-4384 diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini index 793901275..3e11b087b 100644 --- a/test/vinyl/suite.ini +++ b/test/vinyl/suite.ini @@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346 snapshot.test.lua ; gh-4984 write_iterator.test.lua ; gh-4572 errinj_ddl.test.lua ; gh-4993 + gh-4810-dump-during-index-build.test.lua ; gh-5031 + deferred_delete.test.lua ; gh-5089 + gh.test.lua ; gh-5141 -- 2.17.1 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists 2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov @ 2020-07-10 10:01 ` Kirill Yukhin 2020-07-13 12:24 ` Sergey Bronnikov 1 sibling, 0 replies; 4+ messages in thread From: Kirill Yukhin @ 2020-07-10 10:01 UTC (permalink / raw) To: Alexander V. Tikhonov; +Cc: tarantool-patches Hello, On 09 июл 12:46, Alexander V. Tikhonov wrote: > Syncronized suites 'fragile' lists with actual list of flaky tests. > --- > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add I've checked your patch into 1.10, 2.3, 2.4 and master. -- Regards, Kirill Yukhin ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists 2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov 2020-07-10 10:01 ` Kirill Yukhin @ 2020-07-13 12:24 ` Sergey Bronnikov 2020-07-13 13:50 ` Alexander V. Tikhonov 1 sibling, 1 reply; 4+ messages in thread From: Sergey Bronnikov @ 2020-07-13 12:24 UTC (permalink / raw) To: Alexander V. Tikhonov; +Cc: tarantool-patches Thanks for the patch! See 3 comments. On 12:46 Thu 09 Jul , Alexander V. Tikhonov wrote: > Syncronized suites 'fragile' lists with actual list of flaky tests. > --- > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add > > test/app-tap/suite.ini | 1 + > test/box-tap/suite.ini | 3 ++- > test/box/suite.ini | 5 ++++- > test/engine/suite.ini | 3 +-- > test/replication-py/suite.ini | 1 + > test/replication/suite.ini | 10 ++++++---- > test/sql-tap/suite.ini | 1 - > test/sql/suite.ini | 3 +-- > test/vinyl/suite.ini | 3 +++ > 9 files changed, 19 insertions(+), 11 deletions(-) > > diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini > index daababef5..f06f89ed5 100644 > --- a/test/app-tap/suite.ini > +++ b/test/app-tap/suite.ini > @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua > is_parallel = True > pretest_clean = True > use_unix_sockets_iproto = True > +fragile = tarantoolctl.test.lua ; gh-5059 > diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini > index 07f8880b6..0f4f4a566 100644 > --- a/test/box-tap/suite.ini > +++ b/test/box-tap/suite.ini > @@ -4,5 +4,6 @@ description = Database tests with #! using TAP > is_parallel = True > pretest_clean = True > use_unix_sockets_iproto = True > -fragile = cfg.test.lua ; gh-4344 > config = suite.cfg > +fragile = cfg.test.lua ; gh-4344 > + key_def.test.lua ; gh-5049 gh-5049 is a duplicate of gh-4252, that is closed. > diff --git a/test/box/suite.ini b/test/box/suite.ini > index de8f5a70e..7f2f5d898 100644 > --- a/test/box/suite.ini > +++ b/test/box/suite.ini > @@ -13,7 +13,8 @@ pretest_clean = True > fragile = bitset.test.lua ; gh-4095 > func_reload.test.lua ; gh-4425 > function1.test.lua ; gh-4199 > - net.box.test.lua ; gh-3851 gh-4383 > + push.test.lua ; gh-4882 > + tree_pk.test.lua ; gh-4882 > alter_limits.test.lua ; gh-4926 > misc.test.lua ; gh-4982 > tuple.test.lua ; gh-4988 > @@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095 > sequence.test.lua ; gh-4996 > on_replace.test.lua ; gh-4997 > role.test.lua ; gh-4998 > + net.box_reconnect_after_gh-3164.test.lua ; gh-5081 > + net.box_disconnect_gh-3859.test.lua ; gh-5056 gh-5056 is not about flaky test. " gitlab-ci: enable script for VBOX VMs orchestrate #5056" > diff --git a/test/engine/suite.ini b/test/engine/suite.ini > index 3ab3cf108..d170c9f3f 100644 > --- a/test/engine/suite.ini > +++ b/test/engine/suite.ini > @@ -10,5 +10,4 @@ config = engine.cfg > lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua > is_parallel = True > pretest_clean = True > -fragile = recover_wal.test.lua ; gh-3767 > - ddl.test.lua ; gh-4353 > +fragile = ddl.test.lua ; gh-4353 > diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini > index b4caf443b..45add028c 100644 > --- a/test/replication-py/suite.ini > +++ b/test/replication-py/suite.ini > @@ -5,3 +5,4 @@ description = tarantool/box, replication > is_parallel = True > fragile = init_storage.test.py ; gh-4949 > conflict.test.py ; gh-4980 > + cluster.test.py ; gh-5109 > diff --git a/test/replication/suite.ini b/test/replication/suite.ini > index 6119a264b..8d6c9799d 100644 > --- a/test/replication/suite.ini > +++ b/test/replication/suite.ini > @@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870 > long_row_timeout.test.lua ; gh-4351 > misc.test.lua ; gh-4940 > skip_conflict_row.test.lua ; gh-4958 > - sync.test.lua ; gh-3835 gh-3877 > + sync.test.lua ; gh-3835 > transaction.test.lua ; gh-4312 > - wal_rw_stress.test.lua ; gh-4977 > - wal_off.test.lua ; gh-4355 > + autobootstrap.test.lua ; gh-4533 > replica_rejoin.test.lua ; gh-4985 > recover_missing_xlog.test.lua ; gh-4989 > - box_set_replication_stress ; gh-4992 gh-4992 is open, why box_set_replication_stress is not fragile anymore? > + box_set_replication_stress.test.lua ; gh-4992 gh-4986 > + gh-4605-empty-password.test.lua ; gh-5030 > + anon.test.lua ; gh-5058 > + status.test.lua ; gh-5110 > diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini > index 3c0e309ee..bb6772823 100644 > --- a/test/sql-tap/suite.ini > +++ b/test/sql-tap/suite.ini > @@ -30,4 +30,3 @@ config = engine.cfg > show_reproduce_content = False > pretest_clean = True > use_unix_sockets_iproto = True > -fragile = selectG.test.lua ; gh-4458 > diff --git a/test/sql/suite.ini b/test/sql/suite.ini > index 57e78f1c8..57733deec 100644 > --- a/test/sql/suite.ini > +++ b/test/sql/suite.ini > @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua > release_disabled = errinj.test.lua view_delayed_wal.test.lua > disabled = sql-statN-index-drop.test.lua > pretest_clean = True > -fragile = dll.test.lua ; gh-4427 > - func-recreate.test.lua ; gh-4384 > +fragile = func-recreate.test.lua ; gh-4384 > diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini > index 793901275..3e11b087b 100644 > --- a/test/vinyl/suite.ini > +++ b/test/vinyl/suite.ini > @@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346 > snapshot.test.lua ; gh-4984 > write_iterator.test.lua ; gh-4572 > errinj_ddl.test.lua ; gh-4993 > + gh-4810-dump-during-index-build.test.lua ; gh-5031 > + deferred_delete.test.lua ; gh-5089 > + gh.test.lua ; gh-5141 > -- > 2.17.1 > -- sergeyb@ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists 2020-07-13 12:24 ` Sergey Bronnikov @ 2020-07-13 13:50 ` Alexander V. Tikhonov 0 siblings, 0 replies; 4+ messages in thread From: Alexander V. Tikhonov @ 2020-07-13 13:50 UTC (permalink / raw) To: Sergey Bronnikov; +Cc: tarantool-patches Hi Sergey, thanks for the deep review, please check my comments below. On Mon, Jul 13, 2020 at 03:24:23PM +0300, Sergey Bronnikov wrote: > Thanks for the patch! > See 3 comments. > > On 12:46 Thu 09 Jul , Alexander V. Tikhonov wrote: > > Syncronized suites 'fragile' lists with actual list of flaky tests. > > --- > > > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add > > > > test/app-tap/suite.ini | 1 + > > test/box-tap/suite.ini | 3 ++- > > test/box/suite.ini | 5 ++++- > > test/engine/suite.ini | 3 +-- > > test/replication-py/suite.ini | 1 + > > test/replication/suite.ini | 10 ++++++---- > > test/sql-tap/suite.ini | 1 - > > test/sql/suite.ini | 3 +-- > > test/vinyl/suite.ini | 3 +++ > > 9 files changed, 19 insertions(+), 11 deletions(-) > > > > diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini > > index daababef5..f06f89ed5 100644 > > --- a/test/app-tap/suite.ini > > +++ b/test/app-tap/suite.ini > > @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua > > is_parallel = True > > pretest_clean = True > > use_unix_sockets_iproto = True > > +fragile = tarantoolctl.test.lua ; gh-5059 > > diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini > > index 07f8880b6..0f4f4a566 100644 > > --- a/test/box-tap/suite.ini > > +++ b/test/box-tap/suite.ini > > @@ -4,5 +4,6 @@ description = Database tests with #! using TAP > > is_parallel = True > > pretest_clean = True > > use_unix_sockets_iproto = True > > -fragile = cfg.test.lua ; gh-4344 > > config = suite.cfg > > +fragile = cfg.test.lua ; gh-4344 > > + key_def.test.lua ; gh-5049 > > gh-5049 is a duplicate of gh-4252, that is closed. > Right, this test should be removed, due to fixed. > > diff --git a/test/box/suite.ini b/test/box/suite.ini > > index de8f5a70e..7f2f5d898 100644 > > --- a/test/box/suite.ini > > +++ b/test/box/suite.ini > > @@ -13,7 +13,8 @@ pretest_clean = True > > fragile = bitset.test.lua ; gh-4095 > > func_reload.test.lua ; gh-4425 > > function1.test.lua ; gh-4199 > > - net.box.test.lua ; gh-3851 gh-4383 > > + push.test.lua ; gh-4882 > > + tree_pk.test.lua ; gh-4882 > > alter_limits.test.lua ; gh-4926 > > misc.test.lua ; gh-4982 > > tuple.test.lua ; gh-4988 > > @@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095 > > sequence.test.lua ; gh-4996 > > on_replace.test.lua ; gh-4997 > > role.test.lua ; gh-4998 > > + net.box_reconnect_after_gh-3164.test.lua ; gh-5081 > > + net.box_disconnect_gh-3859.test.lua ; gh-5056 > > gh-5056 is not about flaky test. > " gitlab-ci: enable script for VBOX VMs orchestrate #5056" > Right, the issue should be 5156 instead of 5056, corrected. > > diff --git a/test/engine/suite.ini b/test/engine/suite.ini > > index 3ab3cf108..d170c9f3f 100644 > > --- a/test/engine/suite.ini > > +++ b/test/engine/suite.ini > > @@ -10,5 +10,4 @@ config = engine.cfg > > lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua > > is_parallel = True > > pretest_clean = True > > -fragile = recover_wal.test.lua ; gh-3767 > > - ddl.test.lua ; gh-4353 > > +fragile = ddl.test.lua ; gh-4353 > > diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini > > index b4caf443b..45add028c 100644 > > --- a/test/replication-py/suite.ini > > +++ b/test/replication-py/suite.ini > > @@ -5,3 +5,4 @@ description = tarantool/box, replication > > is_parallel = True > > fragile = init_storage.test.py ; gh-4949 > > conflict.test.py ; gh-4980 > > + cluster.test.py ; gh-5109 > > diff --git a/test/replication/suite.ini b/test/replication/suite.ini > > index 6119a264b..8d6c9799d 100644 > > --- a/test/replication/suite.ini > > +++ b/test/replication/suite.ini > > @@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870 > > long_row_timeout.test.lua ; gh-4351 > > misc.test.lua ; gh-4940 > > skip_conflict_row.test.lua ; gh-4958 > > - sync.test.lua ; gh-3835 gh-3877 > > + sync.test.lua ; gh-3835 > > transaction.test.lua ; gh-4312 > > - wal_rw_stress.test.lua ; gh-4977 > > - wal_off.test.lua ; gh-4355 > > + autobootstrap.test.lua ; gh-4533 > > replica_rejoin.test.lua ; gh-4985 > > recover_missing_xlog.test.lua ; gh-4989 > > - box_set_replication_stress ; gh-4992 > > gh-4992 is open, why box_set_replication_stress is not fragile anymore? > No, just added additional issue in comment, please check the line below, also the mistake of the test name was corrected. > > + box_set_replication_stress.test.lua ; gh-4992 gh-4986 > > + gh-4605-empty-password.test.lua ; gh-5030 > > + anon.test.lua ; gh-5058 > > + status.test.lua ; gh-5110 > > diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini > > index 3c0e309ee..bb6772823 100644 > > --- a/test/sql-tap/suite.ini > > +++ b/test/sql-tap/suite.ini > > @@ -30,4 +30,3 @@ config = engine.cfg > > show_reproduce_content = False > > pretest_clean = True > > use_unix_sockets_iproto = True > > -fragile = selectG.test.lua ; gh-4458 > > diff --git a/test/sql/suite.ini b/test/sql/suite.ini > > index 57e78f1c8..57733deec 100644 > > --- a/test/sql/suite.ini > > +++ b/test/sql/suite.ini > > @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua > > release_disabled = errinj.test.lua view_delayed_wal.test.lua > > disabled = sql-statN-index-drop.test.lua > > pretest_clean = True > > -fragile = dll.test.lua ; gh-4427 > > - func-recreate.test.lua ; gh-4384 > > +fragile = func-recreate.test.lua ; gh-4384 > > diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini > > index 793901275..3e11b087b 100644 > > --- a/test/vinyl/suite.ini > > +++ b/test/vinyl/suite.ini > > @@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346 > > snapshot.test.lua ; gh-4984 > > write_iterator.test.lua ; gh-4572 > > errinj_ddl.test.lua ; gh-4993 > > + gh-4810-dump-during-index-build.test.lua ; gh-5031 > > + deferred_delete.test.lua ; gh-5089 > > + gh.test.lua ; gh-5141 > > -- > > 2.17.1 > > > > -- > sergeyb@ ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-07-13 13:50 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov 2020-07-10 10:01 ` Kirill Yukhin 2020-07-13 12:24 ` Sergey Bronnikov 2020-07-13 13:50 ` Alexander V. Tikhonov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox