* [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists
@ 2020-07-09 9:46 Alexander V. Tikhonov
2020-07-10 10:01 ` Kirill Yukhin
2020-07-13 12:24 ` Sergey Bronnikov
0 siblings, 2 replies; 4+ messages in thread
From: Alexander V. Tikhonov @ 2020-07-09 9:46 UTC (permalink / raw)
To: Kirill Yukhin, Sergey Bronnikov; +Cc: tarantool-patches
Syncronized suites 'fragile' lists with actual list of flaky tests.
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add
test/app-tap/suite.ini | 1 +
test/box-tap/suite.ini | 3 ++-
test/box/suite.ini | 5 ++++-
test/engine/suite.ini | 3 +--
test/replication-py/suite.ini | 1 +
test/replication/suite.ini | 10 ++++++----
test/sql-tap/suite.ini | 1 -
test/sql/suite.ini | 3 +--
test/vinyl/suite.ini | 3 +++
9 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini
index daababef5..f06f89ed5 100644
--- a/test/app-tap/suite.ini
+++ b/test/app-tap/suite.ini
@@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua
is_parallel = True
pretest_clean = True
use_unix_sockets_iproto = True
+fragile = tarantoolctl.test.lua ; gh-5059
diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini
index 07f8880b6..0f4f4a566 100644
--- a/test/box-tap/suite.ini
+++ b/test/box-tap/suite.ini
@@ -4,5 +4,6 @@ description = Database tests with #! using TAP
is_parallel = True
pretest_clean = True
use_unix_sockets_iproto = True
-fragile = cfg.test.lua ; gh-4344
config = suite.cfg
+fragile = cfg.test.lua ; gh-4344
+ key_def.test.lua ; gh-5049
diff --git a/test/box/suite.ini b/test/box/suite.ini
index de8f5a70e..7f2f5d898 100644
--- a/test/box/suite.ini
+++ b/test/box/suite.ini
@@ -13,7 +13,8 @@ pretest_clean = True
fragile = bitset.test.lua ; gh-4095
func_reload.test.lua ; gh-4425
function1.test.lua ; gh-4199
- net.box.test.lua ; gh-3851 gh-4383
+ push.test.lua ; gh-4882
+ tree_pk.test.lua ; gh-4882
alter_limits.test.lua ; gh-4926
misc.test.lua ; gh-4982
tuple.test.lua ; gh-4988
@@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095
sequence.test.lua ; gh-4996
on_replace.test.lua ; gh-4997
role.test.lua ; gh-4998
+ net.box_reconnect_after_gh-3164.test.lua ; gh-5081
+ net.box_disconnect_gh-3859.test.lua ; gh-5056
diff --git a/test/engine/suite.ini b/test/engine/suite.ini
index 3ab3cf108..d170c9f3f 100644
--- a/test/engine/suite.ini
+++ b/test/engine/suite.ini
@@ -10,5 +10,4 @@ config = engine.cfg
lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua
is_parallel = True
pretest_clean = True
-fragile = recover_wal.test.lua ; gh-3767
- ddl.test.lua ; gh-4353
+fragile = ddl.test.lua ; gh-4353
diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini
index b4caf443b..45add028c 100644
--- a/test/replication-py/suite.ini
+++ b/test/replication-py/suite.ini
@@ -5,3 +5,4 @@ description = tarantool/box, replication
is_parallel = True
fragile = init_storage.test.py ; gh-4949
conflict.test.py ; gh-4980
+ cluster.test.py ; gh-5109
diff --git a/test/replication/suite.ini b/test/replication/suite.ini
index 6119a264b..8d6c9799d 100644
--- a/test/replication/suite.ini
+++ b/test/replication/suite.ini
@@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870
long_row_timeout.test.lua ; gh-4351
misc.test.lua ; gh-4940
skip_conflict_row.test.lua ; gh-4958
- sync.test.lua ; gh-3835 gh-3877
+ sync.test.lua ; gh-3835
transaction.test.lua ; gh-4312
- wal_rw_stress.test.lua ; gh-4977
- wal_off.test.lua ; gh-4355
+ autobootstrap.test.lua ; gh-4533
replica_rejoin.test.lua ; gh-4985
recover_missing_xlog.test.lua ; gh-4989
- box_set_replication_stress ; gh-4992
+ box_set_replication_stress.test.lua ; gh-4992 gh-4986
+ gh-4605-empty-password.test.lua ; gh-5030
+ anon.test.lua ; gh-5058
+ status.test.lua ; gh-5110
diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini
index 3c0e309ee..bb6772823 100644
--- a/test/sql-tap/suite.ini
+++ b/test/sql-tap/suite.ini
@@ -30,4 +30,3 @@ config = engine.cfg
show_reproduce_content = False
pretest_clean = True
use_unix_sockets_iproto = True
-fragile = selectG.test.lua ; gh-4458
diff --git a/test/sql/suite.ini b/test/sql/suite.ini
index 57e78f1c8..57733deec 100644
--- a/test/sql/suite.ini
+++ b/test/sql/suite.ini
@@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua
release_disabled = errinj.test.lua view_delayed_wal.test.lua
disabled = sql-statN-index-drop.test.lua
pretest_clean = True
-fragile = dll.test.lua ; gh-4427
- func-recreate.test.lua ; gh-4384
+fragile = func-recreate.test.lua ; gh-4384
diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini
index 793901275..3e11b087b 100644
--- a/test/vinyl/suite.ini
+++ b/test/vinyl/suite.ini
@@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346
snapshot.test.lua ; gh-4984
write_iterator.test.lua ; gh-4572
errinj_ddl.test.lua ; gh-4993
+ gh-4810-dump-during-index-build.test.lua ; gh-5031
+ deferred_delete.test.lua ; gh-5089
+ gh.test.lua ; gh-5141
--
2.17.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists
2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov
@ 2020-07-10 10:01 ` Kirill Yukhin
2020-07-13 12:24 ` Sergey Bronnikov
1 sibling, 0 replies; 4+ messages in thread
From: Kirill Yukhin @ 2020-07-10 10:01 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Hello,
On 09 июл 12:46, Alexander V. Tikhonov wrote:
> Syncronized suites 'fragile' lists with actual list of flaky tests.
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add
I've checked your patch into 1.10, 2.3, 2.4 and master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists
2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov
2020-07-10 10:01 ` Kirill Yukhin
@ 2020-07-13 12:24 ` Sergey Bronnikov
2020-07-13 13:50 ` Alexander V. Tikhonov
1 sibling, 1 reply; 4+ messages in thread
From: Sergey Bronnikov @ 2020-07-13 12:24 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Thanks for the patch!
See 3 comments.
On 12:46 Thu 09 Jul , Alexander V. Tikhonov wrote:
> Syncronized suites 'fragile' lists with actual list of flaky tests.
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add
>
> test/app-tap/suite.ini | 1 +
> test/box-tap/suite.ini | 3 ++-
> test/box/suite.ini | 5 ++++-
> test/engine/suite.ini | 3 +--
> test/replication-py/suite.ini | 1 +
> test/replication/suite.ini | 10 ++++++----
> test/sql-tap/suite.ini | 1 -
> test/sql/suite.ini | 3 +--
> test/vinyl/suite.ini | 3 +++
> 9 files changed, 19 insertions(+), 11 deletions(-)
>
> diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini
> index daababef5..f06f89ed5 100644
> --- a/test/app-tap/suite.ini
> +++ b/test/app-tap/suite.ini
> @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua
> is_parallel = True
> pretest_clean = True
> use_unix_sockets_iproto = True
> +fragile = tarantoolctl.test.lua ; gh-5059
> diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini
> index 07f8880b6..0f4f4a566 100644
> --- a/test/box-tap/suite.ini
> +++ b/test/box-tap/suite.ini
> @@ -4,5 +4,6 @@ description = Database tests with #! using TAP
> is_parallel = True
> pretest_clean = True
> use_unix_sockets_iproto = True
> -fragile = cfg.test.lua ; gh-4344
> config = suite.cfg
> +fragile = cfg.test.lua ; gh-4344
> + key_def.test.lua ; gh-5049
gh-5049 is a duplicate of gh-4252, that is closed.
> diff --git a/test/box/suite.ini b/test/box/suite.ini
> index de8f5a70e..7f2f5d898 100644
> --- a/test/box/suite.ini
> +++ b/test/box/suite.ini
> @@ -13,7 +13,8 @@ pretest_clean = True
> fragile = bitset.test.lua ; gh-4095
> func_reload.test.lua ; gh-4425
> function1.test.lua ; gh-4199
> - net.box.test.lua ; gh-3851 gh-4383
> + push.test.lua ; gh-4882
> + tree_pk.test.lua ; gh-4882
> alter_limits.test.lua ; gh-4926
> misc.test.lua ; gh-4982
> tuple.test.lua ; gh-4988
> @@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095
> sequence.test.lua ; gh-4996
> on_replace.test.lua ; gh-4997
> role.test.lua ; gh-4998
> + net.box_reconnect_after_gh-3164.test.lua ; gh-5081
> + net.box_disconnect_gh-3859.test.lua ; gh-5056
gh-5056 is not about flaky test.
" gitlab-ci: enable script for VBOX VMs orchestrate #5056"
> diff --git a/test/engine/suite.ini b/test/engine/suite.ini
> index 3ab3cf108..d170c9f3f 100644
> --- a/test/engine/suite.ini
> +++ b/test/engine/suite.ini
> @@ -10,5 +10,4 @@ config = engine.cfg
> lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua
> is_parallel = True
> pretest_clean = True
> -fragile = recover_wal.test.lua ; gh-3767
> - ddl.test.lua ; gh-4353
> +fragile = ddl.test.lua ; gh-4353
> diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini
> index b4caf443b..45add028c 100644
> --- a/test/replication-py/suite.ini
> +++ b/test/replication-py/suite.ini
> @@ -5,3 +5,4 @@ description = tarantool/box, replication
> is_parallel = True
> fragile = init_storage.test.py ; gh-4949
> conflict.test.py ; gh-4980
> + cluster.test.py ; gh-5109
> diff --git a/test/replication/suite.ini b/test/replication/suite.ini
> index 6119a264b..8d6c9799d 100644
> --- a/test/replication/suite.ini
> +++ b/test/replication/suite.ini
> @@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870
> long_row_timeout.test.lua ; gh-4351
> misc.test.lua ; gh-4940
> skip_conflict_row.test.lua ; gh-4958
> - sync.test.lua ; gh-3835 gh-3877
> + sync.test.lua ; gh-3835
> transaction.test.lua ; gh-4312
> - wal_rw_stress.test.lua ; gh-4977
> - wal_off.test.lua ; gh-4355
> + autobootstrap.test.lua ; gh-4533
> replica_rejoin.test.lua ; gh-4985
> recover_missing_xlog.test.lua ; gh-4989
> - box_set_replication_stress ; gh-4992
gh-4992 is open, why box_set_replication_stress is not fragile anymore?
> + box_set_replication_stress.test.lua ; gh-4992 gh-4986
> + gh-4605-empty-password.test.lua ; gh-5030
> + anon.test.lua ; gh-5058
> + status.test.lua ; gh-5110
> diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini
> index 3c0e309ee..bb6772823 100644
> --- a/test/sql-tap/suite.ini
> +++ b/test/sql-tap/suite.ini
> @@ -30,4 +30,3 @@ config = engine.cfg
> show_reproduce_content = False
> pretest_clean = True
> use_unix_sockets_iproto = True
> -fragile = selectG.test.lua ; gh-4458
> diff --git a/test/sql/suite.ini b/test/sql/suite.ini
> index 57e78f1c8..57733deec 100644
> --- a/test/sql/suite.ini
> +++ b/test/sql/suite.ini
> @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua
> release_disabled = errinj.test.lua view_delayed_wal.test.lua
> disabled = sql-statN-index-drop.test.lua
> pretest_clean = True
> -fragile = dll.test.lua ; gh-4427
> - func-recreate.test.lua ; gh-4384
> +fragile = func-recreate.test.lua ; gh-4384
> diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini
> index 793901275..3e11b087b 100644
> --- a/test/vinyl/suite.ini
> +++ b/test/vinyl/suite.ini
> @@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346
> snapshot.test.lua ; gh-4984
> write_iterator.test.lua ; gh-4572
> errinj_ddl.test.lua ; gh-4993
> + gh-4810-dump-during-index-build.test.lua ; gh-5031
> + deferred_delete.test.lua ; gh-5089
> + gh.test.lua ; gh-5141
> --
> 2.17.1
>
--
sergeyb@
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists
2020-07-13 12:24 ` Sergey Bronnikov
@ 2020-07-13 13:50 ` Alexander V. Tikhonov
0 siblings, 0 replies; 4+ messages in thread
From: Alexander V. Tikhonov @ 2020-07-13 13:50 UTC (permalink / raw)
To: Sergey Bronnikov; +Cc: tarantool-patches
Hi Sergey, thanks for the deep review, please check my comments below.
On Mon, Jul 13, 2020 at 03:24:23PM +0300, Sergey Bronnikov wrote:
> Thanks for the patch!
> See 3 comments.
>
> On 12:46 Thu 09 Jul , Alexander V. Tikhonov wrote:
> > Syncronized suites 'fragile' lists with actual list of flaky tests.
> > ---
> >
> > Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add
> >
> > test/app-tap/suite.ini | 1 +
> > test/box-tap/suite.ini | 3 ++-
> > test/box/suite.ini | 5 ++++-
> > test/engine/suite.ini | 3 +--
> > test/replication-py/suite.ini | 1 +
> > test/replication/suite.ini | 10 ++++++----
> > test/sql-tap/suite.ini | 1 -
> > test/sql/suite.ini | 3 +--
> > test/vinyl/suite.ini | 3 +++
> > 9 files changed, 19 insertions(+), 11 deletions(-)
> >
> > diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini
> > index daababef5..f06f89ed5 100644
> > --- a/test/app-tap/suite.ini
> > +++ b/test/app-tap/suite.ini
> > @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua
> > is_parallel = True
> > pretest_clean = True
> > use_unix_sockets_iproto = True
> > +fragile = tarantoolctl.test.lua ; gh-5059
> > diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini
> > index 07f8880b6..0f4f4a566 100644
> > --- a/test/box-tap/suite.ini
> > +++ b/test/box-tap/suite.ini
> > @@ -4,5 +4,6 @@ description = Database tests with #! using TAP
> > is_parallel = True
> > pretest_clean = True
> > use_unix_sockets_iproto = True
> > -fragile = cfg.test.lua ; gh-4344
> > config = suite.cfg
> > +fragile = cfg.test.lua ; gh-4344
> > + key_def.test.lua ; gh-5049
>
> gh-5049 is a duplicate of gh-4252, that is closed.
>
Right, this test should be removed, due to fixed.
> > diff --git a/test/box/suite.ini b/test/box/suite.ini
> > index de8f5a70e..7f2f5d898 100644
> > --- a/test/box/suite.ini
> > +++ b/test/box/suite.ini
> > @@ -13,7 +13,8 @@ pretest_clean = True
> > fragile = bitset.test.lua ; gh-4095
> > func_reload.test.lua ; gh-4425
> > function1.test.lua ; gh-4199
> > - net.box.test.lua ; gh-3851 gh-4383
> > + push.test.lua ; gh-4882
> > + tree_pk.test.lua ; gh-4882
> > alter_limits.test.lua ; gh-4926
> > misc.test.lua ; gh-4982
> > tuple.test.lua ; gh-4988
> > @@ -22,3 +23,5 @@ fragile = bitset.test.lua ; gh-4095
> > sequence.test.lua ; gh-4996
> > on_replace.test.lua ; gh-4997
> > role.test.lua ; gh-4998
> > + net.box_reconnect_after_gh-3164.test.lua ; gh-5081
> > + net.box_disconnect_gh-3859.test.lua ; gh-5056
>
> gh-5056 is not about flaky test.
> " gitlab-ci: enable script for VBOX VMs orchestrate #5056"
>
Right, the issue should be 5156 instead of 5056, corrected.
> > diff --git a/test/engine/suite.ini b/test/engine/suite.ini
> > index 3ab3cf108..d170c9f3f 100644
> > --- a/test/engine/suite.ini
> > +++ b/test/engine/suite.ini
> > @@ -10,5 +10,4 @@ config = engine.cfg
> > lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua
> > is_parallel = True
> > pretest_clean = True
> > -fragile = recover_wal.test.lua ; gh-3767
> > - ddl.test.lua ; gh-4353
> > +fragile = ddl.test.lua ; gh-4353
> > diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini
> > index b4caf443b..45add028c 100644
> > --- a/test/replication-py/suite.ini
> > +++ b/test/replication-py/suite.ini
> > @@ -5,3 +5,4 @@ description = tarantool/box, replication
> > is_parallel = True
> > fragile = init_storage.test.py ; gh-4949
> > conflict.test.py ; gh-4980
> > + cluster.test.py ; gh-5109
> > diff --git a/test/replication/suite.ini b/test/replication/suite.ini
> > index 6119a264b..8d6c9799d 100644
> > --- a/test/replication/suite.ini
> > +++ b/test/replication/suite.ini
> > @@ -15,10 +15,12 @@ fragile = errinj.test.lua ; gh-3870
> > long_row_timeout.test.lua ; gh-4351
> > misc.test.lua ; gh-4940
> > skip_conflict_row.test.lua ; gh-4958
> > - sync.test.lua ; gh-3835 gh-3877
> > + sync.test.lua ; gh-3835
> > transaction.test.lua ; gh-4312
> > - wal_rw_stress.test.lua ; gh-4977
> > - wal_off.test.lua ; gh-4355
> > + autobootstrap.test.lua ; gh-4533
> > replica_rejoin.test.lua ; gh-4985
> > recover_missing_xlog.test.lua ; gh-4989
> > - box_set_replication_stress ; gh-4992
>
> gh-4992 is open, why box_set_replication_stress is not fragile anymore?
>
No, just added additional issue in comment, please check the line below,
also the mistake of the test name was corrected.
> > + box_set_replication_stress.test.lua ; gh-4992 gh-4986
> > + gh-4605-empty-password.test.lua ; gh-5030
> > + anon.test.lua ; gh-5058
> > + status.test.lua ; gh-5110
> > diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini
> > index 3c0e309ee..bb6772823 100644
> > --- a/test/sql-tap/suite.ini
> > +++ b/test/sql-tap/suite.ini
> > @@ -30,4 +30,3 @@ config = engine.cfg
> > show_reproduce_content = False
> > pretest_clean = True
> > use_unix_sockets_iproto = True
> > -fragile = selectG.test.lua ; gh-4458
> > diff --git a/test/sql/suite.ini b/test/sql/suite.ini
> > index 57e78f1c8..57733deec 100644
> > --- a/test/sql/suite.ini
> > +++ b/test/sql/suite.ini
> > @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua
> > release_disabled = errinj.test.lua view_delayed_wal.test.lua
> > disabled = sql-statN-index-drop.test.lua
> > pretest_clean = True
> > -fragile = dll.test.lua ; gh-4427
> > - func-recreate.test.lua ; gh-4384
> > +fragile = func-recreate.test.lua ; gh-4384
> > diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini
> > index 793901275..3e11b087b 100644
> > --- a/test/vinyl/suite.ini
> > +++ b/test/vinyl/suite.ini
> > @@ -20,3 +20,6 @@ fragile = errinj.test.lua ; gh-4346
> > snapshot.test.lua ; gh-4984
> > write_iterator.test.lua ; gh-4572
> > errinj_ddl.test.lua ; gh-4993
> > + gh-4810-dump-during-index-build.test.lua ; gh-5031
> > + deferred_delete.test.lua ; gh-5089
> > + gh.test.lua ; gh-5141
> > --
> > 2.17.1
> >
>
> --
> sergeyb@
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-07-13 13:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-09 9:46 [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists Alexander V. Tikhonov
2020-07-10 10:01 ` Kirill Yukhin
2020-07-13 12:24 ` Sergey Bronnikov
2020-07-13 13:50 ` Alexander V. Tikhonov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox