Re: [Tarantool-patches] [PATCH v1] test: update suites 'fragile' lists

[Sergey Bronnikov <sergeyb@tarantool.org>]

Thanks for the patch!
See 3 comments.

On 12:46 Thu 09 Jul , Alexander V. Tikhonov wrote:
> Syncronized suites 'fragile' lists with actual list of flaky tests.
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-add
> 
>  test/app-tap/suite.ini        |  1 +
>  test/box-tap/suite.ini        |  3 ++-
>  test/box/suite.ini            |  5 ++++-
>  test/engine/suite.ini         |  3 +--
>  test/replication-py/suite.ini |  1 +
>  test/replication/suite.ini    | 10 ++++++----
>  test/sql-tap/suite.ini        |  1 -
>  test/sql/suite.ini            |  3 +--
>  test/vinyl/suite.ini          |  3 +++
>  9 files changed, 19 insertions(+), 11 deletions(-)
> 
> diff --git a/test/app-tap/suite.ini b/test/app-tap/suite.ini
> index daababef5..f06f89ed5 100644
> --- a/test/app-tap/suite.ini
> +++ b/test/app-tap/suite.ini
> @@ -5,3 +5,4 @@ lua_libs = lua/require_mod.lua lua/serializer_test.lua
>  is_parallel = True
>  pretest_clean = True
>  use_unix_sockets_iproto = True
> +fragile = tarantoolctl.test.lua     ; gh-5059
> diff --git a/test/box-tap/suite.ini b/test/box-tap/suite.ini
> index 07f8880b6..0f4f4a566 100644
> --- a/test/box-tap/suite.ini
> +++ b/test/box-tap/suite.ini
> @@ -4,5 +4,6 @@ description = Database tests with #! using TAP
>  is_parallel = True
>  pretest_clean = True
>  use_unix_sockets_iproto = True
> -fragile = cfg.test.lua     ; gh-4344
>  config = suite.cfg
> +fragile = cfg.test.lua     ; gh-4344
> +          key_def.test.lua ; gh-5049

gh-5049 is a duplicate of gh-4252, that is closed.

> diff --git a/test/box/suite.ini b/test/box/suite.ini
> index de8f5a70e..7f2f5d898 100644
> --- a/test/box/suite.ini
> +++ b/test/box/suite.ini
> @@ -13,7 +13,8 @@ pretest_clean = True
>  fragile = bitset.test.lua      ; gh-4095
>            func_reload.test.lua ; gh-4425
>            function1.test.lua   ; gh-4199
> -          net.box.test.lua     ; gh-3851 gh-4383
> +          push.test.lua        ; gh-4882
> +          tree_pk.test.lua     ; gh-4882
>            alter_limits.test.lua ; gh-4926
>            misc.test.lua        ; gh-4982
>            tuple.test.lua       ; gh-4988
> @@ -22,3 +23,5 @@ fragile = bitset.test.lua      ; gh-4095
>            sequence.test.lua    ; gh-4996
>            on_replace.test.lua  ; gh-4997
>            role.test.lua        ; gh-4998
> +          net.box_reconnect_after_gh-3164.test.lua ; gh-5081
> +          net.box_disconnect_gh-3859.test.lua      ; gh-5056

gh-5056 is not about flaky test.
" gitlab-ci: enable script for VBOX VMs orchestrate #5056"

> diff --git a/test/engine/suite.ini b/test/engine/suite.ini
> index 3ab3cf108..d170c9f3f 100644
> --- a/test/engine/suite.ini
> +++ b/test/engine/suite.ini
> @@ -10,5 +10,4 @@ config = engine.cfg
>  lua_libs = conflict.lua ../box/lua/utils.lua ../box/lua/push.lua
>  is_parallel = True
>  pretest_clean = True
> -fragile = recover_wal.test.lua ; gh-3767
> -          ddl.test.lua         ; gh-4353
> +fragile = ddl.test.lua         ; gh-4353
> diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini
> index b4caf443b..45add028c 100644
> --- a/test/replication-py/suite.ini
> +++ b/test/replication-py/suite.ini
> @@ -5,3 +5,4 @@ description = tarantool/box, replication
>  is_parallel = True
>  fragile = init_storage.test.py        ; gh-4949
>            conflict.test.py            ; gh-4980
> +          cluster.test.py             ; gh-5109
> diff --git a/test/replication/suite.ini b/test/replication/suite.ini
> index 6119a264b..8d6c9799d 100644
> --- a/test/replication/suite.ini
> +++ b/test/replication/suite.ini
> @@ -15,10 +15,12 @@ fragile = errinj.test.lua            ; gh-3870
>            long_row_timeout.test.lua  ; gh-4351
>            misc.test.lua              ; gh-4940
>            skip_conflict_row.test.lua ; gh-4958
> -          sync.test.lua              ; gh-3835 gh-3877
> +          sync.test.lua              ; gh-3835
>            transaction.test.lua       ; gh-4312
> -          wal_rw_stress.test.lua     ; gh-4977
> -          wal_off.test.lua           ; gh-4355
> +          autobootstrap.test.lua     ; gh-4533
>            replica_rejoin.test.lua    ; gh-4985
>            recover_missing_xlog.test.lua ; gh-4989
> -          box_set_replication_stress ; gh-4992

gh-4992 is open, why box_set_replication_stress is not fragile anymore?

> +          box_set_replication_stress.test.lua     ; gh-4992 gh-4986
> +          gh-4605-empty-password.test.lua         ; gh-5030
> +          anon.test.lua              ; gh-5058
> +          status.test.lua            ; gh-5110
> diff --git a/test/sql-tap/suite.ini b/test/sql-tap/suite.ini
> index 3c0e309ee..bb6772823 100644
> --- a/test/sql-tap/suite.ini
> +++ b/test/sql-tap/suite.ini
> @@ -30,4 +30,3 @@ config = engine.cfg
>  show_reproduce_content = False
>  pretest_clean = True
>  use_unix_sockets_iproto = True
> -fragile = selectG.test.lua                        ; gh-4458
> diff --git a/test/sql/suite.ini b/test/sql/suite.ini
> index 57e78f1c8..57733deec 100644
> --- a/test/sql/suite.ini
> +++ b/test/sql/suite.ini
> @@ -10,5 +10,4 @@ lua_libs = lua/sql_tokenizer.lua
>  release_disabled = errinj.test.lua view_delayed_wal.test.lua
>  disabled = sql-statN-index-drop.test.lua
>  pretest_clean = True
> -fragile = dll.test.lua           ; gh-4427
> -          func-recreate.test.lua ; gh-4384
> +fragile = func-recreate.test.lua ; gh-4384
> diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini
> index 793901275..3e11b087b 100644
> --- a/test/vinyl/suite.ini
> +++ b/test/vinyl/suite.ini
> @@ -20,3 +20,6 @@ fragile = errinj.test.lua             ; gh-4346
>            snapshot.test.lua           ; gh-4984
>            write_iterator.test.lua     ; gh-4572
>            errinj_ddl.test.lua         ; gh-4993
> +          gh-4810-dump-during-index-build.test.lua ; gh-5031
> +          deferred_delete.test.lua    ; gh-5089
> +          gh.test.lua                 ; gh-5141
> -- 
> 2.17.1
> 

-- 
sergeyb@