Re: [Tarantool-patches] [PATCH v1] test: fragile flaky tests from parallel runs

[Sergey Bronnikov <sergeyb@tarantool.org>]

Hi, Sasha

in general LGTM and see comment below.

On 12:49 Thu 30 Apr , Alexander V. Tikhonov wrote:
> Fragiled flaky tests from parallel runs to avoid
> of flaky fails in regular testing:

"fragiled flaky test" is a something like "butter oil"?
I propose make commit message more clear. For example:
"Mark tests as fragile in a test's configs"

>   snapshot.test.py            ; gh-4514
>   init_storage.test.py        ; gh-4949
>   stat.test.lua               ; gh-4951
>   xlog/checkpoint_daemon.test.lua ; gh-4952
> 
> Part of #4953
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/fragile-flaky-tests
> Issue: https://github.com/tarantool/tarantool/issues/4953
> 
>  test/box-py/suite.ini         | 1 +
>  test/replication-py/suite.ini | 1 +
>  test/vinyl/suite.ini          | 1 +
>  test/xlog/suite.ini           | 1 +
>  4 files changed, 4 insertions(+)
> 
> diff --git a/test/box-py/suite.ini b/test/box-py/suite.ini
> index 18737e48f..d1ff0fbfd 100644
> --- a/test/box-py/suite.ini
> +++ b/test/box-py/suite.ini
> @@ -5,3 +5,4 @@ script = box.lua
>  lua_libs = lua/fiber.lua lua/fifo.lua
>  use_unix_sockets = True
>  is_parallel = True
> +fragile = snapshot.test.py            ; gh-4514
> diff --git a/test/replication-py/suite.ini b/test/replication-py/suite.ini
> index df2dd8ef0..2658f4816 100644
> --- a/test/replication-py/suite.ini
> +++ b/test/replication-py/suite.ini
> @@ -3,3 +3,4 @@ core = tarantool
>  script =  master.lua
>  description = tarantool/box, replication
>  is_parallel = True
> +fragile = init_storage.test.py        ; gh-4949
> diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini
> index c8bc270f3..44c53dc68 100644
> --- a/test/vinyl/suite.ini
> +++ b/test/vinyl/suite.ini
> @@ -15,3 +15,4 @@ pretest_clean = True
>  fragile = errinj.test.lua             ; gh-4346
>            select_consistency.test.lua ; gh-4385
>            throttle.test.lua           ; gh-4168
> +          stat.test.lua               ; gh-4951
> diff --git a/test/xlog/suite.ini b/test/xlog/suite.ini
> index 635ec53c1..8e4c4d374 100644
> --- a/test/xlog/suite.ini
> +++ b/test/xlog/suite.ini
> @@ -10,3 +10,4 @@ use_unix_sockets_iproto = True
>  long_run = snap_io_rate.test.lua
>  is_parallel = True
>  pretest_clean = True
> +fragile = xlog/checkpoint_daemon.test.lua ; gh-4952
> -- 
> 2.17.1
> 

-- 
sergeyb@