[tarantool-patches] Re: [PATCH] test: enable parallel mode for wall_off tests

[Alexander Turenko <alexander.turenko@tarantool.org>]

Adding cleanup option is okay for me, but parallel run still fails and
it is not the time to enable it. I got two failures:

$ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_BACKTRACE=ON && make -j
$ TEST_RUN_TESTS=$(for i in $(seq 1 100); do echo -n 'wal_off/ '; done) make test

[002] wal_off/lua.test.lua                                            [ fail ]
[002] 
[002] Test failed! Result content mismatch:
[002] --- wal_off/lua.result	Fri Apr 27 14:25:39 2018
[002] +++ wal_off/lua.reject	Mon Sep 17 16:25:40 2018
[002] @@ -91,7 +91,7 @@
[002]  ...
[002]  mktuple(100000)
[002]  ---
[002] -- count 100000 len 368553
[002] +- error: Failed to allocate 368569 bytes in slab allocator for memtx_tuple
[002]  ...
[002]  space:drop()
[002]  ---
[002] 

# reproduce file: /home/alex/projects/tarantool-meta/review/tarantool/test/var/reproduce/002_wal_off.list.yaml
---
- [wal_off/tuple.test.lua, null]
- [wal_off/iterator_lt_gt.test.lua, null]
- [wal_off/snapshot_stress.test.lua, null]
- [wal_off/rtree_benchmark.test.lua, null]
- [wal_off/lua.test.lua, null]

[008] wal_off/snapshot_stress.test.lua                                [ fail ]
[008] 
[008] Test failed! Result content mismatch:
[008] --- wal_off/snapshot_stress.result	Fri Apr 27 14:25:39 2018
[008] +++ wal_off/snapshot_stress.reject	Mon Sep 17 16:25:41 2018
[008] @@ -375,7 +375,7 @@
[008]  ...
[008]  snaps_find_status;
[008]  ---
[008] -- snaps found
[008] +- where are my snapshots?
[008]  ...
[008]  snapshot_check_failed = false
[008]  while #snaps > initial_snap_count do
[008] 

# reproduce file: /home/alex/projects/tarantool-meta/review/tarantool/test/var/reproduce/008_wal_off.list.yaml
---
- [wal_off/lua.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/wal_mode.test.lua, null]
- [wal_off/func_max.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/snapshot_stress.test.lua, null]

Please, reproduce and elaborate.

BTW, you are forgot the branch name: sergw/enable-parallel-test-wal-off.

WBR, Alexander Turenko.

On Mon, Sep 17, 2018 at 04:11:55PM +0300, Sergei Voronezhskii wrote:
> Use the proper way to cleanup tests.
> 
> Part of #2436
> ---
>  test/wal_off/oom.result     | 4 ++--
>  test/wal_off/oom.test.lua   | 4 ++--
>  test/wal_off/suite.ini      | 2 +-
>  test/wal_off/tuple.result   | 2 +-
>  test/wal_off/tuple.test.lua | 2 +-
>  5 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/test/wal_off/oom.result b/test/wal_off/oom.result
> index c47d16c46..65bb8c670 100644
> --- a/test/wal_off/oom.result
> +++ b/test/wal_off/oom.result
> @@ -4,7 +4,7 @@ env = require('test_run')
>  test_run = env.new()
>  ---
>  ...
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
>  ---
>  - true
> @@ -231,7 +231,7 @@ t = nil
>  ---
>  ...
>  -- https://github.com/tarantool/tarantool/issues/962 index:delete() failed
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  arena_bytes = box.cfg.memtx_memory
>  ---
>  ...
> diff --git a/test/wal_off/oom.test.lua b/test/wal_off/oom.test.lua
> index 5c0ab8e73..89dba3f06 100644
> --- a/test/wal_off/oom.test.lua
> +++ b/test/wal_off/oom.test.lua
> @@ -1,6 +1,6 @@
>  env = require('test_run')
>  test_run = env.new()
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
>  
>  space = box.schema.space.create('tweedledum')
> @@ -80,7 +80,7 @@ space:drop()
>  t = nil
>  
>  -- https://github.com/tarantool/tarantool/issues/962 index:delete() failed
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  arena_bytes = box.cfg.memtx_memory
>  str = string.rep('a', 15000) -- about size of index memory block
>  
> diff --git a/test/wal_off/suite.ini b/test/wal_off/suite.ini
> index ad19eab10..cbb7cb341 100644
> --- a/test/wal_off/suite.ini
> +++ b/test/wal_off/suite.ini
> @@ -2,4 +2,4 @@
>  core = tarantool
>  script = wal.lua
>  description = tarantool/box, wal_mode = none
> -is_parallel = False
> +is_parallel = True
> diff --git a/test/wal_off/tuple.result b/test/wal_off/tuple.result
> index fa431e203..dcbd24daf 100644
> --- a/test/wal_off/tuple.result
> +++ b/test/wal_off/tuple.result
> @@ -4,7 +4,7 @@ env = require('test_run')
>  test_run = env.new()
>  ---
>  ...
> -test_run:cmd("restart server default")
> +test_run:cmd("restart server default with cleanup=1")
>  -- 
>  -- Test various tuple bugs which do not require a write ahead log.
>  -- 
> diff --git a/test/wal_off/tuple.test.lua b/test/wal_off/tuple.test.lua
> index 19415a92d..859438f71 100644
> --- a/test/wal_off/tuple.test.lua
> +++ b/test/wal_off/tuple.test.lua
> @@ -1,6 +1,6 @@
>  env = require('test_run')
>  test_run = env.new()
> -test_run:cmd("restart server default")
> +test_run:cmd("restart server default with cleanup=1")
>  -- 
>  -- Test various tuple bugs which do not require a write ahead log.
>  -- 
> -- 
> 2.18.0
>