[tarantool-patches] [PATCH] test: enable parallel mode for wall_off tests

[tarantool-patches] [PATCH] test: enable parallel mode for wall_off tests

[Sergei Voronezhskii]

All tests in one suite must use different spaces, because this can
cause tests to affect each other.

Part of #2436
---
 test/wal_off/expirationd.result   | 2 +-
 test/wal_off/expirationd.test.lua | 2 +-
 test/wal_off/oom.result           | 2 +-
 test/wal_off/oom.test.lua         | 2 +-
 test/wal_off/suite.ini            | 2 +-
 test/wal_off/wal_mode.result      | 2 +-
 test/wal_off/wal_mode.test.lua    | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/test/wal_off/expirationd.result b/test/wal_off/expirationd.result
index affb9b259..a924bf692 100644
--- a/test/wal_off/expirationd.result
+++ b/test/wal_off/expirationd.result
@@ -8,7 +8,7 @@ env = require('test_run')
 test_run = env.new()
 ---
 ...
-s0 = box.schema.space.create('tweedledum')
+s0 = box.schema.space.create('tweedledex')
 ---
 ...
 i0 = s0:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}, unique = true })
diff --git a/test/wal_off/expirationd.test.lua b/test/wal_off/expirationd.test.lua
index 804e9ccb2..bf40259eb 100644
--- a/test/wal_off/expirationd.test.lua
+++ b/test/wal_off/expirationd.test.lua
@@ -2,7 +2,7 @@ fiber = require('fiber')
 -- test for expirationd. iterator must continue iterating after space insert/delete
 env = require('test_run')
 test_run = env.new()
-s0 = box.schema.space.create('tweedledum')
+s0 = box.schema.space.create('tweedledex')
 i0 = s0:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}, unique = true })
 s0:insert{20000}
 test_run:cmd("setopt delimiter ';'")
diff --git a/test/wal_off/oom.result b/test/wal_off/oom.result
index c47d16c46..6e446585f 100644
--- a/test/wal_off/oom.result
+++ b/test/wal_off/oom.result
@@ -9,7 +9,7 @@ test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed
 ---
 - true
 ...
-space = box.schema.space.create('tweedledum')
+space = box.schema.space.create('tweedledoom')
 ---
 ...
 index = space:create_index('primary', { type = 'hash' })
diff --git a/test/wal_off/oom.test.lua b/test/wal_off/oom.test.lua
index 5c0ab8e73..e690966a1 100644
--- a/test/wal_off/oom.test.lua
+++ b/test/wal_off/oom.test.lua
@@ -3,7 +3,7 @@ test_run = env.new()
 test_run:cmd('restart server default')
 test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
 
-space = box.schema.space.create('tweedledum')
+space = box.schema.space.create('tweedledoom')
 index = space:create_index('primary', { type = 'hash' })
 test_run:cmd("setopt delimiter ';'")
 i = 1;
diff --git a/test/wal_off/suite.ini b/test/wal_off/suite.ini
index ad19eab10..cbb7cb341 100644
--- a/test/wal_off/suite.ini
+++ b/test/wal_off/suite.ini
@@ -2,4 +2,4 @@
 core = tarantool
 script = wal.lua
 description = tarantool/box, wal_mode = none
-is_parallel = False
+is_parallel = True
diff --git a/test/wal_off/wal_mode.result b/test/wal_off/wal_mode.result
index c07faf744..3047d80fb 100644
--- a/test/wal_off/wal_mode.result
+++ b/test/wal_off/wal_mode.result
@@ -5,7 +5,7 @@ box.cfg.wal_mode
 ---
 - none
 ...
-space = box.schema.space.create('tweedledum')
+space = box.schema.space.create('tweedledee')
 ---
 ...
 index = space:create_index('primary', { type = 'hash' })
diff --git a/test/wal_off/wal_mode.test.lua b/test/wal_off/wal_mode.test.lua
index 79f4ce967..d9f886e8f 100644
--- a/test/wal_off/wal_mode.test.lua
+++ b/test/wal_off/wal_mode.test.lua
@@ -1,7 +1,7 @@
 test_run = require('test_run').new()
 
 box.cfg.wal_mode
-space = box.schema.space.create('tweedledum')
+space = box.schema.space.create('tweedledee')
 index = space:create_index('primary', { type = 'hash' })
 space:insert{1}
 space:insert{2}
-- 
2.18.0

[tarantool-patches] Re: [PATCH] test: enable parallel mode for wall_off tests

[Alexander Turenko]

Hi!

We look into the problem with Sergei together.

Found failing trace:

- [wal_off/wal_mode.test.lua, null]
- [wal_off/tuple.test.lua, null]
- [wal_off/expirationd.test.lua, null]

wal_mode.test.lua creates a snapshot (it contains space 'tweedledum'),
then tuple.test.lua restarts the default server, it bootstraps from the
snapshot (because WAL is off) and then expirationd.test.lua tries to
create the space again and fails.

The fix:

diff --git a/test/wal_off/tuple.test.lua b/test/wal_off/tuple.test.lua
index 19415a92d..859438f71 100644
--- a/test/wal_off/tuple.test.lua
+++ b/test/wal_off/tuple.test.lua
@@ -1,6 +1,6 @@
 env = require('test_run')
 test_run = env.new()
-test_run:cmd("restart server default")
+test_run:cmd("restart server default with cleanup=1")
 --
 -- Test various tuple bugs which do not require a write ahead log.
 --

WBR, Alexander Turenko.

On Thu, Sep 13, 2018 at 07:30:07PM +0300, Sergei Voronezhskii wrote:
> All tests in one suite must use different spaces, because this can
> cause tests to affect each other.
> 
> Part of #2436
> ---
>  test/wal_off/expirationd.result   | 2 +-
>  test/wal_off/expirationd.test.lua | 2 +-
>  test/wal_off/oom.result           | 2 +-
>  test/wal_off/oom.test.lua         | 2 +-
>  test/wal_off/suite.ini            | 2 +-
>  test/wal_off/wal_mode.result      | 2 +-
>  test/wal_off/wal_mode.test.lua    | 2 +-
>  7 files changed, 7 insertions(+), 7 deletions(-)
> 

[tarantool-patches] Re: [PATCH] test: enable parallel mode for wall_off tests

[Kirill Yukhin]

Hello,
On 17 Sep 16:11, Sergei Voronezhskii wrote:
> Use the proper way to cleanup tests.
> 
> Part of #2436
> ---
>  test/wal_off/oom.result     | 4 ++--
>  test/wal_off/oom.test.lua   | 4 ++--
>  test/wal_off/suite.ini      | 2 +-
>  test/wal_off/tuple.result   | 2 +-
>  test/wal_off/tuple.test.lua | 2 +-
>  5 files changed, 7 insertions(+), 7 deletions(-)

I've checked your patch into 2.1 branch.

--
Regards, Kirill Yukhin

[tarantool-patches] Re: [PATCH] test: enable parallel mode for wall_off tests

[Alexander Turenko]

Adding cleanup option is okay for me, but parallel run still fails and
it is not the time to enable it. I got two failures:

$ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_BACKTRACE=ON && make -j
$ TEST_RUN_TESTS=$(for i in $(seq 1 100); do echo -n 'wal_off/ '; done) make test

[002] wal_off/lua.test.lua                                            [ fail ]
[002] 
[002] Test failed! Result content mismatch:
[002] --- wal_off/lua.result	Fri Apr 27 14:25:39 2018
[002] +++ wal_off/lua.reject	Mon Sep 17 16:25:40 2018
[002] @@ -91,7 +91,7 @@
[002]  ...
[002]  mktuple(100000)
[002]  ---
[002] -- count 100000 len 368553
[002] +- error: Failed to allocate 368569 bytes in slab allocator for memtx_tuple
[002]  ...
[002]  space:drop()
[002]  ---
[002] 

# reproduce file: /home/alex/projects/tarantool-meta/review/tarantool/test/var/reproduce/002_wal_off.list.yaml
---
- [wal_off/tuple.test.lua, null]
- [wal_off/iterator_lt_gt.test.lua, null]
- [wal_off/snapshot_stress.test.lua, null]
- [wal_off/rtree_benchmark.test.lua, null]
- [wal_off/lua.test.lua, null]

[008] wal_off/snapshot_stress.test.lua                                [ fail ]
[008] 
[008] Test failed! Result content mismatch:
[008] --- wal_off/snapshot_stress.result	Fri Apr 27 14:25:39 2018
[008] +++ wal_off/snapshot_stress.reject	Mon Sep 17 16:25:41 2018
[008] @@ -375,7 +375,7 @@
[008]  ...
[008]  snaps_find_status;
[008]  ---
[008] -- snaps found
[008] +- where are my snapshots?
[008]  ...
[008]  snapshot_check_failed = false
[008]  while #snaps > initial_snap_count do
[008] 

# reproduce file: /home/alex/projects/tarantool-meta/review/tarantool/test/var/reproduce/008_wal_off.list.yaml
---
- [wal_off/lua.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/wal_mode.test.lua, null]
- [wal_off/func_max.test.lua, null]
- [wal_off/expirationd.test.lua, null]
- [wal_off/snapshot_stress.test.lua, null]

Please, reproduce and elaborate.

BTW, you are forgot the branch name: sergw/enable-parallel-test-wal-off.

WBR, Alexander Turenko.

On Mon, Sep 17, 2018 at 04:11:55PM +0300, Sergei Voronezhskii wrote:
> Use the proper way to cleanup tests.
> 
> Part of #2436
> ---
>  test/wal_off/oom.result     | 4 ++--
>  test/wal_off/oom.test.lua   | 4 ++--
>  test/wal_off/suite.ini      | 2 +-
>  test/wal_off/tuple.result   | 2 +-
>  test/wal_off/tuple.test.lua | 2 +-
>  5 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/test/wal_off/oom.result b/test/wal_off/oom.result
> index c47d16c46..65bb8c670 100644
> --- a/test/wal_off/oom.result
> +++ b/test/wal_off/oom.result
> @@ -4,7 +4,7 @@ env = require('test_run')
>  test_run = env.new()
>  ---
>  ...
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
>  ---
>  - true
> @@ -231,7 +231,7 @@ t = nil
>  ---
>  ...
>  -- https://github.com/tarantool/tarantool/issues/962 index:delete() failed
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  arena_bytes = box.cfg.memtx_memory
>  ---
>  ...
> diff --git a/test/wal_off/oom.test.lua b/test/wal_off/oom.test.lua
> index 5c0ab8e73..89dba3f06 100644
> --- a/test/wal_off/oom.test.lua
> +++ b/test/wal_off/oom.test.lua
> @@ -1,6 +1,6 @@
>  env = require('test_run')
>  test_run = env.new()
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
>  
>  space = box.schema.space.create('tweedledum')
> @@ -80,7 +80,7 @@ space:drop()
>  t = nil
>  
>  -- https://github.com/tarantool/tarantool/issues/962 index:delete() failed
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
>  arena_bytes = box.cfg.memtx_memory
>  str = string.rep('a', 15000) -- about size of index memory block
>  
> diff --git a/test/wal_off/suite.ini b/test/wal_off/suite.ini
> index ad19eab10..cbb7cb341 100644
> --- a/test/wal_off/suite.ini
> +++ b/test/wal_off/suite.ini
> @@ -2,4 +2,4 @@
>  core = tarantool
>  script = wal.lua
>  description = tarantool/box, wal_mode = none
> -is_parallel = False
> +is_parallel = True
> diff --git a/test/wal_off/tuple.result b/test/wal_off/tuple.result
> index fa431e203..dcbd24daf 100644
> --- a/test/wal_off/tuple.result
> +++ b/test/wal_off/tuple.result
> @@ -4,7 +4,7 @@ env = require('test_run')
>  test_run = env.new()
>  ---
>  ...
> -test_run:cmd("restart server default")
> +test_run:cmd("restart server default with cleanup=1")
>  -- 
>  -- Test various tuple bugs which do not require a write ahead log.
>  -- 
> diff --git a/test/wal_off/tuple.test.lua b/test/wal_off/tuple.test.lua
> index 19415a92d..859438f71 100644
> --- a/test/wal_off/tuple.test.lua
> +++ b/test/wal_off/tuple.test.lua
> @@ -1,6 +1,6 @@
>  env = require('test_run')
>  test_run = env.new()
> -test_run:cmd("restart server default")
> +test_run:cmd("restart server default with cleanup=1")
>  -- 
>  -- Test various tuple bugs which do not require a write ahead log.
>  -- 
> -- 
> 2.18.0
>