Hi!

Thanks for the investigation.

The diff looks good for me and works for me. I have minor comments for
the commit message wording. I'll leave my comments below and the patch
is not needed to re-review with me: please, proceed with Vova and
Kirill.

Vova, can you please confirm that memtx_memory option should be
increased in the test? I mean confirm that it is the test problem and is
not a problem of tarantool itself.

WBR, Alexander Turenko.

On Fri, Sep 21, 2018 at 03:44:42PM +0300, Sergei Voronezhskii wrote:
> Use the proper way to cleanup tests.

'Use the proper way to cleanup tests' is too common and does not give a
reader useful information. It also cannot be a title of the list below,
because not all items below are about clean up. Proposed to remove the
sentence.

> - tuple + lua needs more defragmented memory

Proposed: memtx_memory is increased, because the test '{TEST NAME HERE}'
after '{TEST NAME HERE}' failed with the error '{ERROR HERE}' despite
collectgarbage('collect') calls after cases with huge/many tuples. The
statistics before the allocation fail gives the following values:
{VALUES HERE}. The reason of the fail seems to be a slab memory
fragmentation. It is not clear for now whether we should consider this
as a tarantool issue.

> - snapshot_stress checks for count of checkpoints

Proposed: 'snapshot_stress' test counts snapshot files present in the
working directory and can reach the default 'checkpoint_count' value (2)
if a previous test write its snapshots before.

> - need to cleanup default because of some tests dont drop spaces
>

Proposed: restarting the default server w/o cleaning a working directory
can leave a snapshot that holds a state saved at the middle of a test,
before dropping of the space 'tweedledum' (because WAL is disabled),
that can cause the error '{ERROR HERE}' for a following test.

Sorry for nitpicking, that was to have a clear problem statements and
don't reinvestigate it again in the future (if something like occurs).

> Part of #2436
> ---
>
> BRANCH: sergw/enable-parallel-test-wal-off-clean
>
> test/wal_off/oom.result | 2 +-
> test/wal_off/oom.test.lua | 2 +-
> test/wal_off/suite.ini | 2 +-
> test/wal_off/tuple.result | 2 +-
> test/wal_off/tuple.test.lua | 2 +-
> test/wal_off/wal.lua | 5 +++--
> 6 files changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/test/wal_off/oom.result b/test/wal_off/oom.result
> index c47d16c46..90bc53d22 100644
> --- a/test/wal_off/oom.result
> +++ b/test/wal_off/oom.result
> @@ -4,7 +4,7 @@ env = require('test_run')
> test_run = env.new()
> ---
> ...
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
> test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
> ---
> - true
> diff --git a/test/wal_off/oom.test.lua b/test/wal_off/oom.test.lua
> index 5c0ab8e73..8e6e14046 100644
> --- a/test/wal_off/oom.test.lua
> +++ b/test/wal_off/oom.test.lua
> @@ -1,6 +1,6 @@
> env = require('test_run')
> test_run = env.new()
> -test_run:cmd('restart server default')
> +test_run:cmd('restart server default with cleanup=1')
> test_run:cmd("push filter 'error: Failed to allocate [0-9]+ ' to 'error: Failed to allocate <NUM> '")
>
> space = box.schema.space.create('tweedledum')
> diff --git a/test/wal_off/suite.ini b/test/wal_off/suite.ini
> index ad19eab10..cbb7cb341 100644
> --- a/test/wal_off/suite.ini
> +++ b/test/wal_off/suite.ini
> @@ -2,4 +2,4 @@
> core = tarantool
> script = wal.lua
> description = tarantool/box, wal_mode = none
> -is_parallel = False
> +is_parallel = True
> diff --git a/test/wal_off/tuple.result b/test/wal_off/tuple.result
> index fa431e203..6ea3814fc 100644
> --- a/test/wal_off/tuple.result
> +++ b/test/wal_off/tuple.result
> @@ -4,7 +4,7 @@ env = require('test_run')
> test_run = env.new()
> ---
> ...
> -test_run:cmd("restart server default")
> +test_run:cmd('restart server default with cleanup=1')
> --
> -- Test various tuple bugs which do not require a write ahead log.
> --
> diff --git a/test/wal_off/tuple.test.lua b/test/wal_off/tuple.test.lua
> index 19415a92d..6962f35ad 100644
> --- a/test/wal_off/tuple.test.lua
> +++ b/test/wal_off/tuple.test.lua
> @@ -1,6 +1,6 @@
> env = require('test_run')
> test_run = env.new()
> -test_run:cmd("restart server default")
> +test_run:cmd('restart server default with cleanup=1')
> --
> -- Test various tuple bugs which do not require a write ahead log.
> --
> diff --git a/test/wal_off/wal.lua b/test/wal_off/wal.lua
> index 6e4afbe4d..4005e78cd 100644
> --- a/test/wal_off/wal.lua
> +++ b/test/wal_off/wal.lua
> @@ -2,9 +2,10 @@
>
> box.cfg{
> listen = os.getenv("LISTEN"),
> - memtx_memory = 107374182,
> + memtx_memory = 157374182,
> pid_file = "tarantool.pid",
> - wal_mode = "none"
> + wal_mode = "none",
> + checkpoint_count = 100
> }
>
> require('console').listen(os.getenv('ADMIN'))
> --
> 2.18.0
>