Tarantool development patches archive
 help / color / mirror / Atom feed
From: Alexander Turenko <alexander.turenko@tarantool.org>
To: Nikita Pettik <korablev@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Subject: [Tarantool-patches] [PATCH] test: stabilize flaky fiber memory leak detection
Date: Wed, 29 Jan 2020 20:03:22 +0300	[thread overview]
Message-ID: <ad31974a5490dabd1bb1fb6de51f9c8bc54b52dd.1580304896.git.alexander.turenko@tarantool.org> (raw)

After #4736 regression fix (in fact it just reverts the new logic in
small) it is possible again that a fiber's region may hold a memory for
a while, but release it eventually. When the used memory exceeds 128 KiB
threshold, fiber_gc() puts 'garbage' slabs back to slab_cache and
subtracts them from region_used() metric. But until this point those
slabs are accounted in region_used() and so in fiber.info() metrics.

This commit fixes flakiness of test cases of the following kind:

 | fiber.info()[fiber.self().id()].memory.used -- should be zero
 | <...workload...>
 | fiber.info()[fiber.self().id()].memory.used -- should be zero

The problem is that the first `<...>.memory.used` value may be non-zero.
It depends of previous tests that were executed on this tarantool
instance. It is resolved by restarting of a tarantool instance before
such test cases to ensure that there are no 'garbage' slabs in a current
fiber's region.

Note: This works only if a test case reserves only one slab at the
moment: otherwise some memory may be hold after the case (and so a
memory check after a workload will fail). However it seems that our
cases are small enough to don't trigger this situation.

Call of region_free() would be enough, but we have no Lua API for it.

Fixes #4750.
---

https://github.com/tarantool/tarantool/issues/4750
https://github.com/tarantool/tarantool/tree/Totktonada/gh-4750-stabilize-flaky-memleak-checks

 test/engine/snapshot.result            | 3 +++
 test/engine/snapshot.test.lua          | 4 ++++
 test/sql/gh-3199-no-mem-leaks.result   | 9 ++++++---
 test/sql/gh-3199-no-mem-leaks.test.lua | 5 ++++-
 4 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/test/engine/snapshot.result b/test/engine/snapshot.result
index 9b4824ba3..f4ac7287a 100644
--- a/test/engine/snapshot.result
+++ b/test/engine/snapshot.result
@@ -238,6 +238,9 @@ box.space.test.index.secondary:select{}
 box.space.test:drop()
 ---
 ...
+-- Hard way to flush garbage slabs in the fiber's region. See
+-- gh-4750.
+test_run:cmd('restart server default')
 -- Check that box.snapshot() doesn't leave garbage one the region.
 -- https://github.com/tarantool/tarantool/issues/3732
 fiber = require('fiber')
diff --git a/test/engine/snapshot.test.lua b/test/engine/snapshot.test.lua
index 13eecccc2..3f5a89e0b 100644
--- a/test/engine/snapshot.test.lua
+++ b/test/engine/snapshot.test.lua
@@ -42,6 +42,10 @@ box.space.test.index.primary:select{}
 box.space.test.index.secondary:select{}
 box.space.test:drop()
 
+-- Hard way to flush garbage slabs in the fiber's region. See
+-- gh-4750.
+test_run:cmd('restart server default')
+
 -- Check that box.snapshot() doesn't leave garbage one the region.
 -- https://github.com/tarantool/tarantool/issues/3732
 fiber = require('fiber')
diff --git a/test/sql/gh-3199-no-mem-leaks.result b/test/sql/gh-3199-no-mem-leaks.result
index 35d8572b4..d8590779a 100644
--- a/test/sql/gh-3199-no-mem-leaks.result
+++ b/test/sql/gh-3199-no-mem-leaks.result
@@ -7,13 +7,16 @@ engine = test_run:get_cfg('engine')
 _ = box.space._session_settings:update('sql_default_engine', {{'=', 2, engine}})
 ---
 ...
-fiber = require('fiber')
----
-...
 -- This test checks that no leaks of region memory happens during
 -- executing SQL queries.
 --
 -- box.cfg()
+-- Hard way to flush garbage slabs in the fiber's region. See
+-- gh-4750.
+test_run:cmd('restart server default')
+fiber = require('fiber')
+---
+...
 box.execute('CREATE TABLE test (id INT PRIMARY KEY, x INTEGER, y INTEGER)')
 ---
 - row_count: 1
diff --git a/test/sql/gh-3199-no-mem-leaks.test.lua b/test/sql/gh-3199-no-mem-leaks.test.lua
index 41648d0fc..d517c3373 100644
--- a/test/sql/gh-3199-no-mem-leaks.test.lua
+++ b/test/sql/gh-3199-no-mem-leaks.test.lua
@@ -1,7 +1,6 @@
 test_run = require('test_run').new()
 engine = test_run:get_cfg('engine')
 _ = box.space._session_settings:update('sql_default_engine', {{'=', 2, engine}})
-fiber = require('fiber')
 
 -- This test checks that no leaks of region memory happens during
 -- executing SQL queries.
@@ -9,6 +8,10 @@ fiber = require('fiber')
 
 -- box.cfg()
 
+-- Hard way to flush garbage slabs in the fiber's region. See
+-- gh-4750.
+test_run:cmd('restart server default')
+fiber = require('fiber')
 
 box.execute('CREATE TABLE test (id INT PRIMARY KEY, x INTEGER, y INTEGER)')
 box.execute('INSERT INTO test VALUES (1, 1, 1), (2, 2, 2)')
-- 
2.22.0

             reply	other threads:[~2020-01-29 17:03 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-29 17:03 Alexander Turenko [this message]
2020-01-29 18:46 ` Nikita Pettik
2020-01-29 20:27   ` Alexander Turenko
2020-02-05 15:36     ` Nikita Pettik
2020-02-27 20:22 ` Kirill Yukhin
2020-02-27 22:58   ` Nikita Pettik
2020-02-27 23:07     ` Kirill Yukhin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ad31974a5490dabd1bb1fb6de51f9c8bc54b52dd.1580304896.git.alexander.turenko@tarantool.org \
    --to=alexander.turenko@tarantool.org \
    --cc=korablev@tarantool.org \
    --cc=tarantool-patches@dev.tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH] test: stabilize flaky fiber memory leak detection' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox