Tarantool development patches archive
 help / color / mirror / Atom feed
From: avtikhon <avtikhon@tarantool.org>
To: Vladimir Davydov <vdavydov.dev@gmail.com>,
	Alexander Turenko <alexander.turenko@tarantool.org>
Cc: avtikhon <avtikhon@tarantool.org>, tarantool-patches@freelists.org
Subject: [PATCH v2] test: vinyl/errinj test fails under highload
Date: Fri, 19 Apr 2019 08:27:20 +0300	[thread overview]
Message-ID: <fb7b36b0c9d93ffa7236995e1834ed4d049a6dd1.1555651506.git.avtikhon@tarantool.org> (raw)

Test "check that all dump/compaction tasks that are in progress at
the time when the server stops are aborted immediately.", but in
real the awaiting time of 1 second is not enough due to runs in
parallel and it fails, like:

[009] --- vinyl/errinj.result Tue Apr 16 16:43:36 2019
[009] +++ vinyl/errinj.reject Wed Apr 17 09:42:36 2019
[009] @@ -530,7 +530,7 @@
[009] ...
[009] t2 - t1 < 1
[009] ---
[009] -- true
[009] +- false
[009] ...
[009] test_run:cmd("cleanup server test")
[009] ---
[009]

in 100 parallel runs the failed delays were found:

[002] +- 1.4104716777802
[022] +- 1.3933029174805
[044] +- 1.4296517372131
[033] +- 1.6380662918091
[001] +- 1.9799520969391
[027] +- 1.7067711353302
[043] +- 1.3778221607208
[034] +- 1.3820221424103
[032] +- 1.3820221424103
[020] +- 1.6275615692139
[050] +- 1.6275615692139
[048] +- 1.1880359649658

Decided to avoid of use the time check at all and change the
ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT to ERRINJ_VY_DUMP_DELAY injection.
In this way the time checks were completely removed.

Next issue met was the error:

vy_quota.c:298 !> SystemError Failed to allocate 2097240 bytes
    in lsregion for vinyl transaction: Cannot allocate memory

That is why the merged 2 subtests were divided into 2 standalone
subtests to be able to set the memory limit of the 2nd subtest to
2097240 value.

Close #4169
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4169-vinyl-errinj-highload
Issue: https://github.com/tarantool/tarantool/issues/4169

 test/vinyl/errinj.result   | 46 +++++++++++++++++++++++++++-----------
 test/vinyl/errinj.test.lua | 17 +++++++++-----
 2 files changed, 44 insertions(+), 19 deletions(-)

diff --git a/test/vinyl/errinj.result b/test/vinyl/errinj.result
index 6fc4d9e5d..c69347da6 100644
--- a/test/vinyl/errinj.result
+++ b/test/vinyl/errinj.result
@@ -488,10 +488,40 @@ box.snapshot()
 ---
 - ok
 ...
+test_run:cmd('switch default')
+---
+- true
+...
+test_run:cmd("stop server test") -- don't stuck
+---
+- true
+...
+test_run:cmd("cleanup server test")
+---
+- true
+...
 --
 -- Check that all dump/compaction tasks that are in progress at
 -- the time when the server stops are aborted immediately.
 --
+test_run:cmd("create server double_quota with script='vinyl/low_quota.lua'")
+---
+- true
+...
+test_run:cmd("start server double_quota with args='2097240'")
+---
+- true
+...
+test_run:cmd('switch double_quota')
+---
+- true
+...
+fiber = require 'fiber'
+---
+...
+box.cfg{vinyl_timeout = 0.001}
+---
+...
 s = box.schema.space.create('test', {engine = 'vinyl'})
 ---
 ...
@@ -501,7 +531,7 @@ _ = s:create_index('i1', {parts = {1, 'unsigned'}})
 _ = s:create_index('i2', {parts = {2, 'unsigned'}})
 ---
 ...
-box.error.injection.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 0.01)
+box.error.injection.set('ERRINJ_VY_DUMP_DELAY', true)
 ---
 - ok
 ...
@@ -518,21 +548,11 @@ test_run:cmd('switch default')
 ---
 - true
 ...
-t1 = fiber.time()
----
-...
-test_run:cmd("stop server test")
----
-- true
-...
-t2 = fiber.time()
----
-...
-t2 - t1 < 1
+test_run:cmd("stop server double_quota") -- don't stuck
 ---
 - true
 ...
-test_run:cmd("cleanup server test")
+test_run:cmd("cleanup server double_quota")
 ---
 - true
 ...
diff --git a/test/vinyl/errinj.test.lua b/test/vinyl/errinj.test.lua
index b500b33c6..c3ce9706f 100644
--- a/test/vinyl/errinj.test.lua
+++ b/test/vinyl/errinj.test.lua
@@ -175,24 +175,29 @@ s:drop()
 -- Wait for the dump task to complete.
 box.error.injection.set('ERRINJ_VY_RUN_WRITE_DELAY', false)
 box.snapshot()
+test_run:cmd('switch default')
+test_run:cmd("stop server test") -- don't stuck
+test_run:cmd("cleanup server test")
 
 --
 -- Check that all dump/compaction tasks that are in progress at
 -- the time when the server stops are aborted immediately.
 --
+test_run:cmd("create server double_quota with script='vinyl/low_quota.lua'")
+test_run:cmd("start server double_quota with args='2097240'")
+test_run:cmd('switch double_quota')
+fiber = require 'fiber'
+box.cfg{vinyl_timeout = 0.001}
 s = box.schema.space.create('test', {engine = 'vinyl'})
 _ = s:create_index('i1', {parts = {1, 'unsigned'}})
 _ = s:create_index('i2', {parts = {2, 'unsigned'}})
-box.error.injection.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 0.01)
+box.error.injection.set('ERRINJ_VY_DUMP_DELAY', true)
 for i = 1, 1000 do s:replace{i, i} end
 _ = fiber.create(function() box.snapshot() end)
 fiber.sleep(0.01)
 test_run:cmd('switch default')
-t1 = fiber.time()
-test_run:cmd("stop server test")
-t2 = fiber.time()
-t2 - t1 < 1
-test_run:cmd("cleanup server test")
+test_run:cmd("stop server double_quota") -- don't stuck
+test_run:cmd("cleanup server double_quota")
 
 s = box.schema.space.create('test', {engine = 'vinyl'})
 _ = s:create_index('i1', {parts = {1, 'unsigned'}})
-- 
2.17.1

             reply	other threads:[~2019-04-19  5:27 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-19  5:27 avtikhon [this message]
2019-04-29 17:35 ` Vladimir Davydov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fb7b36b0c9d93ffa7236995e1834ed4d049a6dd1.1555651506.git.avtikhon@tarantool.org \
    --to=avtikhon@tarantool.org \
    --cc=alexander.turenko@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --cc=vdavydov.dev@gmail.com \
    --subject='Re: [PATCH v2] test: vinyl/errinj test fails under highload' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox