From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: avtikhon Subject: [PATCH v2] test: vinyl/errinj test fails under highload Date: Fri, 19 Apr 2019 08:27:20 +0300 Message-Id: To: Vladimir Davydov , Alexander Turenko Cc: avtikhon , tarantool-patches@freelists.org List-ID: Test "check that all dump/compaction tasks that are in progress at the time when the server stops are aborted immediately.", but in real the awaiting time of 1 second is not enough due to runs in parallel and it fails, like: [009] --- vinyl/errinj.result Tue Apr 16 16:43:36 2019 [009] +++ vinyl/errinj.reject Wed Apr 17 09:42:36 2019 [009] @@ -530,7 +530,7 @@ [009] ... [009] t2 - t1 < 1 [009] --- [009] -- true [009] +- false [009] ... [009] test_run:cmd("cleanup server test") [009] --- [009] in 100 parallel runs the failed delays were found: [002] +- 1.4104716777802 [022] +- 1.3933029174805 [044] +- 1.4296517372131 [033] +- 1.6380662918091 [001] +- 1.9799520969391 [027] +- 1.7067711353302 [043] +- 1.3778221607208 [034] +- 1.3820221424103 [032] +- 1.3820221424103 [020] +- 1.6275615692139 [050] +- 1.6275615692139 [048] +- 1.1880359649658 Decided to avoid of use the time check at all and change the ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT to ERRINJ_VY_DUMP_DELAY injection. In this way the time checks were completely removed. Next issue met was the error: vy_quota.c:298 !> SystemError Failed to allocate 2097240 bytes in lsregion for vinyl transaction: Cannot allocate memory That is why the merged 2 subtests were divided into 2 standalone subtests to be able to set the memory limit of the 2nd subtest to 2097240 value. Close #4169 --- Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4169-vinyl-errinj-highload Issue: https://github.com/tarantool/tarantool/issues/4169 test/vinyl/errinj.result | 46 +++++++++++++++++++++++++++----------- test/vinyl/errinj.test.lua | 17 +++++++++----- 2 files changed, 44 insertions(+), 19 deletions(-) diff --git a/test/vinyl/errinj.result b/test/vinyl/errinj.result index 6fc4d9e5d..c69347da6 100644 --- a/test/vinyl/errinj.result +++ b/test/vinyl/errinj.result @@ -488,10 +488,40 @@ box.snapshot() --- - ok ... +test_run:cmd('switch default') +--- +- true +... +test_run:cmd("stop server test") -- don't stuck +--- +- true +... +test_run:cmd("cleanup server test") +--- +- true +... -- -- Check that all dump/compaction tasks that are in progress at -- the time when the server stops are aborted immediately. -- +test_run:cmd("create server double_quota with script='vinyl/low_quota.lua'") +--- +- true +... +test_run:cmd("start server double_quota with args='2097240'") +--- +- true +... +test_run:cmd('switch double_quota') +--- +- true +... +fiber = require 'fiber' +--- +... +box.cfg{vinyl_timeout = 0.001} +--- +... s = box.schema.space.create('test', {engine = 'vinyl'}) --- ... @@ -501,7 +531,7 @@ _ = s:create_index('i1', {parts = {1, 'unsigned'}}) _ = s:create_index('i2', {parts = {2, 'unsigned'}}) --- ... -box.error.injection.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 0.01) +box.error.injection.set('ERRINJ_VY_DUMP_DELAY', true) --- - ok ... @@ -518,21 +548,11 @@ test_run:cmd('switch default') --- - true ... -t1 = fiber.time() ---- -... -test_run:cmd("stop server test") ---- -- true -... -t2 = fiber.time() ---- -... -t2 - t1 < 1 +test_run:cmd("stop server double_quota") -- don't stuck --- - true ... -test_run:cmd("cleanup server test") +test_run:cmd("cleanup server double_quota") --- - true ... diff --git a/test/vinyl/errinj.test.lua b/test/vinyl/errinj.test.lua index b500b33c6..c3ce9706f 100644 --- a/test/vinyl/errinj.test.lua +++ b/test/vinyl/errinj.test.lua @@ -175,24 +175,29 @@ s:drop() -- Wait for the dump task to complete. box.error.injection.set('ERRINJ_VY_RUN_WRITE_DELAY', false) box.snapshot() +test_run:cmd('switch default') +test_run:cmd("stop server test") -- don't stuck +test_run:cmd("cleanup server test") -- -- Check that all dump/compaction tasks that are in progress at -- the time when the server stops are aborted immediately. -- +test_run:cmd("create server double_quota with script='vinyl/low_quota.lua'") +test_run:cmd("start server double_quota with args='2097240'") +test_run:cmd('switch double_quota') +fiber = require 'fiber' +box.cfg{vinyl_timeout = 0.001} s = box.schema.space.create('test', {engine = 'vinyl'}) _ = s:create_index('i1', {parts = {1, 'unsigned'}}) _ = s:create_index('i2', {parts = {2, 'unsigned'}}) -box.error.injection.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 0.01) +box.error.injection.set('ERRINJ_VY_DUMP_DELAY', true) for i = 1, 1000 do s:replace{i, i} end _ = fiber.create(function() box.snapshot() end) fiber.sleep(0.01) test_run:cmd('switch default') -t1 = fiber.time() -test_run:cmd("stop server test") -t2 = fiber.time() -t2 - t1 < 1 -test_run:cmd("cleanup server test") +test_run:cmd("stop server double_quota") -- don't stuck +test_run:cmd("cleanup server double_quota") s = box.schema.space.create('test', {engine = 'vinyl'}) _ = s:create_index('i1', {parts = {1, 'unsigned'}}) -- 2.17.1