* [PATCH] test: fix vinyl/upgrade/fill.lua script
@ 2018-06-13 9:44 Vladimir Davydov
2018-06-13 17:39 ` Konstantin Osipov
0 siblings, 1 reply; 3+ messages in thread
From: Vladimir Davydov @ 2018-06-13 9:44 UTC (permalink / raw)
To: kostja; +Cc: tarantool-patches
Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
not fit in memory"), vinyl won't trigger memory dump if the size of
memory needed by a transaction is greater than the memory limit, instead
it will fail the transaction immediately. This broke the aforementioned
script, which relied on this to trigger system-wide memory dump. Fix it
by reworking the dump trigger logic used by the script: now it tries to
insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.
Closes #3449
---
https://github.com/tarantool/tarantool/issues/3449
https://github.com/tarantool/tarantool/commits/gh-3449-fix-vinyl-upgrade-test-script
test/vinyl/upgrade/fill.lua | 24 +++++++++++++++++-------
1 file changed, 17 insertions(+), 7 deletions(-)
diff --git a/test/vinyl/upgrade/fill.lua b/test/vinyl/upgrade/fill.lua
index 5271a294..dc633b68 100644
--- a/test/vinyl/upgrade/fill.lua
+++ b/test/vinyl/upgrade/fill.lua
@@ -4,22 +4,32 @@
--
fiber = require 'fiber'
-box.cfg{vinyl_memory = 1024 * 1024, vinyl_timeout = 0.1, checkpoint_count = 1}
+box.cfg{vinyl_memory = 1024 * 1024, vinyl_timeout = 1e-9, checkpoint_count = 1}
dump_trigger = box.schema.space.create('dump_trigger', {engine = 'vinyl'})
-dump_trigger:create_index('pk')
+dump_trigger:create_index('pk', {run_count_per_level = 1})
-- Trigger dump of all indexes and wait for it to finish.
--
--- We trigger dump by attempting to insert a huge (> memory limit)
--- tuple into a vinyl memory space. Before failing on timeout this
--- makes the scheduler force dump.
+-- On hitting memory limit, vinyl dumps all existing spaces, so
+-- to trigger system-wide memory dump, it is enough to insert a
+-- huge tuple into one space.
+--
function dump()
- pcall(dump_trigger.insert, dump_trigger,
- {1, string.rep('x', 1024 * 1024)})
+ local pad = string.rep('x', box.cfg.vinyl_memory / 2)
+ dump_trigger:replace{1, pad}
+ -- Must fail due to quota timeout, but still trigger dump.
+ if pcall(dump_trigger.replace, dump_trigger, {1, pad}) then
+ assert(false)
+ end
+ -- Wait for dump to complete.
while box.stat.vinyl().quota.used > 0 do
fiber.sleep(0.1)
end
+ -- Wait for compaction to collect garbage.
+ while dump_trigger.index.pk:stat().run_count > 1 do
+ fiber.sleep(0.1)
+ end
end
--
--
2.11.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] test: fix vinyl/upgrade/fill.lua script
2018-06-13 9:44 [PATCH] test: fix vinyl/upgrade/fill.lua script Vladimir Davydov
@ 2018-06-13 17:39 ` Konstantin Osipov
2018-06-14 8:21 ` Vladimir Davydov
0 siblings, 1 reply; 3+ messages in thread
From: Konstantin Osipov @ 2018-06-13 17:39 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: tarantool-patches
* Vladimir Davydov <vdavydov.dev@gmail.com> [18/06/13 13:42]:
> Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
> not fit in memory"), vinyl won't trigger memory dump if the size of
> memory needed by a transaction is greater than the memory limit, instead
> it will fail the transaction immediately. This broke the aforementioned
> script, which relied on this to trigger system-wide memory dump. Fix it
> by reworking the dump trigger logic used by the script: now it tries to
> insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.
>
> Closes #3449
> while box.stat.vinyl().quota.used > 0 do
> fiber.sleep(0.1)
Let's settle on 0.01 as standard fiber-loop-wait time in tests.
Otherwise it's ok to push.
> end
> + -- Wait for compaction to collect garbage.
> + while dump_trigger.index.pk:stat().run_count > 1 do
> + fiber.sleep(0.1)
> + end
> end
>
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] test: fix vinyl/upgrade/fill.lua script
2018-06-13 17:39 ` Konstantin Osipov
@ 2018-06-14 8:21 ` Vladimir Davydov
0 siblings, 0 replies; 3+ messages in thread
From: Vladimir Davydov @ 2018-06-14 8:21 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: tarantool-patches
On Wed, Jun 13, 2018 at 08:39:12PM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [18/06/13 13:42]:
> > Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
> > not fit in memory"), vinyl won't trigger memory dump if the size of
> > memory needed by a transaction is greater than the memory limit, instead
> > it will fail the transaction immediately. This broke the aforementioned
> > script, which relied on this to trigger system-wide memory dump. Fix it
> > by reworking the dump trigger logic used by the script: now it tries to
> > insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.
> >
> > Closes #3449
>
>
> > while box.stat.vinyl().quota.used > 0 do
> > fiber.sleep(0.1)
>
> Let's settle on 0.01 as standard fiber-loop-wait time in tests.
>
> Otherwise it's ok to push.
Fixed and pushed to 1.10
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-06-14 8:21 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-13 9:44 [PATCH] test: fix vinyl/upgrade/fill.lua script Vladimir Davydov
2018-06-13 17:39 ` Konstantin Osipov
2018-06-14 8:21 ` Vladimir Davydov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox