[PATCH] test: fix vinyl/upgrade/fill.lua script

[PATCH] test: fix vinyl/upgrade/fill.lua script

[Vladimir Davydov]

Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
not fit in memory"), vinyl won't trigger memory dump if the size of
memory needed by a transaction is greater than the memory limit, instead
it will fail the transaction immediately. This broke the aforementioned
script, which relied on this to trigger system-wide memory dump. Fix it
by reworking the dump trigger logic used by the script: now it tries to
insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.

Closes #3449
---
https://github.com/tarantool/tarantool/issues/3449
https://github.com/tarantool/tarantool/commits/gh-3449-fix-vinyl-upgrade-test-script

 test/vinyl/upgrade/fill.lua | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/test/vinyl/upgrade/fill.lua b/test/vinyl/upgrade/fill.lua
index 5271a294..dc633b68 100644
--- a/test/vinyl/upgrade/fill.lua
+++ b/test/vinyl/upgrade/fill.lua
@@ -4,22 +4,32 @@
 --
 fiber = require 'fiber'
 
-box.cfg{vinyl_memory = 1024 * 1024, vinyl_timeout = 0.1, checkpoint_count = 1}
+box.cfg{vinyl_memory = 1024 * 1024, vinyl_timeout = 1e-9, checkpoint_count = 1}
 
 dump_trigger = box.schema.space.create('dump_trigger', {engine = 'vinyl'})
-dump_trigger:create_index('pk')
+dump_trigger:create_index('pk', {run_count_per_level = 1})
 
 -- Trigger dump of all indexes and wait for it to finish.
 --
--- We trigger dump by attempting to insert a huge (> memory limit)
--- tuple into a vinyl memory space. Before failing on timeout this
--- makes the scheduler force dump.
+-- On hitting memory limit, vinyl dumps all existing spaces, so
+-- to trigger system-wide memory dump, it is enough to insert a
+-- huge tuple into one space.
+--
 function dump()
-    pcall(dump_trigger.insert, dump_trigger,
-          {1, string.rep('x', 1024 * 1024)})
+    local pad = string.rep('x', box.cfg.vinyl_memory / 2)
+    dump_trigger:replace{1, pad}
+    -- Must fail due to quota timeout, but still trigger dump.
+    if pcall(dump_trigger.replace, dump_trigger, {1, pad}) then
+        assert(false)
+    end
+    -- Wait for dump to complete.
     while box.stat.vinyl().quota.used > 0 do
         fiber.sleep(0.1)
     end
+    -- Wait for compaction to collect garbage.
+    while dump_trigger.index.pk:stat().run_count > 1 do
+        fiber.sleep(0.1)
+    end
 end
 
 --
-- 
2.11.0

Re: [PATCH] test: fix vinyl/upgrade/fill.lua script

[Konstantin Osipov]

* Vladimir Davydov <vdavydov.dev@gmail.com> [18/06/13 13:42]:
> Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
> not fit in memory"), vinyl won't trigger memory dump if the size of
> memory needed by a transaction is greater than the memory limit, instead
> it will fail the transaction immediately. This broke the aforementioned
> script, which relied on this to trigger system-wide memory dump. Fix it
> by reworking the dump trigger logic used by the script: now it tries to
> insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.
> 
> Closes #3449


>      while box.stat.vinyl().quota.used > 0 do
>          fiber.sleep(0.1)

Let's settle on 0.01 as standard fiber-loop-wait time in tests.

Otherwise it's ok to push.

>      end
> +    -- Wait for compaction to collect garbage.
> +    while dump_trigger.index.pk:stat().run_count > 1 do
> +        fiber.sleep(0.1)
> +    end
>  end
>  

-- 
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov

Re: [PATCH] test: fix vinyl/upgrade/fill.lua script

[Vladimir Davydov]

On Wed, Jun 13, 2018 at 08:39:12PM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [18/06/13 13:42]:
> > Since commit 8f63d5d90 ("vinyl: fail transaction immediately if it does
> > not fit in memory"), vinyl won't trigger memory dump if the size of
> > memory needed by a transaction is greater than the memory limit, instead
> > it will fail the transaction immediately. This broke the aforementioned
> > script, which relied on this to trigger system-wide memory dump. Fix it
> > by reworking the dump trigger logic used by the script: now it tries to
> > insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one.
> > 
> > Closes #3449
> 
> 
> >      while box.stat.vinyl().quota.used > 0 do
> >          fiber.sleep(0.1)
> 
> Let's settle on 0.01 as standard fiber-loop-wait time in tests.
> 
> Otherwise it's ok to push.

Fixed and pushed to 1.10