From: "Alexander V. Tikhonov" <avtikhon@tarantool.org> To: Kirill Yukhin <kyukhin@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: [Tarantool-patches] [PATCH v1 2/4] test: create reproducer for #5141 Date: Fri, 30 Oct 2020 10:00:00 +0300 [thread overview] Message-ID: <27cf951331a1fc00371765d61ae50559f1663d53.1604040010.git.avtikhon@tarantool.org> (raw) In-Reply-To: <cover.1604040010.git.avtikhon@tarantool.org> Created the stable reproducer for the issue #5141: box.snapshot() --- -- ok +- error: 'Invalid VYLOG file: Slice <NUM> deleted but not registered' ... flaky occured in vinyl/ suite tests if running after the test: vinyl/gh-4957-too-many-upserts.test.lua as new standalone test: vinyl/gh-5141-invalid-vylog-file.test.lua based on test: vinyl/gh-4957-too-many-upserts.test.lua Due to issue not reproduced on FreeBSD 12, then test was blocked with: vinyl/gh-5141-invalid-vylog-file.skipcond Needed for #5141 --- test/vinyl/gh-5141-invalid-vylog-file.result | 140 ++++++++++++++++++ .../vinyl/gh-5141-invalid-vylog-file.skipcond | 6 + .../vinyl/gh-5141-invalid-vylog-file.test.lua | 72 +++++++++ test/vinyl/suite.ini | 4 + 4 files changed, 222 insertions(+) create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.result create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.skipcond create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.test.lua diff --git a/test/vinyl/gh-5141-invalid-vylog-file.result b/test/vinyl/gh-5141-invalid-vylog-file.result new file mode 100644 index 000000000..15caa75a9 --- /dev/null +++ b/test/vinyl/gh-5141-invalid-vylog-file.result @@ -0,0 +1,140 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \ + "to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'") + | --- + | - true + | ... + +-- restart the current server to avoid of issues from previous runs +test_run:cmd("restart server default with cleanup=True") + | + +-- Let's test number of upserts in one transaction that exceeds +-- the limit of operations allowed in one update. +-- +ups_cnt = 10000 + | --- + | ... + +s = box.schema.create_space('test', {engine = 'vinyl'}) + | --- + | ... +pk = s:create_index('pk') + | --- + | ... + +tuple = {} + | --- + | ... +for i = 1, ups_cnt do tuple[i] = i end + | --- + | ... +_ = s:insert(tuple) + | --- + | ... +box.snapshot() + | --- + | - ok + | ... + +box.begin() + | --- + | ... +for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end + | --- + | ... +box.commit() + | --- + | ... +-- Upserts are not able to squash, so scheduler will get stuck. +-- So let's not waste much time here, just check that no crash +-- takes place. +-- +box.snapshot() + | --- + | - ok + | ... +require('fiber').sleep(0.01) + | --- + | ... + +s:drop() + | --- + | ... + +-- +-- WARNING: do not split from previous subtest. +-- gh-5141: vinyl: after test with number of upserts in one transaction +-- that exceeded the limit of operations allowed in one update, then +-- box.snapshot() call may fail with error: +-- "Invalid VYLOG file: Slice <NUM> deleted but not registered". +-- To avoid of the issue tarantool server restart is needed. +-- + +-- let's check the issue reproduced +s0 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) + | --- + | ... +i0 = s0:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) + | --- + | ... +cnt = 1 + | --- + | ... +s0:replace{cnt, 'tuple'} + | --- + | - [1, 'tuple'] + | ... +test_run:wait_cond(function() \ + cnt = cnt + 1 \ + s0:replace{cnt, 'tuple ' .. cnt} \ + local ok, err = pcall(box.snapshot) \ + if ok == false then \ + require('log').info( \ + "box.snapshot() returned on loop " .. cnt .. " 'err': " .. err) \ + end \ + return err:match "Invalid VYLOG file: Slice %d+ deleted but not registered" \ +end, 10) + | --- + | - 'Invalid VYLOG file: Slice <NUM> deleted but not registered' + | ... +s0:drop() + | --- + | ... + +-- let's check the issue is stable +s1 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) + | --- + | ... +i1 = s1:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) + | --- + | ... +box.snapshot() + | --- + | - error: 'Invalid VYLOG file: Slice <NUM> deleted but not registered' + | ... +s1:drop() + | --- + | ... + +-- restart the current server to resolve the issue +test_run:cmd("restart server default with cleanup=True") + | + +-- let's check the issue resolved +s2 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) + | --- + | ... +i2 = s2:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) + | --- + | ... +box.snapshot() + | --- + | - ok + | ... +s2:drop() + | --- + | ... diff --git a/test/vinyl/gh-5141-invalid-vylog-file.skipcond b/test/vinyl/gh-5141-invalid-vylog-file.skipcond new file mode 100644 index 000000000..2d688cae1 --- /dev/null +++ b/test/vinyl/gh-5141-invalid-vylog-file.skipcond @@ -0,0 +1,6 @@ +# vim: set ft=python : +import platform + +# Disabled on FreeBSD due to issue #5141 not reproduced. +if platform.system() == 'FreeBSD': + self.skip = 1 diff --git a/test/vinyl/gh-5141-invalid-vylog-file.test.lua b/test/vinyl/gh-5141-invalid-vylog-file.test.lua new file mode 100644 index 000000000..3df4a879e --- /dev/null +++ b/test/vinyl/gh-5141-invalid-vylog-file.test.lua @@ -0,0 +1,72 @@ +test_run = require('test_run').new() +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \ + "to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'") + +-- restart the current server to avoid of issues from previous runs +test_run:cmd("restart server default with cleanup=True") + +-- Let's test number of upserts in one transaction that exceeds +-- the limit of operations allowed in one update. +-- +ups_cnt = 10000 + +s = box.schema.create_space('test', {engine = 'vinyl'}) +pk = s:create_index('pk') + +tuple = {} +for i = 1, ups_cnt do tuple[i] = i end +_ = s:insert(tuple) +box.snapshot() + +box.begin() +for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end +box.commit() +-- Upserts are not able to squash, so scheduler will get stuck. +-- So let's not waste much time here, just check that no crash +-- takes place. +-- +box.snapshot() +require('fiber').sleep(0.01) + +s:drop() + +-- +-- WARNING: do not split from previous subtest. +-- gh-5141: vinyl: after test with number of upserts in one transaction +-- that exceeded the limit of operations allowed in one update, then +-- box.snapshot() call may fail with error: +-- "Invalid VYLOG file: Slice <NUM> deleted but not registered". +-- To avoid of the issue tarantool server restart is needed. +-- + +-- let's check the issue reproduced +s0 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) +i0 = s0:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) +cnt = 1 +s0:replace{cnt, 'tuple'} +test_run:wait_cond(function() \ + cnt = cnt + 1 \ + s0:replace{cnt, 'tuple ' .. cnt} \ + local ok, err = pcall(box.snapshot) \ + if ok == false then \ + require('log').info( \ + "box.snapshot() returned on loop " .. cnt .. " 'err': " .. err) \ + end \ + return err:match "Invalid VYLOG file: Slice %d+ deleted but not registered" \ +end, 10) +s0:drop() + +-- let's check the issue is stable +s1 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) +i1 = s1:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) +box.snapshot() +s1:drop() + +-- restart the current server to resolve the issue +test_run:cmd("restart server default with cleanup=True") + +-- let's check the issue resolved +s2 = box.schema.space.create('tweedledum', {engine = 'vinyl'}) +i2 = s2:create_index('primary', { type = 'tree', parts = {1, 'unsigned'}}) +box.snapshot() +s2:drop() diff --git a/test/vinyl/suite.ini b/test/vinyl/suite.ini index 3ba29f0c6..2d69878af 100644 --- a/test/vinyl/suite.ini +++ b/test/vinyl/suite.ini @@ -55,6 +55,10 @@ fragile = { "issues": [ "gh-5141" ], "checksums": [ "f1286e9e4710062ddfbffb61b2fe2743", "96b22440ab8a881d6b8d14c5ee1672fb" ] }, + "gh-5141-invalid-vylog-file.test.lua": { + "issues": [ "gh-5141" ], + "checksums": [ "1f4d4261fc63291c4020372986c62e2e" ] + }, "iterator.test.lua": { "issues": [ "gh-5336" ], "checksums": [ "f56d6d279692e46d84a06f470af0090d", "a8a2824fb09380330a70a589d4fc545e", "e03548bcb918e824c349dc4c52ae3715", "adbd4ad9878302d570e40aef9a6b92cc", "39ee43ee7b2004166ca54402dfe02238" ] -- 2.25.1
next prev parent reply other threads:[~2020-10-30 7:00 UTC|newest] Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-30 6:59 [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141 Alexander V. Tikhonov 2020-10-30 6:59 ` [Tarantool-patches] [PATCH v1 1/4] test: add test filter for box.snapshot Alexander V. Tikhonov 2020-10-30 7:00 ` Alexander V. Tikhonov [this message] 2020-10-30 7:00 ` [Tarantool-patches] [PATCH v1 3/4] test: fix flaky vinyl/gh-4957-too-many-upserts Alexander V. Tikhonov 2020-10-30 7:00 ` [Tarantool-patches] [PATCH v1 4/4] test: fix hanging of vinyl/gh.test.lua Alexander V. Tikhonov 2020-11-01 8:53 ` [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141 Kirill Yukhin 2020-11-01 21:57 ` Nikita Pettik 2020-11-02 7:33 ` Alexander V. Tikhonov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=27cf951331a1fc00371765d61ae50559f1663d53.1604040010.git.avtikhon@tarantool.org \ --to=avtikhon@tarantool.org \ --cc=kyukhin@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v1 2/4] test: create reproducer for #5141' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox