From: "Alexander V. Tikhonov" <avtikhon@tarantool.org> To: Kirill Yukhin <kyukhin@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141 Date: Fri, 30 Oct 2020 09:59:58 +0300 [thread overview] Message-ID: <cover.1604040010.git.avtikhon@tarantool.org> (raw) To stabilize testing with issue #5141 the following steps had to be done: - Prepare tests - add test-run filter on box.snapshot error message: 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' to avoid of printing changing data in results file to be able to use its checksums in fragile list of test-run to rerun it as flaky issue. - Create the stable reproducer for the issue #5141 as new standalone test vinyl/gh-5141-invalid-vylog-file.test.lua. - Fix flaky issue #5141 in vinyl/gh-4957-too-many-upserts.test.lua. - Fix hanging of vinyl/gh.test.lua. Revert previous fix for #5141 in vinyl/gh.test.lua. To implement it created 4 patches (in reverse order): 4. test: fix hanging of vinyl/gh.test.lua Found that the previously fixed vinyl/gh.test.lua test in commit: 94dc5bddc1fb0f9e6ebebfc5aa6be586e5b6759e ('test: gh test hangs after gh-4957-too-many-upserts') with adding fiber.sleep(1) workaround to avoid of raise from the previously run vinyl/gh-4957-too-many-upserts.test.lua test can be changed in the other way. The new change from one side will leave the found issue untouched to be able to resolve it within opened issue in github. And from the other side it will let the test-run tool to be able to avoid of this issue using fragile list feature to save the stability of testing due to found issue is flaky and can be passed on reruns. The current fix changes the forever waiting loop to especially created for such situations test_run:wait_cond() routine which has timeout in it to avoid of hanging the test till global timeout will occure. It will let the testing to be continued even after the fail. Needed for #5141 3. test: fix flaky vinyl/gh-4957-too-many-upserts Added restart the current server to resolve the issue #5141 which reproduced in test: vinyl/gh-5141-invalid-vylog-file.test.lua Added test-run filter on box.snapshot error message: 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' to avoid of printing changing data in results file to be able to use its checksums in fragile list of test-run to rerun it as flaky issue. Part of #5141 2. test: create reproducer for #5141 Created the stable reproducer for the issue #5141: box.snapshot() --- -- ok +- error: 'Invalid VYLOG file: Slice <NUM> deleted but not registered' ... flaky occured in vinyl/ suite tests if running after the test: vinyl/gh-4957-too-many-upserts.test.lua as new standalone test: vinyl/gh-5141-invalid-vylog-file.test.lua based on test: vinyl/gh-4957-too-many-upserts.test.lua Due to issue not reproduced on FreeBSD 12, then test was blocked with: vinyl/gh-5141-invalid-vylog-file.skipcond Needed for #5141 1. test: add test filter for box.snapshot Added test-run filter on box.snapshot error message: 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' to avoid of printing changing data in results file to be able to use its checksums in fragile list of test-run to rerun it as flaky issue. Also added checksums to fragile list for the following tests: vinyl/iterator.test.lua gh-5141 vinyl/snapshot.test.lua gh-4984 Needed for #5141 Needed for #4984 Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-5141-gh_test Issue: https://github.com/tarantool/tarantool/issues/5141 Alexander V. Tikhonov (4): test: add test filter for box.snapshot test: create reproducer for #5141 test: fix flaky vinyl/gh-4957-too-many-upserts test: fix hanging of vinyl/gh.test.lua test/vinyl/gh-4957-too-many-upserts.result | 17 +++ test/vinyl/gh-4957-too-many-upserts.test.lua | 11 ++ test/vinyl/gh-5141-invalid-vylog-file.result | 140 ++++++++++++++++++ .../vinyl/gh-5141-invalid-vylog-file.skipcond | 6 + .../vinyl/gh-5141-invalid-vylog-file.test.lua | 72 +++++++++ test/vinyl/gh.result | 6 +- test/vinyl/gh.test.lua | 4 +- test/vinyl/iterator.result | 5 + test/vinyl/iterator.test.lua | 3 + test/vinyl/snapshot.result | 5 + test/vinyl/snapshot.test.lua | 3 + test/vinyl/suite.ini | 10 +- 12 files changed, 272 insertions(+), 10 deletions(-) create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.result create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.skipcond create mode 100644 test/vinyl/gh-5141-invalid-vylog-file.test.lua -- 2.25.1
next reply other threads:[~2020-10-30 7:00 UTC|newest] Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-30 6:59 Alexander V. Tikhonov [this message] 2020-10-30 6:59 ` [Tarantool-patches] [PATCH v1 1/4] test: add test filter for box.snapshot Alexander V. Tikhonov 2020-10-30 7:00 ` [Tarantool-patches] [PATCH v1 2/4] test: create reproducer for #5141 Alexander V. Tikhonov 2020-10-30 7:00 ` [Tarantool-patches] [PATCH v1 3/4] test: fix flaky vinyl/gh-4957-too-many-upserts Alexander V. Tikhonov 2020-10-30 7:00 ` [Tarantool-patches] [PATCH v1 4/4] test: fix hanging of vinyl/gh.test.lua Alexander V. Tikhonov 2020-11-01 8:53 ` [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141 Kirill Yukhin 2020-11-01 21:57 ` Nikita Pettik 2020-11-02 7:33 ` Alexander V. Tikhonov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cover.1604040010.git.avtikhon@tarantool.org \ --to=avtikhon@tarantool.org \ --cc=kyukhin@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox