From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp16.mail.ru (smtp16.mail.ru [94.100.176.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id E6FF7469719 for ; Sun, 1 Nov 2020 11:53:15 +0300 (MSK) Date: Sun, 1 Nov 2020 08:53:15 +0000 From: Kirill Yukhin Message-ID: <20201101085315.qapjrnkjqaoryar5@tarantool.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH v1 0/4] test: stabilize testing with issue #5141 List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Alexander V. Tikhonov" Cc: tarantool-patches@dev.tarantool.org Hello, On 30 окт 09:59, Alexander V. Tikhonov wrote: > To stabilize testing with issue #5141 the following steps had to be > done: > > - Prepare tests - add test-run filter on box.snapshot error message: > > 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' > > to avoid of printing changing data in results file to be able to use > its checksums in fragile list of test-run to rerun it as flaky issue. > > - Create the stable reproducer for the issue #5141 as new standalone > test vinyl/gh-5141-invalid-vylog-file.test.lua. > > - Fix flaky issue #5141 in vinyl/gh-4957-too-many-upserts.test.lua. > > - Fix hanging of vinyl/gh.test.lua. Revert previous fix for #5141 in > vinyl/gh.test.lua. > > To implement it created 4 patches (in reverse order): > > 4. test: fix hanging of vinyl/gh.test.lua > > Found that the previously fixed vinyl/gh.test.lua test in commit: > > 94dc5bddc1fb0f9e6ebebfc5aa6be586e5b6759e ('test: gh test hangs after gh-4957-too-many-upserts') > > with adding fiber.sleep(1) workaround to avoid of raise from the > previously run vinyl/gh-4957-too-many-upserts.test.lua test can be > changed in the other way. The new change from one side will leave > the found issue untouched to be able to resolve it within opened > issue in github. And from the other side it will let the test-run > tool to be able to avoid of this issue using fragile list feature > to save the stability of testing due to found issue is flaky and > can be passed on reruns. > > The current fix changes the forever waiting loop to especially > created for such situations test_run:wait_cond() routine which has > timeout in it to avoid of hanging the test till global timeout will > occure. It will let the testing to be continued even after the fail. > > Needed for #5141 > > 3. test: fix flaky vinyl/gh-4957-too-many-upserts > > Added restart the current server to resolve the issue #5141 which > reproduced in test: > > vinyl/gh-5141-invalid-vylog-file.test.lua > > Added test-run filter on box.snapshot error message: > > 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' > > to avoid of printing changing data in results file to be able to use > its checksums in fragile list of test-run to rerun it as flaky issue. > > Part of #5141 > > 2. test: create reproducer for #5141 > > Created the stable reproducer for the issue #5141: > > box.snapshot() > --- > -- ok > +- error: 'Invalid VYLOG file: Slice deleted but not registered' > ... > > flaky occured in vinyl/ suite tests if running after the test: > > vinyl/gh-4957-too-many-upserts.test.lua > > as new standalone test: > > vinyl/gh-5141-invalid-vylog-file.test.lua > > based on test: > > vinyl/gh-4957-too-many-upserts.test.lua > > Due to issue not reproduced on FreeBSD 12, then test was blocked with: > > vinyl/gh-5141-invalid-vylog-file.skipcond > > Needed for #5141 > > 1. test: add test filter for box.snapshot > > Added test-run filter on box.snapshot error message: > > 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered' > > to avoid of printing changing data in results file to be able to use > its checksums in fragile list of test-run to rerun it as flaky issue. > Also added checksums to fragile list for the following tests: > > vinyl/iterator.test.lua gh-5141 > vinyl/snapshot.test.lua gh-4984 > > Needed for #5141 > Needed for #4984 > > Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-5141-gh_test > Issue: https://github.com/tarantool/tarantool/issues/5141 LGTM. I've checked your patchset into 1.10, 2.5, 2.6 and master. -- Regards, Kirill Yukhin