Re: [Tarantool-patches] [PATCH v1] test: add test filter for vinyl tests

["Alexander V. Tikhonov" <avtikhon@tarantool.org>]

Hi All, also force pushed new checksums for the tests after its update.

On Wed, Dec 16, 2020 at 11:07:39AM +0300, Alexander V. Tikhonov wrote:
> Cc: tarantool-patches@dev.tarantool.org
> 
> Added test-run filter on box.snapshot error message:
> 
>   'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'
> 
> to avoid of printing changing data in results file to be able to use
> its checksums in fragile list of test-run to rerun it as flaky issue.
> 
> Found issues:
> 
>  1) vinyl/deferred_delete.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/913623306#L4552
> 
>   [036] 2020-12-15 19:10:01.996 [16602] coio vy_log.c:2202 E> failed to process vylog record: delete_slice{slice_id=744, }
>   [036] 2020-12-15 19:10:01.996 [16602] main/103/vinyl vy_log.c:2068 E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 744 deleted but not registered
> 
>  2) vinyl/gh-4864-stmt-alloc-fail-compact.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/913810422#L4835
> 
>   [052] @@ -56,9 +56,11 @@
>   [052]  --
>   [052]  dump(true)
>   [052]   | ---
>   [052] - | ...
>   [052] -dump()
>   [052] - | ---
>   [052] + | - error: 'Invalid VYLOG file: Slice 253 deleted but not registered'
>   [052] + | ...
> 
>  3) vinyl/misc.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/913727925#L5284
> 
>   [014] @@ -62,14 +62,14 @@
>   [014]  ...
>   [014]  box.snapshot()
>   [014]  ---
>   [014] -- ok
>   [014] +- error: 'Invalid VYLOG file: Slice 1141 deleted but not registered'
>   [014]  ...
> 
>  4) vinyl/quota.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/914016074#L4595
> 
>   [025] 2020-12-15 22:56:50.192 [25576] coio vy_log.c:2202 E> failed to process vylog record: delete_slice{slice_id=522, }
>   [025] 2020-12-15 22:56:50.193 [25576] main/103/vinyl vy_log.c:2068 E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 522 deleted but not registered
> 
>  5) vinyl/update_optimize.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/913728098#L2512
> 
>   [051] 2020-12-15 20:18:43.365 [17147] coio vy_log.c:2202 E> failed to process vylog record: delete_slice{slice_id=350, }
>   [051] 2020-12-15 20:18:43.365 [17147] main/103/vinyl vy_log.c:2068 E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 350 deleted but not registered
> 
>  6) vinyl/upsert.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/913623510#L6132
> 
>   [008] @@ -441,7 +441,7 @@
>   [008]  -- Mem has DELETE
>   [008]  box.snapshot()
>   [008]  ---
>   [008] -- ok
>   [008] +- error: 'Invalid VYLOG file: Slice 1411 deleted but not registered'
>   [008]  ...
> 
>  7) vinyl/replica_quota.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/914272656#L5739
> 
>   [023] @@ -41,7 +41,7 @@
>   [023]  ...
>   [023]  box.snapshot()
>   [023]  ---
>   [023] -- ok
>   [023] +- error: 'Invalid VYLOG file: Slice 232 deleted but not registered'
>   [023]  ...
> 
>  8) vinyl/ddl.test.lua
>     https://gitlab.com/tarantool/tarantool/-/jobs/914309343#L4538
> 
>   [039] @@ -81,7 +81,7 @@
>   [039]  ...
>   [039]  box.snapshot()
>   [039]  ---
>   [039] -- ok
>   [039] +- error: 'Invalid VYLOG file: Slice 206 deleted but not registered'
>   [039]  ...
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/vinyl-filter
> 
>  test/vinyl/ddl.result                               | 5 +++++
>  test/vinyl/ddl.test.lua                             | 3 +++
>  test/vinyl/deferred_delete.result                   | 5 +++++
>  test/vinyl/deferred_delete.test.lua                 | 3 +++
>  test/vinyl/gh-4864-stmt-alloc-fail-compact.result   | 6 ++++++
>  test/vinyl/gh-4864-stmt-alloc-fail-compact.test.lua | 3 +++
>  test/vinyl/misc.result                              | 5 +++++
>  test/vinyl/misc.test.lua                            | 3 +++
>  test/vinyl/quota.result                             | 5 +++++
>  test/vinyl/quota.test.lua                           | 4 ++++
>  test/vinyl/replica_quota.result                     | 5 +++++
>  test/vinyl/replica_quota.test.lua                   | 3 +++
>  test/vinyl/update_optimize.result                   | 5 +++++
>  test/vinyl/update_optimize.test.lua                 | 3 +++
>  test/vinyl/upsert.result                            | 5 +++++
>  test/vinyl/upsert.test.lua                          | 3 +++
>  16 files changed, 66 insertions(+)
> 
> diff --git a/test/vinyl/ddl.result b/test/vinyl/ddl.result
> index 9c453a007..00d76d713 100644
> --- a/test/vinyl/ddl.result
> +++ b/test/vinyl/ddl.result
> @@ -4,6 +4,11 @@ fiber = require('fiber')
>  test_run = require('test_run').new()
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  -- sanity checks
>  space = box.schema.space.create('test', {engine = 'vinyl' })
>  ---
> diff --git a/test/vinyl/ddl.test.lua b/test/vinyl/ddl.test.lua
> index 010ec6d79..6be674638 100644
> --- a/test/vinyl/ddl.test.lua
> +++ b/test/vinyl/ddl.test.lua
> @@ -1,6 +1,9 @@
>  fiber = require('fiber')
>  test_run = require('test_run').new()
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  -- sanity checks
>  space = box.schema.space.create('test', {engine = 'vinyl' })
>  space:create_index('pk', {page_size = 0})
> diff --git a/test/vinyl/deferred_delete.result b/test/vinyl/deferred_delete.result
> index 23c93f0f5..5553a359b 100644
> --- a/test/vinyl/deferred_delete.result
> +++ b/test/vinyl/deferred_delete.result
> @@ -4,6 +4,11 @@ test_run = require('test_run').new()
>  fiber = require('fiber')
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  --
>  -- Create a space with secondary indexes and check that REPLACE and
>  -- DELETE requests do not look up the old tuple in the primary index
> diff --git a/test/vinyl/deferred_delete.test.lua b/test/vinyl/deferred_delete.test.lua
> index 1bce954c5..b1f464adf 100644
> --- a/test/vinyl/deferred_delete.test.lua
> +++ b/test/vinyl/deferred_delete.test.lua
> @@ -1,6 +1,9 @@
>  test_run = require('test_run').new()
>  fiber = require('fiber')
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  --
>  -- Create a space with secondary indexes and check that REPLACE and
>  -- DELETE requests do not look up the old tuple in the primary index
> diff --git a/test/vinyl/gh-4864-stmt-alloc-fail-compact.result b/test/vinyl/gh-4864-stmt-alloc-fail-compact.result
> index 6a1f49626..93c4c80e4 100644
> --- a/test/vinyl/gh-4864-stmt-alloc-fail-compact.result
> +++ b/test/vinyl/gh-4864-stmt-alloc-fail-compact.result
> @@ -9,6 +9,12 @@ digest = require('digest')
>   | ---
>   | ...
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> + | ---
> + | - true
> + | ...
> +
>  s = box.schema.space.create('test', {engine = 'vinyl'})
>   | ---
>   | ...
> diff --git a/test/vinyl/gh-4864-stmt-alloc-fail-compact.test.lua b/test/vinyl/gh-4864-stmt-alloc-fail-compact.test.lua
> index 4b3c55505..1482abecf 100644
> --- a/test/vinyl/gh-4864-stmt-alloc-fail-compact.test.lua
> +++ b/test/vinyl/gh-4864-stmt-alloc-fail-compact.test.lua
> @@ -2,6 +2,9 @@ test_run = require('test_run').new()
>  fiber = require('fiber')
>  digest = require('digest')
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  s = box.schema.space.create('test', {engine = 'vinyl'})
>  _ = s:create_index('pk', {run_count_per_level = 100, page_size = 128, range_size = 1024})
>  
> diff --git a/test/vinyl/misc.result b/test/vinyl/misc.result
> index e647b93c3..15d154854 100644
> --- a/test/vinyl/misc.result
> +++ b/test/vinyl/misc.result
> @@ -4,6 +4,11 @@ test_run = require('test_run').new()
>  fiber = require('fiber')
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  --
>  -- gh-2784: do not validate space formatted but not indexed fields
>  -- in surrogate statements.
> diff --git a/test/vinyl/misc.test.lua b/test/vinyl/misc.test.lua
> index 0a7c7fc99..66bf88f36 100644
> --- a/test/vinyl/misc.test.lua
> +++ b/test/vinyl/misc.test.lua
> @@ -1,6 +1,9 @@
>  test_run = require('test_run').new()
>  fiber = require('fiber')
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  --
>  -- gh-2784: do not validate space formatted but not indexed fields
>  -- in surrogate statements.
> diff --git a/test/vinyl/quota.result b/test/vinyl/quota.result
> index 940df4e49..70e81453d 100644
> --- a/test/vinyl/quota.result
> +++ b/test/vinyl/quota.result
> @@ -4,6 +4,11 @@ env = require('test_run')
>  test_run = env.new()
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  --
>  -- Restart the server because need to reset quota used memory
>  -- after previous tests.
> diff --git a/test/vinyl/quota.test.lua b/test/vinyl/quota.test.lua
> index a2793a015..081c573f3 100644
> --- a/test/vinyl/quota.test.lua
> +++ b/test/vinyl/quota.test.lua
> @@ -1,6 +1,10 @@
>  env = require('test_run')
>  test_run = env.new()
>  
> +
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  --
>  -- Restart the server because need to reset quota used memory
>  -- after previous tests.
> diff --git a/test/vinyl/replica_quota.result b/test/vinyl/replica_quota.result
> index bd09e764d..f7ab2482d 100644
> --- a/test/vinyl/replica_quota.result
> +++ b/test/vinyl/replica_quota.result
> @@ -1,6 +1,11 @@
>  test_run = require('test_run').new()
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  box.schema.user.grant('guest', 'replication')
>  ---
>  ...
> diff --git a/test/vinyl/replica_quota.test.lua b/test/vinyl/replica_quota.test.lua
> index 1f373fd47..b471f623e 100644
> --- a/test/vinyl/replica_quota.test.lua
> +++ b/test/vinyl/replica_quota.test.lua
> @@ -1,5 +1,8 @@
>  test_run = require('test_run').new()
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  box.schema.user.grant('guest', 'replication')
>  
>  s = box.schema.space.create('test', { engine = 'vinyl' })
> diff --git a/test/vinyl/update_optimize.result b/test/vinyl/update_optimize.result
> index 09370e7d5..01ff7e043 100644
> --- a/test/vinyl/update_optimize.result
> +++ b/test/vinyl/update_optimize.result
> @@ -6,6 +6,11 @@ test_run:cmd('restart server default')
>  fiber = require('fiber')
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  -- optimize one index
>  space = box.schema.space.create('test', { engine = 'vinyl' })
>  ---
> diff --git a/test/vinyl/update_optimize.test.lua b/test/vinyl/update_optimize.test.lua
> index a0de6e4cd..d7556560e 100644
> --- a/test/vinyl/update_optimize.test.lua
> +++ b/test/vinyl/update_optimize.test.lua
> @@ -3,6 +3,9 @@ test_run = require('test_run').new()
>  test_run:cmd('restart server default')
>  fiber = require('fiber')
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  -- optimize one index
>  
>  space = box.schema.space.create('test', { engine = 'vinyl' })
> diff --git a/test/vinyl/upsert.result b/test/vinyl/upsert.result
> index fe673ad6f..a935e0911 100644
> --- a/test/vinyl/upsert.result
> +++ b/test/vinyl/upsert.result
> @@ -1,6 +1,11 @@
>  test_run = require('test_run').new()
>  ---
>  ...
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +---
> +- true
> +...
>  -- gh-1671 upsert is broken in a transaction
>  -- upsert after upsert
>  space = box.schema.space.create('test', { engine = 'vinyl' })
> diff --git a/test/vinyl/upsert.test.lua b/test/vinyl/upsert.test.lua
> index b62c19978..36b0f5ec8 100644
> --- a/test/vinyl/upsert.test.lua
> +++ b/test/vinyl/upsert.test.lua
> @@ -1,5 +1,8 @@
>  test_run = require('test_run').new()
>  
> +test_run:cmd("push filter 'Invalid VYLOG file: Slice [0-9]+ deleted but not registered'" .. \
> +             " to 'Invalid VYLOG file: Slice <NUM> deleted but not registered'")
> +
>  -- gh-1671 upsert is broken in a transaction
>  
>  -- upsert after upsert
> -- 
> 2.25.1
>