[Tarantool-patches] [PATCH v1 0/2] fix flaky replication/election_qsync_stress.test.lua test

[Tarantool-patches] [PATCH v1 0/2] fix flaky replication/election_qsync_stress.test.lua test

[Alexander V. Tikhonov]

Created 2 commits:

1. test: fix flaky election_qsync_stress with KILL
    
  Found that replication/election_qsync_stress.test.lua test may fail on
  restating instances. It occures on heavy loaded hosts when its local
  call to stop instance using SIGTERM fails to stop it. Decided to use
  SIGKILL in local stop call options to be sure that the instance will
  be stopped.

2. test: filter output message for test
    
  Added test-run filter for replication/election_qsync_stress.test.lua
  test on error message:
    
    'error: Found uncommitted sync transactions from other instance with id [0-9]+'
    
  to avoid of printing changing data in results file to be able to use
  its checksums in fragile list of test-run to rerun it as flaky issue.

Alexander V. Tikhonov (2):
  test: filter output message for test
  test: fix flaky election_qsync_stress with KILL

 test/replication/election_qsync_stress.result   | 8 +++++++-
 test/replication/election_qsync_stress.test.lua | 5 ++++-
 test/replication/suite.ini                      | 2 +-
 3 files changed, 12 insertions(+), 3 deletions(-)

-- 
2.25.1

[Tarantool-patches] [PATCH v1 1/2] test: filter output message for test

[Alexander V. Tikhonov]

Added test-run filter for replication/election_qsync_stress.test.lua
test on error message:

  'error: Found uncommitted sync transactions from other instance with id [0-9]+'

to avoid of printing changing data in results file to be able to use
its checksums in fragile list of test-run to rerun it as flaky issue.
---
 test/replication/election_qsync_stress.result   | 6 ++++++
 test/replication/election_qsync_stress.test.lua | 3 +++
 test/replication/suite.ini                      | 2 +-
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/test/replication/election_qsync_stress.result b/test/replication/election_qsync_stress.result
index 9fab2f1d7..b54dc23c4 100644
--- a/test/replication/election_qsync_stress.result
+++ b/test/replication/election_qsync_stress.result
@@ -6,6 +6,12 @@ netbox = require('net.box')
  | ---
  | ...
 
+test_run:cmd("push filter 'error: Found uncommitted sync transactions from other instance with id [0-9]+'" .. \
+             "to 'error: Found uncommitted sync transactions from other instance with id <NUM>'")
+ | ---
+ | - true
+ | ...
+
 --
 -- gh-1146: Leader election + Qsync
 --
diff --git a/test/replication/election_qsync_stress.test.lua b/test/replication/election_qsync_stress.test.lua
index 0ba15eef7..873aabf00 100644
--- a/test/replication/election_qsync_stress.test.lua
+++ b/test/replication/election_qsync_stress.test.lua
@@ -1,6 +1,9 @@
 test_run = require('test_run').new()
 netbox = require('net.box')
 
+test_run:cmd("push filter 'error: Found uncommitted sync transactions from other instance with id [0-9]+'" .. \
+             "to 'error: Found uncommitted sync transactions from other instance with id <NUM>'")
+
 --
 -- gh-1146: Leader election + Qsync
 --
diff --git a/test/replication/suite.ini b/test/replication/suite.ini
index 6136c934f..a209f0f3c 100644
--- a/test/replication/suite.ini
+++ b/test/replication/suite.ini
@@ -119,7 +119,7 @@ fragile = {
         },
         "election_qsync_stress.test.lua": {
             "issues": [ "gh-5395" ],
-            "checksums": [ "634bda94accdcdef7b1db3e14f28f445", "36bcdae426c18a60fd13025c09f197d0", "209c865525154a91435c63850f15eca0", "ca38ff2cdfa65b3defb26607b24454c6", "3b6c573d2aa06ee382d6f27b6a76657b", "bcf08e055cf3ccd9958af25d0fba29f8", "411e7462760bc3fc968b68b4b7267ea1", "37e671aea27e3396098f9d13c7fa7316" ]
+            "checksums": [ "133676d72249c570f7124440150a8790", "83e1b4d48bd095590283247352b37ebb", "358bf14addaee9c8f2dbfe64374bb771", "e217688aa8c995123506662a1b07f2d1", "7cbb6b62027df3d1f31ea94f298bc2e0" ]
         },
         "gh-3711-misc-no-restart-on-same-configuration.test.lua": {
             "issues": [ "gh-5407" ],
-- 
2.25.1

[Tarantool-patches] [PATCH v1 2/2] test: fix flaky election_qsync_stress with KILL

[Alexander V. Tikhonov]

Found that replication/election_qsync_stress.test.lua test may fail on
restating instances. It occures on heavy loaded hosts when its local
call to stop instance using SIGTERM fails to stop it. Decided to use
SIGKILL in local stop call options to be sure that the instance will
be stopped.
---
 test/replication/election_qsync_stress.result   | 2 +-
 test/replication/election_qsync_stress.test.lua | 2 +-
 test/replication/suite.ini                      | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/replication/election_qsync_stress.result b/test/replication/election_qsync_stress.result
index b54dc23c4..598e6a9ff 100644
--- a/test/replication/election_qsync_stress.result
+++ b/test/replication/election_qsync_stress.result
@@ -93,7 +93,7 @@ for i = 1,10 do
     c:eval('box.cfg{replication_synchro_quorum=4, replication_synchro_timeout=1000}')
     c.space.test:insert({i}, {is_async=true})
     test_run:wait_cond(function() return c.space.test:get{i} ~= nil end)
-    test_run:cmd('stop server '..old_leader)
+    test_run:cmd('stop server '..old_leader..' with signal=KILL')
     nrs[old_leader_nr] = false
     new_leader_nr = get_leader(nrs)
     new_leader = 'election_replica'..new_leader_nr
diff --git a/test/replication/election_qsync_stress.test.lua b/test/replication/election_qsync_stress.test.lua
index 873aabf00..384e36134 100644
--- a/test/replication/election_qsync_stress.test.lua
+++ b/test/replication/election_qsync_stress.test.lua
@@ -54,7 +54,7 @@ for i = 1,10 do
     c:eval('box.cfg{replication_synchro_quorum=4, replication_synchro_timeout=1000}')
     c.space.test:insert({i}, {is_async=true})
     test_run:wait_cond(function() return c.space.test:get{i} ~= nil end)
-    test_run:cmd('stop server '..old_leader)
+    test_run:cmd('stop server '..old_leader..' with signal=KILL')
     nrs[old_leader_nr] = false
     new_leader_nr = get_leader(nrs)
     new_leader = 'election_replica'..new_leader_nr
diff --git a/test/replication/suite.ini b/test/replication/suite.ini
index a209f0f3c..8d3a330aa 100644
--- a/test/replication/suite.ini
+++ b/test/replication/suite.ini
@@ -119,7 +119,7 @@ fragile = {
         },
         "election_qsync_stress.test.lua": {
             "issues": [ "gh-5395" ],
-            "checksums": [ "133676d72249c570f7124440150a8790", "83e1b4d48bd095590283247352b37ebb", "358bf14addaee9c8f2dbfe64374bb771", "e217688aa8c995123506662a1b07f2d1", "7cbb6b62027df3d1f31ea94f298bc2e0" ]
+            "checksums": [ "133676d72249c570f7124440150a8790", "83e1b4d48bd095590283247352b37ebb", "358bf14addaee9c8f2dbfe64374bb771", "e217688aa8c995123506662a1b07f2d1", "7cbb6b62027df3d1f31ea94f298bc2e0", "353dcc43b70c2b34fdea29c7ab10d018", "82cf01d9b40e877c6a93448acb4f721d", "d02417bf4344dccd14601b3bdc7a4798", "18662765b1a8db977817f0d3527dad2f", "c0710e3023f47aee8785c9096c181f51" ]
         },
         "gh-3711-misc-no-restart-on-same-configuration.test.lua": {
             "issues": [ "gh-5407" ],
-- 
2.25.1

Re: [Tarantool-patches] [PATCH v1 0/2] fix flaky replication/election_qsync_stress.test.lua test

[Serge Petrenko]

05.11.2020 17:33, Alexander V. Tikhonov пишет:
> Created 2 commits:
>
> 1. test: fix flaky election_qsync_stress with KILL
>      
>    Found that replication/election_qsync_stress.test.lua test may fail on
>    restating instances. It occures on heavy loaded hosts when its local
>    call to stop instance using SIGTERM fails to stop it. Decided to use
>    SIGKILL in local stop call options to be sure that the instance will
>    be stopped.
>
> 2. test: filter output message for test
>      
>    Added test-run filter for replication/election_qsync_stress.test.lua
>    test on error message:
>      
>      'error: Found uncommitted sync transactions from other instance with id [0-9]+'
>      
>    to avoid of printing changing data in results file to be able to use
>    its checksums in fragile list of test-run to rerun it as flaky issue.
>
> Alexander V. Tikhonov (2):
>    test: filter output message for test
>    test: fix flaky election_qsync_stress with KILL
>
>   test/replication/election_qsync_stress.result   | 8 +++++++-
>   test/replication/election_qsync_stress.test.lua | 5 ++++-
>   test/replication/suite.ini                      | 2 +-
>   3 files changed, 12 insertions(+), 3 deletions(-)
>

Hi! Thanks for the patchset!

Both patches LGTM.

-- 
Serge Petrenko