From: Cyrill Gorcunov <gorcunov@gmail.com> To: tml <tarantool-patches@dev.tarantool.org> Subject: [Tarantool-patches] [PATCH v7 5/5] test: add replication/applier-rollback Date: Tue, 28 Jan 2020 22:22:49 +0300 [thread overview] Message-ID: <20200128192249.10023-6-gorcunov@gmail.com> (raw) In-Reply-To: <20200128192249.10023-1-gorcunov@gmail.com> In the test force error injection ERRINJ_REPLICA_TXN_WRITE to happen which will initiate applier transaction rollback. Without the fix it will cause SIGSEGV due to lack of error propagation. Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> --- test/replication/applier-rollback-slave.lua | 16 ++ test/replication/applier-rollback.result | 160 ++++++++++++++++++++ test/replication/applier-rollback.test.lua | 79 ++++++++++ test/replication/suite.ini | 2 +- 4 files changed, 256 insertions(+), 1 deletion(-) create mode 100644 test/replication/applier-rollback-slave.lua create mode 100644 test/replication/applier-rollback.result create mode 100644 test/replication/applier-rollback.test.lua diff --git a/test/replication/applier-rollback-slave.lua b/test/replication/applier-rollback-slave.lua new file mode 100644 index 000000000..26fb10055 --- /dev/null +++ b/test/replication/applier-rollback-slave.lua @@ -0,0 +1,16 @@ +-- +-- vim: ts=4 sw=4 et +-- + +print('arg', arg) + +box.cfg({ + replication = os.getenv("MASTER"), + listen = os.getenv("LISTEN"), + memtx_memory = 107374182, + replication_timeout = 0.1, + replication_connect_timeout = 0.5, + read_only = true, +}) + +require('console').listen(os.getenv('ADMIN')) diff --git a/test/replication/applier-rollback.result b/test/replication/applier-rollback.result new file mode 100644 index 000000000..3209fc7fd --- /dev/null +++ b/test/replication/applier-rollback.result @@ -0,0 +1,160 @@ +-- test-run result file version 2 +#!/usr/bin/env tarantool + | --- + | ... +-- +-- vim: ts=4 sw=4 et +-- + +test_run = require('test_run').new() + | --- + | ... + +errinj = box.error.injection + | --- + | ... +engine = test_run:get_cfg('engine') + | --- + | ... + +-- +-- Allow replica to connect to us +box.schema.user.grant('guest', 'replication') + | --- + | ... + +-- +-- Create replica instance, we're the master and +-- start it, no data to sync yet though +test_run:cmd("create server replica_slave with rpl_master=default, script='replication/applier-rollback-slave.lua'") + | --- + | - true + | ... +test_run:cmd("start server replica_slave") + | --- + | - true + | ... + +-- +-- Fill initial data on the master instance +test_run:cmd('switch default') + | --- + | - true + | ... + +_ = box.schema.space.create('test', {engine=engine}) + | --- + | ... +s = box.space.test + | --- + | ... + +s:format({{name = 'id', type = 'unsigned'}, {name = 'band_name', type = 'string'}}) + | --- + | ... + +_ = s:create_index('primary', {type = 'tree', parts = {'id'}}) + | --- + | ... +s:insert({1, '1'}) + | --- + | - [1, '1'] + | ... +s:insert({2, '2'}) + | --- + | - [2, '2'] + | ... +s:insert({3, '3'}) + | --- + | - [3, '3'] + | ... + +-- +-- To make sure we're running +box.info.status + | --- + | - running + | ... + +-- +-- Wait for data from master get propagated +test_run:wait_lsn('replica_slave', 'default') + | --- + | ... + +-- +-- Now inject error into slave instance +test_run:cmd('switch replica_slave') + | --- + | - true + | ... + +-- +-- To make sure we're running +box.info.status + | --- + | - running + | ... + +errinj = box.error.injection + | --- + | ... +errinj.set('ERRINJ_REPLICA_TXN_WRITE', true) + | --- + | - ok + | ... + +-- +-- Jump back to master node and write new +-- entry which should cause error to happen +-- on slave instance +test_run:cmd('switch default') + | --- + | - true + | ... +s:insert({4, '4'}) + | --- + | - [4, '4'] + | ... + +-- +-- Wait for error to trigger +test_run:cmd('switch replica_slave') + | --- + | - true + | ... +fiber = require('fiber') + | --- + | ... +while test_run:grep_log('replica_slave', 'ER_INJECTION:[^\n]*') == nil do fiber.sleep(0.1) end + | --- + | ... + +---- +---- Such error cause the applier to be +---- cancelled and reaped, thus stop the +---- slave node and cleanup +test_run:cmd('switch default') + | --- + | - true + | ... + +-- +-- Cleanup +test_run:cmd("stop server replica_slave") + | --- + | - true + | ... +test_run:cmd("delete server replica_slave") + | --- + | - true + | ... +box.cfg{replication=""} + | --- + | ... +box.space.test:drop() + | --- + | ... +box.schema.user.revoke('guest', 'replication') + | --- + | ... diff --git a/test/replication/applier-rollback.test.lua b/test/replication/applier-rollback.test.lua new file mode 100644 index 000000000..d31eff9f0 --- /dev/null +++ b/test/replication/applier-rollback.test.lua @@ -0,0 +1,79 @@ +#!/usr/bin/env tarantool +-- +-- vim: ts=4 sw=4 et +-- + +test_run = require('test_run').new() + +errinj = box.error.injection +engine = test_run:get_cfg('engine') + +-- +-- Allow replica to connect to us +box.schema.user.grant('guest', 'replication') + +-- +-- Create replica instance, we're the master and +-- start it, no data to sync yet though +test_run:cmd("create server replica_slave with rpl_master=default, script='replication/applier-rollback-slave.lua'") +test_run:cmd("start server replica_slave") + +-- +-- Fill initial data on the master instance +test_run:cmd('switch default') + +_ = box.schema.space.create('test', {engine=engine}) +s = box.space.test + +s:format({{name = 'id', type = 'unsigned'}, {name = 'band_name', type = 'string'}}) + +_ = s:create_index('primary', {type = 'tree', parts = {'id'}}) +s:insert({1, '1'}) +s:insert({2, '2'}) +s:insert({3, '3'}) + +-- +-- To make sure we're running +box.info.status + +-- +-- Wait for data from master get propagated +test_run:wait_lsn('replica_slave', 'default') + +-- +-- Now inject error into slave instance +test_run:cmd('switch replica_slave') + +-- +-- To make sure we're running +box.info.status + +errinj = box.error.injection +errinj.set('ERRINJ_REPLICA_TXN_WRITE', true) + +-- +-- Jump back to master node and write new +-- entry which should cause error to happen +-- on slave instance +test_run:cmd('switch default') +s:insert({4, '4'}) + +-- +-- Wait for error to trigger +test_run:cmd('switch replica_slave') +fiber = require('fiber') +while test_run:grep_log('replica_slave', 'ER_INJECTION:[^\n]*') == nil do fiber.sleep(0.1) end + +---- +---- Such error cause the applier to be +---- cancelled and reaped, thus stop the +---- slave node and cleanup +test_run:cmd('switch default') + +-- +-- Cleanup +test_run:cmd("stop server replica_slave") +test_run:cmd("delete server replica_slave") +box.cfg{replication=""} +box.space.test:drop() +box.schema.user.revoke('guest', 'replication') diff --git a/test/replication/suite.ini b/test/replication/suite.ini index ed1de3140..b804b85f6 100644 --- a/test/replication/suite.ini +++ b/test/replication/suite.ini @@ -3,7 +3,7 @@ core = tarantool script = master.lua description = tarantool/box, replication disabled = consistent.test.lua -release_disabled = catch.test.lua errinj.test.lua gc.test.lua gc_no_space.test.lua before_replace.test.lua quorum.test.lua recover_missing_xlog.test.lua sync.test.lua long_row_timeout.test.lua +release_disabled = catch.test.lua errinj.test.lua gc.test.lua gc_no_space.test.lua before_replace.test.lua quorum.test.lua recover_missing_xlog.test.lua sync.test.lua long_row_timeout.test.lua applier-rollback.test.lua config = suite.cfg lua_libs = lua/fast_replica.lua lua/rlimit.lua use_unix_sockets = True -- 2.20.1
prev parent reply other threads:[~2020-01-28 19:23 UTC|newest] Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-28 19:22 [Tarantool-patches] [PATCH v7 0/5] box/replication: add missing diag set and fix sigsegv Cyrill Gorcunov 2020-01-28 19:22 ` [Tarantool-patches] [PATCH v7 1/5] box/request: add missing OutOfMemory diag_set Cyrill Gorcunov 2020-02-03 14:37 ` Sergey Ostanevich 2020-01-28 19:22 ` [Tarantool-patches] [PATCH v7 2/5] box/applier: add missing diag_set on region_alloc failure Cyrill Gorcunov 2020-02-03 14:39 ` Sergey Ostanevich 2020-02-04 22:15 ` Konstantin Osipov 2020-02-05 7:46 ` Cyrill Gorcunov 2020-02-05 9:49 ` Konstantin Osipov 2020-02-05 10:06 ` Cyrill Gorcunov 2020-01-28 19:22 ` [Tarantool-patches] [PATCH v7 3/5] box/applier: fix nil dereference in applier rollback Cyrill Gorcunov 2020-02-04 22:19 ` Konstantin Osipov 2020-02-05 7:33 ` Cyrill Gorcunov 2020-01-28 19:22 ` [Tarantool-patches] [PATCH v7 4/5] errinj: add ERRINJ_REPLICA_TXN_WRITE Cyrill Gorcunov 2020-02-04 22:45 ` Konstantin Osipov 2020-01-28 19:22 ` Cyrill Gorcunov [this message]
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200128192249.10023-6-gorcunov@gmail.com \ --to=gorcunov@gmail.com \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v7 5/5] test: add replication/applier-rollback' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox