Tarantool development patches archive
 help / color / mirror / Atom feed
* [tarantool-patches] [PATCH] replication: adds replication sync after cfg. update
@ 2018-08-16 19:05 Olga Arkhangelskaia
  2018-08-20 12:36 ` [tarantool-patches] " Georgy Kirichenko
  0 siblings, 1 reply; 2+ messages in thread
From: Olga Arkhangelskaia @ 2018-08-16 19:05 UTC (permalink / raw)
  To: tarantool-patches; +Cc: Olga Arkhangelskaia

When replica reconnects to replica set not for the first time, we
suffer from absence of data sync. Such behavior leads to giving  away outdated
data.

Closes #3427
---
https://github.com/tarantool/tarantool/issues/3427
https://github.com/tarantool/tarantool/tree/OKriw/replication_no_sync-1.9

 src/box/box.cc                      |   2 +
 test/replication/orphan.result      | 103 ++++++++++++++++++++++++++++++++++++
 test/replication/orphan.test.lua    |  47 ++++++++++++++++
 test/replication/replica_orphan.lua |  11 ++++
 4 files changed, 163 insertions(+)
 create mode 100644 test/replication/orphan.result
 create mode 100644 test/replication/orphan.test.lua
 create mode 100644 test/replication/replica_orphan.lua

diff --git a/src/box/box.cc b/src/box/box.cc
index 8d7454d1f..af8a3ccd9 100644
--- a/src/box/box.cc
+++ b/src/box/box.cc
@@ -634,6 +634,8 @@ box_set_replication(void)
 	box_sync_replication(true);
 	/* Follow replica */
 	replicaset_follow();
+	/* Sync replica up to quorum */
+	replicaset_sync();
 }
 
 void
diff --git a/test/replication/orphan.result b/test/replication/orphan.result
new file mode 100644
index 000000000..2230d4546
--- /dev/null
+++ b/test/replication/orphan.result
@@ -0,0 +1,103 @@
+--
+-- gh-3427: no sync after configuration update
+--
+env = require('test_run')
+---
+...
+test_run = env.new()
+---
+...
+engine = test_run:get_cfg('engine')
+---
+...
+box.schema.user.grant('guest', 'read,write,execute', 'universe')
+---
+...
+box.schema.user.grant('guest', 'replication')
+---
+...
+test_run:cmd("create server replica with rpl_master=default, script='replication/replica_orphan.lua'")
+---
+- true
+...
+test_run:cmd("start server replica")
+---
+- true
+...
+repl = test_run:eval('replica', 'return box.cfg.listen')[1]
+---
+...
+box.cfg{replication = repl}
+---
+...
+test_run:cmd("switch replica")
+---
+- true
+...
+test_run:cmd("switch default")
+---
+- true
+...
+s = box.schema.space.create('test', {engine = engine});
+---
+...
+index = s:create_index('primary')
+---
+...
+-- change replica configuration
+test_run:cmd("switch replica")
+---
+- true
+...
+box.cfg{replication={}}
+---
+...
+test_run:cmd("switch default")
+---
+- true
+...
+-- insert values on the master while replica is unconfigured
+a = 100000 while a > 0 do a = a-1 box.space.test:insert{a,'A'..a} end
+---
+...
+test_run:cmd("switch replica")
+---
+- true
+...
+box.cfg{replication = os.getenv("MASTER")}
+---
+...
+test_run:cmd("switch default")
+---
+- true
+...
+test_run:cmd("switch replica")
+---
+- true
+...
+box.info.replication[1].upstream.lag > 1
+---
+- false
+...
+test_run:cmd("switch default")
+---
+- true
+...
+-- cleanup
+test_run:cmd("stop server replica")
+---
+- true
+...
+test_run:cmd("cleanup server replica")
+---
+- true
+...
+box.space.test:drop()
+---
+...
+box.schema.user.revoke('guest', 'replication')
+---
+...
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+---
+...
diff --git a/test/replication/orphan.test.lua b/test/replication/orphan.test.lua
new file mode 100644
index 000000000..25522f5ac
--- /dev/null
+++ b/test/replication/orphan.test.lua
@@ -0,0 +1,47 @@
+--
+-- gh-3427: no sync after configuration update
+--
+
+env = require('test_run')
+test_run = env.new()
+engine = test_run:get_cfg('engine')
+
+box.schema.user.grant('guest', 'read,write,execute', 'universe')
+
+box.schema.user.grant('guest', 'replication')
+test_run:cmd("create server replica with rpl_master=default, script='replication/replica_orphan.lua'")
+test_run:cmd("start server replica")
+
+repl = test_run:eval('replica', 'return box.cfg.listen')[1]
+box.cfg{replication = repl}
+
+test_run:cmd("switch replica")
+test_run:cmd("switch default")
+
+s = box.schema.space.create('test', {engine = engine});
+index = s:create_index('primary')
+
+-- change replica configuration
+test_run:cmd("switch replica")
+box.cfg{replication={}}
+
+test_run:cmd("switch default")
+-- insert values on the master while replica is unconfigured
+a = 100000 while a > 0 do a = a-1 box.space.test:insert{a,'A'..a} end
+
+test_run:cmd("switch replica")
+box.cfg{replication = os.getenv("MASTER")}
+
+test_run:cmd("switch default")
+test_run:cmd("switch replica")
+
+box.info.replication[1].upstream.lag > 1
+test_run:cmd("switch default")
+
+-- cleanup
+test_run:cmd("stop server replica")
+test_run:cmd("cleanup server replica")
+box.space.test:drop()
+box.schema.user.revoke('guest', 'replication')
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+
diff --git a/test/replication/replica_orphan.lua b/test/replication/replica_orphan.lua
new file mode 100644
index 000000000..98573a407
--- /dev/null
+++ b/test/replication/replica_orphan.lua
@@ -0,0 +1,11 @@
+#!/usr/bin/env tarantool
+
+local TIMEOUT = tonumber(arg[1])
+
+box.cfg({
+    listen              = os.getenv("LISTEN"),
+    replication         = os.getenv("MASTER"),
+    replication_connect_timeout = 0.5,
+})
+
+require('console').listen(os.getenv('ADMIN'))
-- 
2.14.3 (Apple Git-98)

^ permalink raw reply	[flat|nested] 2+ messages in thread

* [tarantool-patches] Re: [PATCH] replication: adds replication sync after cfg. update
  2018-08-16 19:05 [tarantool-patches] [PATCH] replication: adds replication sync after cfg. update Olga Arkhangelskaia
@ 2018-08-20 12:36 ` Georgy Kirichenko
  0 siblings, 0 replies; 2+ messages in thread
From: Georgy Kirichenko @ 2018-08-20 12:36 UTC (permalink / raw)
  To: tarantool-patches; +Cc: Olga Arkhangelskaia

[-- Attachment #1: Type: text/plain, Size: 5921 bytes --]

Thanks for the path.
It looks like Ok, but your test is failing for me:

[001] Test failed! Result content mismatch:
[001] --- replication/orphan.result     Mon Aug 20 13:51:43 2018
[001] +++ replication/orphan.reject     Mon Aug 20 15:32:36 2018
[001] @@ -77,7 +77,7 @@
[001]  ...
[001]  box.info.replication[1].upstream.lag > 1
[001]  ---
[001] -- false
[001] +- true
[001]  ...
[001]  test_run:cmd("switch default")
[001]  ---
[001]


On Thursday, August 16, 2018 10:05:21 PM MSK Olga Arkhangelskaia wrote:
> When replica reconnects to replica set not for the first time, we
> suffer from absence of data sync. Such behavior leads to giving  away
> outdated data.
> 
> Closes #3427
> ---
> https://github.com/tarantool/tarantool/issues/3427
> https://github.com/tarantool/tarantool/tree/OKriw/replication_no_sync-1.9
> 
>  src/box/box.cc                      |   2 +
>  test/replication/orphan.result      | 103
> ++++++++++++++++++++++++++++++++++++ test/replication/orphan.test.lua    | 
> 47 ++++++++++++++++
>  test/replication/replica_orphan.lua |  11 ++++
>  4 files changed, 163 insertions(+)
>  create mode 100644 test/replication/orphan.result
>  create mode 100644 test/replication/orphan.test.lua
>  create mode 100644 test/replication/replica_orphan.lua
> 
> diff --git a/src/box/box.cc b/src/box/box.cc
> index 8d7454d1f..af8a3ccd9 100644
> --- a/src/box/box.cc
> +++ b/src/box/box.cc
> @@ -634,6 +634,8 @@ box_set_replication(void)
>  	box_sync_replication(true);
>  	/* Follow replica */
>  	replicaset_follow();
> +	/* Sync replica up to quorum */
> +	replicaset_sync();
>  }
> 
>  void
> diff --git a/test/replication/orphan.result b/test/replication/orphan.result
> new file mode 100644
> index 000000000..2230d4546
> --- /dev/null
> +++ b/test/replication/orphan.result
> @@ -0,0 +1,103 @@
> +--
> +-- gh-3427: no sync after configuration update
> +--
> +env = require('test_run')
> +---
> +...
> +test_run = env.new()
> +---
> +...
> +engine = test_run:get_cfg('engine')
> +---
> +...
> +box.schema.user.grant('guest', 'read,write,execute', 'universe')
> +---
> +...
> +box.schema.user.grant('guest', 'replication')
> +---
> +...
> +test_run:cmd("create server replica with rpl_master=default,
> script='replication/replica_orphan.lua'") +---
> +- true
> +...
> +test_run:cmd("start server replica")
> +---
> +- true
> +...
> +repl = test_run:eval('replica', 'return box.cfg.listen')[1]
> +---
> +...
> +box.cfg{replication = repl}
> +---
> +...
> +test_run:cmd("switch replica")
> +---
> +- true
> +...
> +test_run:cmd("switch default")
> +---
> +- true
> +...
> +s = box.schema.space.create('test', {engine = engine});
> +---
> +...
> +index = s:create_index('primary')
> +---
> +...
> +-- change replica configuration
> +test_run:cmd("switch replica")
> +---
> +- true
> +...
> +box.cfg{replication={}}
> +---
> +...
> +test_run:cmd("switch default")
> +---
> +- true
> +...
> +-- insert values on the master while replica is unconfigured
> +a = 100000 while a > 0 do a = a-1 box.space.test:insert{a,'A'..a} end
> +---
> +...
> +test_run:cmd("switch replica")
> +---
> +- true
> +...
> +box.cfg{replication = os.getenv("MASTER")}
> +---
> +...
> +test_run:cmd("switch default")
> +---
> +- true
> +...
> +test_run:cmd("switch replica")
> +---
> +- true
> +...
> +box.info.replication[1].upstream.lag > 1
> +---
> +- false
> +...
> +test_run:cmd("switch default")
> +---
> +- true
> +...
> +-- cleanup
> +test_run:cmd("stop server replica")
> +---
> +- true
> +...
> +test_run:cmd("cleanup server replica")
> +---
> +- true
> +...
> +box.space.test:drop()
> +---
> +...
> +box.schema.user.revoke('guest', 'replication')
> +---
> +...
> +box.schema.user.revoke('guest', 'read,write,execute', 'universe')
> +---
> +...
> diff --git a/test/replication/orphan.test.lua
> b/test/replication/orphan.test.lua new file mode 100644
> index 000000000..25522f5ac
> --- /dev/null
> +++ b/test/replication/orphan.test.lua
> @@ -0,0 +1,47 @@
> +--
> +-- gh-3427: no sync after configuration update
> +--
> +
> +env = require('test_run')
> +test_run = env.new()
> +engine = test_run:get_cfg('engine')
> +
> +box.schema.user.grant('guest', 'read,write,execute', 'universe')
> +
> +box.schema.user.grant('guest', 'replication')
> +test_run:cmd("create server replica with rpl_master=default,
> script='replication/replica_orphan.lua'") +test_run:cmd("start server
> replica")
> +
> +repl = test_run:eval('replica', 'return box.cfg.listen')[1]
> +box.cfg{replication = repl}
> +
> +test_run:cmd("switch replica")
> +test_run:cmd("switch default")
> +
> +s = box.schema.space.create('test', {engine = engine});
> +index = s:create_index('primary')
> +
> +-- change replica configuration
> +test_run:cmd("switch replica")
> +box.cfg{replication={}}
> +
> +test_run:cmd("switch default")
> +-- insert values on the master while replica is unconfigured
> +a = 100000 while a > 0 do a = a-1 box.space.test:insert{a,'A'..a} end
> +
> +test_run:cmd("switch replica")
> +box.cfg{replication = os.getenv("MASTER")}
> +
> +test_run:cmd("switch default")
> +test_run:cmd("switch replica")
> +
> +box.info.replication[1].upstream.lag > 1
> +test_run:cmd("switch default")
> +
> +-- cleanup
> +test_run:cmd("stop server replica")
> +test_run:cmd("cleanup server replica")
> +box.space.test:drop()
> +box.schema.user.revoke('guest', 'replication')
> +box.schema.user.revoke('guest', 'read,write,execute', 'universe')
> +
> diff --git a/test/replication/replica_orphan.lua
> b/test/replication/replica_orphan.lua new file mode 100644
> index 000000000..98573a407
> --- /dev/null
> +++ b/test/replication/replica_orphan.lua
> @@ -0,0 +1,11 @@
> +#!/usr/bin/env tarantool
> +
> +local TIMEOUT = tonumber(arg[1])
> +
> +box.cfg({
> +    listen              = os.getenv("LISTEN"),
> +    replication         = os.getenv("MASTER"),
> +    replication_connect_timeout = 0.5,
> +})
> +
> +require('console').listen(os.getenv('ADMIN'))


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2018-08-20 12:36 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-16 19:05 [tarantool-patches] [PATCH] replication: adds replication sync after cfg. update Olga Arkhangelskaia
2018-08-20 12:36 ` [tarantool-patches] " Georgy Kirichenko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox