* [PATCH] box: serialize calls to box.cfg
@ 2018-08-07 9:43 Vladimir Davydov
2018-08-07 11:09 ` Vladimir Davydov
2018-08-07 11:39 ` [tarantool-patches] " Kirill Yukhin
0 siblings, 2 replies; 4+ messages in thread
From: Vladimir Davydov @ 2018-08-07 9:43 UTC (permalink / raw)
To: tarantool-patches
It is dangerous to call box.cfg() concurrently from different fibers.
For example, replication configuration uses static variables and yields
so calling it concurrently can result in a crash. To make sure it never
happens, let's protect box.cfg() with a lock.
Closes #3606
---
https://github.com/tarantool/tarantool/issues/3606
https://github.com/tarantool/tarantool/tree/dv/gh-3606-concurrent-replication-cfg-fix
src/box/lua/load_cfg.lua | 21 ++++++++++++++++++---
test/replication/misc.result | 24 ++++++++++++++++++++++++
test/replication/misc.test.lua | 10 ++++++++++
3 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
index 0b668cdc..4de79220 100644
--- a/src/box/lua/load_cfg.lua
+++ b/src/box/lua/load_cfg.lua
@@ -5,6 +5,21 @@ local json = require('json')
local private = require('box.internal')
local urilib = require('uri')
local math = require('math')
+local fiber = require('fiber')
+
+-- Function decorator that is used to prevent box.cfg() from
+-- being called concurrently by different fibers.
+local lock = fiber.channel(1)
+local function locked(f)
+ return function(...)
+ lock:put(true)
+ local status = pcall(f, ...)
+ lock:get()
+ if not status then
+ box.error()
+ end
+ end
+end
-- all available options
local default_cfg = {
@@ -409,7 +424,7 @@ local function load_cfg(cfg)
-- Save new box.cfg
box.cfg = cfg
if not pcall(private.cfg_check) then
- box.cfg = load_cfg -- restore original box.cfg
+ box.cfg = locked(load_cfg) -- restore original box.cfg
return box.error() -- re-throw exception from check_cfg()
end
-- Restore box members after initial configuration
@@ -423,7 +438,7 @@ local function load_cfg(cfg)
__newindex = function(table, index)
error('Attempt to modify a read-only table')
end,
- __call = reload_cfg,
+ __call = locked(reload_cfg),
})
private.cfg_load()
for key, fun in pairs(dynamic_cfg) do
@@ -439,7 +454,7 @@ local function load_cfg(cfg)
box.schema.upgrade{auto = true}
end
end
-box.cfg = load_cfg
+box.cfg = locked(load_cfg)
-- gh-810:
-- hack luajit default cpath
diff --git a/test/replication/misc.result b/test/replication/misc.result
index ff0dbf54..9d9d010c 100644
--- a/test/replication/misc.result
+++ b/test/replication/misc.result
@@ -23,6 +23,30 @@ box.cfg{replication = {'127.0.0.1:12345', box.cfg.listen}}
- error: 'Incorrect value for option ''replication'': failed to connect to one or
more replicas'
...
+-- gh-3606 - Tarantool crashes if box.cfg.replication is updated concurrently
+fiber = require('fiber')
+---
+...
+c = fiber.channel(2)
+---
+...
+f = function() fiber.create(function() pcall(box.cfg, {replication = {12345}}) c:put(true) end) end
+---
+...
+f()
+---
+...
+f()
+---
+...
+c:get()
+---
+- true
+...
+c:get()
+---
+- true
+...
box.cfg{replication_timeout = replication_timeout, replication_connect_timeout = replication_connect_timeout}
---
...
diff --git a/test/replication/misc.test.lua b/test/replication/misc.test.lua
index c05e5216..da5a9023 100644
--- a/test/replication/misc.test.lua
+++ b/test/replication/misc.test.lua
@@ -9,6 +9,16 @@ replication_timeout = box.cfg.replication_timeout
replication_connect_timeout = box.cfg.replication_connect_timeout
box.cfg{replication_timeout=0.05, replication_connect_timeout=0.05, replication={}}
box.cfg{replication = {'127.0.0.1:12345', box.cfg.listen}}
+
+-- gh-3606 - Tarantool crashes if box.cfg.replication is updated concurrently
+fiber = require('fiber')
+c = fiber.channel(2)
+f = function() fiber.create(function() pcall(box.cfg, {replication = {12345}}) c:put(true) end) end
+f()
+f()
+c:get()
+c:get()
+
box.cfg{replication_timeout = replication_timeout, replication_connect_timeout = replication_connect_timeout}
-- gh-3111 - Allow to rebootstrap a replica from a read-only master
--
2.11.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] box: serialize calls to box.cfg
2018-08-07 9:43 [PATCH] box: serialize calls to box.cfg Vladimir Davydov
@ 2018-08-07 11:09 ` Vladimir Davydov
2018-08-07 11:26 ` Georgy Kirichenko
2018-08-07 11:39 ` [tarantool-patches] " Kirill Yukhin
1 sibling, 1 reply; 4+ messages in thread
From: Vladimir Davydov @ 2018-08-07 11:09 UTC (permalink / raw)
To: georgy; +Cc: tarantool-patches
On Tue, Aug 07, 2018 at 12:43:26PM +0300, Vladimir Davydov wrote:
> It is dangerous to call box.cfg() concurrently from different fibers.
> For example, replication configuration uses static variables and yields
> so calling it concurrently can result in a crash. To make sure it never
> happens, let's protect box.cfg() with a lock.
>
> Closes #3606
> ---
> https://github.com/tarantool/tarantool/issues/3606
> https://github.com/tarantool/tarantool/tree/dv/gh-3606-concurrent-replication-cfg-fix
>
> src/box/lua/load_cfg.lua | 21 ++++++++++++++++++---
> test/replication/misc.result | 24 ++++++++++++++++++++++++
> test/replication/misc.test.lua | 10 ++++++++++
> 3 files changed, 52 insertions(+), 3 deletions(-)
>
> diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
> index 0b668cdc..4de79220 100644
> --- a/src/box/lua/load_cfg.lua
> +++ b/src/box/lua/load_cfg.lua
> @@ -5,6 +5,21 @@ local json = require('json')
> local private = require('box.internal')
> local urilib = require('uri')
> local math = require('math')
> +local fiber = require('fiber')
> +
> +-- Function decorator that is used to prevent box.cfg() from
> +-- being called concurrently by different fibers.
> +local lock = fiber.channel(1)
> +local function locked(f)
> + return function(...)
> + lock:put(true)
> + local status = pcall(f, ...)
> + lock:get()
> + if not status then
> + box.error()
> + end
> + end
> +end
Georgy noted that we'd better use error() to re-throw the error returned
by box.cfg(). Updated on the branch, the incremental diff is below:
diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
index 4de79220..c68a3583 100644
--- a/src/box/lua/load_cfg.lua
+++ b/src/box/lua/load_cfg.lua
@@ -13,10 +13,10 @@ local lock = fiber.channel(1)
local function locked(f)
return function(...)
lock:put(true)
- local status = pcall(f, ...)
+ local status, err = pcall(f, ...)
lock:get()
if not status then
- box.error()
+ error(err)
end
end
end
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] box: serialize calls to box.cfg
2018-08-07 11:09 ` Vladimir Davydov
@ 2018-08-07 11:26 ` Georgy Kirichenko
0 siblings, 0 replies; 4+ messages in thread
From: Georgy Kirichenko @ 2018-08-07 11:26 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: tarantool-patches
[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]
Looks good to me
On Tuesday, August 7, 2018 2:09:01 PM MSK Vladimir Davydov wrote:
> On Tue, Aug 07, 2018 at 12:43:26PM +0300, Vladimir Davydov wrote:
> > It is dangerous to call box.cfg() concurrently from different fibers.
> > For example, replication configuration uses static variables and yields
> > so calling it concurrently can result in a crash. To make sure it never
> > happens, let's protect box.cfg() with a lock.
> >
> > Closes #3606
> > ---
> > https://github.com/tarantool/tarantool/issues/3606
> > https://github.com/tarantool/tarantool/tree/dv/gh-3606-concurrent-replicat
> > ion-cfg-fix>
> > src/box/lua/load_cfg.lua | 21 ++++++++++++++++++---
> > test/replication/misc.result | 24 ++++++++++++++++++++++++
> > test/replication/misc.test.lua | 10 ++++++++++
> > 3 files changed, 52 insertions(+), 3 deletions(-)
> >
> > diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
> > index 0b668cdc..4de79220 100644
> > --- a/src/box/lua/load_cfg.lua
> > +++ b/src/box/lua/load_cfg.lua
> > @@ -5,6 +5,21 @@ local json = require('json')
> >
> > local private = require('box.internal')
> > local urilib = require('uri')
> > local math = require('math')
> >
> > +local fiber = require('fiber')
> > +
> > +-- Function decorator that is used to prevent box.cfg() from
> > +-- being called concurrently by different fibers.
> > +local lock = fiber.channel(1)
> > +local function locked(f)
> > + return function(...)
> > + lock:put(true)
> > + local status = pcall(f, ...)
> > + lock:get()
> > + if not status then
> > + box.error()
> > + end
> > + end
> > +end
>
> Georgy noted that we'd better use error() to re-throw the error returned
> by box.cfg(). Updated on the branch, the incremental diff is below:
>
> diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
> index 4de79220..c68a3583 100644
> --- a/src/box/lua/load_cfg.lua
> +++ b/src/box/lua/load_cfg.lua
> @@ -13,10 +13,10 @@ local lock = fiber.channel(1)
> local function locked(f)
> return function(...)
> lock:put(true)
> - local status = pcall(f, ...)
> + local status, err = pcall(f, ...)
> lock:get()
> if not status then
> - box.error()
> + error(err)
> end
> end
> end
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* [tarantool-patches] Re: [PATCH] box: serialize calls to box.cfg
2018-08-07 9:43 [PATCH] box: serialize calls to box.cfg Vladimir Davydov
2018-08-07 11:09 ` Vladimir Davydov
@ 2018-08-07 11:39 ` Kirill Yukhin
1 sibling, 0 replies; 4+ messages in thread
From: Kirill Yukhin @ 2018-08-07 11:39 UTC (permalink / raw)
To: tarantool-patches
Hello,
On 07 авг 12:43, Vladimir Davydov wrote:
> It is dangerous to call box.cfg() concurrently from different fibers.
> For example, replication configuration uses static variables and yields
> so calling it concurrently can result in a crash. To make sure it never
> happens, let's protect box.cfg() with a lock.
>
> Closes #3606
> ---
> https://github.com/tarantool/tarantool/issues/3606
> https://github.com/tarantool/tarantool/tree/dv/gh-3606-concurrent-replication-cfg-fix
I've merged final version of the patch into 1.9 branch.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-08-07 11:39 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-07 9:43 [PATCH] box: serialize calls to box.cfg Vladimir Davydov
2018-08-07 11:09 ` Vladimir Davydov
2018-08-07 11:26 ` Georgy Kirichenko
2018-08-07 11:39 ` [tarantool-patches] " Kirill Yukhin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox