* [PATCH] vinyl: cancel reader and writer threads on shutdown
@ 2019-02-11 17:37 Vladimir Davydov
2019-02-11 18:16 ` Konstantin Osipov
0 siblings, 1 reply; 3+ messages in thread
From: Vladimir Davydov @ 2019-02-11 17:37 UTC (permalink / raw)
To: kostja; +Cc: tarantool-patches
Currently, vinyl won't shutdown until all reader and writer threads
gracefully complete all their pending requests, which may take a while,
especially for writer threads that may happen to be doing compaction at
the time. This is annoying - there's absolutely no reason to delay
termination in such a case. Let's forcefully cancel all threads, like we
do in case of relay threads.
This should fix sporadic vinyl/replica_quota test hang.
Closes #3949
---
https://github.com/tarantool/tarantool/issues/3949
https://github.com/tarantool/tarantool/commits/dv/gh-3949-vy-cancel-threads-on-shutdown
src/box/vy_run.c | 7 ++-----
src/box/vy_scheduler.c | 5 ++---
test/vinyl/errinj.result | 40 ++++++++++++++++++++++++++++++++++++++++
test/vinyl/errinj.test.lua | 20 ++++++++++++++++++++
4 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/src/box/vy_run.c b/src/box/vy_run.c
index cee90458..9aa3beec 100644
--- a/src/box/vy_run.c
+++ b/src/box/vy_run.c
@@ -152,11 +152,8 @@ vy_run_env_stop_readers(struct vy_run_env *env)
{
for (int i = 0; i < env->reader_pool_size; i++) {
struct vy_run_reader *reader = &env->reader_pool[i];
-
- cbus_stop_loop(&reader->reader_pipe);
- cpipe_destroy(&reader->reader_pipe);
- if (cord_join(&reader->cord) != 0)
- panic("failed to join vinyl reader thread");
+ tt_pthread_cancel(reader->cord.id);
+ tt_pthread_join(reader->cord.id, NULL);
}
free(env->reader_pool);
}
diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c
index 5ec6d171..d4047197 100644
--- a/src/box/vy_scheduler.c
+++ b/src/box/vy_scheduler.c
@@ -371,9 +371,8 @@ vy_worker_pool_stop(struct vy_worker_pool *pool)
assert(pool->workers != NULL);
for (int i = 0; i < pool->size; i++) {
struct vy_worker *worker = &pool->workers[i];
- cbus_stop_loop(&worker->worker_pipe);
- cpipe_destroy(&worker->worker_pipe);
- cord_join(&worker->cord);
+ tt_pthread_cancel(worker->cord.id);
+ tt_pthread_join(worker->cord.id, NULL);
}
free(pool->workers);
pool->workers = NULL;
diff --git a/test/vinyl/errinj.result b/test/vinyl/errinj.result
index 990c7e85..248b32c8 100644
--- a/test/vinyl/errinj.result
+++ b/test/vinyl/errinj.result
@@ -1126,3 +1126,43 @@ box.schema.user.revoke('guest', 'replication')
s:drop()
---
...
+--
+-- Check that tarantool stops immediately even if a vinyl worker
+-- thread is blocked (see gh-3225).
+--
+s = box.schema.space.create('test', {engine = 'vinyl'})
+---
+...
+_ = s:create_index('pk')
+---
+...
+s:replace{1, 1}
+---
+- [1, 1]
+...
+box.snapshot()
+---
+- ok
+...
+errinj.set('ERRINJ_VY_READ_PAGE_TIMEOUT', 9000)
+---
+- ok
+...
+_ = fiber.create(function() s:get(1) end)
+---
+...
+s:replace{1, 2}
+---
+- [1, 2]
+...
+errinj.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 9000)
+---
+- ok
+...
+_ = fiber.create(function() box.snapshot() end)
+---
+...
+test_run:cmd("restart server default")
+box.space.test:drop()
+---
+...
diff --git a/test/vinyl/errinj.test.lua b/test/vinyl/errinj.test.lua
index d374a910..eaec52a5 100644
--- a/test/vinyl/errinj.test.lua
+++ b/test/vinyl/errinj.test.lua
@@ -408,3 +408,23 @@ test_run:cmd("delete server replica")
errinj.set('ERRINJ_VYRUN_INDEX_GARBAGE', false)
box.schema.user.revoke('guest', 'replication')
s:drop()
+
+--
+-- Check that tarantool stops immediately even if a vinyl worker
+-- thread is blocked (see gh-3225).
+--
+s = box.schema.space.create('test', {engine = 'vinyl'})
+_ = s:create_index('pk')
+s:replace{1, 1}
+box.snapshot()
+
+errinj.set('ERRINJ_VY_READ_PAGE_TIMEOUT', 9000)
+_ = fiber.create(function() s:get(1) end)
+
+s:replace{1, 2}
+
+errinj.set('ERRINJ_VY_RUN_WRITE_STMT_TIMEOUT', 9000)
+_ = fiber.create(function() box.snapshot() end)
+
+test_run:cmd("restart server default")
+box.space.test:drop()
--
2.11.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] vinyl: cancel reader and writer threads on shutdown
2019-02-11 17:37 [PATCH] vinyl: cancel reader and writer threads on shutdown Vladimir Davydov
@ 2019-02-11 18:16 ` Konstantin Osipov
2019-02-12 9:02 ` Vladimir Davydov
0 siblings, 1 reply; 3+ messages in thread
From: Konstantin Osipov @ 2019-02-11 18:16 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: tarantool-patches
* Vladimir Davydov <vdavydov.dev@gmail.com> [19/02/11 20:41]:
> Currently, vinyl won't shutdown until all reader and writer threads
> gracefully complete all their pending requests, which may take a while,
> especially for writer threads that may happen to be doing compaction at
> the time. This is annoying - there's absolutely no reason to delay
> termination in such a case. Let's forcefully cancel all threads, like we
> do in case of relay threads.
>
> This should fix sporadic vinyl/replica_quota test hang.
>
> Closes #3949
OK to push.
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] vinyl: cancel reader and writer threads on shutdown
2019-02-11 18:16 ` Konstantin Osipov
@ 2019-02-12 9:02 ` Vladimir Davydov
0 siblings, 0 replies; 3+ messages in thread
From: Vladimir Davydov @ 2019-02-12 9:02 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: tarantool-patches
On Mon, Feb 11, 2019 at 09:16:05PM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev@gmail.com> [19/02/11 20:41]:
> > Currently, vinyl won't shutdown until all reader and writer threads
> > gracefully complete all their pending requests, which may take a while,
> > especially for writer threads that may happen to be doing compaction at
> > the time. This is annoying - there's absolutely no reason to delay
> > termination in such a case. Let's forcefully cancel all threads, like we
> > do in case of relay threads.
> >
> > This should fix sporadic vinyl/replica_quota test hang.
> >
> > Closes #3949
>
> OK to push.
Pushed to 2.1 and 1.10.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-02-12 9:02 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-11 17:37 [PATCH] vinyl: cancel reader and writer threads on shutdown Vladimir Davydov
2019-02-11 18:16 ` Konstantin Osipov
2019-02-12 9:02 ` Vladimir Davydov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox