From: Serge Petrenko <sergepetrenko@tarantool.org> To: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>, tarantool-patches@dev.tarantool.org Subject: Re: [Tarantool-patches] [PATCH 8/8] raft: fix crash on death timeout decrease Date: Wed, 16 Dec 2020 16:10:32 +0300 [thread overview] Message-ID: <ddff899c-6c08-910c-6084-e641cf1512d0@tarantool.org> (raw) In-Reply-To: <607c7c67a3d31471a8c8d2fcb13c86615bd48024.1607879643.git.v.shpilevoy@tarantool.org> 13.12.2020 20:15, Vladislav Shpilevoy пишет: > If death timeout was decreased during waiting for leader death or > discovery to a new value making the current death waiting end > immediately, it could crash in libev. > > Because it would mean the remaining time until leader death became > negative. The negative timeout was passed to libev without any > checks, and there is an assertion, that a timeout should always > be >= 0. > > This commit makes raft code covered almost on 100%, not counting > one 'unreachable()' place. > > Closes #5303 LGTM. > --- > src/lib/raft/raft.c | 2 ++ > test/unit/raft.c | 26 +++++++++++++++++++++++++- > test/unit/raft.result | 7 ++++++- > 3 files changed, 33 insertions(+), 2 deletions(-) > > diff --git a/src/lib/raft/raft.c b/src/lib/raft/raft.c > index 4f6a5ee5e..4ea4fc3f8 100644 > --- a/src/lib/raft/raft.c > +++ b/src/lib/raft/raft.c > @@ -924,6 +924,8 @@ raft_cfg_death_timeout(struct raft *raft, double death_timeout) > struct ev_loop *loop = raft_loop(); > double timeout = raft_ev_timer_remaining(loop, &raft->timer) - > raft->timer.at + raft->death_timeout; > + if (timeout < 0) > + timeout = 0; > raft_ev_timer_stop(loop, &raft->timer); > raft_ev_timer_set(&raft->timer, timeout, timeout); > raft_ev_timer_start(loop, &raft->timer); > diff --git a/test/unit/raft.c b/test/unit/raft.c > index 2c3935cbf..11e101777 100644 > --- a/test/unit/raft.c > +++ b/test/unit/raft.c > @@ -971,7 +971,7 @@ raft_test_election_quorum(void) > static void > raft_test_death_timeout(void) > { > - raft_start_test(4); > + raft_start_test(9); > struct raft_node node; > raft_node_create(&node); > > @@ -1018,6 +1018,30 @@ raft_test_death_timeout(void) > "{0: 2}" /* Vclock. */ > ), "enter candidate state when the new death timeout expires"); > > + /* Decrease timeout to earlier than now. */ > + > + is(raft_node_send_leader(&node, > + 3 /* Term. */, > + 2 /* Source. */ > + ), 0, "message from leader"); > + is(node.raft.leader, 2, "leader is accepted"); > + is(node.raft.state, RAFT_STATE_FOLLOWER, "became follower"); > + > + raft_run_for(timeout / 2); > + raft_node_cfg_death_timeout(&node, timeout / 4); > + double ts = raft_time(); > + raft_run_next_event(); > + ok(raft_time() == ts, "death is detected immediately"); > + ok(raft_node_check_full_state(&node, > + RAFT_STATE_CANDIDATE /* State. */, > + 0 /* Leader. */, > + 4 /* Term. */, > + 1 /* Vote. */, > + 4 /* Volatile term. */, > + 1 /* Volatile vote. */, > + "{0: 3}" /* Vclock. */ > + ), "enter candidate state"); > + > raft_node_destroy(&node); > raft_finish_test(); > } > diff --git a/test/unit/raft.result b/test/unit/raft.result > index fcd180cfc..8188d1806 100644 > --- a/test/unit/raft.result > +++ b/test/unit/raft.result > @@ -176,11 +176,16 @@ ok 9 - subtests > ok 10 - subtests > *** raft_test_election_quorum: done *** > *** raft_test_death_timeout *** > - 1..4 > + 1..9 > ok 1 - leader notification > ok 2 - follow the leader > ok 3 - the leader still is considered alive > ok 4 - enter candidate state when the new death timeout expires > + ok 5 - message from leader > + ok 6 - leader is accepted > + ok 7 - became follower > + ok 8 - death is detected immediately > + ok 9 - enter candidate state > ok 11 - subtests > *** raft_test_death_timeout: done *** > *** raft_test_enable_disable *** -- Serge Petrenko
next prev parent reply other threads:[~2020-12-16 13:10 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-13 17:15 [Tarantool-patches] [PATCH 0/8] Raft module, part 4 - unit tests Vladislav Shpilevoy 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 1/8] fakesys: fix ev_is_active not working on fake timers Vladislav Shpilevoy 2020-12-15 9:42 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 2/8] fakesys: introduce fakeev_timer_remaining() Vladislav Shpilevoy 2020-12-15 9:43 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 3/8] raft: introduce raft_ev Vladislav Shpilevoy 2020-12-15 10:02 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 4/8] test: introduce raft unit tests Vladislav Shpilevoy 2020-12-13 18:10 ` Vladislav Shpilevoy 2020-12-16 13:03 ` Serge Petrenko 2020-12-17 22:44 ` Vladislav Shpilevoy 2020-12-18 8:17 ` Serge Petrenko 2020-12-20 17:28 ` Vladislav Shpilevoy 2020-12-21 7:36 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 5/8] raft: fix crash when received 0 term message Vladislav Shpilevoy 2020-12-16 13:05 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 6/8] raft: fix ignorance of bad state receipt Vladislav Shpilevoy 2020-12-16 13:06 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 7/8] raft: fix crash on election timeout decrease Vladislav Shpilevoy 2020-12-16 13:08 ` Serge Petrenko 2020-12-13 17:15 ` [Tarantool-patches] [PATCH 8/8] raft: fix crash on death " Vladislav Shpilevoy 2020-12-16 13:10 ` Serge Petrenko [this message] 2020-12-21 16:50 ` [Tarantool-patches] [PATCH 0/8] Raft module, part 4 - unit tests Vladislav Shpilevoy 2020-12-21 17:29 ` Vladislav Shpilevoy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=ddff899c-6c08-910c-6084-e641cf1512d0@tarantool.org \ --to=sergepetrenko@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH 8/8] raft: fix crash on death timeout decrease' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox