From: Konstantin Osipov <kostja@tarantool.org> To: tarantool-patches@freelists.org Subject: [tarantool-patches] Re: [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err Date: Thu, 28 Jun 2018 18:25:30 +0300 [thread overview] Message-ID: <20180628152530.GA6042@chai> (raw) In-Reply-To: <2d8949f1e20f57f466d5ae56a131b86c5618a110.1530115423.git.k.belyavskiy@tarantool.org> * Konstantin Belyavskiy <k.belyavskiy@tarantool.org> [18/06/28 18:13]: > > +void > +gc_xdir_clean_notify() > +{ > + /* > + * Compare the current time with the time of the last run. > + * This is needed in case of multiple failures to prevent > + * from deleting all replicas. > + */ > + static int prev_time = 0; > + int cur_time = time(NULL); > + if (cur_time - prev_time < 1) Please use fiber time. > + /** > + * Mark consumer with least recent vclock as "dead" and > + * invoke garbage collection. If nothing to delete find > + * next alive consumer etc. Originally created for > + * cases with running out of disk space because of > + * disconnected replica. > + */ > + struct gc_consumer *leftmost = > + gc_tree_first(&gc.consumers); > + /* > + * Exit if no consumers left or if this consumer is > + * not associated with replica (backup for example). > + */ You should delete the consumer *only* if it's going to help with garbage. Running out of space should not lead to deletion of all replicas as consumers if they are not lagging behind. > + if (leftmost == NULL || leftmost->replica == NULL) > + return; > + int64_t signature = leftmost->signature; > + while (true) { > + gc_consumer_unregister(leftmost); > + leftmost = gc_tree_first(&gc.consumers); > + if (leftmost == NULL || leftmost->replica == NULL || > + leftmost->signature > signature) { > + gc_run(); > + return; > + } > + } > +} > + <cut> > const char *wal_mode_STRS[] = { "none", "write", "fsync", NULL }; > @@ -64,6 +65,8 @@ struct wal_thread { > * priority pipe and DOES NOT support yield. > */ > struct cpipe tx_prio_pipe; > + /** Return pipe from 'wal' to tx' */ > + struct cpipe tx_pipe; Why do you need a new pipe? > > +static void > +gc_status_update(struct cmsg *msg) > +{ > + gc_xdir_clean_notify(); > + delete(msg); delete()?! > { > @@ -655,6 +665,15 @@ done: > /* Until we can pass the error to tx, log it and clear. */ > error_log(error); > diag_clear(diag_get()); > + if (errno == ENOSPC) { > + struct cmsg *msg = > + (struct cmsg*)calloc(1, sizeof(struct cmsg)); Please check calloc() return value and do nothing in case it's NULL. > + static const struct cmsg_hop route[] = { > + {gc_status_update, NULL} > + }; > + cmsg_init(msg, route); > + cpipe_push(&wal_thread.tx_pipe, msg); > + } > > index 895d938d5..11f1b7fdc 100644 > +... > +-- add a little timeout so gc could finish job > +require('fiber').sleep(0.01) Please never sleep in a test case unconditionally. It doesn't work reliably, especially in parallel runs. The test case should test that replicas which are severely behind are abandoned, and replicas which are running well are not. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov
next prev parent reply other threads:[~2018-06-28 15:25 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-06-28 15:07 [tarantool-patches] [PATCH v3 0/2] " Konstantin Belyavskiy 2018-06-28 15:07 ` [tarantool-patches] [PATCH v3 1/2] replication: rename thread from tx to tx_prio Konstantin Belyavskiy 2018-06-28 15:07 ` [tarantool-patches] [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err Konstantin Belyavskiy 2018-06-28 15:25 ` Konstantin Osipov [this message] 2018-06-28 21:55 ` [tarantool-patches] Re: [tarantool-patches] " Konstantin Belyavskiy 2018-07-03 8:56 ` Kirill Yukhin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180628152530.GA6042@chai \ --to=kostja@tarantool.org \ --cc=tarantool-patches@freelists.org \ --subject='[tarantool-patches] Re: [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox