Tarantool development patches archive
 help / color / mirror / Atom feed
From: Konstantin Osipov <kostja@tarantool.org>
To: tarantool-patches@freelists.org
Subject: [tarantool-patches] Re: [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err
Date: Thu, 28 Jun 2018 18:25:30 +0300	[thread overview]
Message-ID: <20180628152530.GA6042@chai> (raw)
In-Reply-To: <2d8949f1e20f57f466d5ae56a131b86c5618a110.1530115423.git.k.belyavskiy@tarantool.org>

* Konstantin Belyavskiy <k.belyavskiy@tarantool.org> [18/06/28 18:13]:
> 
> +void
> +gc_xdir_clean_notify()
> +{
> +	/*
> +	 * Compare the current time with the time of the last run.
> +	 * This is needed in case of multiple failures to prevent
> +	 * from deleting all replicas.
> +	 */
> +	static int prev_time = 0;
> +	int cur_time = time(NULL);
> +	if (cur_time - prev_time < 1)

Please use fiber time.

> +	/**
> +	 * Mark consumer with least recent vclock as "dead" and
> +	 * invoke garbage collection. If nothing to delete find
> +	 * next alive consumer etc. Originally created for
> +	 * cases with running out of disk space because of
> +	 * disconnected replica.
> +	 */
> +	struct gc_consumer *leftmost =
> +	    gc_tree_first(&gc.consumers);
> +	/*
> +	 * Exit if no consumers left or if this consumer is
> +	 * not associated with replica (backup for example).
> +	 */

You should delete the consumer *only* if it's going to help with
garbage. Running out of space should not lead to deletion of all
replicas as consumers if they are not lagging behind.

> +	if (leftmost == NULL || leftmost->replica == NULL)
> +		return;
> +	int64_t signature = leftmost->signature;
> +	while (true) {
> +		gc_consumer_unregister(leftmost);
> +		leftmost = gc_tree_first(&gc.consumers);
> +		if (leftmost == NULL || leftmost->replica == NULL ||
> +		    leftmost->signature > signature) {
> +			gc_run();
> +			return;
> +		}
> +	}
> +}
> +

<cut>

>  const char *wal_mode_STRS[] = { "none", "write", "fsync", NULL };
> @@ -64,6 +65,8 @@ struct wal_thread {
>  	 * priority pipe and DOES NOT support yield.
>  	 */
>  	struct cpipe tx_prio_pipe;
> +	/** Return pipe from 'wal' to tx' */
> +	struct cpipe tx_pipe;

Why do you need a new pipe?

> 
> +static void
> +gc_status_update(struct cmsg *msg)
> +{
> +	gc_xdir_clean_notify();
> +	delete(msg);

delete()?!


>  {
> @@ -655,6 +665,15 @@ done:
>  		/* Until we can pass the error to tx, log it and clear. */
>  		error_log(error);
>  		diag_clear(diag_get());
> +		if (errno == ENOSPC) {
> +			struct cmsg *msg =
> +			    (struct cmsg*)calloc(1, sizeof(struct cmsg));

Please check calloc() return value and do nothing 
in case it's NULL.

> +			static const struct cmsg_hop route[] = {
> +				{gc_status_update, NULL}
> +			};
> +			cmsg_init(msg, route);
> +			cpipe_push(&wal_thread.tx_pipe, msg);
> +		}
> 
> index 895d938d5..11f1b7fdc 100644
> +...
> +-- add a little timeout so gc could finish job
> +require('fiber').sleep(0.01)

Please never sleep in a test case unconditionally. It doesn't work
reliably, especially in parallel runs.

The test case should test that replicas which are severely behind 
are abandoned, and replicas which are running well are not.

-- 
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov

  reply	other threads:[~2018-06-28 15:25 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-28 15:07 [tarantool-patches] [PATCH v3 0/2] " Konstantin Belyavskiy
2018-06-28 15:07 ` [tarantool-patches] [PATCH v3 1/2] replication: rename thread from tx to tx_prio Konstantin Belyavskiy
2018-06-28 15:07 ` [tarantool-patches] [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err Konstantin Belyavskiy
2018-06-28 15:25   ` Konstantin Osipov [this message]
2018-06-28 21:55     ` [tarantool-patches] Re: [tarantool-patches] " Konstantin Belyavskiy
2018-07-03  8:56       ` Kirill Yukhin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180628152530.GA6042@chai \
    --to=kostja@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='[tarantool-patches] Re: [PATCH v3 2/2] replication: force gc to clean xdir on ENOSPC err' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox