[tarantool-patches] Re: [PATCH v2 4/4] replication: use wal memory buffer to fetch rows

[Konstantin Osipov]

* Georgy Kirichenko <georgy at tarantool.org> [19/09/18 12:39]:
> A relay tries to create a wal memory cursor and follow them to relay
> data to it's replica. If a relay failed to attach to a wal memory buffer
> or went out of the buffer then the relay recovers xlogs from files and
> the makes a new try to attach.

I find this approach very, very elegant. I would add this comment to the
loop as well:

>  	while (!fiber_is_cancelled()) {
> +		/* Try to relay direct from wal memory buffer. */
> +		if (wal_relay(&relay->wal_relay, &relay->relay_vclock,
> +			      relay_send_cb, relay,
> +			      tt_sprintf("relay_%p", relay)) != 0) {
> +			relay_set_error(relay, diag_last_error(&fiber()->diag));
> +			break;
> +		/* Recover xlogs from files. */
> +		try {
> +			relay->r = recovery_new(relay->wal_dir, false,
> +					        &relay->relay_vclock);
> +			auto relay_guard = make_scoped_guard([&] {
> +				recovery_delete(relay->r);
> +				relay->r = NULL;
> +			});
> +			recover_remaining_wals(relay->r, &relay->stream,
> +					       NULL, true);
> +		} catch (Exception *e) {
> +			relay_set_error(relay, e);
> +			break;
> +		}

What I don't understand is what happens with relay thread during
the switch. Who owns the socket when the relaying is done from
in-memory buffer? How is the ownership passed back and forth? 
Does the relay thread suspend while in-memory replication is done,
or it is still responsible for receiving the acks? Please note
that to implement true sync replication you will need to process
acks in the wal thread as well.

Could you please document all of the above? I would prefer
comments in the code, not in the changeset - this way they are
more easily accessible to whoever has to work with this in the
future.

-- 
Konstantin Osipov, Moscow, Russia