Re: [Tarantool-patches] [PATCH 5/5] replication: introduce anonymous replica.

[Vladislav Shpilevoy <v.shpilevoy@tarantool.org>]

Hi! Thanks for the fixes!

>> See 16 comments/questions below.
>>
>> On 15/12/2019 21:58, sergepetrenko wrote:
>>> This commit introduces anonymous replicas. Such replicas do not pollute
>>> _cluster table (they can only be read-only and have a zero id in return).
>>> An anonymous replica can be promoted to a normal one if needed.
>>
>> 1. Do we need the promotion? Seems like it was not asked for, but we will
>> need to support it forever in case we allow this now. Also it looks not
>> even, when I can promote, but can't demote.
> 
> Well, one of the use cases for anonymous replica, in my opinion, is a backup
> instance, which can be promoted once one of the normal instances gets down.
> (Almost like a hot standby instance).

Fair, then lets keep it.

> Regarding demotion, what if I add it later after giving it some thought?

Sounds good.

>>> +relay_final_join(io->fd, header->sync, &start_vclock, &stop_vclock);
>>> +say_info("final data sent.");
>>> +
>>> +struct xrow_header row;
>>> +/* Send end of WAL stream marker */
>>> +xrow_encode_vclock_xc(&row, &replicaset.vclock);
>>> +row.sync = header->sync;
>>> +coio_write_xrow(io, &row);
>>> +
>>> +/*
>>> + * Advance the WAL consumer state to the position where
>>> + * FINAL JOIN ended and assign it to the replica.
>>> + */
>>> +gc_consumer_advance(gc, &stop_vclock);
>>> +replica = replica_by_uuid(&instance_uuid);
>>> +if (replica->gc != NULL)
>>> +gc_consumer_unregister(replica->gc);
>>
>> 9. How is that possible that the former anon replica has gc?
> 
> Anon replica receives a gc registration in replica_set_id, which is called
> above.

I missed that, thanks. Then it looks ok.

>>>  /** Vinyl run info stored in .index file */
>>>  VY_INDEX_RUN_INFO = 100,
>>> diff --git a/src/box/relay.cc b/src/box/relay.cc
>>> index e849fcf4f..14644716d 100644
>>> --- a/src/box/relay.cc
>>> +++ b/src/box/relay.cc
>>> @@ -569,11 +569,17 @@ relay_subscribe_f(va_list ap)
>>>  cbus_pair("tx", relay->endpoint.name, &relay->tx_pipe,
>>>    &relay->relay_pipe, NULL, NULL, cbus_process);
>>>
>>> -/* Setup garbage collection trigger. */
>>> +/*
>>> + * Setup garbage collection trigger.
>>> + * Not needed for anonymous replicas, since they
>>> + * aren't registered with gc at all.
>>> + */
>>
>> 14. If a master does not register an anon replica as a gc consumer,
>> it will remove xlogs even if the replica didn't get them yet. Not
>> sure, if we want that behaviour. AFAIU, purpose of anon replicas
>> is to break the limit on vlock size about 32 instances only.
> 
> True, but if we do register replicas as gc consumers there is no way
> to exclude them, so such a replica, in case of failure, will stall gc on master
> forever. We need some mechanism, to remove anonymous replicas from gc
> consumers. Normal replicas can be removed by deleting their entries from
> _cluster, we cannot do this for anonymous replicas. And if we just remove
> replica from gc consumer on every disconnect, there is no point in registering
> it with gc at all, since the point of gc consumer, among others, is to wait for
> dead replicas to reconnect and collect xlogs.

How anon replicas are removed now? I mean when are they dropped automatically
from the master? Is it done on each disconnect?