From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-f196.google.com (mail-oi1-f196.google.com [209.85.167.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C4288469719 for ; Mon, 24 Feb 2020 15:31:47 +0300 (MSK) Received: by mail-oi1-f196.google.com with SMTP id c16so8768953oic.3 for ; Mon, 24 Feb 2020 04:31:47 -0800 (PST) MIME-Version: 1.0 References: <2175956.ElGaqSPkdT@localhost> <20200222204930.GA23200@atlas> <9655697.nUPlyArG6x@localhost> <20200224101848.GE18378@atlas> In-Reply-To: <20200224101848.GE18378@atlas> From: =?UTF-8?B?0JPQtdC+0YDQs9C40Lkg0JrQuNGA0LjRh9C10L3QutC+?= Date: Mon, 24 Feb 2020 15:31:34 +0300 Message-ID: Content-Type: multipart/alternative; boundary="000000000000e82da4059f518db9" Subject: Re: [Tarantool-patches] [PATCH v3 0/4] replication: fix applying of rows originating from local instance List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Konstantin Osipov , Georgy Kirichenko , tarantool-patches@dev.tarantool.org, Serge Petrenko , Vladislav Shpilevoy , alexander.turenko@tarantool.org --000000000000e82da4059f518db9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Please read messages before answering. I did never say that: > You've been suggesting that filtering on the master is safer. I said it safer do to it on the replica side and replica should not rely on master correctness. > I pointed out it's not, there is no way to guarantee (even in theory) correctness/safety if replica if master is malfunctioning. Excuse my but this is demagogy, we talk about what is more safer but not absolutely safety. >The situation is symmetrical. Both peers do not have the whole >picture. You can make either of the peers responsible for the >decision, then the other peer will need to supply the missing >bits. No, you are wrong. A master has only one information source about the stream it should send to a replica whereas a replica could connect to many masters to fetch proper data (from one or many masters). And we already implemented similar logic - a voting protocol and yoh should known about it.Additionally my approach allows to collect all corresponding logic as filtering of concurrent streams, vclock following, subcriptions and replication groups which are not implemented yet, registration and whatever else in one module at replica side. >I do not think the scope of this issue has ever been protecting >against hacked masters. It has never been a goal of the protocol >either. A hacked master could be a master with an implementation error and we should be able to detech such error as soon as possible. But if a replica will not check an incomming stream there is no way to prevent fatal data losses. >This was added for specific reasons. There is no known reason the >master should send unnecessary data to replica or replica fast >path should get slower. I am afraid you did not understand me. I did not ever said that I am against any optimization which could make replication faster. I completely against any attempts to rely on an optimiztion logic. If a master allows to skip unrequired rows then replica should not rely on this code corectness. In other words, if some input stream could broke replica the replica should protect itself agains such data. This is not the replicas master responsibility. =D0=BF=D0=BD, 24 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 13:18, Konst= antin Osipov : > * Georgy Kirichenko [20/02/23 12:21]: > > > Please do not think you are the only person who knows about byzantine > faults. > > Also there is little relevance between byzantine faults and my > suggestion to > > enforce replica-side checking. > > You've been suggesting that filtering on the master is safer. I > pointed out it's not, there is no way to guarantee > (even in theory) correctness/safety if replica if master is > malfunctioning. > > I merely pointed out that your safety argument has no merit. > > There are no other practical advantages of filtering on replica > either: there is a disadvantage, more traffic and more filtering work to > do > inside tx thread (as opposed to relay/wal thread if done on > master). > > It is also against the current responsibilities of IPROTO_SUBSCRIBE: the > concept of a subscription is that replica specifies what it is > interested in. Specifically, it specifies vclock components it's. > You suggest to make the replica responsible for > submitting its vclock, but the master decide what to do with it - > this splits the decision making logic between the two, making the > whole thing harder to understand. > > IPROTO_SUBSCRIBE responsibility layout today is typical for a > request-response protocol: the master, being the server, executes > the command as specified by the client (the replica), and the > replica runs the logic to decide what command to issue. > > You suggest to change it because of some theoretical concerns you > have. > > > In any case filtering on the master side is the most worst thing we > could do. > > In this case master has only one peer and have no chance to make a > proper > > decision if replica is broken. And we have no chance to know about it > (except > > assert which are excluded from release builds, or panic messages). For > > instance if master skipped some rows then there are no any tracks of th= e > > situation we could detect. > > The situation is symmetrical. Both peers do not have the whole > picture. You can make either of the peers responsible for the > decision, then the other peer will need to supply the missing > bits. There is no way you can make it safer by changing who makes > the decision, but you can certainly make it more messed up by > splitting this logic or going against an established layout. > > If you have a specific example why things will improve if done > otherwise - in the number of packets, or traffic, or some other > measurable way, you should point it out. > > > In the opposite case a replica could connect to as many masters as they > need > > to filter out all invalid data or hacked masters. At least we could > enforce > > replication stream meta checking. > > I do not think the scope of this issue has ever been protecting > against hacked masters. It has never been a goal of the protocol > either. > > > Two major point I would like to mention are: > > 1. Replica could consistently follow all vclock members and apply all > > transactions without gaps (I already got rid of them, I hope you > remember) > > 2. Replica could protect itself against concurrent local writes (one wa= s > made > > locally, the second one is returned from master) > > This was added for specific reasons. There is no known reason the > master should send unnecessary data to replica or replica fast > path should get slower. > > -- > Konstantin Osipov, Moscow, Russia > https://scylladb.com > --000000000000e82da4059f518db9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Please read messages before answering. I did never sa= y that:=C2=A0
> You've been suggesting that filtering on t= he master is safer.
I said it safer do to it on the replica side = and replica should not rely on master correctness.
>= ; I pointed out it's not, there is no way to guarantee
(even in th= eory) correctness/safety if replica if master is
malfunctioning.
Excuse my but this is demagogy, we talk about what is more safer but not a= bsolutely=C2=A0safety.
>The situation is symmetrical. Both= peers do not have the whole
>picture. You can make either of the p= eers responsible for the
>decision, then the other peer will need to = supply the missing
>bits.
No, you are wrong. A master has only on= e information source about the stream it should send to a replica whereas
=C2=A0a replica could=C2=A0connect to many masters to fetch proper= data (from one or many masters). And we already implemented similar logic = -=C2=A0
a voting protocol and yoh should known about it.Additiona= lly my approach=C2=A0allows to collect all corresponding logic as filtering=
=C2=A0of concurrent streams, vclock following, subcriptions and = replication groups which are not implemented yet, registration and whatever= else in one module at replica side.
>I do not think = the scope of this issue has ever been protecting
>against hacked ma= sters. It has never been a goal of the protocol
>either.
A = hacked master could be a master with an implementation error and we should = be able to detech such error as soon as possible. But if a replica will not=
check an incomming stream there is no way to prevent fatal data = losses.
>This was added for specific reasons. There is no know= n reason the
>master should send unnecessary data to replica or rep= lica fast
>path should get slower.
I am afraid you did not unders= tand me. I did not ever said that I am against any optimization which could= make replication faster.
I completely against any attempts to re= ly on an optimiztion logic. If a master allows to skip unrequired rows then= replica should not rely on this code corectness.
=C2=A0In other = words, if some input stream could broke replica the replica should protect = itself agains such data.=C2=A0This is not the replicas master responsibilit= y.

=D0=BF=D0=BD, 24 =D1=84=D0=B5=D0=B2=D1=80. 2020 =D0=B3. =D0=B2 = 13:18, Konstantin Osipov <kos= tja.osipov@gmail.com>:
* Georgy Kirichenko <kirichenkoga@gmail.com> [20/02/23 12:21]:

> Please do not think you are the only person who knows about byzantine = faults.
> Also there is little relevance between byzantine faults and my suggest= ion to
> enforce replica-side checking.

You've been suggesting that filtering on the master is safer. I
pointed out it's not, there is no way to guarantee
(even in theory) correctness/safety if replica if master is
malfunctioning.

I merely pointed out that your safety argument has no merit.

There are no other practical advantages of filtering on replica
either: there is a disadvantage, more traffic and more filtering work to do=
inside tx thread (as opposed to relay/wal thread if done on
master).

It is also against the current responsibilities of IPROTO_SUBSCRIBE: the concept of a subscription is that replica specifies what it is
interested in. Specifically, it specifies vclock components it's.
You suggest to make the replica responsible for
submitting its vclock, but the master decide what to do with it -
this splits the decision making logic between the two, making the
whole thing harder to understand.

IPROTO_SUBSCRIBE responsibility layout today is typical for a
request-response protocol: the master, being the server, executes
the command as specified by the client (the replica), and the
replica runs the logic to decide what command to issue.

You suggest to change it because of some theoretical concerns you
have.

> In any case filtering on the master side is the most worst=C2=A0 thing= we could do.
> In this case master has only one peer and have no chance to make a pro= per
> decision if replica is broken. And we have no chance to know about it = (except
> assert which are excluded from release builds, or panic messages). For=
> instance if master skipped some rows then there are no any tracks of t= he
> situation we could detect.

The situation is symmetrical. Both peers do not have the whole
picture. You can make either of the peers responsible for the
decision, then the other peer will need to supply the missing
bits. There is no way you can make it safer by changing who makes
the decision, but you can certainly make it more messed up by
splitting this logic or going against an established layout.

If you have a specific example why things will improve if done
otherwise - in the number of packets, or traffic, or some other
measurable way, you should point it out.

> In the opposite case a replica could connect to as many masters as the= y need
> to filter out all invalid data or hacked masters. At least we could en= force
> replication stream meta checking.

I do not think the scope of this issue has ever been protecting
against hacked masters. It has never been a goal of the protocol
either.

> Two major point I would like to mention are:
> 1. Replica could consistently follow all vclock members and apply all =
> transactions without gaps (I already got rid of them, I hope you remem= ber)
> 2. Replica could protect itself against concurrent local writes (one w= as made
> locally, the second one is returned from master)

This was added for specific reasons. There is no known reason the
master should send unnecessary data to replica or replica fast
path should get slower.

--
Konstantin Osipov, Moscow, Russia
https= ://scylladb.com
--000000000000e82da4059f518db9--