From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp3.mail.ru (smtp3.mail.ru [94.100.179.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id DD543452566 for ; Sat, 16 Nov 2019 23:34:16 +0300 (MSK) From: Georgy Kirichenko Date: Sat, 16 Nov 2019 23:34:15 +0300 Message-ID: <13345393.mysJK4fUO5@home.lan> In-Reply-To: <20191116115607.GE14490@atlas> References: <20191114125705.26760-1-maria.khaydich@tarantool.org> <3021966.aeNJFYEL58@home.lan> <20191116115607.GE14490@atlas> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Subject: Re: [Tarantool-patches] [PATCH] Trigger on vclock change List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Konstantin Osipov Cc: tarantool-patches@dev.tarantool.org On Saturday, November 16, 2019 2:56:07 PM MSK Konstantin Osipov wrote: > * Georgy Kirichenko [19/11/15 22:59]: > > Please, point me out first how your claims related to my approach. Beca= use > > you made no effort to understand the approach. Even did not ask for very > > brief explanation. >=20 > Alright, you claim I am not asking for a brief explanation. I am > asking for an RFC, which is the best way to explain ideas we have. Yes, of course, the RFC is the best way to explain my ideas, but today I ha= ve=20 only a proof of concepts which requires some further investigation to=20 understand how can I implement it (as I pretty sure the RFC should contain = an=20 implementation plan). Just after =D0=B5he raw patch would be working I defi= nitely=20 will write a RFC and any other documentation. The only thing I asked you to= =20 discuss this proof of concept with you as the research is not done yet. >=20 > But fine, I am asking you: > > > If you have a miracle design, and you happen to not want to send > > > an RFC, you still can prove it by sending a patch. > >=20 > > The next wrong suggestion. I have a concrete design which was shared and > > discussed. > >=20 > > > Last time it didn't work: your refused to send an RFC for in-memory = WAL > > > - > >=20 > > and the patch can't pass the code review for over 3 months. > > Please read my previous message and find out why this patchset is on ho= ld. > > To be concrete, the patch is not passed the review because of: > > 1. Bad gc design which I want to fix first, and I already answered why > > your > > approach to fix it is not even working. Yes, you could not / did not wa= nt > > to object. >=20 > What is wrong with GC and how exactly do you want to "fix" it? We have discussed some points with you verbally (about 3-4 month ago). The main point is: the way of information processing is weird: 1. WAL has the full information about the wal directory (xlogs and their=20 boundaries) 2. WAL process the wal directory cleanup 3. We filter out all this information while relaying (as a relay has only a= =20 stream of rows) 4. We try to restore some of this information using on_close_log recovery=20 trigger. 5. We send recovered boundaries to TX and tx tread reconstruct the relay or= der=20 loosing really relay vclocks (as they mapped to the local xlog history) 6. TX sends the oldest vclock back to wal 7. There is some issues with making a consumer inactive. For instance if we= =20 deactivated a consumer could survive, for instance if deleted xlog was alre= ady=20 send by an applier but not reported yet (I do not even know how it could be= =20 fixed in the current design). Maybe it is working, but I afraid, this was done without any thinking about= =20 the future development (I mean the synchronous replication). Let me explain= =20 why. 1. WAL should receive all relay states as soon as possible. 2. The set of relay vclocks is enough to perform garbage collection (as we= =20 could form a vclock with is the lower bound of the set) So I wish the garbage collection would be implemented using direct relay to= =20 wal reporting. In this circumstances I was in need to implement a structure= (I=20 named it as matrix clock - mclcok) which able to contain relay vclocks and= =20 evaluate a vclock which is lower bound of n-members of the mclcock. The mclock could be used to get the n-majority vclock as wel as the lowest= =20 boundary of all vclock alive. The mclock is already implemented as well as new gc design (wal knows about= =20 all relay vclock and the first vclock locked by TX - checkpoint or join rea= d=20 view).