From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Mon, 23 Jul 2018 14:02:23 +0300 From: Vladimir Davydov Subject: Re: [PATCH 6/6] Introduce blackhole engine Message-ID: <20180723110223.amuv2i4op3yh63u2@esperanza> References: <638ad828056ebe88fba0692963fde70a52768e69.1532107326.git.vdavydov.dev@gmail.com> <20180720184247.GG4827@chai> <20180721123531.zzxiq2e2cnkt3wnt@esperanza> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180721123531.zzxiq2e2cnkt3wnt@esperanza> To: Konstantin Osipov Cc: tarantool-patches@freelists.org List-ID: On Sat, Jul 21, 2018 at 03:35:31PM +0300, Vladimir Davydov wrote: > > Please document how it could be used for logical replication. > > Will do. Come to think of it, using a blackhole space for replicating CALLs with the weaponry we have at hand isn't as simple as I thought it would be. The problem is we need to receive the snapshot when a replica is joined, but then unsubscribe from all changes except those done to the blackhole space that is used for logging CALLs. That is, we can't just mark all spaces except DD as replica local, because they won't be sent in a snapshot then. The only solution I can come up with now is using before_replace trigger for filtering out DML requests received from the master, but this isn't the best practice, because before_replace trigger is somewhat slow. If we had replication groups things would be different though. Logical replication would consist of the following steps then: 1. Create a new replication group on the master. Use the new group for creating all user spaces. 2. Create a blackhole space for logging CALLs. Do NOT include it in the new replication group (mark it global). 3. Join a replica to the master as a member of the replication group created on step 1 so that it receives the whole user data set in a snapshot. 4. Stop replication temporarily. 5. Install a trigger on the space used for logging CALLs. 6. Remove the replica from the replication group so that it won't receive any changes from the master except CALL log and resume replication. (ideally, the process should be automated by tarantool as much as possible) That said, I don't think it's worth exposing blackhole to the user until we have replication groups and some guidelines for logical replication. What about leaving blackhole undocumented for now to prevent users from abusing it in ways that we haven't thought through?