From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 22 Aug 2019 11:05:54 +0300 From: Vladimir Davydov Subject: Re: [tarantool-patches] Re: [PATCH v2 7/7] relay: join new replicas off read view Message-ID: <20190822080554.GM13834@esperanza> References: <9ed5cd28b89751e0f4edd8f8c9a57d66ba2bd5f8.1566233187.git.vdavydov.dev@gmail.com> <20190819205721.GH21602@atlas> <20190820081655.GR13834@esperanza> <20190820115333.GC8932@atlas> <20190820120518.GU13834@esperanza> <20190820135007.GC12831@atlas> <20190820140307.GV13834@esperanza> <20190821220807.GA8614@atlas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190821220807.GA8614@atlas> To: Konstantin Osipov Cc: tarantool-patches@freelists.org List-ID: On Thu, Aug 22, 2019 at 01:08:07AM +0300, Konstantin Osipov wrote: > * Vladimir Davydov [19/08/20 17:08]: > > On Tue, Aug 20, 2019 at 04:50:07PM +0300, Konstantin Osipov wrote: > > > * Vladimir Davydov [19/08/20 15:10]: > > > > Well, yeah, kinda. OTOH they do relay data to a replica that's why I > > > > named them relay_something :-/ Also, those functions live in relay.cc, > > > > which is consistent with the relay_ prefix. > > > > > > engine_send? I'll think about it. > > > > But the code is engine-agnostic now - it just opens read-view iterators > > and sends whatever they return to a replica :-/ > > OK. Let's move it out of relay then. It's OK to put it in a > special file or move to engine.cc. It's OK that it's > engine-agnostic- engine.cc can have code common to all engines. Okay, will do. > > relay prefix is also confusing since in replication "relaying" is > used to refer to re-sending some existing files or logs. > > The only issue about this patch is that since, unlike a snapshot > iterator, the code runs in tx thread, it may require some > throttling. If the network is fast enough it the send loop may run > at very high speeds. It could even hog 100% of CPU time and nearly > never yield CPU to other transactions. Good point. > > I see two options here: move it to a separate thread (memtx > iterators allow it, so it would be preferred), or throttle. Unfortunately, I don't think we can move this code out of the tx thread, because vinyl iterator must be run from tx (well, we could iterate over run files in a thread, but not over the memory level; and we don't want to suspend dumps/compaction until we're done joining a replica). So I guess I will simply call fiber_sleep() periodically, similarly to how we handle index build.