From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Thu, 6 Jun 2019 16:15:15 +0300 From: Vladimir Davydov Subject: Re: [tarantool-patches] Re: [PATCH 10/10] vinyl: get rid of the latch protecting vylog buffer Message-ID: <20190606131514.uhybh3rbgnmodjqw@esperanza> References: <56d1e35f6d54b87ed8b359af3ddfe4845246dd85.1558103547.git.vdavydov.dev@gmail.com> <20190601084436.GH29429@atlas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190601084436.GH29429@atlas> To: Konstantin Osipov Cc: tarantool-patches@freelists.org List-ID: On Sat, Jun 01, 2019 at 11:44:36AM +0300, Konstantin Osipov wrote: > * Vladimir Davydov [19/05/17 17:54]: > > The latch is needed solely to synchronize access to the transaction > > write buffer, which is shared by all transactions. We don't need it to > > sync vylog readers vs writers as everything is done from the same > > thread. So to get rid of the latch, which is a prerequisite for > > transactional DDL, as it makes even simply Vinyl DDL operations > > yielding, we need to rework the buffer management. > > > > To achieve that, this patch introduces a separate object for each vylog > > transaction, which stores the list of records to be committed. Once a > > transaction is ready to be committed, it is sent to the vylog thread, > > which writes it to vylog and then wakes up the awaiting fiber. For > > non-discardable transactions, it doesn't wakeup any fiber, the fiber > > returns immediately and the transaction is freed automatically. > > > > To allow that, transactions are now allocated on lsregion, which acts > > as a queue: new transactions are appended to the tail, committed > > transactions are popped from the head. > > The same could be achieved with xlog savepoints that you > introduced earlier, if you allow multiple "savepoints" to be present > in xlog tx, each of which could be rolled back independently (the > tail of the buffer would have to be moved on rollback). Before writing a transaction to the xlog buffer, we need to accumulate its statements in a buffer in the tx thread. We can either malloc() them and then free(), but I considered it to have too much overhead. So I decided to allocate all vylog statements using the same allocator - lsregion - acting as a FIFO. Possibly, I'm wrong about it, and using malloc() is just fine. > > I mean, honestly, if we're getting rid of the latch, there is no > point of having adding a separate thread on the way, it just takes > a little extra effort. And better yet, we could kill entire vy_log > altogether and store everything in xlog. We need to discuss that f2f, I think.