From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Fri, 5 Oct 2018 01:05:09 +0300 From: Konstantin Osipov Subject: Re: [PATCH 11/13] gc: separate checkpoint references from wal consumers Message-ID: <20181004220509.GK22855@chai> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: To: Vladimir Davydov Cc: tarantool-patches@freelists.org List-ID: * Vladimir Davydov [18/10/05 00:11]: Pinning and referencing are different concepts :) I agree with your reasoning and the patch is OK to push. > Initially, gc_consumer object was used for pinning both checkpoint and > WAL files, but commit 9c5d851d7830 ("replication: remove old snapshot > files not needed by replicas") changed that. Now whether a consumer pins > WALs or checkpoints or both depends on gc_consumer_type. This was done > so that replicas wouldn't prevent garbage collection of checkpoint > files, which they don't need after initial join is complete. > > The way the feature was implemented is rather questionable though: > - Since consumers of both types are stored in the same binary search > tree, we have to iterate through the tree to find the leftmost > checkpoint consumer, see gc_tree_first_checkpoint. This looks > inefficient and ugly. > - The notion of advancing a checkpoint consumer (gc_consumer_advance) > is dubious: there's no point to move on to the next checkpoint after > reading one - instead the consumer needs incremental changes, i.e. > WALs. > > To eliminate those questionable aspects and make the code easier for > understanding, let's separate WAL and checkpoint consumers. We do this > by removing gc_consumer_type and making gc_consumer track WALs only. > For pinning the files corresponding to a checkpoint a new object class > is introduced, gc_checkpoint_ref. To pin a checkpoint, gc_ref_checkpoint > needs to be called. It is passed the gc_checkpoint object to pin, the > consumer name, and the gc_checkpoint_ref to store the ref in. To unpin a > previously pinned checkpoint, gc_checkpoint_unref should be called. > > References are listed by box.info.gc() for each checkpoint under > 'references' key. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov