From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Fri, 1 Jun 2018 21:51:35 +0300 From: Konstantin Osipov Subject: Re: [PATCH] vinyl: fix compaction vs checkpoint race resulting in invalid gc Message-ID: <20180601185135.GC22817@atlas> References: <09ab67b920b5207ac3b47d797412da94b1fe8336.1527769684.git.vdavydov.dev@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <09ab67b920b5207ac3b47d797412da94b1fe8336.1527769684.git.vdavydov.dev@gmail.com> To: Vladimir Davydov Cc: tarantool-patches@freelists.org List-ID: * Vladimir Davydov [18/05/31 15:51]: > The callback invoked upon compaction completion uses checkpoint_last() > to determine whether compacted runs may be deleted: if the max LSN > stored in a compacted run (run->dump_lsn) is greater than the LSN of the > last checkpoint (gc_lsn) then the run doesn't belong to the last > checkpoint and hence is safe to delete, see commit 35db70fac55c ("vinyl: > remove runs not referenced by any checkpoint immediately"). > > The problem is checkpoint_last() isn't synced with vylog rotation - it > returns the signature of the last successfully created memtx snapshot > and is updated in memtx_engine_commit_checkpoint() after vylog is > rotated. If a compaction task completes after vylog is rotated but > before snap file is renamed, it will assume that compacted runs do not > belong to the last checkpoint, although they do (as they have been > appended to the rotated vylog), and delete them. > > To eliminate this race, let's use vylog signature instead of snap > signature in vy_task_compact_complete(). > > Closes #3437 Pushed. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov