From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 19A162F99A for ; Sat, 18 May 2019 14:47:18 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0hlPvAgNJHKn for ; Sat, 18 May 2019 14:47:18 -0400 (EDT) Received: from smtp16.mail.ru (smtp16.mail.ru [94.100.176.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id CB25A2EEF5 for ; Sat, 18 May 2019 14:47:17 -0400 (EDT) Received: by smtp16.mail.ru with esmtpa (envelope-from ) id 1hS4MV-0006G3-Tq for tarantool-patches@freelists.org; Sat, 18 May 2019 21:47:16 +0300 Date: Sat, 18 May 2019 21:47:15 +0300 From: Konstantin Osipov Subject: [tarantool-patches] Re: [PATCH 05/10] vinyl: don't purge deleted runs from vylog on compaction Message-ID: <20190518184715.GD9448@atlas> References: <67424f76249971cf6c4fc5a361f294a0b3c88283.1558103547.git.vdavydov.dev@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <67424f76249971cf6c4fc5a361f294a0b3c88283.1558103547.git.vdavydov.dev@gmail.com> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org * Vladimir Davydov [19/05/17 17:54]: > After compacting runs, we first mark them as dropped (VY_LOG_DROP_RUN), > then try to delete their files unless they are needed for recovery from > the checkpoint, and finally mark them as not needed in the vylog > (VY_LOG_FORGET_RUN). There's a potential race sitting here: the problem > is the garbage collector might kick in after files are dropped, but > before they are marked as not needed. If this happens, there will be > runs that have two VY_LOG_FORGET_RUN records, which will break recovery: > > Run XX is forgotten, but not registered > > The following patches make the race more likely to happen so let's > eliminate it by making the garbage collector the only one who can mark > runs as not needed (i.e. write VY_LOG_FORGET_RUN record). There will > be no warnings, because the garbage collector silently ignores ENOENT > errors, see vy_gc(). > > Another good thing about this patch is that now we never yield inside > a vylog transaction, which makes it easier to remove the vylog latch > blocking implementation of transactional DDL. Should be a separate commit with a test? -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32