[tarantool-patches] [xlog 1/1] xlog: Remove inprogress files on start
Vladimir Davydov
vdavydov.dev at gmail.com
Mon Jun 18 12:44:07 MSK 2018
0. Please carefully read and follow our guideline on how to submit
patches:
https://tarantool.io/en/doc/1.9/dev_guide/developer_guidelines.html#how-to-submit-a-patch-for-review
In particular, the subject prefix should be PATCH (default if you use
git-format-patch), not subsystem name, and the branch name as well as
the GitHub ticket should be given as hyperlinks.
When you resubmit a new version of your patch, please don't forget to
append the version to the subject prefix (e.g. PATCH v2), and write a
brief change log, e.g. see
https://www.freelists.org/post/tarantool-patches/PATCH-v2-box-make-gc-info-public
I'm kinda tired of reiterating those simple rules over and over
again. I think I'll simply ignore all patches that don't conform to
the guideline...
Also, before submitting a patch for review, please make sure there
are no stray hunks, like adding/removing blank lines. Your patch has
a few. Fixing a comment or removing a blank line is OK only if it
goes to a hunk you need anyway.
1. engine_vtab::commit_checkpoint() must not fail, because if it does,
it may leave the box in inconsistent state, when there's no vinyl
checkpoint corresponding to a memtx snapshot. An attempt to recover
from such a state will fail. That's why we panic in
memtx_engine_commit_checkpoint().
To test deletion of snap.inprogress files, you don't need to change
that - it's enough to set the error injection that delays renaming of
the snap file (it already exists), start checkpoint in a fiber, and
then restart the server.
2. Removal of engine-specific files should be done in engine callbacks.
For instance, snap.inprogress files should probably be removed from
memtx_engine_end_recovery().
3. Scanning vinyl_dir is totally wrong, because information about all
vinyl files, even .inprogress ones, is stored in vylog. Moreover,
code that removes incomplete vinyl files already exists - see
vinyl_engine_end_recovery() - all you need to do is implement removal
of .inprogress files there.
On Sat, Jun 09, 2018 at 05:46:37PM +0300, Ilya Markov wrote:
> When tarantool crashes during writing to vylog, index, run or snapshot,
> inprogress files remain. But garbage collector doesn't take into account
> these files. So they remain until they are removed manually.
>
> Fix this with adding function to box_cfg which traverses memtx_dir and
> vinyl_dir and removes inprogress files.
>
> * Add 4 errinj which simulate the crash before renaming inprogress
> files.
> * Change signature of engine_commit_checkpoint function in order to
> support error injection in snapshot commit.
>
> Closes #3406
> ---
> branch: gh-3406-remove-inprogress-files
> src/box/box.cc | 20 +++++-
> src/box/engine.c | 3 +-
> src/box/engine.h | 2 +-
> src/box/memtx_engine.c | 21 +++++-
> src/box/sysview_engine.c | 3 +-
> src/box/vinyl.c | 3 +-
> src/box/vy_log.c | 4 ++
> src/box/vy_run.c | 14 ++++
> src/box/xlog.c | 73 +++++++++++++++++++-
> src/box/xlog.h | 13 ++++
> src/errinj.h | 4 ++
> test/box/errinj.result | 171 ++++++++++++++++++++++++++++++++++++++++++++++-
> test/box/errinj.test.lua | 59 ++++++++++++++++
> 13 files changed, 378 insertions(+), 12 deletions(-)
More information about the Tarantool-patches
mailing list