From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: kostja@tarantool.org
Cc: tarantool-patches@freelists.org
Subject: [PATCH 0/5] Delete old WAL files if running out of disk space
Date: Sun, 7 Oct 2018 23:27:13 +0300 [thread overview]
Message-ID: <cover.1538942600.git.vdavydov.dev@gmail.com> (raw)
If a replica permanently stops working for some reason, it will pin WAL
files it would need to resume until it is deleted from the _cluster
system space or the master is restarted. This happens in production when
an admin drops a replica and forgets to remove it from the master, and
this is quite annoying, because it may result in ENOSPC errors on the
master.
This patch set attempts to mitigate this problem by making the WAL
thread delete old WAL files and shoot off old replicas automatically
when it runs out of disk space.
https://github.com/tarantool/tarantool/issues/3397
https://github.com/tarantool/tarantool/commits/dv/gh-3397-wal-auto-deletion
Vladimir Davydov (5):
xlog: fix filename in error messages
wal: preallocate disk space before writing rows
xlog: allow to limit number of files deleted by xdir_collect_garbage
wal: notify watchers about wal file removal
wal: delete old wal files when running out of disk space
CMakeLists.txt | 1 +
src/box/box.cc | 9 +-
src/box/gc.c | 67 +++++++++-
src/box/gc.h | 31 +++++
src/box/journal.c | 1 +
src/box/journal.h | 4 +
src/box/memtx_engine.c | 2 +-
src/box/relay.cc | 8 +-
src/box/txn.c | 1 +
src/box/vy_log.c | 2 +-
src/box/wal.c | 122 ++++++++++++++----
src/box/wal.h | 35 +++--
src/box/xlog.c | 121 ++++++++++++++++--
src/box/xlog.h | 43 ++++++-
src/box/xrow.h | 13 ++
src/errinj.h | 1 +
src/trivia/config.h.cmake | 1 +
test/box/errinj.result | 2 +
test/replication/gc_no_space.result | 234 ++++++++++++++++++++++++++++++++++
test/replication/gc_no_space.test.lua | 103 +++++++++++++++
test/replication/suite.ini | 2 +-
21 files changed, 739 insertions(+), 64 deletions(-)
create mode 100644 test/replication/gc_no_space.result
create mode 100644 test/replication/gc_no_space.test.lua
--
2.11.0
next reply other threads:[~2018-10-07 20:27 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-07 20:27 Vladimir Davydov [this message]
2018-10-07 20:27 ` [PATCH 1/5] xlog: fix filename in error messages Vladimir Davydov
2018-10-12 8:19 ` Vladimir Davydov
2018-10-16 19:07 ` [tarantool-patches] " Konstantin Osipov
2018-10-07 20:27 ` [PATCH 2/5] wal: preallocate disk space before writing rows Vladimir Davydov
2018-10-16 19:09 ` [tarantool-patches] " Konstantin Osipov
2018-10-07 20:27 ` [PATCH 3/5] xlog: allow to limit number of files deleted by xdir_collect_garbage Vladimir Davydov
2018-10-07 20:27 ` [PATCH 4/5] wal: notify watchers about wal file removal Vladimir Davydov
2018-10-07 20:27 ` [PATCH 5/5] wal: delete old wal files when running out of disk space Vladimir Davydov
2018-10-16 19:05 ` [tarantool-patches] Re: [PATCH 0/5] Delete old WAL files if " Konstantin Osipov
2018-10-17 8:20 ` Vladimir Davydov
2018-10-23 8:37 ` Vladimir Davydov
2018-10-23 8:46 ` Konstantin Osipov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1538942600.git.vdavydov.dev@gmail.com \
--to=vdavydov.dev@gmail.com \
--cc=kostja@tarantool.org \
--cc=tarantool-patches@freelists.org \
--subject='Re: [PATCH 0/5] Delete old WAL files if running out of disk space' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox