Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v4 0/3] gc/xlog: delay xlog cleanup until relays are subscribed
@ 2021-03-24 16:37 Cyrill Gorcunov via Tarantool-patches
  2021-03-24 16:37 ` [Tarantool-patches] [PATCH v4 1/3] " Cyrill Gorcunov via Tarantool-patches
                   ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: Cyrill Gorcunov via Tarantool-patches @ 2021-03-24 16:37 UTC (permalink / raw)
  To: tml; +Cc: Vladislav Shpilevoy

Take a look please.

v2:
 - rebase code to the fresh master branch
 - keep wal_cleanup_delay option name
 - pass wal_cleanup_delay as an option to gc_init, so it
   won't be dependent on cfg engine
 - add comment about gc_delay_unref in plain bootstrap mode
 - allow to setup wal_cleanup_delay dynamically
 - update comment in gc_wait_cleanup and call it conditionally
 - declare wal_cleanup_delay as a double
 - rename gc.cleanup_is_paused to gc.is_paused and update output
 - do not show ref counter in box.info.gc() output
 - update documentation
 - move gc_delay_unref inside relay_subscribe call which runs
   in tx context (instead of relay's context)
 - update tests:
   - add a comment why we need a temp space on replica node
   - use explicit insert/snapshot operations
   - shrkink the number of insert/snapshot to speedup testing
   - use "restart" instead of stop/start pair
   - use wait_log helper instead of own function
   - add is_paused test
v3:
 - fix changelog
 - rework box_check_wal_cleanup_delay, the replication_anon
   setting is considered only in box_set_wal_cleanup_delay,
   ie when config is checked and parsed, moreover the order
   of setup is set to be behind "replication_anon" option
   processing
 - delay cycle now considers deadline instead of per cycle
   calculation
 - use `double` type for timestamp
 - test update
   - verify `.is_paused` value
   - minimize number of inserts
   - no need to use temporary space, regular space works as well
   - add comments on why we should restart the master node
v4:
 - drop argument from gc_init(), since we're configuring delay
   value from load_cfg.lua script there is no need to read the
   delay early, simply start gc paused and unpause it on demand
 - move unpause message to main wait cycle
 - test update:
   - verify tests and fix replication/replica_rejoin since it waits
     for xlogs to be cleaned up too early
   - use 10 seconds for XlogGapError instead of 0.1 second, this is
     a common deadline value

issue https://github.com/tarantool/tarantool/issues/5806
branch gorcunov/gh-5806-xlog-gc-4

Cyrill Gorcunov (3):
  gc/xlog: delay xlog cleanup until relays are subscribed
  test: add a test for wal_cleanup_delay option
  test: box-tap/gc -- add test for is_paused field

 .../unreleased/add-wal_cleanup_delay.md       |   5 +
 src/box/box.cc                                |  41 ++
 src/box/box.h                                 |   1 +
 src/box/gc.c                                  |  85 ++++-
 src/box/gc.h                                  |  36 ++
 src/box/lua/cfg.cc                            |   9 +
 src/box/lua/info.c                            |   4 +
 src/box/lua/load_cfg.lua                      |   5 +
 src/box/relay.cc                              |   1 +
 src/box/replication.cc                        |   2 +
 test/app-tap/init_script.result               |   1 +
 test/box-tap/gc.test.lua                      |   3 +-
 test/box/admin.result                         |   2 +
 test/box/cfg.result                           |   4 +
 test/replication/gh-5806-master.lua           |   8 +
 test/replication/gh-5806-slave.lua            |   8 +
 test/replication/gh-5806-xlog-cleanup.result  | 356 ++++++++++++++++++
 .../replication/gh-5806-xlog-cleanup.test.lua | 155 ++++++++
 test/replication/replica_rejoin.result        |  16 +
 test/replication/replica_rejoin.test.lua      |   9 +
 20 files changed, 746 insertions(+), 5 deletions(-)
 create mode 100644 changelogs/unreleased/add-wal_cleanup_delay.md
 create mode 100644 test/replication/gh-5806-master.lua
 create mode 100644 test/replication/gh-5806-slave.lua
 create mode 100644 test/replication/gh-5806-xlog-cleanup.result
 create mode 100644 test/replication/gh-5806-xlog-cleanup.test.lua


base-commit: f4e248c0c13a46beee238fbebc38ef687ef09d02
-- 
2.30.2


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2021-03-26  7:05 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-24 16:37 [Tarantool-patches] [PATCH v4 0/3] gc/xlog: delay xlog cleanup until relays are subscribed Cyrill Gorcunov via Tarantool-patches
2021-03-24 16:37 ` [Tarantool-patches] [PATCH v4 1/3] " Cyrill Gorcunov via Tarantool-patches
2021-03-24 22:10   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-25 11:25     ` Cyrill Gorcunov via Tarantool-patches
2021-03-25 19:59       ` Vladislav Shpilevoy via Tarantool-patches
2021-03-25 21:02         ` Cyrill Gorcunov via Tarantool-patches
2021-03-25 21:29           ` Cyrill Gorcunov via Tarantool-patches
2021-03-25 23:50             ` Vladislav Shpilevoy via Tarantool-patches
2021-03-26  7:04               ` Cyrill Gorcunov via Tarantool-patches
2021-03-25 23:51           ` Vladislav Shpilevoy via Tarantool-patches
2021-03-26  7:04             ` Cyrill Gorcunov via Tarantool-patches
2021-03-24 16:37 ` [Tarantool-patches] [PATCH v4 2/3] test: add a test for wal_cleanup_delay option Cyrill Gorcunov via Tarantool-patches
2021-03-24 22:10   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-25 12:07     ` Cyrill Gorcunov via Tarantool-patches
2021-03-25 19:56       ` Vladislav Shpilevoy via Tarantool-patches
2021-03-25 20:19         ` Cyrill Gorcunov via Tarantool-patches
2021-03-24 16:37 ` [Tarantool-patches] [PATCH v4 3/3] test: box-tap/gc -- add test for is_paused field Cyrill Gorcunov via Tarantool-patches

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox