Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v14 0/6] qsync: implement packets filtering
@ 2021-09-10 15:29 Cyrill Gorcunov via Tarantool-patches
  2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 1/6] qsync: track confirmed lsn number on reads Cyrill Gorcunov via Tarantool-patches
                   ` (6 more replies)
  0 siblings, 7 replies; 21+ messages in thread
From: Cyrill Gorcunov via Tarantool-patches @ 2021-09-10 15:29 UTC (permalink / raw)
  To: tml; +Cc: Vladislav Shpilevoy

Guys, take a look please, once time permit. The questionable moments:

- use filter disabling procedure for join/recovery: we make it so since
  snapshot has promote record which fills initial limbo state

- need more tests to cover all possible scenarios

- I keep filter_confirm_rollback() as is but rereading Vlad's comment
  >
  > 9. What if rollback is for LSN > limbo's last LSN? It
  > also means nothing to do. The same for confirm LSN < limbo's
  > first LSN.
  >
  I presume I need to traverse limbo and test if incoming LSN is
  present inside current queue.

Anyway I send this version early to gather more comments, I hope
not that much left to implement to be ready for merging.

previous series https://lists.tarantool.org/tarantool-patches/20210804190752.488147-1-gorcunov@gmail.com/
branch gorcunov/gh-6036-rollback-confirm-14
issue https://github.com/tarantool/tarantool/issues/6036

v6:
 - use txn_limbo_terms name for structure
 - rebase on fresh sp/gh-6034-empty-limbo-transition branch
 - rework filtering chains
v8:
 - add ability to disable filtering for local recovery
   and join stages
 - update tests
v9:
 - opencode terms tracking
 - fix tests to use wait function since log output might
   be deferred by OS
v10:
 - rework FILTER_IN and FILTER_PROMOTE chains with more
   detailed packets inspection
 - preserve old naming for terms manipulations
 - require the packet's replica_id to match limbo owner_id
   all the time
v11-13: internal
v14:
 - use straightforward packet inspection by their type
   without more general type routing
 - tried to hide locking api inside limbo level but since
   journal completion is called from inside of sched fiber
   the lock owner get migrated which cause error thus leave
   explicit locking instead
 - added updating of limbo::confirmed_lsn since we need it
   for proper validation
 - added new error code to distinguish filter errors from
   anything else
 - use say_error instead of say_info
 - keep disabling of filtration inside initial join/recovery
   because we're filling initial limbo state

Cyrill Gorcunov (6):
  qsync: track confirmed lsn number on reads
  qsync: update confirmed lsn on initial promote request
  latch: add latch_is_locked helper
  qsync: order access to the limbo terms
  qsync: filter incoming synchro requests
  test: add replication/gh-6036-rollback-confirm

 .../gh-6036-qsync-filter-packets.md           |   9 +
 src/box/applier.cc                            |  26 +-
 src/box/box.cc                                |  30 +-
 src/box/errcode.h                             |   1 +
 src/box/memtx_engine.cc                       |   3 +-
 src/box/txn_limbo.c                           | 337 +++++++++++++++---
 src/box/txn_limbo.h                           |  85 ++++-
 src/lib/core/latch.h                          |  11 +
 test/box/error.result                         |   1 +
 test/replication/gh-6036-master.lua           |   1 +
 test/replication/gh-6036-node.lua             |  33 ++
 test/replication/gh-6036-replica.lua          |   1 +
 .../gh-6036-rollback-confirm.result           | 180 ++++++++++
 .../gh-6036-rollback-confirm.test.lua         |  92 +++++
 14 files changed, 747 insertions(+), 63 deletions(-)
 create mode 100644 changelogs/unreleased/gh-6036-qsync-filter-packets.md
 create mode 120000 test/replication/gh-6036-master.lua
 create mode 100644 test/replication/gh-6036-node.lua
 create mode 120000 test/replication/gh-6036-replica.lua
 create mode 100644 test/replication/gh-6036-rollback-confirm.result
 create mode 100644 test/replication/gh-6036-rollback-confirm.test.lua


base-commit: b0431cf8f47e9d081f6a402bc18edb1d6ad49847
-- 
2.31.1


^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2021-09-14 19:41 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-10 15:29 [Tarantool-patches] [PATCH v14 0/6] qsync: implement packets filtering Cyrill Gorcunov via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 1/6] qsync: track confirmed lsn number on reads Cyrill Gorcunov via Tarantool-patches
2021-09-12 15:44   ` Vladislav Shpilevoy via Tarantool-patches
2021-09-12 22:18     ` Cyrill Gorcunov via Tarantool-patches
2021-09-13  8:33     ` Serge Petrenko via Tarantool-patches
2021-09-13  8:50   ` Serge Petrenko via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 2/6] qsync: update confirmed lsn on initial promote request Cyrill Gorcunov via Tarantool-patches
2021-09-12 15:44   ` Vladislav Shpilevoy via Tarantool-patches
2021-09-12 22:25     ` Cyrill Gorcunov via Tarantool-patches
2021-09-13  8:52       ` Serge Petrenko via Tarantool-patches
2021-09-13 14:20         ` [Tarantool-patches] [RFC] qsync: overall design Cyrill Gorcunov via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 3/6] latch: add latch_is_locked helper Cyrill Gorcunov via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 4/6] qsync: order access to the limbo terms Cyrill Gorcunov via Tarantool-patches
2021-09-12 15:44   ` Vladislav Shpilevoy via Tarantool-patches
2021-09-13 10:52     ` Cyrill Gorcunov via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 5/6] qsync: filter incoming synchro requests Cyrill Gorcunov via Tarantool-patches
2021-09-12 15:44   ` Vladislav Shpilevoy via Tarantool-patches
2021-09-14 19:41     ` Cyrill Gorcunov via Tarantool-patches
2021-09-10 15:29 ` [Tarantool-patches] [PATCH v14 6/6] test: add replication/gh-6036-rollback-confirm Cyrill Gorcunov via Tarantool-patches
2021-09-12 15:44   ` Vladislav Shpilevoy via Tarantool-patches
2021-09-12 15:43 ` [Tarantool-patches] [PATCH v14 0/6] qsync: implement packets filtering Vladislav Shpilevoy via Tarantool-patches

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox