From: Cyrill Gorcunov via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: tml <tarantool-patches@dev.tarantool.org> Cc: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Subject: [Tarantool-patches] [PATCH v9 0/2] relay: provide downstream lag information Date: Thu, 17 Jun 2021 18:48:33 +0300 [thread overview] Message-ID: <20210617154835.315576-1-gorcunov@gmail.com> (raw) Guys, take a look once time permit, hopefully manage to address all comments. Previous series at https://lists.tarantool.org/tarantool-patches/20210607155519.109626-1-gorcunov@gmail.com/ v4 (by Vlad): - add a test case - add docbot request - dropped off xrow_encode_vclock_timed, we use opencoded assignment for tm value when send ack - struct awstat renamed to applier_wal_stat. Vlad I think this is better name than "applier_lag" because this is statistics on WAL, we simply track remote WAL propagation here, so more general name is better for grep sake and for future extensions - instead of passing applier structure we pass replica_id - the real keeper of this statistics comes into "replica" structure thus unbound of applier itself - for synchro entries we pass a pointer to the applier_wal_stat instead of using replica_id = 0 as a sign that we don't need to update statistics for initial and final join cases - to write and read statistics we provide wal_stat_update and wal_stat_ack helpers to cover the case where single ACK spans several transactions v8: - make one branch less in apply_synchro_row() - keep applier_txn_start_tm inside replica stucture - rename wal_stat to replica_cb_data since this is more logical for case where we have no general stat engine - make applier to send timestamp so that relay will compute delta upon the read, the lag is kept permanently until new write happens - extend doc and changelog a bit - keep reading of relay's lag from TX thread without any modifications because relay get deleted from TX thread and set to non-RELAY_FOLLOW state, thus any attempt to read it won't success. To be honest there is a small race window present: write doubles are not atomic operation thus we might read partially updated timestamp similarly as we have with @idle field already. I think this should be addressed separately and better without heavy cmsg engine involved but with rw lock instead or plain atomics. v9 (Vlad and Serge): - update of transaction lag for reading by TX thread done via cbus message - use last timestamp from transaction to account - verify that we really need to test for replica being non-nil in applier reader - update docs - update a testcase branch gorcunov/gh-5447-relay-lag-9 issue https://github.com/tarantool/tarantool/issues/5447 Cyrill Gorcunov (2): applier: send transaction's first row WAL time in the applier_writer_f relay: provide information about downstream lag .../unreleased/gh-5447-downstream-lag.md | 6 + src/box/applier.cc | 97 +++++++++++-- src/box/lua/info.c | 3 + src/box/relay.cc | 94 ++++++++++++- src/box/relay.h | 6 + src/box/replication.cc | 1 + src/box/replication.h | 5 + .../replication/gh-5447-downstream-lag.result | 128 ++++++++++++++++++ .../gh-5447-downstream-lag.test.lua | 57 ++++++++ 9 files changed, 378 insertions(+), 19 deletions(-) create mode 100644 changelogs/unreleased/gh-5447-downstream-lag.md create mode 100644 test/replication/gh-5447-downstream-lag.result create mode 100644 test/replication/gh-5447-downstream-lag.test.lua base-commit: b5f0dc4db9aef9618f56b0bcb4a7b82a59591784 -- 2.31.1
next reply other threads:[~2021-06-17 15:48 UTC|newest] Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-17 15:48 Cyrill Gorcunov via Tarantool-patches [this message] 2021-06-17 15:48 ` [Tarantool-patches] [PATCH v9 1/2] applier: send transaction's first row WAL time in the applier_writer_f Cyrill Gorcunov via Tarantool-patches 2021-06-18 9:51 ` Serge Petrenko via Tarantool-patches 2021-06-18 18:06 ` Cyrill Gorcunov via Tarantool-patches 2021-06-21 8:35 ` Serge Petrenko via Tarantool-patches 2021-06-17 15:48 ` [Tarantool-patches] [PATCH v9 2/2] relay: provide information about downstream lag Cyrill Gorcunov via Tarantool-patches 2021-06-18 9:50 ` Serge Petrenko via Tarantool-patches 2021-06-20 14:37 ` Vladislav Shpilevoy via Tarantool-patches 2021-06-21 8:44 ` Cyrill Gorcunov via Tarantool-patches 2021-06-21 16:17 ` Cyrill Gorcunov via Tarantool-patches 2021-06-21 21:16 ` Vladislav Shpilevoy via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210617154835.315576-1-gorcunov@gmail.com \ --to=tarantool-patches@dev.tarantool.org \ --cc=gorcunov@gmail.com \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v9 0/2] relay: provide downstream lag information' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox