From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 54EAF6EC5F; Fri, 30 Apr 2021 18:39:45 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 54EAF6EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1619797185; bh=iSOTrp3lhlxYrExfk1/Ztw/8IVzB6k6e0mlsxzEuWNs=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=CfcpfWp0pfctn+2JVxB+mjTA5Wg8+RTIbJ29tlUKH7BLTPcz4u9cCWRia1K8HTIxg L1kiA9j6tli6PX4iSzDQJQCY8g5FGLs4js1enDBKrXpEfDveOzhhYRvFqRgQRlvVvF yn17gbFVHpyimr8/uR6ZV8oDpryl4+ot/wejOu5Y= Received: from mail-lj1-f182.google.com (mail-lj1-f182.google.com [209.85.208.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 922936EC5F for ; Fri, 30 Apr 2021 18:39:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 922936EC5F Received: by mail-lj1-f182.google.com with SMTP id p12so4386265ljg.1 for ; Fri, 30 Apr 2021 08:39:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=DC83LpSVXsNYDoLYYH4PQ59bw5i+f2y/QIoN2SpOlVU=; b=H4igJc03jAWs1FDSvvyXPaS0Q3npfwSRStKw6pCW9spzfEib0/ppt3n0PLywju9/wH Kl/BxzyhF+b/iZqxF7VQa2XW62l1XZf96Sor6pfqI5y/jlqAhlHK+RRZUl43jrI4gf9a ySCIEduaRxk38Z0daGrkA02A9zGyDGKyMOdkLa8xihl14bB2dIMkay00sVp7WRuiPHmS e9ddVkvftonfdiNaThJPRPs8a1lRVVLvTUixl1igITGCI2oz41PvzYrc1IKfLgC+H9/4 gwBPzRdMywmbMh/hgKpb0ZiG2bYXy2n6TjRRwHlVukOwWwpHtJnhqQlGwNNEeGHWNKmc 39HA== X-Gm-Message-State: AOAM5316URFFsWCSIGFeoc1J1aMlV6xQSzzQ//oopjHJh88SL8Ot7vm8 wID8wspvd+Q3X9NGtwzo1+s= X-Google-Smtp-Source: ABdhPJzOlVpgw+cH2KH7gNQWLlOUH2BF6uDWZZWl2tXjfAn5sjg+EetdvdeN3gfD++wr3YsM1ph5CA== X-Received: by 2002:a05:651c:1243:: with SMTP id h3mr4202581ljh.128.1619797184117; Fri, 30 Apr 2021 08:39:44 -0700 (PDT) Received: from grain.localdomain ([5.18.199.94]) by smtp.gmail.com with ESMTPSA id u2sm317946lfr.272.2021.04.30.08.39.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 08:39:42 -0700 (PDT) Received: by grain.localdomain (Postfix, from userid 1000) id 6DB94560111; Fri, 30 Apr 2021 18:39:41 +0300 (MSK) To: tml Date: Fri, 30 Apr 2021 18:39:37 +0300 Message-Id: <20210430153940.121271-1-gorcunov@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [RFC v3 0/3] relay: provide downstream lag information X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Cyrill Gorcunov via Tarantool-patches Reply-To: Cyrill Gorcunov Cc: Mons Anderson , Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Guys, this is *NOT* for merging but rather to gather comments on the code structure and overall idea. Here is a code flow for memory refresh MASTER NODE =========== TX == main.sched | `- box_process_rw ^ `- txn_commit | `- alloc xrow | `- journal_write | `- wal_assign_lsn | `- write to disk | `- wal_notify_watchers | | +---------------+ wakeup relay thread | v RELAY THREAD ============ relay_subscribe_f `- relay_reader_f | `- coio_read_xrow_timeout_xc <------------------+ | | `- relay_process_wal_event | `- recover_remaining_wals | `- relay_send | | | | read xrows from disk | | and send them to replica's | | applier | | | | | REPLICA NODE | | ============ | ^ TX | | == | | main.sched | | `- applier_apply_tx <---+ | | `- apply_synchro_row (if CONFIRM | ROLLBACK) | | | `- journal_write | | | `- applier->first_row_wal_time from xrow::tm | | `- apply_plain_tx | | `- txn_commit_try_async | | `- applier_txn_wal_write_cb | | `- applier->first_row_wal_time from xrow::tm | | | `- applier_writer_f | `- xrow_encode_vclock_timed(applier->first_row_wal_time) | `- coio_write_xrow -----------------------------------------+ Typical output is something like (freshly started) |tarantool> box.info.replication |--- |- 1: | id: 1 | uuid: f94edca8-71d4-46c9-b9d2-620a6a2bd977 | lsn: 121 | 2: | id: 2 | uuid: f6ac84e1-a040-48d9-a9c7-f8147b8e2c9e | lsn: 0 | upstream: | status: follow | idle: 0.56554910800332 | peer: replicator@127.0.0.1:3302 | lag: 0.00021719932556152 | downstream: | status: follow | idle: 0.52823433600133 | vclock: {1: 121} | lag: 0 |... The new data sent |tarantool> box.space.sync:insert{55} |--- |- [55] |... | tarantool> box.info.replication | --- | - 1: | id: 1 | uuid: f94edca8-71d4-46c9-b9d2-620a6a2bd977 | lsn: 123 | 2: | id: 2 | uuid: f6ac84e1-a040-48d9-a9c7-f8147b8e2c9e | lsn: 0 | upstream: | status: follow | idle: 0.96756215799542 | peer: replicator@127.0.0.1:3302 | lag: 0.0002143383026123 | downstream: | status: follow | idle: 0.31903971399879 | vclock: {1: 123} | lag: 0.0010807514190674 | ... Please take a look on applier notifications structure and naming. Actually I don't really like `downstream.lag` name either because this is not a counterpart for `upstream.lag` as far as I understand but rather packet traverse so maybe `dowstream.wal-lag` would be more suitable? Also in idle cycles downstream.lag is not changed which might confuse the readers because `upstream.lag` does. Anyway any kind of comments and code structure would be highly appreciated. Again, this series is not for merging because there is no docs, no tests yet, I did manual testing only. Previous version https://lists.tarantool.org/tarantool-patches/20210201100037.212301-1-gorcunov@gmail.com/ branch: gorcunov/gh-5447-relay-lag-3 issue: https://github.com/tarantool/tarantool/issues/5447 Cyrill Gorcunov (3): xrow: allow to pass timestamp via xrow_encode_vclock_timed helper applier: send first row's WAL time in the applier_writer_f relay: provide information about downstream lag src/box/applier.cc | 84 ++++++++++++++++++++++++++++++++++++++-------- src/box/applier.h | 5 +++ src/box/lua/info.c | 3 ++ src/box/relay.cc | 46 ++++++++++++++++++++++--- src/box/relay.h | 3 ++ src/box/xrow.c | 5 ++- src/box/xrow.h | 21 ++++++++++-- 7 files changed, 146 insertions(+), 21 deletions(-) base-commit: 7fd53b4c5264bdbc8f01858409abe52bc38764c8 -- 2.30.2