From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <tarantool-patches-bounces@dev.tarantool.org> Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id ACEBB6EC58; Wed, 4 Aug 2021 22:07:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org ACEBB6EC58 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628104078; bh=RekFC1CJBoJtsh/RUa2v4AIbbjrgHy+2VnUwF/3xWCM=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=TRGHQIcrUIPjKsyqPQvTAdRpvnBl63phWZ+fuFPr9BlSGGG//bmPCVBT/qP61KWFv SyLXwLHrd8IDEt5rAWebFsDvDsfMZEMnSiSi5PkswDXZezR2IkK0S5/LfdsmahVK79 1F7iQ33q5IXXGRU1uhoFw4DyfoVriAsAIned3AJA= Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 22A4A6EC58 for <tarantool-patches@dev.tarantool.org>; Wed, 4 Aug 2021 22:07:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 22A4A6EC58 Received: by mail-lj1-f171.google.com with SMTP id a7so3724810ljq.11 for <tarantool-patches@dev.tarantool.org>; Wed, 04 Aug 2021 12:07:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=UsiEO7Jv2ptzqliGVV+T7YZGWgVPzryB3i8kB4N0a5Q=; b=KfJm6+tjtERl1AbR/WFoYCcOOGvQZU+XbGYvJ+z+vtdFWgQp8nZ08hGNDVLezv2sTF YGq7q22pOg3ccnnFuUQkww3KCi7UEPFa08V0FNI1OpXidfAH0LFiQgR7cfSKaEfkzfeY VI6qUCRAL2t+rS+dHenfvLNDEqbFfzqg86RAfoO6pGF5fSkL4VQhtesMvOWefOaS5Lpj Kd8NVZgkuImi/8B25jMaUqKJjAf/5uFd3REP99RuGoHCYPRMlkJtRDXT5+rAcHRFV4M/ fnm+GhkDimm5j7BJjuZPThysaInlSQxznHfj2MAj8SLzk/xU6I58W71vJ7pEzRxXwVIi UDaw== X-Gm-Message-State: AOAM530n7AkXLYlmOPjBNK8YVnmU5LfjjQmytJiPCWQHSTR+4yInqlsg XOL+KgO2TjwyOh4ngIyWZzLv3f79gMY= X-Google-Smtp-Source: ABdhPJxba9KhpUyUyixfz8PTIh+N/kRH/CZgoGeXe59JlzPadt8cRXOHO7ZJiXMxEnInOZjYyOORrg== X-Received: by 2002:a2e:9c01:: with SMTP id s1mr562273lji.356.1628104076610; Wed, 04 Aug 2021 12:07:56 -0700 (PDT) Received: from grain.localdomain ([5.18.255.97]) by smtp.gmail.com with ESMTPSA id p27sm274150lfo.296.2021.08.04.12.07.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Aug 2021 12:07:54 -0700 (PDT) Received: by grain.localdomain (Postfix, from userid 1000) id D2FD55A001E; Wed, 4 Aug 2021 22:07:53 +0300 (MSK) To: tml <tarantool-patches@dev.tarantool.org> Date: Wed, 4 Aug 2021 22:07:48 +0300 Message-Id: <20210804190752.488147-1-gorcunov@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH v10 0/4] limbo: implement packets filtering X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches <tarantool-patches.dev.tarantool.org> List-Unsubscribe: <https://lists.tarantool.org/mailman/options/tarantool-patches>, <mailto:tarantool-patches-request@dev.tarantool.org?subject=unsubscribe> List-Archive: <https://lists.tarantool.org/pipermail/tarantool-patches/> List-Post: <mailto:tarantool-patches@dev.tarantool.org> List-Help: <mailto:tarantool-patches-request@dev.tarantool.org?subject=help> List-Subscribe: <https://lists.tarantool.org/mailman/listinfo/tarantool-patches>, <mailto:tarantool-patches-request@dev.tarantool.org?subject=subscribe> From: Cyrill Gorcunov via Tarantool-patches <tarantool-patches@dev.tarantool.org> Reply-To: Cyrill Gorcunov <gorcunov@gmail.com> Cc: Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" <tarantool-patches-bounces@dev.tarantool.org> Replication tests are passing but gihutb tests are not: the series is on top of Serge's branch `sp/gh-6034-empty-limbo-transition` which is failing yet. Vlad, back to your question about one single test for CONFIRM/ROLLBACK packet filtering > Is it the only issue? What about ROLLBACK coming to > an instance, which already made PROMOTE on the rolled back > data? That is a part of the original problem in the ticket. if any confirm/rollback comes at the moment when our limbo is already empty I think we can't do anything, there is no data to investigate and the node should rejoin from the scratch, or you have something else in mind? If yes, then we can extend our confirm/rollback filter chain with more cases. We've a long discussion with Serge about current packet validation in PROMOTE/DEMOTE filter so I reworked the validation a lot, please take a look. previous series https://lists.tarantool.org/tarantool-patches/20210730113539.563318-1-gorcunov@gmail.com/ branch gorcunov/gh-6036-rollback-confirm-10-notest issue https://github.com/tarantool/tarantool/issues/6036 v6: - use txn_limbo_terms name for structure - rebase on fresh sp/gh-6034-empty-limbo-transition branch - rework filtering chains v8: - add ability to disable filtering for local recovery and join stages - update tests v9: - opencode terms tracking - fix tests to use wait function since log output might be deferred by OS v10: - rework FILTER_IN and FILTER_PROMOTE chains with more detailed packets inspection - preserve old naming for terms manipulations - require the packet's replica_id to match limbo owner_id all the time Cyrill Gorcunov (4): latch: add latch_is_locked helper limbo: order access to the limbo terms limbo: filter incoming synchro requests test: add replication/gh-6036-rollback-confirm src/box/applier.cc | 31 +- src/box/box.cc | 11 +- src/box/memtx_engine.c | 3 +- src/box/txn_limbo.c | 367 ++++++++++++++++-- src/box/txn_limbo.h | 79 +++- src/lib/core/latch.h | 11 + test/replication/gh-6036-master.lua | 1 + test/replication/gh-6036-node.lua | 33 ++ test/replication/gh-6036-replica.lua | 1 + .../gh-6036-rollback-confirm.result | 180 +++++++++ .../gh-6036-rollback-confirm.test.lua | 92 +++++ 11 files changed, 757 insertions(+), 52 deletions(-) create mode 120000 test/replication/gh-6036-master.lua create mode 100644 test/replication/gh-6036-node.lua create mode 120000 test/replication/gh-6036-replica.lua create mode 100644 test/replication/gh-6036-rollback-confirm.result create mode 100644 test/replication/gh-6036-rollback-confirm.test.lua base-commit: 9b668f4f8508ee890f4b643b0f9ec16024939bff -- 2.31.1