From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 56A6F6EC40; Tue, 21 Sep 2021 19:31:20 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 56A6F6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1632241880; bh=vyDFQgF/P968dz/97YajeYUgGnbXJVOXoLH4ZDk4La4=; h=To:Cc:Date:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=Gef8zLsX6/DtBjadknjmPPk1RBoMhl8OKuJnEUMLJh1zo3kbtD6ShhMA1/QgvVsOW PQqG4/p1BfGvMOt4ob+qvtT8WheO5i/7mpNwQDZdDrpWKnqy2c74H7Jt+8gk9RYaF1 a0zOQGr122FIOHBWBTUBekjucFvCFxu82QqleLLs= Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 06AFD6EC40 for ; Tue, 21 Sep 2021 19:31:18 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 06AFD6EC40 Received: by mail-lf1-f53.google.com with SMTP id y28so162883lfb.0 for ; Tue, 21 Sep 2021 09:31:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=em+qEpE/B5e0XdOyefSFocGC7T+BEhikgZ/lPd/wxsE=; b=u6mYdOZc1nyqmsyNqm6B2mevuXNg/VMQhRkb95nS2nGQBEqpjzVTUcWAXsWJAo5YOu GSLdEm6EP2XEASljQ/QDO76JzEg0IVg4C5GDW/Z0mnYYHhWKhYxb5W10cnzb4LWSJtL4 OLr11jDiA+BVeqWzbfNYO18sQrji4ST5FwzsWZLFXXeA9wAo+MZXsMEMPgl5HKeCpI56 +SizCo7u4i8rRG8DLZP8MaRQOLCubkXo5EvBAk7fFg6q1QrJsFlLQ3cK9TXFX4Chv/1y rFiO88/yexHYebME44gVSY/Koj2FVEPcy2NDo7LUC+IR0TJ6lmPsP4WyJyBa3YAgIgvz +few== X-Gm-Message-State: AOAM531/6K51h1cukNECFEXuFpUXhi1R5jzvI+GLW865b1spz2XsP1S0 RdNKydxA6PFhTTOi5Q/45sM= X-Google-Smtp-Source: ABdhPJwDh2nwjD0WXvOgkc8R7TRYRsYFlZHV1rqC2bXv1cIEanhMZ5CxFBMhjh5McJb0GDshYNLm6Q== X-Received: by 2002:a05:6512:238a:: with SMTP id c10mr9027539lfv.353.1632241874970; Tue, 21 Sep 2021 09:31:14 -0700 (PDT) Received: from localhost.localdomain (broadband-46-242-13-79.ip.moscow.rt.ru. [46.242.13.79]) by smtp.gmail.com with ESMTPSA id t11sm464682lfr.281.2021.09.21.09.31.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Sep 2021 09:31:14 -0700 (PDT) To: sergepetrenko@tarantool.org, shemeneval@gmail.com Cc: tarantool-patches@dev.tarantool.org, Yan Shtunder Date: Tue, 21 Sep 2021 19:31:10 +0300 Message-Id: <20210921163110.39841-1-ya.shtunder@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH] replication: the truncate method called from within a transaction X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Yan Shtunder via Tarantool-patches Reply-To: Yan Shtunder Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" The truncate method could be called from within a transaction. The flag of GROUP_LOCAL was set in truncate method after statement row had been being checked on the GROUP_LOCAL. Accordingly, after a local transaction NOP row was not appended. Closes #6123 --- Issue: https://github.com/tarantool/tarantool/issues/6123 Patch: https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction src/box/alter.cc | 11 +++ test/replication-luatest/gh_6123_test.lua | 61 +++++++++++++ .../instance_files/master.lua | 22 +++++ .../instance_files/replica.lua | 21 +++++ test/replication-luatest/suite.ini | 3 + ...-6123-truncate-is-local-transaction.result | 90 +++++++++++++++++++ ...123-truncate-is-local-transaction.test.lua | 36 ++++++++ 7 files changed, 244 insertions(+) create mode 100644 test/replication-luatest/gh_6123_test.lua create mode 100755 test/replication-luatest/instance_files/master.lua create mode 100755 test/replication-luatest/instance_files/replica.lua create mode 100644 test/replication-luatest/suite.ini create mode 100644 test/replication/gh-6123-truncate-is-local-transaction.result create mode 100644 test/replication/gh-6123-truncate-is-local-transaction.test.lua diff --git a/src/box/alter.cc b/src/box/alter.cc index 89bb5946c..35bd603d0 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -2901,6 +2901,17 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) if (space_is_temporary(old_space) || space_group_id(old_space) == GROUP_LOCAL) { stmt->row->group_id = GROUP_LOCAL; + /* + * In order to append a dummy NOP statement + * to preserve the tx replication boundaries, + * it is necessary that the condition + * txn->n_local_rows > 0 must be true. + * Therefore it is necessary to increase the + * value of n_local_rows, because the checking + * on the flag of GROUP_LOCAL will occurs + * before it is set. + */ + ++txn->n_local_rows; } try { diff --git a/test/replication-luatest/gh_6123_test.lua b/test/replication-luatest/gh_6123_test.lua new file mode 100644 index 000000000..7341ccc95 --- /dev/null +++ b/test/replication-luatest/gh_6123_test.lua @@ -0,0 +1,61 @@ +local fio = require('fio') +local log = require('log') + +local t = require('luatest') +local g = t.group() + +local Server = t.Server + +g.before_all(function() + g.master = Server:new({ + alias = 'master', + command = './test/replication-luatest/instance_files/master.lua', + workdir = fio.tempdir(), + env = {TARANTOOL_REPLICA = '13302'}, + http_port = 8081, + net_box_port = 13301, + }) + + g.replica = Server:new({ + alias = 'replica', + command = './test/replication-luatest/instance_files/replica.lua', + workdir = fio.tempdir(), + env = {TARANTOOL_MASTER = '13301'}, + http_port = 8082, + net_box_port = 13302, + }) + + + g.master:start() + g.replica:start() + + t.helpers.retrying({}, function() g.master:connect_net_box() end) + t.helpers.retrying({}, function() g.replica:connect_net_box() end) + + log.info('Everything is started') +end) + +g.after_all(function() + g.replica:stop() + g.master:stop() + fio.rmtree(g.master.workdir) + fio.rmtree(g.replica.workdir) +end) + +g.test_truncate_is_local_transaction = function() + g.master:eval("s = box.schema.space.create('temp', {temporary = true})") + g.master:eval("s:create_index('pk')") + + g.master:eval("s:insert{1, 2}") + g.master:eval("s:insert{4}") + t.assert_equals(g.master:eval("return s:select()"), {{1, 2}, {4}}) + + g.master:eval("box.begin() box.space._schema:replace{'smth'} s:truncate() box.commit()") + t.assert_equals(g.master:eval("return s:select()"), {}) + t.assert_equals(g.master:eval("return box.space._schema:select{'smth'}"), {{'smth'}}) + + -- Checking that replica has received the last transaction, + -- and that replication isn't broken. + t.assert_equals(g.replica:eval("return box.space._schema:select{'smth'}"), {{'smth'}}) + t.assert_equals(g.replica:eval("return box.info.replication[1].upstream.status"), 'follow') +end diff --git a/test/replication-luatest/instance_files/master.lua b/test/replication-luatest/instance_files/master.lua new file mode 100755 index 000000000..9063ad1db --- /dev/null +++ b/test/replication-luatest/instance_files/master.lua @@ -0,0 +1,22 @@ +#!/usr/bin/env tarantool + +local function instance_uri(instance) + local port = os.getenv(instance) + return 'localhost:' .. port +end + +box.cfg({ + --log_level = 7, + work_dir = os.getenv('TARANTOOL_WORKDIR'), + listen = os.getenv('TARANTOOL_LISTEN'), + replication = { + instance_uri('TARANTOOL_LISTEN'), + instance_uri('TARANTOOL_REPLICA') + }, + memtx_memory = 107374182, + replication_timeout = 0.1, + read_only = false +}) + +box.schema.user.grant('guest', 'read, write, execute, create', 'universe', nil, {if_not_exists=true}) +require('log').warn("master is ready") diff --git a/test/replication-luatest/instance_files/replica.lua b/test/replication-luatest/instance_files/replica.lua new file mode 100755 index 000000000..493ec3aab --- /dev/null +++ b/test/replication-luatest/instance_files/replica.lua @@ -0,0 +1,21 @@ +#!/usr/bin/env tarantool + +local function instance_uri(instance) + local port = os.getenv(instance) + return 'localhost:' .. port +end + +box.cfg({ + work_dir = os.getenv('TARANTOOL_WORKDIR'), + listen = os.getenv('TARANTOOL_LISTEN'), + replication = { + instance_uri('TARANTOOL_MASTER'), + instance_uri('TARANTOOL_LISTEN') + }, + memtx_memory = 107374182, + replication_timeout = 0.1, + replication_connect_timeout = 0.5, + read_only = true +}) + +require('log').warn("replica is ready") diff --git a/test/replication-luatest/suite.ini b/test/replication-luatest/suite.ini new file mode 100644 index 000000000..ccd624099 --- /dev/null +++ b/test/replication-luatest/suite.ini @@ -0,0 +1,3 @@ +[default] +core = luatest +description = first try of using luatest diff --git a/test/replication/gh-6123-truncate-is-local-transaction.result b/test/replication/gh-6123-truncate-is-local-transaction.result new file mode 100644 index 000000000..db57d37d8 --- /dev/null +++ b/test/replication/gh-6123-truncate-is-local-transaction.result @@ -0,0 +1,90 @@ +-- test-run result file version 2 +test_run = require('test_run').new() + | --- + | ... + + +-- Step 1 +box.schema.user.grant("guest", "replication") + | --- + | ... +s = box.schema.space.create("temp", {temporary=true}) + | --- + | ... +_ = s:create_index('pk') + | --- + | ... +s:insert{1,2} + | --- + | - [1, 2] + | ... +s:insert{4} + | --- + | - [4] + | ... +s:select() + | --- + | - - [1, 2] + | - [4] + | ... + + +-- Step 2 +test_run:cmd('create server replica with rpl_master=default,\ + script="replication/replica.lua"') + | --- + | - true + | ... +test_run:cmd('start server replica') + | --- + | - true + | ... + + +-- Step 3 +box.begin() box.space._schema:replace{"smth"} s:truncate() box.commit() + | --- + | ... +s:select() + | --- + | - [] + | ... +box.space._schema:select{'smth'} + | --- + | - - ['smth'] + | ... + + +-- Step 4 +-- Checking that replica has received the last transaction, +-- and that replication isn't broken. +test_run:switch('replica') + | --- + | - true + | ... +box.space._schema:select{'smth'} + | --- + | - - ['smth'] + | ... +box.info.replication[1].upstream.status + | --- + | - follow + | ... + + +test_run:switch('default') + | --- + | - true + | ... +test_run:cmd('stop server replica') + | --- + | - true + | ... +test_run:cmd('cleanup server replica') + | --- + | - true + | ... +test_run:cmd('delete server replica') + | --- + | - true + | ... diff --git a/test/replication/gh-6123-truncate-is-local-transaction.test.lua b/test/replication/gh-6123-truncate-is-local-transaction.test.lua new file mode 100644 index 000000000..336c0c44d --- /dev/null +++ b/test/replication/gh-6123-truncate-is-local-transaction.test.lua @@ -0,0 +1,36 @@ +test_run = require('test_run').new() + + +-- Step 1 +box.schema.user.grant("guest", "replication") +s = box.schema.space.create("temp", {temporary=true}) +_ = s:create_index('pk') +s:insert{1,2} +s:insert{4} +s:select() + + +-- Step 2 +test_run:cmd('create server replica with rpl_master=default,\ + script="replication/replica.lua"') +test_run:cmd('start server replica') + + +-- Step 3 +box.begin() box.space._schema:replace{"smth"} s:truncate() box.commit() +s:select() +box.space._schema:select{'smth'} + + +-- Step 4 +-- Checking that replica has received the last transaction, +-- and that replication isn't broken. +test_run:switch('replica') +box.space._schema:select{'smth'} +box.info.replication[1].upstream.status + + +test_run:switch('default') +test_run:cmd('stop server replica') +test_run:cmd('cleanup server replica') +test_run:cmd('delete server replica') -- 2.25.1