[Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Yan Shtunder via Tarantool-patches]

The truncate method could be called from within a transaction. The flag
of GROUP_LOCAL was set in truncate method after statement row had been
being checked on the GROUP_LOCAL. Accordingly, after a local transaction
NOP row was not appended.

Closes #6123
---
Issue: https://github.com/tarantool/tarantool/issues/6123
Patch: https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction

 src/box/alter.cc                          |  5 ++
 test/replication-luatest/gh_6123_test.lua | 62 +++++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 test/replication-luatest/gh_6123_test.lua

diff --git a/src/box/alter.cc b/src/box/alter.cc
index 31ac82fc5..52b9ea4d2 100644
--- a/src/box/alter.cc
+++ b/src/box/alter.cc
@@ -2914,6 +2914,11 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event)
 	if (space_is_temporary(old_space) ||
 	    space_group_id(old_space) == GROUP_LOCAL) {
 		stmt->row->group_id = GROUP_LOCAL;
+		/*
+		 * The trigger is invoked after txn->n_local_rows
+		 * is counted, so don't forget to update it here.
+		 */
+		++txn->n_local_rows;
 	}

 	try {
diff --git a/test/replication-luatest/gh_6123_test.lua b/test/replication-luatest/gh_6123_test.lua
new file mode 100644
index 000000000..d0ffa0c8b
--- /dev/null
+++ b/test/replication-luatest/gh_6123_test.lua
@@ -0,0 +1,62 @@
+local fio = require('fio')
+local log = require('log')
+local t = require('luatest')
+local cluster = require('test.luatest_helpers.cluster')
+local helpers = require('test.luatest_helpers.helpers')
+
+local g = t.group('gh-6123')
+
+g.before_test('test_truncate_is_local_transaction', function()
+    g.cluster = cluster:new({})
+
+    local box_cfg = {
+        replication         = {
+            helpers.instance_uri('master')
+        },
+        replication_timeout = 0.1,
+        read_only           = false
+    }
+
+    g.master = g.cluster:build_server({alias = 'master'}, engine, box_cfg)
+
+    local box_cfg = {
+        replication         = {
+            helpers.instance_uri('master'),
+            helpers.instance_uri('replica')
+        },
+        replication_timeout = 0.1,
+        replication_connect_timeout = 0.5,
+        read_only           = true
+    }
+
+    g.replica = g.cluster:build_server({alias = 'replica'}, engine, box_cfg)
+
+    g.cluster:join_server(g.master)
+    g.cluster:join_server(g.replica)
+    g.cluster:start()
+    log.info('Everything is started')
+end)
+
+g.after_test('test_truncate_is_local_transaction', function()
+    g.cluster:stop()
+    fio.rmtree(g.master.workdir)
+    fio.rmtree(g.replica.workdir)
+end)
+
+g.test_truncate_is_local_transaction = function()
+    g.master:eval("s = box.schema.space.create('temp', {temporary = true})")
+    g.master:eval("s:create_index('pk')")
+
+    g.master:eval("s:insert{1, 2}")
+    g.master:eval("s:insert{4}")
+    t.assert_equals(g.master:eval("return s:select()"), {{1, 2}, {4}})
+
+    g.master:eval("box.begin() box.space._schema:replace{'smth'} s:truncate() box.commit()")
+    t.assert_equals(g.master:eval("return s:select()"), {})
+    t.assert_equals(g.master:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
+
+    -- Checking that replica has received the last transaction,
+    -- and that replication isn't broken.
+    t.assert_equals(g.replica:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
+    t.assert_equals(g.replica:eval("return box.info.replication[1].upstream.status"), 'follow')
+end
--
2.25.1

Re: [Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Serge Petrenko via Tarantool-patches]

25.10.2021 13:44, Yan Shtunder via Tarantool-patches пишет:
> The truncate method could be called from within a transaction. The flag
> of GROUP_LOCAL was set in truncate method after statement row had been
> being checked on the GROUP_LOCAL. Accordingly, after a local transaction
> NOP row was not appended.
>
> Closes #6123

Thanks for the patch!

Please, find 2 comments below.
> ---
> Issue: https://github.com/tarantool/tarantool/issues/6123
> Patch: https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction
>
>   src/box/alter.cc                          |  5 ++
>   test/replication-luatest/gh_6123_test.lua | 62 +++++++++++++++++++++++
>   2 files changed, 67 insertions(+)
>   create mode 100644 test/replication-luatest/gh_6123_test.lua
>
> diff --git a/src/box/alter.cc b/src/box/alter.cc
> index 31ac82fc5..52b9ea4d2 100644
> --- a/src/box/alter.cc
> +++ b/src/box/alter.cc
> @@ -2914,6 +2914,11 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event)
>   	if (space_is_temporary(old_space) ||
>   	    space_group_id(old_space) == GROUP_LOCAL) {
>   		stmt->row->group_id = GROUP_LOCAL;
> +		/*
> +		 * The trigger is invoked after txn->n_local_rows
> +		 * is counted, so don't forget to update it here.
> +		 */
> +		++txn->n_local_rows;
>   	}
>
>   	try {

The patch doesn't build on my machine. For the same reason as the other 
patch.
I suppose you'll fix that with the other patch.

> diff --git a/test/replication-luatest/gh_6123_test.lua b/test/replication-luatest/gh_6123_test.lua
> new file mode 100644
> index 000000000..d0ffa0c8b
> --- /dev/null
> +++ b/test/replication-luatest/gh_6123_test.lua

Please, find a better test name.
Like "gh_6123_truncate_temp_space_test.lua"
> @@ -0,0 +1,62 @@
> +local fio = require('fio')
> +local log = require('log')
> +local t = require('luatest')
> +local cluster = require('test.luatest_helpers.cluster')
> +local helpers = require('test.luatest_helpers.helpers')
> +
> +local g = t.group('gh-6123')
> +
> +g.before_test('test_truncate_is_local_transaction', function()
> +    g.cluster = cluster:new({})
> +
> +    local box_cfg = {
> +        replication         = {
> +            helpers.instance_uri('master')
> +        },
> +        replication_timeout = 0.1,
> +        read_only           = false
> +    }
> +
> +    g.master = g.cluster:build_server({alias = 'master'}, engine, box_cfg)
> +
> +    local box_cfg = {
> +        replication         = {
> +            helpers.instance_uri('master'),
> +            helpers.instance_uri('replica')
> +        },
> +        replication_timeout = 0.1,
> +        replication_connect_timeout = 0.5,
> +        read_only           = true
> +    }
> +
> +    g.replica = g.cluster:build_server({alias = 'replica'}, engine, box_cfg)
> +
> +    g.cluster:join_server(g.master)
> +    g.cluster:join_server(g.replica)
> +    g.cluster:start()
> +    log.info('Everything is started')
> +end)
> +
> +g.after_test('test_truncate_is_local_transaction', function()
> +    g.cluster:stop()
> +    fio.rmtree(g.master.workdir)
> +    fio.rmtree(g.replica.workdir)
> +end)
> +
> +g.test_truncate_is_local_transaction = function()
> +    g.master:eval("s = box.schema.space.create('temp', {temporary = true})")
> +    g.master:eval("s:create_index('pk')")
> +
> +    g.master:eval("s:insert{1, 2}")
> +    g.master:eval("s:insert{4}")
> +    t.assert_equals(g.master:eval("return s:select()"), {{1, 2}, {4}})
> +
> +    g.master:eval("box.begin() box.space._schema:replace{'smth'} s:truncate() box.commit()")
> +    t.assert_equals(g.master:eval("return s:select()"), {})
> +    t.assert_equals(g.master:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
> +
> +    -- Checking that replica has received the last transaction,
> +    -- and that replication isn't broken.
> +    t.assert_equals(g.replica:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
> +    t.assert_equals(g.replica:eval("return box.info.replication[1].upstream.status"), 'follow')
> +end
> --
> 2.25.1
>

-- 
Serge Petrenko

Re: [Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Serge Petrenko via Tarantool-patches]

28.10.2021 18:48, Ян Штундер пишет:
> Hi! Thank you for the review!
> I have fixed the errors
>
>     Please, find a better test name.
>     Like "gh_6123_truncate_temp_space_test.lua"
>
>
> gh_6123_test.lua -> gh_6123_truncate_temp_space_test.lua
>
> --
> Yan Shtunder
>
>

Thanks  for the fixes!

One comment left. I'm pasting part of the patch here to review.

In test:

+g.after_each(function(cg)
+    cg.cluster.servers = nil
+    cg.cluster:drop()
+    fio.rmtree(cg.master.workdir)
+    fio.rmtree(cg.replica.workdir)
+end)
+
+
+g.before_test('test_truncate_is_local_transaction', function(cg)
+    cg.cluster:add_server(cg.master)
+    cg.cluster:add_server(cg.replica)
+    cg.cluster:start()
+end)
+
+g.after_test('test_truncate_is_local_transaction', function(cg)
+    cg.cluster:drop({cg.master, cg.replica})
+end)
+

You have a single test: test_truncate_is_local_transaction.

So g.after_each and g.after_test both have the same meaning in your case.

I propose to dorp g.after_test, since g.after_each supersedes it.

Also, you don't need to call fio.rmtree in g.after_each.
luatest does this for you, when needed.
>
> пн, 25 окт. 2021 г. в 16:34, Serge Petrenko <sergepetrenko@tarantool.org>:
>
>
>
>     25.10.2021 13:44, Yan Shtunder via Tarantool-patches пишет:
>     > The truncate method could be called from within a transaction.
>     The flag
>     > of GROUP_LOCAL was set in truncate method after statement row
>     had been
>     > being checked on the GROUP_LOCAL. Accordingly, after a local
>     transaction
>     > NOP row was not appended.
>     >
>     > Closes #6123
>
>     Thanks for the patch!
>
>     Please, find 2 comments below.
>     > ---
>     > Issue: https://github.com/tarantool/tarantool/issues/6123
>     > Patch:
>     https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction
>     >
>     >   src/box/alter.cc                          |  5 ++
>     >   test/replication-luatest/gh_6123_test.lua | 62
>     +++++++++++++++++++++++
>     >   2 files changed, 67 insertions(+)
>     >   create mode 100644 test/replication-luatest/gh_6123_test.lua
>     >
>     > diff --git a/src/box/alter.cc b/src/box/alter.cc
>     > index 31ac82fc5..52b9ea4d2 100644
>     > --- a/src/box/alter.cc
>     > +++ b/src/box/alter.cc
>     > @@ -2914,6 +2914,11 @@ on_replace_dd_truncate(struct trigger *
>     /* trigger */, void *event)
>     >       if (space_is_temporary(old_space) ||
>     >           space_group_id(old_space) == GROUP_LOCAL) {
>     >               stmt->row->group_id = GROUP_LOCAL;
>     > +             /*
>     > +              * The trigger is invoked after txn->n_local_rows
>     > +              * is counted, so don't forget to update it here.
>     > +              */
>     > +             ++txn->n_local_rows;
>     >       }
>     >
>     >       try {
>
>     The patch doesn't build on my machine. For the same reason as the
>     other
>     patch.
>     I suppose you'll fix that with the other patch.
>
>     > diff --git a/test/replication-luatest/gh_6123_test.lua
>     b/test/replication-luatest/gh_6123_test.lua
>     > new file mode 100644
>     > index 000000000..d0ffa0c8b
>     > --- /dev/null
>     > +++ b/test/replication-luatest/gh_6123_test.lua
>
>     Please, find a better test name.
>     Like "gh_6123_truncate_temp_space_test.lua"
>     > @@ -0,0 +1,62 @@
>     > +local fio = require('fio')
>     > +local log = require('log')
>     > +local t = require('luatest')
>     > +local cluster = require('test.luatest_helpers.cluster')
>     > +local helpers = require('test.luatest_helpers.helpers')
>     > +
>     > +local g = t.group('gh-6123')
>     > +
>     > +g.before_test('test_truncate_is_local_transaction', function()
>     > +    g.cluster = cluster:new({})
>     > +
>     > +    local box_cfg = {
>     > +        replication         = {
>     > +            helpers.instance_uri('master')
>     > +        },
>     > +        replication_timeout = 0.1,
>     > +        read_only           = false
>     > +    }
>     > +
>     > +    g.master = g.cluster:build_server({alias = 'master'},
>     engine, box_cfg)
>     > +
>     > +    local box_cfg = {
>     > +        replication         = {
>     > +            helpers.instance_uri('master'),
>     > +            helpers.instance_uri('replica')
>     > +        },
>     > +        replication_timeout = 0.1,
>     > +        replication_connect_timeout = 0.5,
>     > +        read_only           = true
>     > +    }
>     > +
>     > +    g.replica = g.cluster:build_server({alias = 'replica'},
>     engine, box_cfg)
>     > +
>     > +    g.cluster:join_server(g.master)
>     > +    g.cluster:join_server(g.replica)
>     > +    g.cluster:start()
>     > + log.info <http://log.info>('Everything is started')
>     > +end)
>     > +
>     > +g.after_test('test_truncate_is_local_transaction', function()
>     > +    g.cluster:stop()
>     > +    fio.rmtree(g.master.workdir)
>     > +    fio.rmtree(g.replica.workdir)
>     > +end)
>     > +
>     > +g.test_truncate_is_local_transaction = function()
>     > +    g.master:eval("s = box.schema.space.create('temp',
>     {temporary = true})")
>     > +    g.master:eval("s:create_index('pk')")
>     > +
>     > +    g.master:eval("s:insert{1, 2}")
>     > +    g.master:eval("s:insert{4}")
>     > +    t.assert_equals(g.master:eval("return s:select()"), {{1,
>     2}, {4}})
>     > +
>     > +    g.master:eval("box.begin()
>     box.space._schema:replace{'smth'} s:truncate() box.commit()")
>     > +    t.assert_equals(g.master:eval("return s:select()"), {})
>     > +    t.assert_equals(g.master:eval("return
>     box.space._schema:select{'smth'}"), {{'smth'}})
>     > +
>     > +    -- Checking that replica has received the last transaction,
>     > +    -- and that replication isn't broken.
>     > +    t.assert_equals(g.replica:eval("return
>     box.space._schema:select{'smth'}"), {{'smth'}})
>     > +    t.assert_equals(g.replica:eval("return
>     box.info.replication[1].upstream.status"), 'follow')
>     > +end
>     > --
>     > 2.25.1
>     >
>
>     -- 
>     Serge Petrenko
>

-- 
Serge Petrenko

Re: [Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Serge Petrenko via Tarantool-patches]

03.11.2021 13:03, Ян Штундер пишет:
> Thanks for the comments!
>  I corrected your remarks.

Thanks for the changes!
LGTM now.

>
> Diff:
>
> diff --git 
> a/test/replication-luatest/gh_6123_truncate_temp_space_test.lua 
> b/test/replication-luatest/gh_6123_truncate_temp_space_test.lua
> new file mode 100644
> index 000000000..7940e722f
> --- /dev/null
> +++ b/test/replication-luatest/gh_6123_truncate_temp_space_test.lua
> @@ -0,0 +1,62 @@
> +local t = require('luatest')
> +local cluster = require('test.luatest_helpers.cluster')
> +local helpers = require('test.luatest_helpers')
> +
> +local g = t.group('gh_6123', {{engine = 'memtx'}, {engine = 'vinyl'}})
> +
> +g.before_each(function(cg)
> +    local engine = cg.params.engine
> +
> +    cg.cluster = cluster:new({})
> +
> +    local box_cfg = {
> +        replication         = {
> +            helpers.instance_uri('master')
> +        },
> +        replication_timeout = 1,
> +        read_only           = false
> +    }
> +
> +    cg.master = cg.cluster:build_server({alias = 'master', engine = 
> engine, box_cfg = box_cfg})
> +
> +    local box_cfg = {
> +        replication         = {
> +            helpers.instance_uri('master'),
> +            helpers.instance_uri('replica')
> +        },
> +        replication_timeout = 1,
> +        replication_connect_timeout = 4,
> +        read_only           = true
> +    }
> +
> +    cg.replica = cg.cluster:build_server({alias = 'replica', engine = 
> engine, box_cfg = box_cfg})
> +
> +    cg.cluster:add_server(cg.master)
> +    cg.cluster:add_server(cg.replica)
> +    cg.cluster:start()
> +end)
> +
> +
> +g.after_each(function(cg)
> +    cg.cluster.servers = nil
> +    cg.cluster:drop()
> +end)
> +
> +
> +g.test_truncate_is_local_transaction = function(cg)
> +    cg.master:eval("s = box.schema.space.create('temp', {temporary = 
> true})")
> +    cg.master:eval("s:create_index('pk')")
> +
> +    cg.master:eval("s:insert{1, 2}")
> +    cg.master:eval("s:insert{4}")
> +    t.assert_equals(cg.master:eval("return s:select()"), {{1, 2}, {4}})
> +
> +    cg.master:eval("box.begin() box.space._schema:replace{'smth'} 
> s:truncate() box.commit()")
> +    t.assert_equals(cg.master:eval("return s:select()"), {})
> +    t.assert_equals(cg.master:eval("return 
> box.space._schema:select{'smth'}"), {{'smth'}})
> +
> +    -- Checking that replica has received the last transaction,
> +    -- and that replication isn't broken.
> +    t.assert_equals(cg.replica:eval("return 
> box.space._schema:select{'smth'}"), {{'smth'}})
> +    t.assert_equals(cg.replica:eval("return 
> box.info.replication[1].upstream.status"), 'follow')
> +end
>
> --
> Yan Shtunder
>
> пт, 29 окт. 2021 г. в 12:20, Serge Petrenko <sergepetrenko@tarantool.org>:
>
>
>
>     28.10.2021 18:48, Ян Штундер пишет:
>     > Hi! Thank you for the review!
>     > I have fixed the errors
>     >
>     >     Please, find a better test name.
>     >     Like "gh_6123_truncate_temp_space_test.lua"
>     >
>     >
>     > gh_6123_test.lua -> gh_6123_truncate_temp_space_test.lua
>     >
>     > --
>     > Yan Shtunder
>     >
>     >
>
>     Thanks  for the fixes!
>
>     One comment left. I'm pasting part of the patch here to review.
>
>     In test:
>
>     +g.after_each(function(cg)
>     +    cg.cluster.servers = nil
>     +    cg.cluster:drop()
>     +    fio.rmtree(cg.master.workdir)
>     +    fio.rmtree(cg.replica.workdir)
>     +end)
>     +
>     +
>     +g.before_test('test_truncate_is_local_transaction', function(cg)
>     +    cg.cluster:add_server(cg.master)
>     +    cg.cluster:add_server(cg.replica)
>     +    cg.cluster:start()
>     +end)
>     +
>     +g.after_test('test_truncate_is_local_transaction', function(cg)
>     +    cg.cluster:drop({cg.master, cg.replica})
>     +end)
>     +
>
>     You have a single test: test_truncate_is_local_transaction.
>
>     So g.after_each and g.after_test both have the same meaning in
>     your case.
>
>     I propose to dorp g.after_test, since g.after_each supersedes it.
>
>     Also, you don't need to call fio.rmtree in g.after_each.
>     luatest does this for you, when needed.
>     >
>     > пн, 25 окт. 2021 г. в 16:34, Serge Petrenko
>     <sergepetrenko@tarantool.org>:
>     >
>     >
>     >
>     >     25.10.2021 13:44, Yan Shtunder via Tarantool-patches пишет:
>     >     > The truncate method could be called from within a transaction.
>     >     The flag
>     >     > of GROUP_LOCAL was set in truncate method after statement row
>     >     had been
>     >     > being checked on the GROUP_LOCAL. Accordingly, after a local
>     >     transaction
>     >     > NOP row was not appended.
>     >     >
>     >     > Closes #6123
>     >
>     >     Thanks for the patch!
>     >
>     >     Please, find 2 comments below.
>     >     > ---
>     >     > Issue: https://github.com/tarantool/tarantool/issues/6123
>     >     > Patch:
>     >
>     https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction
>     >     >
>     >     >   src/box/alter.cc                          | 5 ++
>     >     >   test/replication-luatest/gh_6123_test.lua | 62
>     >     +++++++++++++++++++++++
>     >     >   2 files changed, 67 insertions(+)
>     >     >   create mode 100644 test/replication-luatest/gh_6123_test.lua
>     >     >
>     >     > diff --git a/src/box/alter.cc b/src/box/alter.cc
>     >     > index 31ac82fc5..52b9ea4d2 100644
>     >     > --- a/src/box/alter.cc
>     >     > +++ b/src/box/alter.cc
>     >     > @@ -2914,6 +2914,11 @@ on_replace_dd_truncate(struct trigger *
>     >     /* trigger */, void *event)
>     >     >       if (space_is_temporary(old_space) ||
>     >     >           space_group_id(old_space) == GROUP_LOCAL) {
>     >     >               stmt->row->group_id = GROUP_LOCAL;
>     >     > +             /*
>     >     > +              * The trigger is invoked after
>     txn->n_local_rows
>     >     > +              * is counted, so don't forget to update it
>     here.
>     >     > +              */
>     >     > +             ++txn->n_local_rows;
>     >     >       }
>     >     >
>     >     >       try {
>     >
>     >     The patch doesn't build on my machine. For the same reason
>     as the
>     >     other
>     >     patch.
>     >     I suppose you'll fix that with the other patch.
>     >
>     >     > diff --git a/test/replication-luatest/gh_6123_test.lua
>     >     b/test/replication-luatest/gh_6123_test.lua
>     >     > new file mode 100644
>     >     > index 000000000..d0ffa0c8b
>     >     > --- /dev/null
>     >     > +++ b/test/replication-luatest/gh_6123_test.lua
>     >
>     >     Please, find a better test name.
>     >     Like "gh_6123_truncate_temp_space_test.lua"
>     >     > @@ -0,0 +1,62 @@
>     >     > +local fio = require('fio')
>     >     > +local log = require('log')
>     >     > +local t = require('luatest')
>     >     > +local cluster = require('test.luatest_helpers.cluster')
>     >     > +local helpers = require('test.luatest_helpers.helpers')
>     >     > +
>     >     > +local g = t.group('gh-6123')
>     >     > +
>     >     > +g.before_test('test_truncate_is_local_transaction',
>     function()
>     >     > +    g.cluster = cluster:new({})
>     >     > +
>     >     > +    local box_cfg = {
>     >     > +        replication         = {
>     >     > +            helpers.instance_uri('master')
>     >     > +        },
>     >     > +        replication_timeout = 0.1,
>     >     > +        read_only           = false
>     >     > +    }
>     >     > +
>     >     > +    g.master = g.cluster:build_server({alias = 'master'},
>     >     engine, box_cfg)
>     >     > +
>     >     > +    local box_cfg = {
>     >     > +        replication         = {
>     >     > +            helpers.instance_uri('master'),
>     >     > +            helpers.instance_uri('replica')
>     >     > +        },
>     >     > +        replication_timeout = 0.1,
>     >     > +        replication_connect_timeout = 0.5,
>     >     > +        read_only           = true
>     >     > +    }
>     >     > +
>     >     > +    g.replica = g.cluster:build_server({alias = 'replica'},
>     >     engine, box_cfg)
>     >     > +
>     >     > +    g.cluster:join_server(g.master)
>     >     > +    g.cluster:join_server(g.replica)
>     >     > +    g.cluster:start()
>     >     > + log.info <http://log.info> <http://log.info>('Everything
>     is started')
>     >     > +end)
>     >     > +
>     >     > +g.after_test('test_truncate_is_local_transaction', function()
>     >     > +    g.cluster:stop()
>     >     > +    fio.rmtree(g.master.workdir)
>     >     > +    fio.rmtree(g.replica.workdir)
>     >     > +end)
>     >     > +
>     >     > +g.test_truncate_is_local_transaction = function()
>     >     > +    g.master:eval("s = box.schema.space.create('temp',
>     >     {temporary = true})")
>     >     > +    g.master:eval("s:create_index('pk')")
>     >     > +
>     >     > +    g.master:eval("s:insert{1, 2}")
>     >     > +    g.master:eval("s:insert{4}")
>     >     > +    t.assert_equals(g.master:eval("return s:select()"), {{1,
>     >     2}, {4}})
>     >     > +
>     >     > +    g.master:eval("box.begin()
>     >     box.space._schema:replace{'smth'} s:truncate() box.commit()")
>     >     > +    t.assert_equals(g.master:eval("return s:select()"), {})
>     >     > +    t.assert_equals(g.master:eval("return
>     >     box.space._schema:select{'smth'}"), {{'smth'}})
>     >     > +
>     >     > +    -- Checking that replica has received the last
>     transaction,
>     >     > +    -- and that replication isn't broken.
>     >     > +    t.assert_equals(g.replica:eval("return
>     >     box.space._schema:select{'smth'}"), {{'smth'}})
>     >     > +    t.assert_equals(g.replica:eval("return
>     >     box.info.replication[1].upstream.status"), 'follow')
>     >     > +end
>     >     > --
>     >     > 2.25.1
>     >     >
>     >
>     >     --
>     >     Serge Petrenko
>     >
>
>     -- 
>     Serge Petrenko
>

-- 
Serge Petrenko

Re: [Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[sergos via Tarantool-patches]

Hi!

Thanks for the patch!

LGTM.

Sergos

> On 3 Nov 2021, at 17:37, Serge Petrenko via Tarantool-patches <tarantool-patches@dev.tarantool.org> wrote:
> 
> 
> 
> 03.11.2021 13:03, Ян Штундер пишет:
>> Thanks for the comments!
>>  I corrected your remarks.
> 
> Thanks for the changes!
> LGTM now.
> 
>> 
>> Diff:
>> 
>> diff --git a/test/replication-luatest/gh_6123_truncate_temp_space_test.lua b/test/replication-luatest/gh_6123_truncate_temp_space_test.lua
>> new file mode 100644
>> index 000000000..7940e722f
>> --- /dev/null
>> +++ b/test/replication-luatest/gh_6123_truncate_temp_space_test.lua
>> @@ -0,0 +1,62 @@
>> +local t = require('luatest')
>> +local cluster = require('test.luatest_helpers.cluster')
>> +local helpers = require('test.luatest_helpers')
>> +
>> +local g = t.group('gh_6123', {{engine = 'memtx'}, {engine = 'vinyl'}})
>> +
>> +g.before_each(function(cg)
>> +    local engine = cg.params.engine
>> +
>> +    cg.cluster = cluster:new({})
>> +
>> +    local box_cfg = {
>> +        replication         = {
>> +            helpers.instance_uri('master')
>> +        },
>> +        replication_timeout = 1,
>> +        read_only           = false
>> +    }
>> +
>> +    cg.master = cg.cluster:build_server({alias = 'master', engine = engine, box_cfg = box_cfg})
>> +
>> +    local box_cfg = {
>> +        replication         = {
>> +            helpers.instance_uri('master'),
>> +            helpers.instance_uri('replica')
>> +        },
>> +        replication_timeout = 1,
>> +        replication_connect_timeout = 4,
>> +        read_only           = true
>> +    }
>> +
>> +    cg.replica = cg.cluster:build_server({alias = 'replica', engine = engine, box_cfg = box_cfg})
>> +
>> +    cg.cluster:add_server(cg.master)
>> +    cg.cluster:add_server(cg.replica)
>> +    cg.cluster:start()
>> +end)
>> +
>> +
>> +g.after_each(function(cg)
>> +    cg.cluster.servers = nil
>> +    cg.cluster:drop()
>> +end)
>> +
>> +
>> +g.test_truncate_is_local_transaction = function(cg)
>> +    cg.master:eval("s = box.schema.space.create('temp', {temporary = true})")
>> +    cg.master:eval("s:create_index('pk')")
>> +
>> +    cg.master:eval("s:insert{1, 2}")
>> +    cg.master:eval("s:insert{4}")
>> +    t.assert_equals(cg.master:eval("return s:select()"), {{1, 2}, {4}})
>> +
>> +    cg.master:eval("box.begin() box.space._schema:replace{'smth'} s:truncate() box.commit()")
>> +    t.assert_equals(cg.master:eval("return s:select()"), {})
>> +    t.assert_equals(cg.master:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
>> +
>> +    -- Checking that replica has received the last transaction,
>> +    -- and that replication isn't broken.
>> +    t.assert_equals(cg.replica:eval("return box.space._schema:select{'smth'}"), {{'smth'}})
>> +    t.assert_equals(cg.replica:eval("return box.info.replication[1].upstream.status"), 'follow')
>> +end
>> 
>> --
>> Yan Shtunder
>> 
>> пт, 29 окт. 2021 г. в 12:20, Serge Petrenko <sergepetrenko@tarantool.org>:
>> 
>> 
>> 
>>    28.10.2021 18:48, Ян Штундер пишет:
>>    > Hi! Thank you for the review!
>>    > I have fixed the errors
>>    >
>>    >     Please, find a better test name.
>>    >     Like "gh_6123_truncate_temp_space_test.lua"
>>    >
>>    >
>>    > gh_6123_test.lua -> gh_6123_truncate_temp_space_test.lua
>>    >
>>    > --
>>    > Yan Shtunder
>>    >
>>    >
>> 
>>    Thanks  for the fixes!
>> 
>>    One comment left. I'm pasting part of the patch here to review.
>> 
>>    In test:
>> 
>>    +g.after_each(function(cg)
>>    +    cg.cluster.servers = nil
>>    +    cg.cluster:drop()
>>    +    fio.rmtree(cg.master.workdir)
>>    +    fio.rmtree(cg.replica.workdir)
>>    +end)
>>    +
>>    +
>>    +g.before_test('test_truncate_is_local_transaction', function(cg)
>>    +    cg.cluster:add_server(cg.master)
>>    +    cg.cluster:add_server(cg.replica)
>>    +    cg.cluster:start()
>>    +end)
>>    +
>>    +g.after_test('test_truncate_is_local_transaction', function(cg)
>>    +    cg.cluster:drop({cg.master, cg.replica})
>>    +end)
>>    +
>> 
>>    You have a single test: test_truncate_is_local_transaction.
>> 
>>    So g.after_each and g.after_test both have the same meaning in
>>    your case.
>> 
>>    I propose to dorp g.after_test, since g.after_each supersedes it.
>> 
>>    Also, you don't need to call fio.rmtree in g.after_each.
>>    luatest does this for you, when needed.
>>    >
>>    > пн, 25 окт. 2021 г. в 16:34, Serge Petrenko
>>    <sergepetrenko@tarantool.org>:
>>    >
>>    >
>>    >
>>    >     25.10.2021 13:44, Yan Shtunder via Tarantool-patches пишет:
>>    >     > The truncate method could be called from within a transaction.
>>    >     The flag
>>    >     > of GROUP_LOCAL was set in truncate method after statement row
>>    >     had been
>>    >     > being checked on the GROUP_LOCAL. Accordingly, after a local
>>    >     transaction
>>    >     > NOP row was not appended.
>>    >     >
>>    >     > Closes #6123
>>    >
>>    >     Thanks for the patch!
>>    >
>>    >     Please, find 2 comments below.
>>    >     > ---
>>    >     > Issue: https://github.com/tarantool/tarantool/issues/6123
>>    >     > Patch:
>>    >
>>    https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction
>>    >     >
>>    >     >   src/box/alter.cc                          | 5 ++
>>    >     >   test/replication-luatest/gh_6123_test.lua | 62
>>    >     +++++++++++++++++++++++
>>    >     >   2 files changed, 67 insertions(+)
>>    >     >   create mode 100644 test/replication-luatest/gh_6123_test.lua
>>    >     >
>>    >     > diff --git a/src/box/alter.cc b/src/box/alter.cc
>>    >     > index 31ac82fc5..52b9ea4d2 100644
>>    >     > --- a/src/box/alter.cc
>>    >     > +++ b/src/box/alter.cc
>>    >     > @@ -2914,6 +2914,11 @@ on_replace_dd_truncate(struct trigger *
>>    >     /* trigger */, void *event)
>>    >     >       if (space_is_temporary(old_space) ||
>>    >     >           space_group_id(old_space) == GROUP_LOCAL) {
>>    >     >               stmt->row->group_id = GROUP_LOCAL;
>>    >     > +             /*
>>    >     > +              * The trigger is invoked after
>>    txn->n_local_rows
>>    >     > +              * is counted, so don't forget to update it
>>    here.
>>    >     > +              */
>>    >     > +             ++txn->n_local_rows;
>>    >     >       }
>>    >     >
>>    >     >       try {
>>    >
>>    >     The patch doesn't build on my machine. For the same reason
>>    as the
>>    >     other
>>    >     patch.
>>    >     I suppose you'll fix that with the other patch.
>>    >
>>    >     > diff --git a/test/replication-luatest/gh_6123_test.lua
>>    >     b/test/replication-luatest/gh_6123_test.lua
>>    >     > new file mode 100644
>>    >     > index 000000000..d0ffa0c8b
>>    >     > --- /dev/null
>>    >     > +++ b/test/replication-luatest/gh_6123_test.lua
>>    >
>>    >     Please, find a better test name.
>>    >     Like "gh_6123_truncate_temp_space_test.lua"
>>    >     > @@ -0,0 +1,62 @@
>>    >     > +local fio = require('fio')
>>    >     > +local log = require('log')
>>    >     > +local t = require('luatest')
>>    >     > +local cluster = require('test.luatest_helpers.cluster')
>>    >     > +local helpers = require('test.luatest_helpers.helpers')
>>    >     > +
>>    >     > +local g = t.group('gh-6123')
>>    >     > +
>>    >     > +g.before_test('test_truncate_is_local_transaction',
>>    function()
>>    >     > +    g.cluster = cluster:new({})
>>    >     > +
>>    >     > +    local box_cfg = {
>>    >     > +        replication         = {
>>    >     > +            helpers.instance_uri('master')
>>    >     > +        },
>>    >     > +        replication_timeout = 0.1,
>>    >     > +        read_only           = false
>>    >     > +    }
>>    >     > +
>>    >     > +    g.master = g.cluster:build_server({alias = 'master'},
>>    >     engine, box_cfg)
>>    >     > +
>>    >     > +    local box_cfg = {
>>    >     > +        replication         = {
>>    >     > +            helpers.instance_uri('master'),
>>    >     > +            helpers.instance_uri('replica')
>>    >     > +        },
>>    >     > +        replication_timeout = 0.1,
>>    >     > +        replication_connect_timeout = 0.5,
>>    >     > +        read_only           = true
>>    >     > +    }
>>    >     > +
>>    >     > +    g.replica = g.cluster:build_server({alias = 'replica'},
>>    >     engine, box_cfg)
>>    >     > +
>>    >     > +    g.cluster:join_server(g.master)
>>    >     > +    g.cluster:join_server(g.replica)
>>    >     > +    g.cluster:start()
>>    >     > + log.info <http://log.info> <http://log.info>('Everything
>>    is started')
>>    >     > +end)
>>    >     > +
>>    >     > +g.after_test('test_truncate_is_local_transaction', function()
>>    >     > +    g.cluster:stop()
>>    >     > +    fio.rmtree(g.master.workdir)
>>    >     > +    fio.rmtree(g.replica.workdir)
>>    >     > +end)
>>    >     > +
>>    >     > +g.test_truncate_is_local_transaction = function()
>>    >     > +    g.master:eval("s = box.schema.space.create('temp',
>>    >     {temporary = true})")
>>    >     > +    g.master:eval("s:create_index('pk')")
>>    >     > +
>>    >     > +    g.master:eval("s:insert{1, 2}")
>>    >     > +    g.master:eval("s:insert{4}")
>>    >     > +    t.assert_equals(g.master:eval("return s:select()"), {{1,
>>    >     2}, {4}})
>>    >     > +
>>    >     > +    g.master:eval("box.begin()
>>    >     box.space._schema:replace{'smth'} s:truncate() box.commit()")
>>    >     > +    t.assert_equals(g.master:eval("return s:select()"), {})
>>    >     > +    t.assert_equals(g.master:eval("return
>>    >     box.space._schema:select{'smth'}"), {{'smth'}})
>>    >     > +
>>    >     > +    -- Checking that replica has received the last
>>    transaction,
>>    >     > +    -- and that replication isn't broken.
>>    >     > +    t.assert_equals(g.replica:eval("return
>>    >     box.space._schema:select{'smth'}"), {{'smth'}})
>>    >     > +    t.assert_equals(g.replica:eval("return
>>    >     box.info.replication[1].upstream.status"), 'follow')
>>    >     > +end
>>    >     > --
>>    >     > 2.25.1
>>    >     >
>>    >
>>    >     --
>>    >     Serge Petrenko
>>    >
>> 
>>    --     Serge Petrenko
>> 
> 
> -- 
> Serge Petrenko
> 

Re: [Tarantool-patches] [PATCH v3] replication: the truncate method called from within a transaction

[Kirill Yukhin via Tarantool-patches]

Hello,

On 25 окт 13:44, Yan Shtunder via Tarantool-patches wrote:
> The truncate method could be called from within a transaction. The flag
> of GROUP_LOCAL was set in truncate method after statement row had been
> being checked on the GROUP_LOCAL. Accordingly, after a local transaction
> NOP row was not appended.
> 
> Closes #6123
> ---
> Issue: https://github.com/tarantool/tarantool/issues/6123
> Patch: https://github.com/tarantool/tarantool/tree/yshtunder/gh-6123-truncate-is-local-transaction

I've checked your patch into master.


--
Regards, Kirill Yukhin