* [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
@ 2020-11-03 17:37 Nikita Pettik
2020-11-11 21:55 ` Vladislav Shpilevoy
2020-11-17 19:51 ` Alexander V. Tikhonov
0 siblings, 2 replies; 6+ messages in thread
From: Nikita Pettik @ 2020-11-03 17:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy
After upsert rework in 5a61c4717 (#5107 issue) update operations are
applied consistently corresponding to upserts they belong to: if update
operation from single upsert fails - all update operations from that
upsert are skipped. But the rest of update operations related to other
upserts (after squashing two upserts) are applied. So let's update #4957
test case: now upsert operation can be processed only if it contains
more than BOX_UPDATE_OP_CNT_MAX (4000) operations before (before
squashing with any other upsert).
Follow-up #4957
Follow-up #5107
---
Branch: https://github.com/tarantool/tarantool/tree/np/gh-4957-test-follow-up
test/vinyl/gh-4957-too-many-upserts.result | 49 +++++++++++++++-----
test/vinyl/gh-4957-too-many-upserts.test.lua | 29 ++++++++----
2 files changed, 58 insertions(+), 20 deletions(-)
diff --git a/test/vinyl/gh-4957-too-many-upserts.result b/test/vinyl/gh-4957-too-many-upserts.result
index c942e62c8..d70a2e668 100644
--- a/test/vinyl/gh-4957-too-many-upserts.result
+++ b/test/vinyl/gh-4957-too-many-upserts.result
@@ -32,26 +32,24 @@ ups_cnt = 5000
box.begin()
| ---
| ...
-for i = 1, ups_cnt do s:upsert({1}, {{'&', 2, 1}}) end
+for i = 1, ups_cnt do s:upsert({1}, {{'|', 2, i}}) end
| ---
| ...
box.commit()
| ---
| ...
--- Upserts are not able to squash, so scheduler will get stuck.
--- So let's not waste much time here, just check that no crash
--- takes place.
+-- Since 2.6 update operations of single upsert are applied consistently.
+-- So despite upserts' update operations can be squashed, they are anyway
+-- applied as they come in corresponding upserts. The same concerns the
+-- second test.
--
box.snapshot()
| ---
| - ok
| ...
-
-fiber = require('fiber')
- | ---
- | ...
-fiber.sleep(0.01)
+s:select()
| ---
+ | - - [1, 8191]
| ...
s:drop()
@@ -82,7 +80,7 @@ box.snapshot()
box.begin()
| ---
| ...
-for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
+for k = 2, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
| ---
| ...
box.commit()
@@ -92,10 +90,39 @@ box.snapshot()
| ---
| - ok
| ...
-fiber.sleep(0.01)
+s:select()[1][ups_cnt]
+ | ---
+ | - 5001
+ | ...
+
+s:drop()
| ---
| ...
+-- Test that single upsert with too many (> 4000) operations doesn't
+-- pass check so it is rejected.
+--
+s = box.schema.create_space('test', {engine = 'vinyl'})
+ | ---
+ | ...
+pk = s:create_index('pk')
+ | ---
+ | ...
+
+ups_ops = {}
+ | ---
+ | ...
+for k = 2, ups_cnt do ups_ops[k] = {'+', k, 1} end
+ | ---
+ | ...
+s:upsert({1}, ups_ops)
+ | ---
+ | - error: Illegal parameters, too many operations for update
+ | ...
+s:select()
+ | ---
+ | - []
+ | ...
s:drop()
| ---
| ...
diff --git a/test/vinyl/gh-4957-too-many-upserts.test.lua b/test/vinyl/gh-4957-too-many-upserts.test.lua
index 572540a4c..48e923bf1 100644
--- a/test/vinyl/gh-4957-too-many-upserts.test.lua
+++ b/test/vinyl/gh-4957-too-many-upserts.test.lua
@@ -12,16 +12,15 @@ box.snapshot()
--
ups_cnt = 5000
box.begin()
-for i = 1, ups_cnt do s:upsert({1}, {{'&', 2, 1}}) end
+for i = 1, ups_cnt do s:upsert({1}, {{'|', 2, i}}) end
box.commit()
--- Upserts are not able to squash, so scheduler will get stuck.
--- So let's not waste much time here, just check that no crash
--- takes place.
+-- Since 2.6 update operations of single upsert are applied consistently.
+-- So despite upserts' update operations can be squashed, they are anyway
+-- applied as they come in corresponding upserts. The same concerns the
+-- second test.
--
box.snapshot()
-
-fiber = require('fiber')
-fiber.sleep(0.01)
+s:select()
s:drop()
@@ -34,11 +33,23 @@ _ = s:insert(tuple)
box.snapshot()
box.begin()
-for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
+for k = 2, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
box.commit()
box.snapshot()
-fiber.sleep(0.01)
+s:select()[1][ups_cnt]
+
+s:drop()
+
+-- Test that single upsert with too many (> 4000) operations doesn't
+-- pass check so it is rejected.
+--
+s = box.schema.create_space('test', {engine = 'vinyl'})
+pk = s:create_index('pk')
+ups_ops = {}
+for k = 2, ups_cnt do ups_ops[k] = {'+', k, 1} end
+s:upsert({1}, ups_ops)
+s:select()
s:drop()
-- restart the current server to resolve the issue #5141
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
2020-11-03 17:37 [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test Nikita Pettik
@ 2020-11-11 21:55 ` Vladislav Shpilevoy
2020-11-13 0:22 ` Nikita Pettik
2020-11-17 19:51 ` Alexander V. Tikhonov
1 sibling, 1 reply; 6+ messages in thread
From: Vladislav Shpilevoy @ 2020-11-11 21:55 UTC (permalink / raw)
To: Nikita Pettik, tarantool-patches
Hi! Thanks for the patch!
On 03.11.2020 18:37, Nikita Pettik wrote:
> After upsert rework in 5a61c4717 (#5107 issue) update operations are
> applied consistently corresponding to upserts they belong to: if update
> operation from single upsert fails - all update operations from that
> upsert are skipped. But the rest of update operations related to other
> upserts (after squashing two upserts) are applied. So let's update #4957
> test case: now upsert operation can be processed only if it contains
> more than BOX_UPDATE_OP_CNT_MAX (4000) operations before (before
'not more' instead of 'more'? Otherwise I don't understand.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
2020-11-11 21:55 ` Vladislav Shpilevoy
@ 2020-11-13 0:22 ` Nikita Pettik
2020-11-16 21:24 ` Vladislav Shpilevoy
0 siblings, 1 reply; 6+ messages in thread
From: Nikita Pettik @ 2020-11-13 0:22 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
On 11 Nov 22:55, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
>
> On 03.11.2020 18:37, Nikita Pettik wrote:
> > After upsert rework in 5a61c4717 (#5107 issue) update operations are
> > applied consistently corresponding to upserts they belong to: if update
> > operation from single upsert fails - all update operations from that
> > upsert are skipped. But the rest of update operations related to other
> > upserts (after squashing two upserts) are applied. So let's update #4957
> > test case: now upsert operation can be processed only if it contains
> > more than BOX_UPDATE_OP_CNT_MAX (4000) operations before (before
>
> 'not more' instead of 'more'? Otherwise I don't understand.
For sure. Fixed 'can' -> 'can't'
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
2020-11-13 0:22 ` Nikita Pettik
@ 2020-11-16 21:24 ` Vladislav Shpilevoy
0 siblings, 0 replies; 6+ messages in thread
From: Vladislav Shpilevoy @ 2020-11-16 21:24 UTC (permalink / raw)
To: Nikita Pettik; +Cc: tarantool-patches
Hi! Thanks for the fix!
LGTM.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
2020-11-03 17:37 [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test Nikita Pettik
2020-11-11 21:55 ` Vladislav Shpilevoy
@ 2020-11-17 19:51 ` Alexander V. Tikhonov
2020-11-17 20:26 ` Nikita Pettik
1 sibling, 1 reply; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-11-17 19:51 UTC (permalink / raw)
To: Nikita Pettik; +Cc: tarantool-patches
Hi Nikita, thanks for the patch, as I see no new degradations found in
gitlab-ci testing commit criteria pipeline [1], patch LGTM.
[1] - https://gitlab.com/tarantool/tarantool/-/pipelines/217447464
On Tue, Nov 03, 2020 at 08:37:33PM +0300, Nikita Pettik wrote:
> After upsert rework in 5a61c4717 (#5107 issue) update operations are
> applied consistently corresponding to upserts they belong to: if update
> operation from single upsert fails - all update operations from that
> upsert are skipped. But the rest of update operations related to other
> upserts (after squashing two upserts) are applied. So let's update #4957
> test case: now upsert operation can be processed only if it contains
> more than BOX_UPDATE_OP_CNT_MAX (4000) operations before (before
> squashing with any other upsert).
>
> Follow-up #4957
> Follow-up #5107
> ---
> Branch: https://github.com/tarantool/tarantool/tree/np/gh-4957-test-follow-up
>
> test/vinyl/gh-4957-too-many-upserts.result | 49 +++++++++++++++-----
> test/vinyl/gh-4957-too-many-upserts.test.lua | 29 ++++++++----
> 2 files changed, 58 insertions(+), 20 deletions(-)
>
> diff --git a/test/vinyl/gh-4957-too-many-upserts.result b/test/vinyl/gh-4957-too-many-upserts.result
> index c942e62c8..d70a2e668 100644
> --- a/test/vinyl/gh-4957-too-many-upserts.result
> +++ b/test/vinyl/gh-4957-too-many-upserts.result
> @@ -32,26 +32,24 @@ ups_cnt = 5000
> box.begin()
> | ---
> | ...
> -for i = 1, ups_cnt do s:upsert({1}, {{'&', 2, 1}}) end
> +for i = 1, ups_cnt do s:upsert({1}, {{'|', 2, i}}) end
> | ---
> | ...
> box.commit()
> | ---
> | ...
> --- Upserts are not able to squash, so scheduler will get stuck.
> --- So let's not waste much time here, just check that no crash
> --- takes place.
> +-- Since 2.6 update operations of single upsert are applied consistently.
> +-- So despite upserts' update operations can be squashed, they are anyway
> +-- applied as they come in corresponding upserts. The same concerns the
> +-- second test.
> --
> box.snapshot()
> | ---
> | - ok
> | ...
> -
> -fiber = require('fiber')
> - | ---
> - | ...
> -fiber.sleep(0.01)
> +s:select()
> | ---
> + | - - [1, 8191]
> | ...
>
> s:drop()
> @@ -82,7 +80,7 @@ box.snapshot()
> box.begin()
> | ---
> | ...
> -for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
> +for k = 2, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
> | ---
> | ...
> box.commit()
> @@ -92,10 +90,39 @@ box.snapshot()
> | ---
> | - ok
> | ...
> -fiber.sleep(0.01)
> +s:select()[1][ups_cnt]
> + | ---
> + | - 5001
> + | ...
> +
> +s:drop()
> | ---
> | ...
>
> +-- Test that single upsert with too many (> 4000) operations doesn't
> +-- pass check so it is rejected.
> +--
> +s = box.schema.create_space('test', {engine = 'vinyl'})
> + | ---
> + | ...
> +pk = s:create_index('pk')
> + | ---
> + | ...
> +
> +ups_ops = {}
> + | ---
> + | ...
> +for k = 2, ups_cnt do ups_ops[k] = {'+', k, 1} end
> + | ---
> + | ...
> +s:upsert({1}, ups_ops)
> + | ---
> + | - error: Illegal parameters, too many operations for update
> + | ...
> +s:select()
> + | ---
> + | - []
> + | ...
> s:drop()
> | ---
> | ...
> diff --git a/test/vinyl/gh-4957-too-many-upserts.test.lua b/test/vinyl/gh-4957-too-many-upserts.test.lua
> index 572540a4c..48e923bf1 100644
> --- a/test/vinyl/gh-4957-too-many-upserts.test.lua
> +++ b/test/vinyl/gh-4957-too-many-upserts.test.lua
> @@ -12,16 +12,15 @@ box.snapshot()
> --
> ups_cnt = 5000
> box.begin()
> -for i = 1, ups_cnt do s:upsert({1}, {{'&', 2, 1}}) end
> +for i = 1, ups_cnt do s:upsert({1}, {{'|', 2, i}}) end
> box.commit()
> --- Upserts are not able to squash, so scheduler will get stuck.
> --- So let's not waste much time here, just check that no crash
> --- takes place.
> +-- Since 2.6 update operations of single upsert are applied consistently.
> +-- So despite upserts' update operations can be squashed, they are anyway
> +-- applied as they come in corresponding upserts. The same concerns the
> +-- second test.
> --
> box.snapshot()
> -
> -fiber = require('fiber')
> -fiber.sleep(0.01)
> +s:select()
>
> s:drop()
>
> @@ -34,11 +33,23 @@ _ = s:insert(tuple)
> box.snapshot()
>
> box.begin()
> -for k = 1, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
> +for k = 2, ups_cnt do s:upsert({1}, {{'+', k, 1}}) end
> box.commit()
> box.snapshot()
> -fiber.sleep(0.01)
> +s:select()[1][ups_cnt]
> +
> +s:drop()
> +
> +-- Test that single upsert with too many (> 4000) operations doesn't
> +-- pass check so it is rejected.
> +--
> +s = box.schema.create_space('test', {engine = 'vinyl'})
> +pk = s:create_index('pk')
>
> +ups_ops = {}
> +for k = 2, ups_cnt do ups_ops[k] = {'+', k, 1} end
> +s:upsert({1}, ups_ops)
> +s:select()
> s:drop()
>
> -- restart the current server to resolve the issue #5141
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test
2020-11-17 19:51 ` Alexander V. Tikhonov
@ 2020-11-17 20:26 ` Nikita Pettik
0 siblings, 0 replies; 6+ messages in thread
From: Nikita Pettik @ 2020-11-17 20:26 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
On 17 Nov 22:51, Alexander V. Tikhonov wrote:
> Hi Nikita, thanks for the patch, as I see no new degradations found in
> gitlab-ci testing commit criteria pipeline [1], patch LGTM.
>
> [1] - https://gitlab.com/tarantool/tarantool/-/pipelines/217447464
>
Pushed to master, branch is dropped.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-11-17 20:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-03 17:37 [Tarantool-patches] [PATCH] vinyl: update gh-4957-too-many-upserts test Nikita Pettik
2020-11-11 21:55 ` Vladislav Shpilevoy
2020-11-13 0:22 ` Nikita Pettik
2020-11-16 21:24 ` Vladislav Shpilevoy
2020-11-17 19:51 ` Alexander V. Tikhonov
2020-11-17 20:26 ` Nikita Pettik
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox