[Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Maxim Kokryashkin via Tarantool-patches]

From: Mike Pall <mike>

Thanks to doujiang24.

(cherry-picked from commit 3f9389edc6cdf3f78a6896d550c236860aed62b2)

The Lua stack is changed in the `recff_stitch`, so if the trace
is aborted, for example, when the `maxsnap` is reached, and an
error is thrown from there, the execution continues with an
unbalanced stack.

This patch adds a check for snapshot overflow to the
`recff_stitch` to overcome the issue for the `LJ_TRERR_SNAPOV`
case, but other cases remain unresolved and should be fixed in
other series. For a detailed description of the remaining cases,
see the issue[1].

[1]: https://github.com/LuaJIT/LuaJIT/issues/1166

Maxim Kokryashkin:
* added the description and the test for the problem

Part of tarantool/tarantool#9595
---
Changes in v2:
- Fixed comments as per review by Sergey Kaplun

Branch: https://github.com/tarantool/luajit/tree/fckxorg/lj-720-errors-before-stitch

 src/lj_ffrecord.c                             |  4 ++++
 .../lj-720-errors-before-stitch.test.lua      | 20 +++++++++++++++++++
 2 files changed, 24 insertions(+)
 create mode 100644 test/tarantool-tests/lj-720-errors-before-stitch.test.lua

diff --git a/src/lj_ffrecord.c b/src/lj_ffrecord.c
index 1b2ddb72..e3ed80fb 100644
--- a/src/lj_ffrecord.c
+++ b/src/lj_ffrecord.c
@@ -107,6 +107,10 @@ static void recff_stitch(jit_State *J)
   const BCIns *pc = frame_pc(base-1);
   TValue *pframe = frame_prevl(base-1);

+  /* Check for this now. Throwing in lj_record_stop messes up the stack. */
+  if (J->cur.nsnap >= (MSize)J->param[JIT_P_maxsnap])
+    lj_trace_err(J, LJ_TRERR_SNAPOV);
+
   /* Move func + args up in Lua stack and insert continuation. */
   memmove(&base[1], &base[-1-LJ_FR2], sizeof(TValue)*nslot);
   setframe_ftsz(nframe, ((char *)nframe - (char *)pframe) + FRAME_CONT);
diff --git a/test/tarantool-tests/lj-720-errors-before-stitch.test.lua b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
new file mode 100644
index 00000000..00fb0b9b
--- /dev/null
+++ b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
@@ -0,0 +1,20 @@
+local tap = require('tap')
+local test = tap.test('lj-720-errors-before-stitch'):skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})
+test:plan(1)
+
+-- `math.modf` recording is NYI.
+local modf = math.modf
+jit.opt.start('hotloop=1', 'maxsnap=1')
+
+-- The loop has only two iterations: the first to detect its
+-- hotness and the second to record it. The snapshot limit is
+-- set to one and is certainly reached.
+for _ = 1, 2 do
+  -- Forcify stitch.
+  modf(1.2)
+end
+
+test:ok(true, 'stack is balanced')
+test:done(true)
--
2.43.0

Re: [Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Sergey Bronnikov via Tarantool-patches]

Hi, Max

thanks for the patch! LGTM with a minor comment

On 3/1/24 16:21, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
>
> Thanks to doujiang24.
>
> (cherry-picked from commit 3f9389edc6cdf3f78a6896d550c236860aed62b2)
>
> The Lua stack is changed in the `recff_stitch`, so if the trace
> is aborted, for example, when the `maxsnap` is reached, and an
> error is thrown from there, the execution continues with an
> unbalanced stack.
>
> This patch adds a check for snapshot overflow to the
> `recff_stitch` to overcome the issue for the `LJ_TRERR_SNAPOV`
> case, but other cases remain unresolved and should be fixed in
> other series. For a detailed description of the remaining cases,
> see the issue[1].
>
> [1]: https://github.com/LuaJIT/LuaJIT/issues/1166

Probably it is better to remove a link to the issue to avoid spam

in GH notifications. Or replace it with "LJ#1166".


<snipped>

Re: [Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Sergey Kaplun via Tarantool-patches]

Hi, Maxim!
Thanks for the fixes!
LGTM, after fixing the comments below.

On 01.03.24, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
> 
> Thanks to doujiang24.
> 
> (cherry-picked from commit 3f9389edc6cdf3f78a6896d550c236860aed62b2)
> 
> The Lua stack is changed in the `recff_stitch`, so if the trace
> is aborted, for example, when the `maxsnap` is reached, and an
> error is thrown from there, the execution continues with an
> unbalanced stack.
> 
> This patch adds a check for snapshot overflow to the
> `recff_stitch` to overcome the issue for the `LJ_TRERR_SNAPOV`
> case, but other cases remain unresolved and should be fixed in
> other series. For a detailed description of the remaining cases,
> see the issue[1].
> 
> [1]: https://github.com/LuaJIT/LuaJIT/issues/1166

I suppose it is better to drop the link (or just mention it in the test)
to avoid excess notification in the upstream repository.

> 
> Maxim Kokryashkin:
> * added the description and the test for the problem
> 
> Part of tarantool/tarantool#9595
> ---
> Changes in v2:
> - Fixed comments as per review by Sergey Kaplun
> 
> Branch: https://github.com/tarantool/luajit/tree/fckxorg/lj-720-errors-before-stitch
> 
>  src/lj_ffrecord.c                             |  4 ++++
>  .../lj-720-errors-before-stitch.test.lua      | 20 +++++++++++++++++++
>  2 files changed, 24 insertions(+)
>  create mode 100644 test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> 
> diff --git a/src/lj_ffrecord.c b/src/lj_ffrecord.c
> index 1b2ddb72..e3ed80fb 100644
> --- a/src/lj_ffrecord.c
> +++ b/src/lj_ffrecord.c

<snipped>

> diff --git a/test/tarantool-tests/lj-720-errors-before-stitch.test.lua b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> new file mode 100644
> index 00000000..00fb0b9b
> --- /dev/null
> +++ b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> @@ -0,0 +1,20 @@
> +local tap = require('tap')
> +local test = tap.test('lj-720-errors-before-stitch'):skipcond({
> +  ['Test requires JIT enabled'] = not jit.status(),
> +})
> +test:plan(1)
> +
> +-- `math.modf` recording is NYI.
> +local modf = math.modf

Why do we need to the use local value here?

> +jit.opt.start('hotloop=1', 'maxsnap=1')
> +
> +-- The loop has only two iterations: the first to detect its
> +-- hotness and the second to record it. The snapshot limit is
> +-- set to one and is certainly reached.
> +for _ = 1, 2 do
> +  -- Forcify stitch.
> +  modf(1.2)
> +end
> +
> +test:ok(true, 'stack is balanced')
> +test:done(true)
> --
> 2.43.0
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Maxim Kokryashkin via Tarantool-patches]

Hi, Sergey!
Thanks for the review!

Here is the new commit message:
===
Throw any errors before stack changes in trace stitching.

Thanks to doujiang24.

(cherry-picked from commit 3f9389edc6cdf3f78a6896d550c236860aed62b2)

The Lua stack is changed in the `recff_stitch`, so if the trace
is aborted, for example, when the `maxsnap` is reached, and an
error is thrown from there, the execution continues with an
unbalanced stack.

This patch adds a check for snapshot overflow to the
`recff_stitch` to overcome the issue for the `LJ_TRERR_SNAPOV`
case, but other cases remain unresolved and should be fixed in
other series. For a detailed description of the remaining cases,
see the issue LJ#1166.

Maxim Kokryashkin:
* added the description and the test for the problem

Part of tarantool/tarantool#9595
===

And here is the diff with changes:
===
diff --git a/test/tarantool-tests/lj-720-errors-before-stitch.test.lua b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
index 00fb0b9b..a2c6eef2 100644
--- a/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
+++ b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
@@ -5,6 +5,7 @@ local test = tap.test('lj-720-errors-before-stitch'):skipcond({
 test:plan(1)

 -- `math.modf` recording is NYI.
+-- Local `modf` simplifies jdump output.
 local modf = math.modf
 jit.opt.start('hotloop=1', 'maxsnap=1')

===

Branch is force-pushed.

On Mon, Mar 04, 2024 at 05:13:58PM +0300, Sergey Kaplun via Tarantool-patches wrote:
> Hi, Maxim!
> Thanks for the fixes!
> LGTM, after fixing the comments below.
>
> On 01.03.24, Maxim Kokryashkin wrote:
> > From: Mike Pall <mike>
> >
> > Thanks to doujiang24.
> >
> > (cherry-picked from commit 3f9389edc6cdf3f78a6896d550c236860aed62b2)
> >
> > The Lua stack is changed in the `recff_stitch`, so if the trace
> > is aborted, for example, when the `maxsnap` is reached, and an
> > error is thrown from there, the execution continues with an
> > unbalanced stack.
> >
> > This patch adds a check for snapshot overflow to the
> > `recff_stitch` to overcome the issue for the `LJ_TRERR_SNAPOV`
> > case, but other cases remain unresolved and should be fixed in
> > other series. For a detailed description of the remaining cases,
> > see the issue[1].
> >
> > [1]: https://github.com/LuaJIT/LuaJIT/issues/1166
>
> I suppose it is better to drop the link (or just mention it in the test)
> to avoid excess notification in the upstream repository.
>
> >
> > Maxim Kokryashkin:
> > * added the description and the test for the problem
> >
> > Part of tarantool/tarantool#9595
> > ---
> > Changes in v2:
> > - Fixed comments as per review by Sergey Kaplun
> >
> > Branch: https://github.com/tarantool/luajit/tree/fckxorg/lj-720-errors-before-stitch
> >
> >  src/lj_ffrecord.c                             |  4 ++++
> >  .../lj-720-errors-before-stitch.test.lua      | 20 +++++++++++++++++++
> >  2 files changed, 24 insertions(+)
> >  create mode 100644 test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> >
> > diff --git a/src/lj_ffrecord.c b/src/lj_ffrecord.c
> > index 1b2ddb72..e3ed80fb 100644
> > --- a/src/lj_ffrecord.c
> > +++ b/src/lj_ffrecord.c
>
> <snipped>
>
> > diff --git a/test/tarantool-tests/lj-720-errors-before-stitch.test.lua b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> > new file mode 100644
> > index 00000000..00fb0b9b
> > --- /dev/null
> > +++ b/test/tarantool-tests/lj-720-errors-before-stitch.test.lua
> > @@ -0,0 +1,20 @@
> > +local tap = require('tap')
> > +local test = tap.test('lj-720-errors-before-stitch'):skipcond({
> > +  ['Test requires JIT enabled'] = not jit.status(),
> > +})
> > +test:plan(1)
> > +
> > +-- `math.modf` recording is NYI.
> > +local modf = math.modf
>
> Why do we need to the use local value here?
>
> > +jit.opt.start('hotloop=1', 'maxsnap=1')
> > +
> > +-- The loop has only two iterations: the first to detect its
> > +-- hotness and the second to record it. The snapshot limit is
> > +-- set to one and is certainly reached.
> > +for _ = 1, 2 do
> > +  -- Forcify stitch.
> > +  modf(1.2)
> > +end
> > +
> > +test:ok(true, 'stack is balanced')
> > +test:done(true)
> > --
> > 2.43.0
> >
>
> --
> Best regards,
> Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit v2] Throw any errors before stack changes in trace stitching.

[Sergey Kaplun via Tarantool-patches]

Maxim,

I've checked the patchset into all long-term branches in
tarantool/luajit and bumped a new version in master [1], release/3.0 [2]
and release/2.11 [3].

[1]: https://github.com/tarantool/tarantool/pull/9832
[2]: https://github.com/tarantool/tarantool/pull/9833
[3]: https://github.com/tarantool/tarantool/pull/9834

-- 
Best regards,
Sergey Kaplun