[Tarantool-patches] [PATCH luajit] Check frame size limit before returning to a lower frame.

[Maxim Kokryashkin]

Hi, Sergey!
Thanks for the patch!
LGTM, except for the single comment below.
On Tue, Mar 12, 2024 at 08:26:27AM +0300, Sergey Kaplun wrote:
> From: Mike Pall <mike>
>
> Thanks to Sergey Kaplun.
>
> (cherry picked from commit 302366a33853b730f1b7eb61d792abc4f84f0caa)
>
> When compiling a stitched (or side) trace, there is no check for the
> frame size of the current prototype during recording. Hence, when we
> return (for example, after stitching) to the lower frame with a maximum
> possible frame size (249), the 251 = `baseslot` (2) + `maxslot` (249)
> slot for GC64 mode may be used. This leads to the corresponding assertion
> failure in `rec_check_slots()`.
>
> This patch adds the corresponding check.
>
> Sergey Kaplun:
> * added the description and the test for the problem
>
> Part of tarantool/tarantool#9595
> ---
>
> Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1173-frame-limit-lower-frame
> Tarantool PR: https://github.com/tarantool/tarantool/pull/9791
> Related issues:
> * https://github.com/tarantool/tarantool/issues/9595
> * https://github.com/LuaJIT/LuaJIT/issues/1173
>
>  src/lj_record.c                               |  2 +
>  .../lj-1173-frame-limit-lower-frame.test.lua  | 83 +++++++++++++++++++
>  2 files changed, 85 insertions(+)
>  create mode 100644 test/tarantool-tests/lj-1173-frame-limit-lower-frame.test.lua
>
> diff --git a/src/lj_record.c b/src/lj_record.c
> index c01c1f0b..e3590b1a 100644
> --- a/src/lj_record.c
> +++ b/src/lj_record.c
<snipped>

> diff --git a/test/tarantool-tests/lj-1173-frame-limit-lower-frame.test.lua b/test/tarantool-tests/lj-1173-frame-limit-lower-frame.test.lua
> new file mode 100644
> index 00000000..91e2c603
> --- /dev/null
> +++ b/test/tarantool-tests/lj-1173-frame-limit-lower-frame.test.lua
> @@ -0,0 +1,83 @@
<snipped>
> +local chunk = 'local uv = {key = 1}\n'
> +chunk = chunk .. 'return function()\n'
> +chunk = chunk .. 'local r = retf()\n'
Kind of a strange way to define a chunk. I believe that multiline
is better here.
> +
> +-- Each `UGET` occupies 1 slot, `KNIL` occupies the same amount.
> +-- 1 slot is reserved (`r` variable), 1 pair is set outside the
> +-- cycle. 249 slots (the maximum available amount, see
> +-- <src/lj_parse.c>, `bcreg_bump()` for details) are occupied in
> +-- total.
> +for _ = 1, LJ_MAX_JSLOTS / 2 - 2 do
> +  chunk = chunk .. ('uv.key, ')
> +end
> +chunk = chunk .. 'uv.key = nil\n'
> +chunk = chunk .. 'end\n'
Same applies here.
> +
> +local get_func = assert(loadstring(chunk))
> +local function_max_framesize = get_func()
> +
> +jit.opt.start('hotloop=1', 'hotexit=1')
> +
> +-- Compile the parent trace first.
> +retf()
> +retf()
> +
> +-- Try to compile the side trace with a return to a lower frame
> +-- with a huge frame size.
> +function_max_framesize()
> +function_max_framesize()
> +
> +-- XXX: The limit check is OK with default defines for non-GC64
> +-- mode, the trace is compiled for it. The test fails only with
> +-- GC64 mode enabled. Still run the test for non-GC64 mode to
> +-- avoid regressions.
> +
> +test:ok(true, 'no assertion failure during recording')
> +
> +test:done(true)
> --
> 2.44.0
>