[Tarantool-patches] [PATCH luajit] memprof: report stack resizing as internal event

[Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Resizing of the Lua stack is not reported as internal allocation.
Moreover, it may lead to crash inside Lua or FF frames.

Profiler performs reallocation first and after reports corresponding
event. When the stack is resized for local function arguments, the link
to previous frame is invalid in the cause of reallocation. Therefore,
assertion in `debug_framepc()` failes, because of invalid function
reference at previous frame.

Resolves tarantool/tarantool#5842
Follows up tarantool/tarantool#5442
---
Branch: https://github.com/tarantool/luajit/tree/skaplun/gh-5842-memprof-core-on-resizestack
Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5842-memprof-core-on-resizestack
Issue: https://github.com/tarantool/tarantool/issues/5842


 src/lj_state.c                                 |  6 ++++++
 .../misclib-memprof-lapi.test.lua              | 18 ++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/src/lj_state.c b/src/lj_state.c
index 1ed79a5..ea9abd4 100644
--- a/src/lj_state.c
+++ b/src/lj_state.c
@@ -64,7 +64,11 @@ static void resizestack(lua_State *L, MSize n)
   MSize oldsize = L->stacksize;
   MSize realsize = n + 1 + LJ_STACK_EXTRA;
   GCobj *up;
+  int32_t old_vmstate = G(L)->vmstate;
+
   lua_assert((MSize)(tvref(L->maxstack)-oldst)==L->stacksize-LJ_STACK_EXTRA-1);
+
+  setvmstate(G(L), INTERP);
   st = (TValue *)lj_mem_realloc(L, tvref(L->stack),
 				(MSize)(oldsize*sizeof(TValue)),
 				(MSize)(realsize*sizeof(TValue)));
@@ -80,6 +84,8 @@ static void resizestack(lua_State *L, MSize n)
   L->top = (TValue *)((char *)L->top + delta);
   for (up = gcref(L->openupval); up != NULL; up = gcnext(up))
     setmref(gco2uv(up)->v, (TValue *)((char *)uvval(gco2uv(up)) + delta));
+
+  G(L)->vmstate = old_vmstate;
 }
 
 /* Relimit stack after error, in case the limit was overdrawn. */
diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
index 1c36c8a..93cc348 100644
--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
@@ -125,5 +125,23 @@ test:ok(check_alloc_report(alloc, 25, 18, 100))
 -- Collect all previous allocated objects.
 test:ok(free.INTERNAL.num == 102)
 
+-- Test for https://github.com/tarantool/tarantool/issues/5842.
+-- We do not interested in report itself.
+misc.memprof.start("/dev/null")
+-- We need to cause stack resize for local variables at function
+-- call. Let's create a new coroutine (all slots are free).
+-- It has 1 slot for dummy frame + 39 free slots + 5 extra slots
+-- (so-called red zone) + 2 * LJ_FR2 slots. So 50 local variables
+-- is enough.
+local payload_str = ""
+for i = 1, 50 do
+  payload_str = payload_str..("local v%d = %d\n"):format(i, i)
+end
+local f, errmsg = loadstring(payload_str)
+assert(f, errmsg)
+local co = coroutine.create(f)
+coroutine.resume(co)
+misc.memprof.stop()
+
 jit.on()
 os.exit(test:check() and 0 or 1)
-- 
2.28.0