[Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

From: Mike Pall <mike>

Reported by Changochen.

(cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)

Additional stack traversal is needed to find an error function set for
handling runtime errors. cframe unwinding is missed for a C protected
frame during this stack traversal. It leads to undefined behaviour or
crash, when raising a runtime error on stack with the CP frame before an
error function handler (for example, an error in __gc handler).

This patch adds missing unwinding for CP frame.

Sergey Kaplun:
* added the description and the test for the problem
---

Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601

 src/lj_err.c                                  |  1 +
 .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
 2 files changed, 26 insertions(+)
 create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua

diff --git a/src/lj_err.c b/src/lj_err.c
index b6be357e..b520b3d3 100644
--- a/src/lj_err.c
+++ b/src/lj_err.c
@@ -585,6 +585,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
       if (cframe_canyield(cf)) return 0;
       if (cframe_errfunc(cf) >= 0)
 	return cframe_errfunc(cf);
+      cf = cframe_prev(cf);
       frame = frame_prevd(frame);
       break;
     case FRAME_PCALL:
diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
new file mode 100644
index 00000000..d8d79100
--- /dev/null
+++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
@@ -0,0 +1,25 @@
+local tap = require('tap')
+
+local test = tap.test('lj-601-fix-gc-finderrfunc')
+test:plan(1)
+
+-- Test file to demonstrate LuaJIT incorrect behaviour, when
+-- throwing error in __gc finalizer.
+-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
+
+collectgarbage()
+
+local a = newproxy(true)
+getmetatable(a).__gc = function()
+  -- Function to raise error via `lj_err_run()` inside __gc.
+  local _ = load(function() collectgarbage()() end)
+end
+
+-- XXX: Generate a small bunch of proxies. Need several to call
+-- `collectgarbage()` on another proxy inside __gc. N cycles is
+-- empirical number.
+for _ = 1, 4 do newproxy(a) end
+collectgarbage('collect')
+
+test:ok(true, 'successfully collectgarbage with error')
+os.exit(test:check() and 0 or 1)
-- 
2.31.0