[Tarantool-patches] [PATCH luajit 0/5] jit: add exception unwinding

[Tarantool-patches] [PATCH luajit 0/5] jit: add exception unwinding

[Maxim Kokryashkin via Tarantool-patches]

This patchset adopts the LuaJIT machinery for unwinding
exceptions raised while trace execution.

The snap-restore test is disabled, since after this patchset
it requires dynamic calculation for amount of allocated
stack slots. That test should be rewritten in a
less fragile manner.

Branch: https://github.com/tarantool/luajit/tree/fckxorg/gh-7745-exceptions-on-traces
Issue: https://github.com/tarantool/tarantool/issues/7745
PR: https://github.com/tarantool/tarantool/pull/8308

Maxim Kokryashkin (1):
  test: disable `lj-603-snap-restore` test

Mike Pall (4):
  Handle on-trace OOM errors from helper functions.
  Disable unreliable assertion for external frame unwinding.
  OSX: Disable unreliable assertion for external frame unwinding.
  Fix IR_RENAME snapshot number. Follow-up fix for a32aeadc.

 doc/status.html                               |   7 -
 src/lj_arch.h                                 |  12 +
 src/lj_asm.c                                  |  86 ++++--
 src/lj_dispatch.h                             |   4 +-
 src/lj_err.c                                  | 279 +++++++++++++++++-
 src/lj_err.h                                  |  19 +-
 src/lj_ffrecord.c                             |   2 +
 src/lj_jit.h                                  |   2 +
 src/lj_mcode.c                                |   5 +-
 src/lj_opt_loop.c                             |   1 +
 src/lj_record.c                               |   3 +-
 src/lj_snap.c                                 |   1 +
 src/lj_state.c                                |   1 +
 src/lj_target_x86.h                           |   2 +
 src/lj_trace.c                                |  61 +++-
 src/lj_trace.h                                |   3 +
 src/lj_vm.h                                   |   3 +
 src/vm_arm.dasc                               |   3 +-
 src/vm_arm64.dasc                             |   4 +-
 src/vm_mips.dasc                              |   9 +-
 src/vm_mips64.dasc                            |  10 +-
 src/vm_ppc.dasc                               |   3 +-
 src/vm_x64.dasc                               |   6 +-
 src/vm_x86.dasc                               |   4 +-
 .../gh-7745-oom-on-trace.test.lua             |  20 ++
 .../lj-603-err-snap-restore.test.lua          |   1 +
 26 files changed, 492 insertions(+), 59 deletions(-)
 create mode 100644 test/tarantool-tests/gh-7745-oom-on-trace.test.lua

-- 
2.39.0

[Tarantool-patches] [PATCH luajit 1/5] test: disable `lj-603-snap-restore` test

[Maxim Kokryashkin via Tarantool-patches]

The test is extremely fragile and it is burdening
to fix it in scope of every major change.
---
 test/tarantool-tests/lj-603-err-snap-restore.test.lua | 1 +
 1 file changed, 1 insertion(+)

diff --git a/test/tarantool-tests/lj-603-err-snap-restore.test.lua b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
index b5353e85..9dbd9cc2 100644
--- a/test/tarantool-tests/lj-603-err-snap-restore.test.lua
+++ b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
@@ -1,3 +1,4 @@
+require('utils').skipcond(true, 'too fragile, temporarily disabled')
 local tap = require('tap')
 
 -- Test to demonstrate the incorrect JIT behaviour when an error
-- 
2.39.0

[Tarantool-patches] [PATCH luajit 2/5] Handle on-trace OOM errors from helper functions.

[Maxim Kokryashkin via Tarantool-patches]

From: Mike Pall <mike>

(cherry-picked from commit 4bba29e697d00df5f020e76c2003bb9ce51c5d38)

This patch introduces handling of errors from internal helper
functions on traces. FFI C++ exception interoperability is
not yet implemented.

For each throwing trace, its mcode entry is augmented with a
DWARF2 frame description entry. After that, a dynamic DWARF2
frame info is registered based on that entry with
`__register_frame()`. Because the ARM32 architecture lacks
the __register_frame, unwinding is not supported on it.

For each throwing function call, a snapshot is allocated.
When we have a parent trace, our side trace head requires
additional snapshot allocation, so the additional
`asm_snap_prev()` call is added.

Unwinding on traces is disabled on Darwin, due to quirks of
Apple's version of libunwind.

Maxim Kokryashkin:
* added the description and the test for the problem

Part of tarantool/tarantool#7745
---
 doc/status.html                               |   7 -
 src/lj_arch.h                                 |  12 +
 src/lj_asm.c                                  |  77 ++++-
 src/lj_dispatch.h                             |   4 +-
 src/lj_err.c                                  | 274 +++++++++++++++++-
 src/lj_err.h                                  |  19 +-
 src/lj_ffrecord.c                             |   2 +
 src/lj_jit.h                                  |   2 +
 src/lj_mcode.c                                |   5 +-
 src/lj_opt_loop.c                             |   1 +
 src/lj_record.c                               |   3 +-
 src/lj_snap.c                                 |   1 +
 src/lj_state.c                                |   1 +
 src/lj_target_x86.h                           |   2 +
 src/lj_trace.c                                |  61 +++-
 src/lj_trace.h                                |   3 +
 src/lj_vm.h                                   |   3 +
 src/vm_arm.dasc                               |   3 +-
 src/vm_arm64.dasc                             |   4 +-
 src/vm_mips.dasc                              |   9 +-
 src/vm_mips64.dasc                            |  10 +-
 src/vm_ppc.dasc                               |   3 +-
 src/vm_x64.dasc                               |   6 +-
 src/vm_x86.dasc                               |   4 +-
 .../gh-7745-oom-on-trace.test.lua             |  20 ++
 25 files changed, 478 insertions(+), 58 deletions(-)
 create mode 100644 test/tarantool-tests/gh-7745-oom-on-trace.test.lua

diff --git a/doc/status.html b/doc/status.html
index c89255b6..c9cd9071 100644
--- a/doc/status.html
+++ b/doc/status.html
@@ -90,13 +90,6 @@ The Lua <b>debug API</b> is missing a couple of features (return
 hooks for non-Lua functions) and shows slightly different behavior
 in LuaJIT (no per-coroutine hooks, no tail call counting).
 </li>
-<li>
-Currently some <b>out-of-memory</b> errors from <b>on-trace code</b> are not
-handled correctly. The error may fall through an on-trace
-<tt>pcall</tt> or it may be passed on to the function set with
-<tt>lua_atpanic</tt> on x64. This issue will be fixed with the new
-garbage collector.
-</li>
 </ul>
 <br class="flush">
 </div>
diff --git a/src/lj_arch.h b/src/lj_arch.h
index 2e458d20..8643aa4a 100644
--- a/src/lj_arch.h
+++ b/src/lj_arch.h
@@ -155,6 +155,7 @@
 #define LJ_TARGET_X86		1
 #define LJ_TARGET_X86ORX64	1
 #define LJ_TARGET_EHRETREG	0
+#define LJ_TARGET_EHRAREG	8
 #define LJ_TARGET_MASKSHIFT	1
 #define LJ_TARGET_MASKROT	1
 #define LJ_TARGET_UNALIGNED	1
@@ -168,6 +169,7 @@
 #define LJ_TARGET_X64		1
 #define LJ_TARGET_X86ORX64	1
 #define LJ_TARGET_EHRETREG	0
+#define LJ_TARGET_EHRAREG	16
 #define LJ_TARGET_JUMPRANGE	31	/* +-2^31 = +-2GB */
 #define LJ_TARGET_MASKSHIFT	1
 #define LJ_TARGET_MASKROT	1
@@ -193,6 +195,7 @@
 #define LJ_ABI_EABI		1
 #define LJ_TARGET_ARM		1
 #define LJ_TARGET_EHRETREG	0
+#define LJ_TARGET_EHRAREG	14
 #define LJ_TARGET_JUMPRANGE	25	/* +-2^25 = +-32MB */
 #define LJ_TARGET_MASKSHIFT	0
 #define LJ_TARGET_MASKROT	1
@@ -226,6 +229,7 @@
 #endif
 #define LJ_TARGET_ARM64		1
 #define LJ_TARGET_EHRETREG	0
+#define LJ_TARGET_EHRAREG	30
 #define LJ_TARGET_JUMPRANGE	27	/* +-2^27 = +-128MB */
 #define LJ_TARGET_MASKSHIFT	1
 #define LJ_TARGET_MASKROT	1
@@ -262,6 +266,7 @@
 
 #define LJ_TARGET_PPC		1
 #define LJ_TARGET_EHRETREG	3
+#define LJ_TARGET_EHRAREG	65
 #define LJ_TARGET_JUMPRANGE	25	/* +-2^25 = +-32MB */
 #define LJ_TARGET_MASKSHIFT	0
 #define LJ_TARGET_MASKROT	1
@@ -353,6 +358,7 @@
 #endif
 #define LJ_TARGET_MIPS		1
 #define LJ_TARGET_EHRETREG	4
+#define LJ_TARGET_EHRAREG	31
 #define LJ_TARGET_JUMPRANGE	27	/* 2*2^27 = 256MB-aligned region */
 #define LJ_TARGET_MASKSHIFT	1
 #define LJ_TARGET_MASKROT	1
@@ -574,6 +580,12 @@
 #define LJ_UNWIND_EXT		0
 #endif
 
+#if LJ_UNWIND_EXT && LJ_HASJIT && !LJ_TARGET_ARM && !(LJ_ABI_WIN && LJ_TARGET_X86) && !(LJ_TARGET_OSX)
+#define LJ_UNWIND_JIT		1
+#else
+#define LJ_UNWIND_JIT		0
+#endif
+
 /* Compatibility with Lua 5.1 vs. 5.2. */
 #ifdef LUAJIT_ENABLE_LUA52COMPAT
 #define LJ_52			1
diff --git a/src/lj_asm.c b/src/lj_asm.c
index a154547b..adfaf286 100644
--- a/src/lj_asm.c
+++ b/src/lj_asm.c
@@ -73,6 +73,7 @@ typedef struct ASMState {
   SnapNo snapno;	/* Current snapshot number. */
   SnapNo loopsnapno;	/* Loop snapshot number. */
   BloomFilter snapfilt1, snapfilt2;	/* Filled with snapshot refs. */
+  int snapalloc;	/* Current snapshot needs allocation. */
 
   IRRef fuseref;	/* Fusion limit (loopref, 0 or FUSE_DISABLED). */
   IRRef sectref;	/* Section base reference (loopref or 0). */
@@ -86,6 +87,7 @@ typedef struct ASMState {
 
   MCode *mcbot;		/* Bottom of reserved MCode. */
   MCode *mctop;		/* Top of generated MCode. */
+  MCode *mctoporig;	/* Original top of generated MCode. */
   MCode *mcloop;	/* Pointer to loop MCode (or NULL). */
   MCode *invmcp;	/* Points to invertible loop branch (or NULL). */
   MCode *flagmcp;	/* Pending opportunity to merge flag setting ins. */
@@ -932,9 +934,9 @@ static void asm_snap_alloc1(ASMState *as, IRRef ref)
 }
 
 /* Allocate refs escaping to a snapshot. */
-static void asm_snap_alloc(ASMState *as)
+static void asm_snap_alloc(ASMState *as, int snapno)
 {
-  SnapShot *snap = &as->T->snap[as->snapno];
+  SnapShot *snap = &as->T->snap[snapno];
   SnapEntry *map = &as->T->snapmap[snap->mapofs];
   MSize n, nent = snap->nent;
   as->snapfilt1 = as->snapfilt2 = 0;
@@ -944,6 +946,14 @@ static void asm_snap_alloc(ASMState *as)
     if (!irref_isk(ref)) {
       asm_snap_alloc1(as, ref);
       if (LJ_SOFTFP && (sn & SNAP_SOFTFPNUM)) {
+        /*
+        ** FIXME: The following assert was replaced with
+        ** the conventional `lua_assert`.
+        **
+        ** lj_assertA(irt_type(IR(ref+1)->t) == IRT_SOFTFP,
+		    ** "snap %d[%d] points to bad SOFTFP IR %04d",
+		    ** snapno, n, ref - REF_BIAS);
+        */
 	lua_assert(irt_type(IR(ref+1)->t) == IRT_SOFTFP);
 	asm_snap_alloc1(as, ref+1);
       }
@@ -970,19 +980,16 @@ static int asm_snap_checkrename(ASMState *as, IRRef ren)
   return 0;  /* Not found. */
 }
 
-/* Prepare snapshot for next guard instruction. */
+/* Prepare snapshot for next guard or throwing instruction. */
 static void asm_snap_prep(ASMState *as)
 {
-  if (as->curins < as->snapref) {
-    do {
-      if (as->snapno == 0) return;  /* Called by sunk stores before snap #0. */
-      as->snapno--;
-      as->snapref = as->T->snap[as->snapno].ref;
-    } while (as->curins < as->snapref);
-    asm_snap_alloc(as);
+  if (as->snapalloc) {
+    /* Alloc on first invocation for each snapshot. */
+    as->snapalloc = 0;
+    asm_snap_alloc(as, as->snapno);
     as->snaprename = as->T->nins;
   } else {
-    /* Process any renames above the highwater mark. */
+    /* Check any renames above the highwater mark. */
     for (; as->snaprename < as->T->nins; as->snaprename++) {
       IRIns *ir = &as->T->ir[as->snaprename];
       if (asm_snap_checkrename(as, ir->op1))
@@ -991,6 +998,35 @@ static void asm_snap_prep(ASMState *as)
   }
 }
 
+/* Move to previous snapshot when we cross the current snapshot ref. */
+static void asm_snap_prev(ASMState *as)
+{
+  if (as->curins < as->snapref) {
+    ptrdiff_t ofs = as->mctoporig - as->mcp;
+    if (ofs >= 0x10000) lj_trace_err(as->J, LJ_TRERR_MCODEOV);
+    do {
+      if (as->snapno == 0) return;
+      as->snapno--;
+      as->snapref = as->T->snap[as->snapno].ref;
+      as->T->snap[as->snapno].mcofs = ofs;  /* Remember mcode offset. */
+    } while (as->curins < as->snapref);  /* May have no ins inbetween. */
+    as->snapalloc = 1;
+  }
+}
+
+/* Fixup snapshot mcode offsetst. */
+static void asm_snap_fixup_mcofs(ASMState *as)
+{
+  uint32_t sz = (uint32_t)(as->mctoporig - as->mcp);
+  SnapShot *snap = as->T->snap;
+  SnapNo i;
+  for (i = as->T->nsnap-1; i > 0; i--) {
+    /* Compute offset from mcode start and store in correct snapshot. */
+    snap[i].mcofs = (uint16_t)(sz - snap[i-1].mcofs);
+  }
+  snap[0].mcofs = 0;
+}
+
 /* -- Miscellaneous helpers ----------------------------------------------- */
 
 /* Calculate stack adjustment. */
@@ -1034,6 +1070,7 @@ static void asm_snew(ASMState *as, IRIns *ir)
 {
   const CCallInfo *ci = &lj_ir_callinfo[IRCALL_lj_str_new];
   IRRef args[3];
+  asm_snap_prep(as);
   args[0] = ASMREF_L;  /* lua_State *L    */
   args[1] = ir->op1;   /* const char *str */
   args[2] = ir->op2;   /* size_t len      */
@@ -1046,6 +1083,7 @@ static void asm_tnew(ASMState *as, IRIns *ir)
 {
   const CCallInfo *ci = &lj_ir_callinfo[IRCALL_lj_tab_new1];
   IRRef args[2];
+  asm_snap_prep(as);
   args[0] = ASMREF_L;     /* lua_State *L    */
   args[1] = ASMREF_TMP1;  /* uint32_t ahsize */
   as->gcsteps++;
@@ -1058,6 +1096,7 @@ static void asm_tdup(ASMState *as, IRIns *ir)
 {
   const CCallInfo *ci = &lj_ir_callinfo[IRCALL_lj_tab_dup];
   IRRef args[2];
+  asm_snap_prep(as);
   args[0] = ASMREF_L;  /* lua_State *L    */
   args[1] = ir->op1;   /* const GCtab *kt */
   as->gcsteps++;
@@ -1176,6 +1215,7 @@ static void asm_tostr(ASMState *as, IRIns *ir)
 {
   const CCallInfo *ci;
   IRRef args[2];
+  asm_snap_prep(as);
   args[0] = ASMREF_L;
   as->gcsteps++;
   if (ir->op2 == IRTOSTR_NUM) {
@@ -1231,6 +1271,7 @@ static void asm_newref(ASMState *as, IRIns *ir)
   IRRef args[3];
   if (ir->r == RID_SINK)
     return;
+  asm_snap_prep(as);
   args[0] = ASMREF_L;     /* lua_State *L */
   args[1] = ir->op1;      /* GCtab *t     */
   args[2] = ASMREF_TMP1;  /* cTValue *key */
@@ -1769,8 +1810,7 @@ static void asm_head_side(ASMState *as)
 
   if (as->snapno && as->topslot > as->parent->topslot) {
     /* Force snap #0 alloc to prevent register overwrite in stack check. */
-    as->snapno = 0;
-    asm_snap_alloc(as);
+    asm_snap_alloc(as, 0);
   }
   allow = asm_head_side_base(as, irp, allow);
 
@@ -2028,6 +2068,7 @@ static void asm_setup_regsp(ASMState *as)
   as->snaprename = nins;
   as->snapref = nins;
   as->snapno = T->nsnap;
+  as->snapalloc = 0;
 
   as->stopins = REF_BASE;
   as->orignins = nins;
@@ -2249,7 +2290,6 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
 {
   ASMState as_;
   ASMState *as = &as_;
-  MCode *origtop;
 
   /* Remove nops/renames left over from ASM restart due to LJ_TRERR_MCODELM. */
   {
@@ -2277,7 +2317,7 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
   as->parent = J->parent ? traceref(J, J->parent) : NULL;
 
   /* Reserve MCode memory. */
-  as->mctop = origtop = lj_mcode_reserve(J, &as->mcbot);
+  as->mctop = as->mctoporig = lj_mcode_reserve(J, &as->mcbot);
   as->mcp = as->mctop;
   as->mclim = as->mcbot + MCLIM_REDZONE;
   asm_setup_target(as);
@@ -2336,6 +2376,7 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
     for (as->curins--; as->curins > as->stopins; as->curins--) {
       IRIns *ir = IR(as->curins);
       lua_assert(!(LJ_32 && irt_isint64(ir->t)));  /* Handled by SPLIT. */
+      asm_snap_prev(as);
       if (!ra_used(ir) && !ir_sideeff(ir) && (as->flags & JIT_F_OPT_DCE))
 	continue;  /* Dead-code elimination can be soooo easy. */
       if (irt_isguard(ir->t))
@@ -2369,6 +2410,9 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
       memcpy(J->curfinal->ir + as->orignins, T->ir + as->orignins,
 	     (T->nins - as->orignins) * sizeof(IRIns));  /* Copy RENAMEs. */
       T->nins = J->curfinal->nins;
+      /* Fill mcofs of any unprocessed snapshots. */
+      as->curins = REF_FIRST;
+      asm_snap_prev(as);
       break;  /* Done. */
     }
 
@@ -2390,10 +2434,11 @@ void lj_asm_trace(jit_State *J, GCtrace *T)
   if (!as->loopref)
     asm_tail_fixup(as, T->link);  /* Note: this may change as->mctop! */
   T->szmcode = (MSize)((char *)as->mctop - (char *)as->mcp);
+  asm_snap_fixup_mcofs(as);
 #if LJ_TARGET_MCODE_FIXUP
   asm_mcode_fixup(T->mcode, T->szmcode);
 #endif
-  lj_mcode_sync(T->mcode, origtop);
+  lj_mcode_sync(T->mcode, as->mctoporig);
 }
 
 #undef IR
diff --git a/src/lj_dispatch.h b/src/lj_dispatch.h
index addf5572..b8bc2594 100644
--- a/src/lj_dispatch.h
+++ b/src/lj_dispatch.h
@@ -31,7 +31,7 @@ extern double __divdf3(double a, double b);
 #define SFGOTDEF(_)
 #endif
 #if LJ_HASJIT
-#define JITGOTDEF(_)	_(lj_trace_exit) _(lj_trace_hot)
+#define JITGOTDEF(_)	_(lj_err_trace) _(lj_trace_exit) _(lj_trace_hot)
 #else
 #define JITGOTDEF(_)
 #endif
@@ -46,7 +46,7 @@ extern double __divdf3(double a, double b);
   _(asin) _(acos) _(atan) _(sinh) _(cosh) _(tanh) _(frexp) _(modf) _(atan2) \
   _(pow) _(fmod) _(ldexp) _(lj_vm_modi) \
   _(lj_dispatch_call) _(lj_dispatch_ins) _(lj_dispatch_stitch) \
-  _(lj_dispatch_profile) _(lj_err_throw) _(lj_err_run) \
+  _(lj_dispatch_profile) _(lj_err_throw) \
   _(lj_ffh_coroutine_wrap_err) _(lj_func_closeuv) _(lj_func_newL_gc) \
   _(lj_gc_barrieruv) _(lj_gc_step) _(lj_gc_step_fixtop) _(lj_meta_arith) \
   _(lj_meta_call) _(lj_meta_cat) _(lj_meta_comp) _(lj_meta_equal) \
diff --git a/src/lj_err.c b/src/lj_err.c
index d0223384..c7fd9e65 100644
--- a/src/lj_err.c
+++ b/src/lj_err.c
@@ -52,6 +52,11 @@
 **   the wrapper function feature. Lua errors thrown through C++ frames
 **   cannot be caught by C++ code and C++ destructors are not run.
 **
+** - EXT can handle errors from internal helper functions that are called
+**   from JIT-compiled code (except for Windows/x86 and 32 bit ARM).
+**   INT has no choice but to call the panic handler, if this happens.
+**   Note: this is mainly relevant for out-of-memory errors.
+**
 ** EXT is the default on all systems where the toolchain produces unwind
 ** tables by default (*). This is hard-coded and/or detected in src/Makefile.
 ** You can thwart the detection with: TARGET_XCFLAGS=-DLUAJIT_UNWIND_INTERNAL
@@ -304,12 +309,59 @@ LJ_FUNCA int lj_err_unwind_win(EXCEPTION_RECORD *rec,
   return 1;  /* ExceptionContinueSearch */
 }
 
+#if LJ_UNWIND_JIT
+
+#if LJ_TARGET_X64
+#define CONTEXT_REG_PC	Rip
+#elif LJ_TARGET_ARM64
+#define CONTEXT_REG_PC	Pc
+#else
+#error "NYI: Windows arch-specific unwinder for JIT-compiled code"
+#endif
+
+/* Windows unwinder for JIT-compiled code. */
+static void err_unwind_win_jit(global_State *g, int errcode)
+{
+  CONTEXT ctx;
+  UNWIND_HISTORY_TABLE hist;
+
+  memset(&hist, 0, sizeof(hist));
+  RtlCaptureContext(&ctx);
+  while (1) {
+    uintptr_t frame, base, addr = ctx.CONTEXT_REG_PC;
+    void *hdata;
+    PRUNTIME_FUNCTION func = RtlLookupFunctionEntry(addr, &base, &hist);
+    if (!func) {  /* Found frame without .pdata: must be JIT-compiled code. */
+      ExitNo exitno;
+      uintptr_t stub = lj_trace_unwind(G2J(g), addr - sizeof(MCode), &exitno);
+      if (stub) {  /* Jump to side exit to unwind the trace. */
+	ctx.CONTEXT_REG_PC = stub;
+	G2J(g)->exitcode = errcode;
+	RtlRestoreContext(&ctx, NULL);  /* Does not return. */
+      }
+      break;
+    }
+    RtlVirtualUnwind(UNW_FLAG_NHANDLER, base, addr, func,
+		     &ctx, &hdata, &frame, NULL);
+    if (!addr) break;
+  }
+  /* Unwinding failed, if we end up here. */
+}
+#endif
+
 /* Raise Windows exception. */
 static void err_raise_ext(global_State *g, int errcode)
 {
-#if LJ_HASJIT
+#if LJ_UNWIND_JIT
+  if (tvref(g->jit_base)) {
+    err_unwind_win_jit(g, errcode);
+    return;  /* Unwinding failed. */
+  }
+#elif LJ_HASJIT
+  /* Cannot catch on-trace errors for Windows/x86 SEH. Unwind to interpreter. */
   setmref(g->jit_base, NULL);
 #endif
+  UNUSED(g);
   RaiseException(LJ_EXCODE_MAKE(errcode), 1 /* EH_NONCONTINUABLE */, 0, NULL);
 }
 
@@ -323,6 +375,7 @@ static void err_raise_ext(global_State *g, int errcode)
 typedef struct _Unwind_Context _Unwind_Context;
 
 #define _URC_OK			0
+#define _URC_FATAL_PHASE2_ERROR	2
 #define _URC_FATAL_PHASE1_ERROR	3
 #define _URC_HANDLER_FOUND	6
 #define _URC_INSTALL_CONTEXT	7
@@ -342,9 +395,11 @@ typedef struct _Unwind_Exception
   void (*excleanup)(int, struct _Unwind_Exception *);
   uintptr_t p1, p2;
 } __attribute__((__aligned__)) _Unwind_Exception;
+#define UNWIND_EXCEPTION_TYPE	_Unwind_Exception
 
 extern uintptr_t _Unwind_GetCFA(_Unwind_Context *);
 extern void _Unwind_SetGR(_Unwind_Context *, int, uintptr_t);
+extern uintptr_t _Unwind_GetIP(_Unwind_Context *);
 extern void _Unwind_SetIP(_Unwind_Context *, uintptr_t);
 extern void _Unwind_DeleteException(_Unwind_Exception *);
 extern int _Unwind_RaiseException(_Unwind_Exception *);
@@ -417,13 +472,150 @@ LJ_FUNCA int lj_err_unwind_dwarf(int version, int actions,
   return _URC_CONTINUE_UNWIND;
 }
 
-#if LJ_UNWIND_EXT
-#if LJ_TARGET_OSX || defined(__OpenBSD__)
-/* Sorry, no thread safety for OSX. Complain to Apple, not me. */
-static _Unwind_Exception static_uex;
+#if LJ_UNWIND_EXT && defined(LUA_USE_ASSERT)
+struct dwarf_eh_bases { void *tbase, *dbase, *func; };
+extern const void *_Unwind_Find_FDE(void *pc, struct dwarf_eh_bases *bases);
+
+/* Verify that external error handling actually has a chance to work. */
+void lj_err_verify(void)
+{
+  struct dwarf_eh_bases ehb;
+  /*
+  ** FIXME: The following assertions were replaced with
+  ** the conventional `lua_assert` ones.
+  **
+  ** lj_assertX(_Unwind_Find_FDE((void *)lj_err_throw, &ehb), "broken build: external frame unwinding enabled, but missing -funwind-tables");
+  ** lj_assertX(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb), "broken build: external frame unwinding enabled, but system libraries have no unwind tables");
+  */
+  lua_assert(_Unwind_Find_FDE((void *)lj_err_throw, &ehb));
+  lua_assert(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb));
+}
+#endif
+
+#if LJ_UNWIND_JIT
+/* DWARF2 personality handler for JIT-compiled code. */
+static int err_unwind_jit(int version, int actions,
+  uint64_t uexclass, _Unwind_Exception *uex, _Unwind_Context *ctx)
+{
+  /* NYI: FFI C++ exception interoperability. */
+  if (version != 1 || !LJ_UEXCLASS_CHECK(uexclass))
+    return _URC_FATAL_PHASE1_ERROR;
+  if ((actions & _UA_SEARCH_PHASE)) {
+    return _URC_HANDLER_FOUND;
+  }
+  if ((actions & _UA_CLEANUP_PHASE)) {
+    global_State *g = *(global_State **)(uex+1);
+    ExitNo exitno;
+    uintptr_t addr = _Unwind_GetIP(ctx);  /* Return address _after_ call. */
+    uintptr_t stub = lj_trace_unwind(G2J(g), addr - sizeof(MCode), &exitno);
+    /*
+    ** FIXME: The following assert was replaced with
+    ** the conventional `lua_assert`.
+    **
+    ** lj_assertG(tvref(g->jit_base), "unexpected throw across mcode frame");
+    */
+    lua_assert(tvref(g->jit_base));
+    if (stub) {  /* Jump to side exit to unwind the trace. */
+      G2J(g)->exitcode = LJ_UEXCLASS_ERRCODE(uexclass);
+#ifdef LJ_TARGET_MIPS
+      _Unwind_SetGR(ctx, 4, stub);
+      _Unwind_SetGR(ctx, 5, exitno);
+      _Unwind_SetIP(ctx, (uintptr_t)(void *)lj_vm_unwind_stub);
+#else
+      _Unwind_SetIP(ctx, stub);
+#endif
+      return _URC_INSTALL_CONTEXT;
+    }
+    return _URC_FATAL_PHASE2_ERROR;
+  }
+  return _URC_FATAL_PHASE1_ERROR;
+}
+
+/* DWARF2 template frame info for JIT-compiled code.
+**
+** After copying the template to the start of the mcode segment,
+** the frame handler function and the code size is patched.
+** The frame handler always installs a new context to jump to the exit,
+** so don't bother to add any unwind opcodes.
+*/
+static const uint8_t err_frame_jit_template[] = {
+#if LJ_BE
+  0,0,0,
+#endif
+  LJ_64 ? 0x1c : 0x14,  /* CIE length. */
+#if LJ_LE
+  0,0,0,
+#endif
+  0,0,0,0, 1, 'z','P','R',0,  /* CIE mark, CIE version, augmentation. */
+  1, LJ_64 ? 0x78 : 0x7c, LJ_TARGET_EHRAREG,  /* Code/data align, RA. */
+#if LJ_64
+  10, 0, 0,0,0,0,0,0,0,0, 0x1b,  /* Aug. data ABS handler, PCREL|SDATA4 code. */
+  0,0,0,0,0,  /* Alignment. */
+#else
+  6, 0, 0,0,0,0, 0x1b,  /* Aug. data ABS handler, PCREL|SDATA4 code. */
+  0,  /* Alignment. */
+#endif
+#if LJ_BE
+  0,0,0,
+#endif
+  LJ_64 ? 0x14 : 0x10,  /* FDE length. */
+  0,0,0,
+  LJ_64 ? 0x24 : 0x1c,  /* CIE offset. */
+  0,0,0,
+  LJ_64 ? 0x14 : 0x10,  /* Code offset. After Final FDE. */
+#if LJ_LE
+  0,0,0,
+#endif
+  0,0,0,0, 0, 0,0,0, /* Code size, augmentation length, alignment. */
+#if LJ_64
+  0,0,0,0,  /* Alignment. */
+#endif
+  0,0,0,0  /* Final FDE. */
+};
+
+#define ERR_FRAME_JIT_OFS_HANDLER	0x12
+#define ERR_FRAME_JIT_OFS_FDE		(LJ_64 ? 0x20 : 0x18)
+#define ERR_FRAME_JIT_OFS_CODE_SIZE	(LJ_64 ? 0x2c : 0x24)
+#if LJ_TARGET_OSX
+#define ERR_FRAME_JIT_OFS_REGISTER	ERR_FRAME_JIT_OFS_FDE
 #else
-static __thread _Unwind_Exception static_uex;
+#define ERR_FRAME_JIT_OFS_REGISTER	0
 #endif
+
+extern void __register_frame(const void *);
+extern void __deregister_frame(const void *);
+
+uint8_t *lj_err_register_mcode(void *base, size_t sz, uint8_t *info)
+{
+  void **handler;
+  memcpy(info, err_frame_jit_template, sizeof(err_frame_jit_template));
+  handler = (void *)err_unwind_jit;
+  memcpy(info + ERR_FRAME_JIT_OFS_HANDLER, &handler, sizeof(handler));
+  *(uint32_t *)(info + ERR_FRAME_JIT_OFS_CODE_SIZE) =
+    (uint32_t)(sz - sizeof(err_frame_jit_template) - (info - (uint8_t *)base));
+  __register_frame(info + ERR_FRAME_JIT_OFS_REGISTER);
+#ifdef LUA_USE_ASSERT
+  {
+    struct dwarf_eh_bases ehb;
+    /*
+    ** FIXME: The following assert was replaced with
+    ** the conventional `lua_assert`.
+    **
+    ** lj_assertX(_Unwind_Find_FDE(info + sizeof(err_frame_jit_template)+1, &ehb),
+    **      "bad JIT unwind table registration");
+    */
+    lua_assert(_Unwind_Find_FDE(info + sizeof(err_frame_jit_template)+1,
+               &ehb));
+  }
+#endif
+  return info + sizeof(err_frame_jit_template);
+}
+
+void lj_err_deregister_mcode(void *base, size_t sz, uint8_t *info)
+{
+  UNUSED(base); UNUSED(sz);
+  __deregister_frame(info + ERR_FRAME_JIT_OFS_REGISTER);
+}
 #endif
 
 #else /* LJ_TARGET_ARM */
@@ -434,6 +626,7 @@ static __thread _Unwind_Exception static_uex;
 #define _US_FORCE_UNWIND		8
 
 typedef struct _Unwind_Control_Block _Unwind_Control_Block;
+#define UNWIND_EXCEPTION_TYPE	_Unwind_Control_Block
 
 struct _Unwind_Control_Block {
   uint64_t exclass;
@@ -492,25 +685,68 @@ LJ_FUNCA int lj_err_unwind_arm(int state, _Unwind_Control_Block *ucb,
   }
   if (__gnu_unwind_frame(ucb, ctx) != _URC_OK)
     return _URC_FAILURE;
+#ifdef LUA_USE_ASSERT
+  /* We should never get here unless this is a forced unwind aka backtrace. */
+  if (_Unwind_GetGR(ctx, 0) == 0xff33aa77) {
+    _Unwind_SetGR(ctx, 0, 0xff33aa88);
+  }
+#endif
   return _URC_CONTINUE_UNWIND;
 }
 
-#if LJ_UNWIND_EXT
-static __thread _Unwind_Control_Block static_uex;
+#if LJ_UNWIND_EXT && defined(LUA_USE_ASSERT)
+typedef int (*_Unwind_Trace_Fn)(_Unwind_Context *, void *);
+extern int _Unwind_Backtrace(_Unwind_Trace_Fn, void *);
+
+static int err_verify_bt(_Unwind_Context *ctx, int *got)
+{
+  if (_Unwind_GetGR(ctx, 0) == 0xff33aa88) { *got = 2; }
+  else if (*got == 0) { *got = 1; _Unwind_SetGR(ctx, 0, 0xff33aa77); }
+  return _URC_OK;
+}
+
+/* Verify that external error handling actually has a chance to work. */
+void lj_err_verify(void)
+{
+  int got = 0;
+  _Unwind_Backtrace((_Unwind_Trace_Fn)err_verify_bt, &got);
+  /*
+  ** FIXME: The following assert was replaced with
+  ** the conventional `lua_assert`.
+  **
+  ** lj_assertX(got == 2, "broken build: external frame unwinding enabled, but missing -funwind-tables");
+  */
+  lua_assert(got == 2);
+}
 #endif
+
+/*
+** Note: LJ_UNWIND_JIT is not implemented for 32 bit ARM.
+**
+** The quirky ARM unwind API doesn't have __register_frame().
+** A potential workaround might involve _Unwind_Backtrace.
+** But most 32 bit ARM targets don't qualify for LJ_UNWIND_EXT, anyway,
+** since they are built without unwind tables by default.
+*/
+
 #endif /* LJ_TARGET_ARM */
 
+
 #if LJ_UNWIND_EXT
+static __thread struct {
+  UNWIND_EXCEPTION_TYPE ex;
+  global_State *g;
+} static_uex;
+
 /* Raise external exception. */
 static void err_raise_ext(global_State *g, int errcode)
 {
-#if LJ_HASJIT
-  setmref(g->jit_base, NULL);
-#endif
   memset(&static_uex, 0, sizeof(static_uex));
-  static_uex.exclass = LJ_UEXCLASS_MAKE(errcode);
-  _Unwind_RaiseException(&static_uex);
+  static_uex.ex.exclass = LJ_UEXCLASS_MAKE(errcode);
+  static_uex.g = g;
+  _Unwind_RaiseException(&static_uex.ex);
 }
+
 #endif
 
 #endif
@@ -620,7 +856,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
 /* Runtime error. */
 LJ_NOINLINE void LJ_FASTCALL lj_err_run(lua_State *L)
 {
-  ptrdiff_t ef = finderrfunc(L);
+  ptrdiff_t ef = (LJ_HASJIT && tvref(G(L)->jit_base)) ? 0 : finderrfunc(L);
   if (ef) {
     TValue *errfunc = restorestack(L, ef);
     TValue *top = L->top;
@@ -639,6 +875,16 @@ LJ_NOINLINE void LJ_FASTCALL lj_err_run(lua_State *L)
   lj_err_throw(L, LUA_ERRRUN);
 }
 
+#if LJ_HASJIT
+LJ_NOINLINE void LJ_FASTCALL lj_err_trace(lua_State *L, int errcode)
+{
+  if (errcode == LUA_ERRRUN)
+    lj_err_run(L);
+  else
+    lj_err_throw(L, errcode);
+}
+#endif
+
 /* Formatted runtime error message. */
 LJ_NORET LJ_NOINLINE static void err_msgv(lua_State *L, ErrMsg em, ...)
 {
diff --git a/src/lj_err.h b/src/lj_err.h
index aa4b7e0d..b0c72c24 100644
--- a/src/lj_err.h
+++ b/src/lj_err.h
@@ -23,7 +23,10 @@ LJ_DATA const char *lj_err_allmsg;
 LJ_FUNC GCstr *lj_err_str(lua_State *L, ErrMsg em);
 LJ_FUNCA_NORET void LJ_FASTCALL lj_err_throw(lua_State *L, int errcode);
 LJ_FUNC_NORET void lj_err_mem(lua_State *L);
-LJ_FUNCA_NORET void LJ_FASTCALL lj_err_run(lua_State *L);
+LJ_FUNC_NORET void LJ_FASTCALL lj_err_run(lua_State *L);
+#if LJ_HASJIT
+LJ_FUNCA_NORET void LJ_FASTCALL lj_err_trace(lua_State *L, int errcode);
+#endif
 LJ_FUNC_NORET void lj_err_msg(lua_State *L, ErrMsg em);
 LJ_FUNC_NORET void lj_err_lex(lua_State *L, GCstr *src, const char *tok,
 			      BCLine line, ErrMsg em, va_list argp);
@@ -38,4 +41,18 @@ LJ_FUNC_NORET void lj_err_argv(lua_State *L, int narg, ErrMsg em, ...);
 LJ_FUNC_NORET void lj_err_argtype(lua_State *L, int narg, const char *xname);
 LJ_FUNC_NORET void lj_err_argt(lua_State *L, int narg, int tt);
 
+#if LJ_UNWIND_JIT && !LJ_ABI_WIN
+LJ_FUNC uint8_t *lj_err_register_mcode(void *base, size_t sz, uint8_t *info);
+LJ_FUNC void lj_err_deregister_mcode(void *base, size_t sz, uint8_t *info);
+#else
+#define lj_err_register_mcode(base, sz, info)	(info)
+#define lj_err_deregister_mcode(base, sz, info)	UNUSED(base)
+#endif
+
+#if LJ_UNWIND_EXT && !LJ_ABI_WIN && defined(LUA_USE_ASSERT)
+LJ_FUNC void lj_err_verify(void);
+#else
+#define lj_err_verify()		((void)0)
+#endif
+
 #endif
diff --git a/src/lj_ffrecord.c b/src/lj_ffrecord.c
index 649ac705..8af9da1d 100644
--- a/src/lj_ffrecord.c
+++ b/src/lj_ffrecord.c
@@ -455,6 +455,7 @@ static void LJ_FASTCALL recff_pcall(jit_State *J, RecordFFData *rd)
 #endif
     lj_record_call(J, 0, J->maxslot - 1);
     rd->nres = -1;  /* Pending call. */
+    J->needsnap = 1;  /* Start catching on-trace errors. */
   }  /* else: Interpreter will throw. */
 }
 
@@ -490,6 +491,7 @@ static void LJ_FASTCALL recff_xpcall(jit_State *J, RecordFFData *rd)
     if (errcode)
       lj_err_throw(J->L, errcode);  /* Propagate errors. */
     rd->nres = -1;  /* Pending call. */
+    J->needsnap = 1;  /* Start catching on-trace errors. */
   }  /* else: Interpreter will throw. */
 }
 
diff --git a/src/lj_jit.h b/src/lj_jit.h
index d82292f8..f2ad3c6e 100644
--- a/src/lj_jit.h
+++ b/src/lj_jit.h
@@ -162,6 +162,7 @@ typedef uint32_t MCode;
 typedef struct SnapShot {
   uint32_t mapofs;	/* Offset into snapshot map. */
   IRRef1 ref;		/* First IR ref for this snapshot. */
+  uint16_t mcofs;	/* Offset into machine code in MCode units. */
   uint8_t nslots;	/* Number of valid slots. */
   uint8_t topslot;	/* Maximum frame extent. */
   uint8_t nent;		/* Number of compressed entries. */
@@ -464,6 +465,7 @@ typedef struct jit_State {
   const BCIns *startpc;	/* Bytecode PC of starting instruction. */
   TraceNo parent;	/* Parent of current side trace (0 for root traces). */
   ExitNo exitno;	/* Exit number in parent of current side trace. */
+  int exitcode;		/* Exit code from unwound trace. */
 
   BCIns *patchpc;	/* PC for pending re-patch. */
   BCIns patchins;	/* Instruction for pending re-patch. */
diff --git a/src/lj_mcode.c b/src/lj_mcode.c
index 77035bf7..7184d3b4 100644
--- a/src/lj_mcode.c
+++ b/src/lj_mcode.c
@@ -292,6 +292,7 @@ static void mcode_allocarea(jit_State *J)
   ((MCLink *)J->mcarea)->next = oldarea;
   ((MCLink *)J->mcarea)->size = sz;
   J->szallmcarea += sz;
+  J->mcbot = (MCode *)lj_err_register_mcode(J->mcarea, sz, (uint8_t *)J->mcbot);
 }
 
 /* Free all MCode areas. */
@@ -302,7 +303,9 @@ void lj_mcode_free(jit_State *J)
   J->szallmcarea = 0;
   while (mc) {
     MCode *next = ((MCLink *)mc)->next;
-    mcode_free(J, mc, ((MCLink *)mc)->size);
+    size_t sz = ((MCLink *)mc)->size;
+    lj_err_deregister_mcode(mc, sz, (uint8_t *)mc + sizeof(MCLink));
+    mcode_free(J, mc, sz);
     mc = next;
   }
 }
diff --git a/src/lj_opt_loop.c b/src/lj_opt_loop.c
index 441b8add..10613641 100644
--- a/src/lj_opt_loop.c
+++ b/src/lj_opt_loop.c
@@ -225,6 +225,7 @@ static void loop_subst_snap(jit_State *J, SnapShot *osnap,
   /* Setup new snapshot. */
   snap->mapofs = (uint32_t)nmapofs;
   snap->ref = (IRRef1)J->cur.nins;
+  snap->mcofs = 0;
   snap->nslots = nslots;
   snap->topslot = osnap->topslot;
   snap->count = 0;
diff --git a/src/lj_record.c b/src/lj_record.c
index 9e2e1d9e..e7dac7ac 100644
--- a/src/lj_record.c
+++ b/src/lj_record.c
@@ -800,6 +800,7 @@ void lj_record_ret(jit_State *J, BCReg rbase, ptrdiff_t gotresults)
     J->base -= cbase;
     J->base[--rbase] = TREF_TRUE;  /* Prepend true to results. */
     frame = frame_prevd(frame);
+    J->needsnap = 1;  /* Stop catching on-trace errors. */
   }
   /* Return to lower frame via interpreter for unhandled cases. */
   if (J->framedepth == 0 && J->pt && bc_isret(bc_op(*J->pc)) &&
@@ -2021,7 +2022,7 @@ void lj_record_ins(jit_State *J)
   /* Need snapshot before recording next bytecode (e.g. after a store). */
   if (J->needsnap) {
     J->needsnap = 0;
-    lj_snap_purge(J);
+    if (J->pt) lj_snap_purge(J);
     lj_snap_add(J);
     J->mergesnap = 1;
   }
diff --git a/src/lj_snap.c b/src/lj_snap.c
index 2f7cf80a..a8b49fcb 100644
--- a/src/lj_snap.c
+++ b/src/lj_snap.c
@@ -163,6 +163,7 @@ static void snapshot_stack(jit_State *J, SnapShot *snap, MSize nsnapmap)
   nent += snapshot_framelinks(J, p + nent, &snap->topslot);
   snap->mapofs = (uint32_t)nsnapmap;
   snap->ref = (IRRef1)J->cur.nins;
+  snap->mcofs = 0;
   snap->nslots = (uint8_t)nslots;
   snap->count = 0;
   J->cur.nsnapmap = (uint32_t)(nsnapmap + nent);
diff --git a/src/lj_state.c b/src/lj_state.c
index cc6f92f1..4add3d65 100644
--- a/src/lj_state.c
+++ b/src/lj_state.c
@@ -173,6 +173,7 @@ static TValue *cpluaopen(lua_State *L, lua_CFunction dummy, void *ud)
   fixstring(lj_err_str(L, LJ_ERR_ERRMEM));  /* Preallocate memory error msg. */
   g->gc.threshold = 4*g->gc.total;
   lj_trace_initstate(g);
+  lj_err_verify();
   return NULL;
 }
 
diff --git a/src/lj_target_x86.h b/src/lj_target_x86.h
index 194f8e70..4efb566b 100644
--- a/src/lj_target_x86.h
+++ b/src/lj_target_x86.h
@@ -165,6 +165,8 @@ typedef struct {
 #define EXITSTUB_SPACING	(2+2)
 #define EXITSTUBS_PER_GROUP	32
 
+#define EXITTRACE_VMSTATE	1	/* g->vmstate has traceno on exit. */
+
 /* -- x86 ModRM operand encoding ------------------------------------------ */
 
 typedef enum {
diff --git a/src/lj_trace.c b/src/lj_trace.c
index c6e2f72e..17743159 100644
--- a/src/lj_trace.c
+++ b/src/lj_trace.c
@@ -838,7 +838,7 @@ static void trace_exit_regs(lua_State *L, ExitState *ex)
 }
 #endif
 
-#ifdef EXITSTATE_PCREG
+#if defined(EXITSTATE_PCREG) || (LJ_UNWIND_JIT && !EXITTRACE_VMSTATE)
 /* Determine trace number from pc of exit instruction. */
 static TraceNo trace_exit_find(jit_State *J, MCode *pc)
 {
@@ -860,10 +860,18 @@ int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr)
   lua_State *L = J->L;
   ExitState *ex = (ExitState *)exptr;
   ExitDataCP exd;
-  int errcode;
+  int errcode, exitcode = J->exitcode;
+  TValue exiterr;
   const BCIns *pc;
   void *cf;
   GCtrace *T;
+
+  setnilV(&exiterr);
+  if (exitcode) {  /* Trace unwound with error code. */
+    J->exitcode = 0;
+    copyTV(L, &exiterr, L->top-1);
+  }
+
 #ifdef EXITSTATE_PCREG
   J->parent = trace_exit_find(J, (MCode *)(intptr_t)ex->gpr[EXITSTATE_PCREG]);
 #endif
@@ -883,6 +891,8 @@ int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr)
   if (errcode)
     return -errcode;  /* Return negated error code. */
 
+  if (exitcode) copyTV(L, L->top++, &exiterr);  /* Anchor the error object. */
+
   if (!(LJ_HASPROFILE && (G(L)->hookmask & HOOK_PROFILE)))
     lj_vmevent_send(L, TEXIT,
       lj_state_checkstack(L, 4+RID_NUM_GPR+RID_NUM_FPR+LUA_MINSTACK);
@@ -894,7 +904,9 @@ int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr)
   pc = exd.pc;
   cf = cframe_raw(L->cframe);
   setcframe_pc(cf, pc);
-  if (LJ_HASPROFILE && (G(L)->hookmask & HOOK_PROFILE)) {
+  if (exitcode) {
+    return -exitcode;
+  } else if (LJ_HASPROFILE && (G(L)->hookmask & HOOK_PROFILE)) {
     /* Just exit to interpreter. */
   } else if (G(L)->gc.state == GCSatomic || G(L)->gc.state == GCSfinalize) {
     if (!(G(L)->hookmask & HOOK_GC))
@@ -932,4 +944,47 @@ int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr)
   }
 }
 
+#if LJ_UNWIND_JIT
+/* Given an mcode address determine trace exit address for unwinding. */
+uintptr_t LJ_FASTCALL lj_trace_unwind(jit_State *J, uintptr_t addr, ExitNo *ep)
+{
+#if EXITTRACE_VMSTATE
+  TraceNo traceno = J2G(J)->vmstate;
+#else
+  TraceNo traceno = trace_exit_find(J, (MCode *)addr);
+#endif
+  GCtrace *T = traceref(J, traceno);
+  if (T
+#if EXITTRACE_VMSTATE
+      && addr >= (uintptr_t)T->mcode && addr < (uintptr_t)T->mcode + T->szmcode
+#endif
+     ) {
+    SnapShot *snap = T->snap;
+    SnapNo lo = 0, exitno = T->nsnap;
+    uintptr_t ofs = (uintptr_t)((MCode *)addr - T->mcode);  /* MCode units! */
+    /* Rightmost binary search for mcode offset to determine exit number. */
+    do {
+      SnapNo mid = (lo+exitno) >> 1;
+      if (ofs < snap[mid].mcofs) exitno = mid; else lo = mid + 1;
+    } while (lo < exitno);
+    exitno--;
+    *ep = exitno;
+#ifdef EXITSTUBS_PER_GROUP
+    return (uintptr_t)exitstub_addr(J, exitno);
+#else
+    return (uintptr_t)exitstub_trace_addr(T, exitno);
+#endif
+  }
+  /* Cannot correlate addr with trace/exit. This will be fatal. */
+  /*
+  ** FIXME: The following assert was replaced with
+  ** the conventional `lua_assert`.
+  **
+  ** lj_assertJ(0, "bad exit pc");
+  */
+  lua_assert(0);
+  return 0;
+}
+#endif
+
 #endif
diff --git a/src/lj_trace.h b/src/lj_trace.h
index 22cae741..0bfb606f 100644
--- a/src/lj_trace.h
+++ b/src/lj_trace.h
@@ -37,6 +37,9 @@ LJ_FUNC void lj_trace_ins(jit_State *J, const BCIns *pc);
 LJ_FUNCA void LJ_FASTCALL lj_trace_hot(jit_State *J, const BCIns *pc);
 LJ_FUNCA void LJ_FASTCALL lj_trace_stitch(jit_State *J, const BCIns *pc);
 LJ_FUNCA int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr);
+#if LJ_UNWIND_EXT
+LJ_FUNC uintptr_t LJ_FASTCALL lj_trace_unwind(jit_State *J, uintptr_t addr, ExitNo *ep);
+#endif
 
 /* Signal asynchronous abort of trace or end of trace. */
 #define lj_trace_abort(g)	(G2J(g)->state &= ~LJ_TRACE_ACTIVE)
diff --git a/src/lj_vm.h b/src/lj_vm.h
index 1cc7eed7..411caafa 100644
--- a/src/lj_vm.h
+++ b/src/lj_vm.h
@@ -26,6 +26,9 @@ LJ_ASMF void lj_vm_unwind_ff_eh(void);
 #if LJ_TARGET_X86ORX64
 LJ_ASMF void lj_vm_unwind_rethrow(void);
 #endif
+#if LJ_TARGET_MIPS
+LJ_ASMF void lj_vm_unwind_stub(void);
+#endif
 
 /* Miscellaneous functions. */
 #if LJ_TARGET_X86ORX64
diff --git a/src/vm_arm.dasc b/src/vm_arm.dasc
index a29292f1..436a9fd7 100644
--- a/src/vm_arm.dasc
+++ b/src/vm_arm.dasc
@@ -2247,8 +2247,9 @@ static void build_subroutines(BuildCtx *ctx)
   |  b <2
   |
   |9:  // Rethrow error from the right C frame.
+  |  rsb CARG2, CARG1, #0
   |  mov CARG1, L
-  |  bl extern lj_err_run		// (lua_State *L)
+  |  bl extern lj_err_trace		// (lua_State *L, int errcode)
   |.endif
   |
   |//-----------------------------------------------------------------------
diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc
index f517a808..7066b051 100644
--- a/src/vm_arm64.dasc
+++ b/src/vm_arm64.dasc
@@ -2040,9 +2040,9 @@ static void build_subroutines(BuildCtx *ctx)
   |  b <2
   |
   |9:  // Rethrow error from the right C frame.
-  |  neg CARG2, CARG1
+  |  neg CARG2w, CARG1w
   |  mov CARG1, L
-  |  bl extern lj_err_throw		// (lua_State *L, int errcode)
+  |  bl extern lj_err_trace		// (lua_State *L, int errcode)
   |.endif
   |
   |//-----------------------------------------------------------------------
diff --git a/src/vm_mips.dasc b/src/vm_mips.dasc
index 93c772ff..32caabf7 100644
--- a/src/vm_mips.dasc
+++ b/src/vm_mips.dasc
@@ -501,6 +501,10 @@ static void build_subroutines(BuildCtx *ctx)
   |  b ->vm_returnc
   |.  li RD, 16				// 2 results: false + error message.
   |
+  |->vm_unwind_stub:			// Jump to exit stub from unwinder.
+  |  jr CARG1
+  |.  move ra, CARG2
+  |
   |//-----------------------------------------------------------------------
   |//-- Grow stack for calls -----------------------------------------------
   |//-----------------------------------------------------------------------
@@ -2512,8 +2516,9 @@ static void build_subroutines(BuildCtx *ctx)
   |.  addu RA, RA, BASE
   |
   |9:  // Rethrow error from the right C frame.
-  |  load_got lj_err_run
-  |  call_intern lj_err_run		// (lua_State *L)
+  |  load_got lj_err_trace
+  |  sub CARG2, r0, CRET1
+  |  call_intern lj_err_trace		// (lua_State *L, int errcode)
   |.  move CARG1, L
   |.endif
   |
diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc
index 9a749f93..04be38f0 100644
--- a/src/vm_mips64.dasc
+++ b/src/vm_mips64.dasc
@@ -545,6 +545,10 @@ static void build_subroutines(BuildCtx *ctx)
   |  b ->vm_returnc
   |.  li RD, 16				// 2 results: false + error message.
   |
+  |->vm_unwind_stub:			// Jump to exit stub from unwinder.
+  |  jr CARG1
+  |.  move ra, CARG2
+  |
   |//-----------------------------------------------------------------------
   |//-- Grow stack for calls -----------------------------------------------
   |//-----------------------------------------------------------------------
@@ -2470,9 +2474,9 @@ static void build_subroutines(BuildCtx *ctx)
   |.  daddu RA, RA, BASE
   |
   |9:  // Rethrow error from the right C frame.
-  |  load_got lj_err_throw
-  |  negu CARG2, CRET1
-  |  call_intern lj_err_throw		// (lua_State *L, int errcode)
+  |  load_got lj_err_trace
+  |  sub CARG2, r0, CRET1
+  |  call_intern lj_err_trace		// (lua_State *L, int errcode)
   |.  move CARG1, L
   |.endif
   |
diff --git a/src/vm_ppc.dasc b/src/vm_ppc.dasc
index 980176a2..7ad8df37 100644
--- a/src/vm_ppc.dasc
+++ b/src/vm_ppc.dasc
@@ -2707,8 +2707,9 @@ static void build_subroutines(BuildCtx *ctx)
   |  bctr
   |
   |9:  // Rethrow error from the right C frame.
+  |  neg CARG2, CARG1
   |  mr CARG1, L
-  |  bl extern lj_err_run		// (lua_State *L)
+  |  bl extern lj_err_trace		// (lua_State *L, int errcode)
   |.endif
   |
   |//-----------------------------------------------------------------------
diff --git a/src/vm_x64.dasc b/src/vm_x64.dasc
index 59f117ba..27af164a 100644
--- a/src/vm_x64.dasc
+++ b/src/vm_x64.dasc
@@ -2565,10 +2565,10 @@ static void build_subroutines(BuildCtx *ctx)
   |  jmp <2
   |
   |9:  // Rethrow error from the right C frame.
-  |  neg RD
+  |  mov CARG2d, RDd
   |  mov CARG1, L:RB
-  |  mov CARG2, RD
-  |  call extern lj_err_throw		// (lua_State *L, int errcode)
+  |  neg CARG2d
+  |  call extern lj_err_trace		// (lua_State *L, int errcode)
   |.endif
   |
   |//-----------------------------------------------------------------------
diff --git a/src/vm_x86.dasc b/src/vm_x86.dasc
index f7ffe5d2..85807129 100644
--- a/src/vm_x86.dasc
+++ b/src/vm_x86.dasc
@@ -3048,8 +3048,10 @@ static void build_subroutines(BuildCtx *ctx)
   |  jmp <2
   |
   |9:  // Rethrow error from the right C frame.
+  |  mov FCARG2, RD
   |  mov FCARG1, L:RB
-  |  call extern lj_err_run@4		// (lua_State *L)
+  |  neg FCARG2
+  |  call extern lj_err_trace@8		// (lua_State *L, int errcode)
   |.endif
   |
   |//-----------------------------------------------------------------------
diff --git a/test/tarantool-tests/gh-7745-oom-on-trace.test.lua b/test/tarantool-tests/gh-7745-oom-on-trace.test.lua
new file mode 100644
index 00000000..1366af30
--- /dev/null
+++ b/test/tarantool-tests/gh-7745-oom-on-trace.test.lua
@@ -0,0 +1,20 @@
+require('utils').skipcond(jit.os == 'OSX', 'Disabled due to incorrect behavior of unwinder in tarantool_panic_handler')
+local ffi = require('ffi')
+
+local tap = require('tap')
+
+local test = tap.test('OOM on trace')
+test:plan(1)
+
+local function memory_payload()
+  local t = {}
+  for i = 1, 1e10 do
+    t[ffi.new("uint64_t")] = i
+  end
+  print(t)
+end
+
+local res = pcall(memory_payload)
+test:ok(res == false)
+
+os.exit(test:check() and 0 or 1)
-- 
2.39.0

[Tarantool-patches] [PATCH luajit 3/5] Disable unreliable assertion for external frame unwinding.

[Maxim Kokryashkin via Tarantool-patches]

From: Mike Pall <mike>

Broken on Fedora/ARM64. Reported by Yichun Zhang.

(cherry-picked from commit e957737650e060d5bf1c2909b741cc3dffe073ac)

This patch disables the assertion that failed because of
incorrectly constructed unwind information.
That debug info generation was fixed in the scope
of tarantool/tarantool#6096. This patch is backported
only for consistency.

Maxim Kokryashkin:
* added the description for the problem

Part of tarantool/tarantool#7745
Relates to tarantool/tarantool#6096
---
 src/lj_err.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/lj_err.c b/src/lj_err.c
index c7fd9e65..f6200233 100644
--- a/src/lj_err.c
+++ b/src/lj_err.c
@@ -488,7 +488,9 @@ void lj_err_verify(void)
   ** lj_assertX(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb), "broken build: external frame unwinding enabled, but system libraries have no unwind tables");
   */
   lua_assert(_Unwind_Find_FDE((void *)lj_err_throw, &ehb));
+  /* Check disabled, because of broken Fedora/ARM64. See #722.
   lua_assert(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb));
+  */
 }
 #endif
 
-- 
2.39.0

[Tarantool-patches] [PATCH luajit 4/5] OSX: Disable unreliable assertion for external frame unwinding.

[Maxim Kokryashkin via Tarantool-patches]

From: Mike Pall <mike>

(cherry-picked from commit be251d9149b386ca0d4b51106be14366c5dbdf14)

`_Unwind_Find_FDE()` will locate the FDE if the pc is in some
function that has an associated FDE. Note, Mac OS X 10.6 and
later, introduces "compact unwind info" which the runtime uses in
preference to DWARF unwind info. This function will only work if
the target function has an FDE but no compact unwind info.

Maxim Kokryashkin:
* added the description for the problem

Part of tarantool/tarantool#7745
---
 src/lj_err.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/lj_err.c b/src/lj_err.c
index f6200233..975b5621 100644
--- a/src/lj_err.c
+++ b/src/lj_err.c
@@ -479,6 +479,8 @@ extern const void *_Unwind_Find_FDE(void *pc, struct dwarf_eh_bases *bases);
 /* Verify that external error handling actually has a chance to work. */
 void lj_err_verify(void)
 {
+#if !LJ_TARGET_OSX
+  /* Check disabled on MacOS due to brilliant software engineering at Apple. */
   struct dwarf_eh_bases ehb;
   /*
   ** FIXME: The following assertions were replaced with
@@ -488,6 +490,7 @@ void lj_err_verify(void)
   ** lj_assertX(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb), "broken build: external frame unwinding enabled, but system libraries have no unwind tables");
   */
   lua_assert(_Unwind_Find_FDE((void *)lj_err_throw, &ehb));
+#endif
   /* Check disabled, because of broken Fedora/ARM64. See #722.
   lua_assert(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb));
   */
-- 
2.39.0

[Tarantool-patches] [PATCH luajit 5/5] Fix IR_RENAME snapshot number. Follow-up fix for a32aeadc.

[Maxim Kokryashkin via Tarantool-patches]

From: Mike Pall <mike>

Reported by Victor Bombi, analyzed by XmiliaH. Thanks!

(cherry-picked from commit bf51d3535109c4745bfbbe19a5587a9eac00259a)

If the `snapalloc` flag is set, then the allocation hasn't
occurred yet, meaning that rename is applied to the next
snapshot. Otherwise, refs are already allocated and rename
is applied to current applied to current snapshot.

Maxim Kokryashkin:
* added the description for the problem

Part of tarantool/tarantool#7745
---
 src/lj_asm.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/lj_asm.c b/src/lj_asm.c
index adfaf286..929a6da6 100644
--- a/src/lj_asm.c
+++ b/src/lj_asm.c
@@ -682,7 +682,14 @@ static void ra_rename(ASMState *as, Reg down, Reg up)
   RA_DBGX((as, "rename    $f $r $r", regcost_ref(as->cost[up]), down, up));
   emit_movrr(as, ir, down, up);  /* Backwards codegen needs inverse move. */
   if (!ra_hasspill(IR(ref)->s)) {  /* Add the rename to the IR. */
-    ra_addrename(as, down, ref, as->snapno);
+    /*
+    ** The rename is effective at the subsequent (already emitted) exit
+    ** branch. This is for the current snapshot (as->snapno). Except if we
+    ** haven't yet allocated any refs for the snapshot (as->snapalloc == 1),
+    ** then it belongs to the next snapshot.
+    ** See also the discussion at asm_snap_checkrename().
+    */
+    ra_addrename(as, down, ref, as->snapno + as->snapalloc);
   }
 }
 
-- 
2.39.0

Re: [Tarantool-patches] [PATCH luajit 4/5] OSX: Disable unreliable assertion for external frame unwinding.

[Sergey Kaplun via Tarantool-patches]

Hi, Maxim!
Thanks for the patch!
LGTM, except a few nits regarding the commit message.

On 15.02.23, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
> 
> (cherry-picked from commit be251d9149b386ca0d4b51106be14366c5dbdf14)
> 
> `_Unwind_Find_FDE()` will locate the FDE if the pc is in some
> function that has an associated FDE. Note, Mac OS X 10.6 and
> later, introduces "compact unwind info" which the runtime uses in
> preference to DWARF unwind info. This function will only work if
> the target function has an FDE but no compact unwind info.

Please, mention that our VM has "compact unwind info" (see
`BUILD_machasm` case), so this is the reason why DWARF unwind info isn't
found by unwinder's `_Unwind_Find_FDE()`.

Does it mean, that all `pcall()` related tests will fail on macOS with
enabled external unwinder (this is why we don't need a test case)?

> 
> Maxim Kokryashkin:
> * added the description for the problem
> 
> Part of tarantool/tarantool#7745

I suppose we should add:
| Part of tarantool/tarantool#8069

> ---
>  src/lj_err.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/src/lj_err.c b/src/lj_err.c
> index f6200233..975b5621 100644
> --- a/src/lj_err.c
> +++ b/src/lj_err.c

<snipped>

> -- 
> 2.39.0
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 3/5] Disable unreliable assertion for external frame unwinding.

[Sergey Kaplun via Tarantool-patches]

Hi, Maxim!
Thanks for the patch!
LGTM, except a few comments below.

On 15.02.23, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
> 
> Broken on Fedora/ARM64. Reported by Yichun Zhang.
> 
> (cherry-picked from commit e957737650e060d5bf1c2909b741cc3dffe073ac)
> 
> This patch disables the assertion that failed because of
> incorrectly constructed unwind information.
> That debug info generation was fixed in the scope
> of tarantool/tarantool#6096. This patch is backported
> only for consistency.
> 

I suggest to menion the commit directly:
b731df1c0392cf7008be2e3c4112e1edb2392de4 ("ARM64: Reorder interpreter
stack frame and fix unwinding.") NB: I mention _our_ commit intense --
so we can follow this commit chain in our repo and cherry-picked notes
link commits to the upstream.

> Maxim Kokryashkin:
> * added the description for the problem
> 
> Part of tarantool/tarantool#7745
> Relates to tarantool/tarantool#6096

I suppose that we should add:
| Part of tarantool/tarantool#8069

> ---
>  src/lj_err.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/src/lj_err.c b/src/lj_err.c
> index c7fd9e65..f6200233 100644
> --- a/src/lj_err.c
> +++ b/src/lj_err.c
> @@ -488,7 +488,9 @@ void lj_err_verify(void)
>    ** lj_assertX(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb), "broken build: external frame unwinding enabled, but system libraries have no unwind tables");
>    */
>    lua_assert(_Unwind_Find_FDE((void *)lj_err_throw, &ehb));
> +  /* Check disabled, because of broken Fedora/ARM64. See #722.

I see that this line is present in LuaJIT's master. Should it be removed
as far as this issue is resolved?

>    lua_assert(_Unwind_Find_FDE((void *)_Unwind_RaiseException, &ehb));
> +  */
>  }
>  #endif
>  
> -- 
> 2.39.0
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 5/5] Fix IR_RENAME snapshot number. Follow-up fix for a32aeadc.

[Sergey Kaplun via Tarantool-patches]

Hi, Maxim!
Thanks for the patch!

Please consider my comments below.

On 15.02.23, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
> 
> Reported by Victor Bombi, analyzed by XmiliaH. Thanks!
> 
> (cherry-picked from commit bf51d3535109c4745bfbbe19a5587a9eac00259a)
> 
> If the `snapalloc` flag is set, then the allocation hasn't
> occurred yet, meaning that rename is applied to the next
> snapshot. Otherwise, refs are already allocated and rename
> is applied to current applied to current snapshot.

Typo: s<applied to current applied to current snapshot>
       <applied to the current snapshot>

> 
> Maxim Kokryashkin:
> * added the description for the problem
> 
> Part of tarantool/tarantool#7745

I suggest to add the following:
| Part of tarantool/tarantool#8069

> ---
>  src/lj_asm.c | 9 ++++++++-

It will be nice to see the testcase, anyway.

>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/src/lj_asm.c b/src/lj_asm.c
> index adfaf286..929a6da6 100644
> --- a/src/lj_asm.c
> +++ b/src/lj_asm.c
> @@ -682,7 +682,14 @@ static void ra_rename(ASMState *as, Reg down, Reg up)
>    RA_DBGX((as, "rename    $f $r $r", regcost_ref(as->cost[up]), down, up));
>    emit_movrr(as, ir, down, up);  /* Backwards codegen needs inverse move. */
>    if (!ra_hasspill(IR(ref)->s)) {  /* Add the rename to the IR. */
> -    ra_addrename(as, down, ref, as->snapno);
> +    /*
> +    ** The rename is effective at the subsequent (already emitted) exit
> +    ** branch. This is for the current snapshot (as->snapno). Except if we
> +    ** haven't yet allocated any refs for the snapshot (as->snapalloc == 1),
> +    ** then it belongs to the next snapshot.
> +    ** See also the discussion at asm_snap_checkrename().
> +    */
> +    ra_addrename(as, down, ref, as->snapno + as->snapalloc);
>    }
>  }
>  
> -- 
> 2.39.0
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 2/5] Handle on-trace OOM errors from helper functions.

[Sergey Kaplun via Tarantool-patches]

Hi, Maxim!
Thanks for the patch!
It's really well described and generally LGTM, but I still have some
more minor questions and comments.

On 15.02.23, Maxim Kokryashkin wrote:
> From: Mike Pall <mike>
> 
> (cherry-picked from commit 4bba29e697d00df5f020e76c2003bb9ce51c5d38)
> 
> This patch introduces handling of errors from internal helper
> functions on traces. FFI C++ exception interoperability is
> not yet implemented.
> 
> For each throwing trace, its mcode entry is augmented with a
> DWARF2 frame description entry. After that, a dynamic DWARF2

Minor: Common information entry and frame description entry.

> frame info is registered based on that entry with
> `__register_frame()`. Because the ARM32 architecture lacks

Is there any documentation links for this function (I see only something
like: "document me!")?
If not, may you please describe it shortly?

> the __register_frame, unwinding is not supported on it.
> 
> For each throwing function call, a snapshot is allocated.
> When we have a parent trace, our side trace head requires
> additional snapshot allocation, so the additional

Typo: s/additional/an additional/

> `asm_snap_prev()` call is added.
> 
> Unwinding on traces is disabled on Darwin, due to quirks of
> Apple's version of libunwind.

I don't see those changes, what are you talking about?

> 
> Maxim Kokryashkin:
> * added the description and the test for the problem
> 
> Part of tarantool/tarantool#7745

I suggest to add here:
| Part of tarantool/tarantool#8069

Also, it will be nice to describe new fields in structures (`asm_State`,
`SnapShot`) introduced by Mike.

Also, please mention what registers are chosen as EHRAREG (you can use a
bullet list with arches, numbers and names of registers).

Also, is good to mention why do we now need snapshots right after each
`pcall()` and `xpcall()` frame.

> ---
>  doc/status.html                               |   7 -
>  src/lj_arch.h                                 |  12 +
>  src/lj_asm.c                                  |  77 ++++-
>  src/lj_dispatch.h                             |   4 +-
>  src/lj_err.c                                  | 274 +++++++++++++++++-
>  src/lj_err.h                                  |  19 +-
>  src/lj_ffrecord.c                             |   2 +
>  src/lj_jit.h                                  |   2 +
>  src/lj_mcode.c                                |   5 +-
>  src/lj_opt_loop.c                             |   1 +
>  src/lj_record.c                               |   3 +-
>  src/lj_snap.c                                 |   1 +
>  src/lj_state.c                                |   1 +
>  src/lj_target_x86.h                           |   2 +
>  src/lj_trace.c                                |  61 +++-
>  src/lj_trace.h                                |   3 +
>  src/lj_vm.h                                   |   3 +
>  src/vm_arm.dasc                               |   3 +-
>  src/vm_arm64.dasc                             |   4 +-
>  src/vm_mips.dasc                              |   9 +-
>  src/vm_mips64.dasc                            |  10 +-
>  src/vm_ppc.dasc                               |   3 +-
>  src/vm_x64.dasc                               |   6 +-
>  src/vm_x86.dasc                               |   4 +-
>  .../gh-7745-oom-on-trace.test.lua             |  20 ++
>  25 files changed, 478 insertions(+), 58 deletions(-)
>  create mode 100644 test/tarantool-tests/gh-7745-oom-on-trace.test.lua
> 
> diff --git a/doc/status.html b/doc/status.html
> index c89255b6..c9cd9071 100644
> --- a/doc/status.html
> +++ b/doc/status.html

<snipped>

> diff --git a/src/lj_arch.h b/src/lj_arch.h
> index 2e458d20..8643aa4a 100644
> --- a/src/lj_arch.h
> +++ b/src/lj_arch.h

<snipped>

> diff --git a/src/lj_asm.c b/src/lj_asm.c
> index a154547b..adfaf286 100644
> --- a/src/lj_asm.c
> +++ b/src/lj_asm.c

<snipped>

> @@ -944,6 +946,14 @@ static void asm_snap_alloc(ASMState *as)
>      if (!irref_isk(ref)) {
>        asm_snap_alloc1(as, ref);
>        if (LJ_SOFTFP && (sn & SNAP_SOFTFPNUM)) {
> +        /*
> +        ** FIXME: The following assert was replaced with

Minor: I'm not so sure about this heavy comment. I suggest just to use
`lua_assert()` as is, it will be replaced back after backporting
"Improve assertions".
Feel free to ignore.

> +        ** the conventional `lua_assert`.
> +        **
> +        ** lj_assertA(irt_type(IR(ref+1)->t) == IRT_SOFTFP,
> +		    ** "snap %d[%d] points to bad SOFTFP IR %04d",
> +		    ** snapno, n, ref - REF_BIAS);
> +        */
>  	lua_assert(irt_type(IR(ref+1)->t) == IRT_SOFTFP);
>  	asm_snap_alloc1(as, ref+1);
>        }
> @@ -970,19 +980,16 @@ static int asm_snap_checkrename(ASMState *as, IRRef ren)

<snipped>

> diff --git a/src/lj_dispatch.h b/src/lj_dispatch.h
> index addf5572..b8bc2594 100644
> --- a/src/lj_dispatch.h
> +++ b/src/lj_dispatch.h

<snipped>

> diff --git a/src/lj_err.c b/src/lj_err.c
> index d0223384..c7fd9e65 100644
> --- a/src/lj_err.c
> +++ b/src/lj_err.c

<snipped>

> @@ -304,12 +309,59 @@ LJ_FUNCA int lj_err_unwind_win(EXCEPTION_RECORD *rec,

<snipped>

> +
> +#if LJ_UNWIND_JIT
> +/* DWARF2 personality handler for JIT-compiled code. */
> +static int err_unwind_jit(int version, int actions,
> +  uint64_t uexclass, _Unwind_Exception *uex, _Unwind_Context *ctx)
> +{
> +  /* NYI: FFI C++ exception interoperability. */
> +  if (version != 1 || !LJ_UEXCLASS_CHECK(uexclass))
> +    return _URC_FATAL_PHASE1_ERROR;
> +  if ((actions & _UA_SEARCH_PHASE)) {
> +    return _URC_HANDLER_FOUND;
> +  }
> +  if ((actions & _UA_CLEANUP_PHASE)) {
> +    global_State *g = *(global_State **)(uex+1);

Side note: Just get the next field of the structure, nothing to see
here (cust to the whole structure is too easy way...).

> +    ExitNo exitno;
> +    uintptr_t addr = _Unwind_GetIP(ctx);  /* Return address _after_ call. */
> +    uintptr_t stub = lj_trace_unwind(G2J(g), addr - sizeof(MCode), &exitno);
> +    /*
> +    ** FIXME: The following assert was replaced with
> +    ** the conventional `lua_assert`.
> +    **
> +    ** lj_assertG(tvref(g->jit_base), "unexpected throw across mcode frame");
> +    */
> +    lua_assert(tvref(g->jit_base));
> +    if (stub) {  /* Jump to side exit to unwind the trace. */
> +      G2J(g)->exitcode = LJ_UEXCLASS_ERRCODE(uexclass);
> +#ifdef LJ_TARGET_MIPS
> +      _Unwind_SetGR(ctx, 4, stub);
> +      _Unwind_SetGR(ctx, 5, exitno);
> +      _Unwind_SetIP(ctx, (uintptr_t)(void *)lj_vm_unwind_stub);
> +#else
> +      _Unwind_SetIP(ctx, stub);
> +#endif
> +      return _URC_INSTALL_CONTEXT;
> +    }
> +    return _URC_FATAL_PHASE2_ERROR;
> +  }
> +  return _URC_FATAL_PHASE1_ERROR;
> +}
> +
> +/* DWARF2 template frame info for JIT-compiled code.
> +**
> +** After copying the template to the start of the mcode segment,
> +** the frame handler function and the code size is patched.
> +** The frame handler always installs a new context to jump to the exit,
> +** so don't bother to add any unwind opcodes.
> +*/
> +static const uint8_t err_frame_jit_template[] = {

Side note: Still don't want to use defines (or structures:)) for all
those constatns and strings...

> +#if LJ_BE
> +  0,0,0,
> +#endif
> +  LJ_64 ? 0x1c : 0x14,  /* CIE length. */
> +#if LJ_LE
> +  0,0,0,
> +#endif
> +  0,0,0,0, 1, 'z','P','R',0,  /* CIE mark, CIE version, augmentation. */
> +  1, LJ_64 ? 0x78 : 0x7c, LJ_TARGET_EHRAREG,  /* Code/data align, RA. */

Side note:
Code alignment factor is 1 (uleb128).
Data alignment factor is LJ_64 ? 8 : 4 (sleb128).

> +#if LJ_64
> +  10, 0, 0,0,0,0,0,0,0,0, 0x1b,  /* Aug. data ABS handler, PCREL|SDATA4 code. */

Don't get this. Why do we use PCREL|SDATA4 code for 8-byte pointers
(instead PCREL|SDATA8) (I mean we already reserve room for it)?

What is "ABS handler"? I don't see any docs about it.

> +  0,0,0,0,0,  /* Alignment. */
> +#else
> +  6, 0, 0,0,0,0, 0x1b,  /* Aug. data ABS handler, PCREL|SDATA4 code. */
> +  0,  /* Alignment. */
> +#endif
> +#if LJ_BE
> +  0,0,0,
> +#endif
> +  LJ_64 ? 0x14 : 0x10,  /* FDE length. */

Side note: Is differ due to "Alignment".

> +  0,0,0,
> +  LJ_64 ? 0x24 : 0x1c,  /* CIE offset. */
> +  0,0,0,
> +  LJ_64 ? 0x14 : 0x10,  /* Code offset. After Final FDE. */

Side note: Equals to FDE length, because this entry is right before the
corresponding MCode.

> +#if LJ_LE
> +  0,0,0,
> +#endif
> +  0,0,0,0, 0, 0,0,0, /* Code size, augmentation length, alignment. */
> +#if LJ_64
> +  0,0,0,0,  /* Alignment. */
> +#endif
> +  0,0,0,0  /* Final FDE. */
> +};
> +

<snipped>

> +#define ERR_FRAME_JIT_OFS_HANDLER       0x12
> +#define ERR_FRAME_JIT_OFS_FDE           (LJ_64 ? 0x20 : 0x18)
> +#define ERR_FRAME_JIT_OFS_CODE_SIZE     (LJ_64 ? 0x2c : 0x24)
> +#if LJ_TARGET_OSX
> +#define ERR_FRAME_JIT_OFS_REGISTER      ERR_FRAME_JIT_OFS_FDE

Why this field is different for macOS and Linux (macOS uses FDE start, and
Linux CIE start as an argument)?
Please describe it in the commit message.
(May be the link to `__register_frame()` documentation is enough).

> +#else
> +#define ERR_FRAME_JIT_OFS_REGISTER      0
> +#endif
> +
> +extern void __register_frame(const void *);
> +extern void __deregister_frame(const void *);
> +
> +uint8_t *lj_err_register_mcode(void *base, size_t sz, uint8_t *info)
> +{
> +  void **handler;
> +  memcpy(info, err_frame_jit_template, sizeof(err_frame_jit_template));
> +  handler = (void *)err_unwind_jit;
> +  memcpy(info + ERR_FRAME_JIT_OFS_HANDLER, &handler, sizeof(handler));
> +  *(uint32_t *)(info + ERR_FRAME_JIT_OFS_CODE_SIZE) =
> +    (uint32_t)(sz - sizeof(err_frame_jit_template) - (info - (uint8_t *)base));
> +  __register_frame(info + ERR_FRAME_JIT_OFS_REGISTER);
> +#ifdef LUA_USE_ASSERT
> +  {
> +    struct dwarf_eh_bases ehb;
> +    lj_assertX(_Unwind_Find_FDE(info + sizeof(err_frame_jit_template)+1, &ehb),
> +               "bad JIT unwind table registration");
> +  }
> +#endif
> +  return info + sizeof(err_frame_jit_template);
> +}
> +
> +void lj_err_deregister_mcode(void *base, size_t sz, uint8_t *info)
> +{
> +  UNUSED(base); UNUSED(sz);
> +  __deregister_frame(info + ERR_FRAME_JIT_OFS_REGISTER);

Should it be the opposite assert here?

> +}

<snipped>

> @@ -492,25 +685,68 @@ LJ_FUNCA int lj_err_unwind_arm(int state, _Unwind_Control_Block *ucb,
>    }
>    if (__gnu_unwind_frame(ucb, ctx) != _URC_OK)
>      return _URC_FAILURE;
> +#ifdef LUA_USE_ASSERT
> +  /* We should never get here unless this is a forced unwind aka backtrace. */
> +  if (_Unwind_GetGR(ctx, 0) == 0xff33aa77) {
> +    _Unwind_SetGR(ctx, 0, 0xff33aa88);
> +  }
> +#endif
>    return _URC_CONTINUE_UNWIND;
>  }
>  
> -#if LJ_UNWIND_EXT
> -static __thread _Unwind_Control_Block static_uex;
> +#if LJ_UNWIND_EXT && defined(LUA_USE_ASSERT)
> +typedef int (*_Unwind_Trace_Fn)(_Unwind_Context *, void *);
> +extern int _Unwind_Backtrace(_Unwind_Trace_Fn, void *);
> +
> +static int err_verify_bt(_Unwind_Context *ctx, int *got)
> +{
> +  if (_Unwind_GetGR(ctx, 0) == 0xff33aa88) { *got = 2; }
> +  else if (*got == 0) { *got = 1; _Unwind_SetGR(ctx, 0, 0xff33aa77); }

Side note: OMG we totally, don't want to use defines with meaningfull
names here, just use random constants:)...

> +  return _URC_OK;
> +}
> +
> +/* Verify that external error handling actually has a chance to work. */
> +void lj_err_verify(void)
> +{
> +  int got = 0;
> +  _Unwind_Backtrace((_Unwind_Trace_Fn)err_verify_bt, &got);
> +  /*
> +  ** FIXME: The following assert was replaced with
> +  ** the conventional `lua_assert`.
> +  **
> +  ** lj_assertX(got == 2, "broken build: external frame unwinding enabled, but missing -funwind-tables");
> +  */
> +  lua_assert(got == 2);
> +}
>  #endif

<snipped>

> @@ -620,7 +856,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
>  /* Runtime error. */
>  LJ_NOINLINE void LJ_FASTCALL lj_err_run(lua_State *L)
>  {
> -  ptrdiff_t ef = finderrfunc(L);
> +  ptrdiff_t ef = (LJ_HASJIT && tvref(G(L)->jit_base)) ? 0 : finderrfunc(L);

If we don't want to find error function when we on trace why do we take
`if (errcode == LUA_ERRRUN)` below?
Does it mean that we will still try to rethrow error raising inside
`lj_trace_exit()` (AFAICS this is the only place when `jit_base == 0`
and we call `lj_err_trace()`)?
If so, please mention this in the commit message.

Also, please mention that `lj_err_trace()` is introduced to use instead
`lj_err_run()` in rethrowing of the error.

>    if (ef) {
>      TValue *errfunc = restorestack(L, ef);
>      TValue *top = L->top;
> @@ -639,6 +875,16 @@ LJ_NOINLINE void LJ_FASTCALL lj_err_run(lua_State *L)
>    lj_err_throw(L, LUA_ERRRUN);
>  }
>  
> +#if LJ_HASJIT
> +LJ_NOINLINE void LJ_FASTCALL lj_err_trace(lua_State *L, int errcode)
> +{
> +  if (errcode == LUA_ERRRUN)
> +    lj_err_run(L);
> +  else
> +    lj_err_throw(L, errcode);
> +}
> +#endif
> +
>  /* Formatted runtime error message. */
>  LJ_NORET LJ_NOINLINE static void err_msgv(lua_State *L, ErrMsg em, ...)
>  {
> diff --git a/src/lj_err.h b/src/lj_err.h
> index aa4b7e0d..b0c72c24 100644
> --- a/src/lj_err.h
> +++ b/src/lj_err.h

<snipped>

> diff --git a/src/lj_ffrecord.c b/src/lj_ffrecord.c
> index 649ac705..8af9da1d 100644
> --- a/src/lj_ffrecord.c
> +++ b/src/lj_ffrecord.c

<snipped>

> diff --git a/src/lj_jit.h b/src/lj_jit.h
> index d82292f8..f2ad3c6e 100644
> --- a/src/lj_jit.h
> +++ b/src/lj_jit.h

<snipped>

> diff --git a/src/lj_mcode.c b/src/lj_mcode.c
> index 77035bf7..7184d3b4 100644
> --- a/src/lj_mcode.c
> +++ b/src/lj_mcode.c

<snipped>

> diff --git a/src/lj_opt_loop.c b/src/lj_opt_loop.c
> index 441b8add..10613641 100644
> --- a/src/lj_opt_loop.c
> +++ b/src/lj_opt_loop.c

<snipped>

> diff --git a/src/lj_record.c b/src/lj_record.c
> index 9e2e1d9e..e7dac7ac 100644
> --- a/src/lj_record.c
> +++ b/src/lj_record.c
> @@ -800,7 +800,7 @@ void lj_record_ret(jit_State *J, BCReg rbase, ptrdiff_t gotresults)
>      J->base -= cbase;
>      J->base[--rbase] = TREF_TRUE;  /* Prepend true to results. */
>      frame = frame_prevd(frame);
> +    J->needsnap = 1;  /* Stop catching on-trace errors. */
>    }
>    /* Return to lower frame via interpreter for unhandled cases. */
>    if (J->framedepth == 0 && J->pt && bc_isret(bc_op(*J->pc)) &&
> @@ -2021,7 +2022,7 @@ void lj_record_ins(jit_State *J)
>    /* Need snapshot before recording next bytecode (e.g. after a store). */
>    if (J->needsnap) {
>      J->needsnap = 0;
> -    lj_snap_purge(J);
> +    if (J->pt) lj_snap_purge(J);

In which cases there is no prototype?

>      lj_snap_add(J);
>      J->mergesnap = 1;
>    }
> diff --git a/src/lj_snap.c b/src/lj_snap.c
> index 2f7cf80a..a8b49fcb 100644
> --- a/src/lj_snap.c
> +++ b/src/lj_snap.c

<snipped>

> diff --git a/src/lj_state.c b/src/lj_state.c
> index cc6f92f1..4add3d65 100644
> --- a/src/lj_state.c
> +++ b/src/lj_state.c

<snipped>

> diff --git a/src/lj_target_x86.h b/src/lj_target_x86.h
> index 194f8e70..4efb566b 100644
> --- a/src/lj_target_x86.h
> +++ b/src/lj_target_x86.h

<snipped>

> diff --git a/src/lj_trace.c b/src/lj_trace.c
> index c6e2f72e..17743159 100644
> --- a/src/lj_trace.c
> +++ b/src/lj_trace.c

<snipped>

> @@ -932,4 +944,47 @@ int LJ_FASTCALL lj_trace_exit(jit_State *J, void *exptr)
>    }
>  }
>  
> +#if LJ_UNWIND_JIT
> +/* Given an mcode address determine trace exit address for unwinding. */
> +uintptr_t LJ_FASTCALL lj_trace_unwind(jit_State *J, uintptr_t addr, ExitNo *ep)
> +{
> +#if EXITTRACE_VMSTATE
> +  TraceNo traceno = J2G(J)->vmstate;
> +#else
> +  TraceNo traceno = trace_exit_find(J, (MCode *)addr);
> +#endif

Side note: Looks like we should use the same approach for our symtab
dumping in profilers.

> +  GCtrace *T = traceref(J, traceno);
> +  if (T
> +#if EXITTRACE_VMSTATE
> +      && addr >= (uintptr_t)T->mcode && addr < (uintptr_t)T->mcode + T->szmcode
> +#endif
> +     ) {

<snipped>

> +}
> +#endif
> +
>  #endif
> diff --git a/src/lj_trace.h b/src/lj_trace.h
> index 22cae741..0bfb606f 100644
> --- a/src/lj_trace.h
> +++ b/src/lj_trace.h

<snipped>

> diff --git a/src/lj_vm.h b/src/lj_vm.h
> index 1cc7eed7..411caafa 100644
> --- a/src/lj_vm.h
> +++ b/src/lj_vm.h

<snipped>

> diff --git a/src/vm_arm.dasc b/src/vm_arm.dasc
> index a29292f1..436a9fd7 100644
> --- a/src/vm_arm.dasc
> +++ b/src/vm_arm.dasc

<snipped>

> diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc
> index f517a808..7066b051 100644
> --- a/src/vm_arm64.dasc
> +++ b/src/vm_arm64.dasc

<snipped>

> diff --git a/src/vm_mips.dasc b/src/vm_mips.dasc
> index 93c772ff..32caabf7 100644
> --- a/src/vm_mips.dasc
> +++ b/src/vm_mips.dasc

<snipped>

> diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc
> index 9a749f93..04be38f0 100644
> --- a/src/vm_mips64.dasc
> +++ b/src/vm_mips64.dasc
> @@ -545,6 +545,10 @@ static void build_subroutines(BuildCtx *ctx)

<snipped>

> diff --git a/src/vm_ppc.dasc b/src/vm_ppc.dasc
> index 980176a2..7ad8df37 100644
> --- a/src/vm_ppc.dasc
> +++ b/src/vm_ppc.dasc

<snipped>

> diff --git a/src/vm_x64.dasc b/src/vm_x64.dasc
> index 59f117ba..27af164a 100644
> --- a/src/vm_x64.dasc
> +++ b/src/vm_x64.dasc

<snipped>

> diff --git a/src/vm_x86.dasc b/src/vm_x86.dasc
> index f7ffe5d2..85807129 100644
> --- a/src/vm_x86.dasc
> +++ b/src/vm_x86.dasc

<snipped>

> diff --git a/test/tarantool-tests/gh-7745-oom-on-trace.test.lua b/test/tarantool-tests/gh-7745-oom-on-trace.test.lua
> new file mode 100644
> index 00000000..1366af30
> --- /dev/null
> +++ b/test/tarantool-tests/gh-7745-oom-on-trace.test.lua
> @@ -0,0 +1,20 @@
> +require('utils').skipcond(jit.os == 'OSX', 'Disabled due to incorrect behavior of unwinder in tarantool_panic_handler')

Line length is more than 80 symbols.

> +local ffi = require('ffi')
> +
> +local tap = require('tap')
> +
> +local test = tap.test('OOM on trace')
> +test:plan(1)
> +
> +local function memory_payload()
> +  local t = {}
> +  for i = 1, 1e10 do
> +    t[ffi.new("uint64_t")] = i

Typo: s/"uint64_t"/'uint64_t'/

This case returns OOM, error (and may return the TABOV error).
Can we check exactly TABOV error too(may be with the help of `table.new()`?)?

> +  end
> +  print(t)
> +end
> +
> +local res = pcall(memory_payload)
> +test:ok(res == false)
> +
> +os.exit(test:check() and 0 or 1)
> -- 
> 2.39.0
> 

-- 
Best regards,
Sergey Kaplun