From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9D21A6ECCD; Mon, 20 Jul 2026 11:53:13 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9D21A6ECCD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1784537593; bh=GFOpAIMxZWngno4/KtCF83d5DyBj/+eHgb7Zmox4agA=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=f2Hroknd19ZyJlKW6bpuzjBzCQnl6G1yq1wrh/vKWa+GHwNUJRjXfjTa7kYshp5pC GqV8P0qgc9Sr5LaM0QY1/MFqO0QID2rJdGTm1QephvNj7nnE9cVYPeFsq2phyudXDm VEJi7MBUE89PJm+cIby7564KGO7abXGz5i53ckp4= Received: from send172.i.mail.ru (send172.i.mail.ru [95.163.59.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id A187B6ECCD for ; Mon, 20 Jul 2026 11:53:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A187B6ECCD Received: by exim-smtp-77575cb4cb-kh8k7 with esmtpa (envelope-from ) id 1wljkM-000000004kC-2eNS; Mon, 20 Jul 2026 11:53:11 +0300 To: Sergey Bronnikov , Evgeniy Temirgaleev Date: Mon, 20 Jul 2026 11:52:54 +0300 Message-ID: <20260720085254.2452841-1-skaplun@tarantool.org> X-Mailer: git-send-email 2.55.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9EA1C1F5BBAB1DAA0015BD83715E84368EF2F423F54D9BB15182A05F538085040B494F6DE96AC20B33DE06ABAFEAF67052146A0937FF9854B8DC4C741CE6B1926A316B52795F7D3CA X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE70C5E0F71D77D667BEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB5533756661848894E0D3EACE5164530273012661FE79A3885742FFE816DCF83D72C2D751389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0DEC8C2C8BCD2534D8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6E232F00D8D26902CA471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FC9DF7DD15B8400A713AA81AA40904B5D9CF19DD082D7633A0C84D3B47A649675F3AA81AA40904B5D98AA50765F7900637CF9CBD2CBC4A4D55D81D268191BDAD3D3666184CF4C3C14F3FC91FA280E0CE3D1A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89F616AD31D0D18CD5C5E1C53F199C2BB95B5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A5FC897C3187014F855002B1117B3ED69630CE842C668BB0E81E49B01306B5E3AD823CB91A9FED034534781492E4B8EEADA757276DBF662F3EC79554A2A72441328621D336A7BC284946AD531847A6065A535571D14F44ED41 X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE1918E10F71CB4DF9F9677DD89D51EBB774225B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D6596DF0F0E22CF139D85FFBA84CFC471074CE344859B3F4F994A5FC0AE32704E8CC1EF631181B0D2B03B8341EE9D5BE9A0A248D5DF6CDE5D80E1E556436CBDC5D9BB7934370933876CDC7CEAA0681F5848F4C41F94D744909CECFA6C6B0C050A61A8CAF69B82BA93681CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVWmxowtcrDwUEevbqzVW8Tk= X-Mailru-Sender: 520A125C2F17F0B17094CDC02B85F11BBBCA26E9D63B36FE3DE06ABAFEAF67052146A0937FF9854BB7CBEF92542CD7C88B0A2698F12F5C9EC77752E0C033A69E86920BD37369036789A8C6A0E60D2BB63A5DB60FBEB33A8A0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit] Fix stack overflow relimit handling. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mike Pall Thanks to Sergey Kaplun. (cherry picked from commit d2c1327f57dc96f6ed8b11474f3bc5e09a9d227a) If the error handler can't fit to the top of the Lua stack, the stack is resized up to `LJ_STACK_MAXEX + 1 + 2 * LUA_MINSTACK` hoping that `lj_state_relimitstack()` shrinks it back. In case when another error is (gently) handled in the error handler (`pcall(error)` or failed `load()`) the stack is shrunk to `LJ_STACK_MAX` leaving the rest of the handler to be executed on the clipped stack, which leads to the heap-overflow or a crash. This patch fixes the issue by hardening the relimiting border to match the extra space allocated before the stack overflow. Also, it fixes some typos in the comments. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12880 --- The test is leading to the core dump on GC64 build. Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1471-fix-stackov-relimit Related issues: * https://github.com/LuaJIT/LuaJIT/issues/1471 * https://github.com/tarantool/tarantool/issues/12880 src/lj_state.c | 17 ++++--- .../lj-1471-fix-stackov-relimit.test.lua | 48 +++++++++++++++++++ 2 files changed, 59 insertions(+), 6 deletions(-) create mode 100644 test/tarantool-tests/lj-1471-fix-stackov-relimit.test.lua diff --git a/src/lj_state.c b/src/lj_state.c index 2fa6a2f6..ea0f24f8 100644 --- a/src/lj_state.c +++ b/src/lj_state.c @@ -44,8 +44,9 @@ #define LJ_STACK_MAX LUAI_MAXSTACK /* Max. stack size. */ #define LJ_STACK_START (2*LJ_STACK_MIN) /* Starting stack size. */ #define LJ_STACK_MAXEX (LJ_STACK_MAX + 1 + LJ_STACK_EXTRA) +#define LJ_STACK_ERREX (1 + 2*LJ_STACK_MIN) /* Extra for error handling. */ -/* Explanation of LJ_STACK_EXTRA: +/* Explanation for LJ_STACK_EXTRA: ** ** Calls to metamethods store their arguments beyond the current top ** without checking for the stack limit. This avoids stack resizes which @@ -58,6 +59,11 @@ ** slots above top, but then mobj is always a function. So we can get by ** with 5 extra slots. ** LJ_FR2: We need 2 more slots for the frame PC and the continuation PC. +** +** Explanation for LJ_STACK_ERREX: +** +** The 1 is space for the error message, and 2 * LJ_STACK_MIN is for +** the lj_state_checkstack() call in lj_err_run(). */ /* Resize stack slots and adjust pointers in state. */ @@ -101,7 +107,8 @@ static void resizestack(lua_State *L, MSize n) /* Relimit stack after error, in case the limit was overdrawn. */ void lj_state_relimitstack(lua_State *L) { - if (L->stacksize > LJ_STACK_MAXEX && L->top-tvref(L->stack) < LJ_STACK_MAX-1) + if (L->stacksize > LJ_STACK_MAXEX && + L->top-tvref(L->stack) < LJ_STACK_MAX - 1 - LJ_STACK_ERREX) resizestack(L, LJ_STACK_MAX); } @@ -147,11 +154,9 @@ void LJ_FASTCALL lj_state_growstack(lua_State *L, MSize need) /* An error handler might want to inspect the stack overflow error, but ** will need some stack space to run in. We give it a stack size beyond ** the normal limit in order to do so, then rely on lj_state_relimitstack - ** calls during unwinding to bring us back to a convential stack size. - ** The + 1 is space for the error message, and 2 * LUA_MINSTACK is for - ** the lj_state_checkstack() call in lj_err_run(). + ** calls during unwinding to bring us back to a conventional stack size. */ - resizestack(L, LJ_STACK_MAX + 1 + 2 * LUA_MINSTACK); + resizestack(L, LJ_STACK_MAX + LJ_STACK_ERREX); lj_err_stkov(L); /* May invoke an error handler. */ } else { /* If we're here, then the stack overflow error handler is requesting diff --git a/test/tarantool-tests/lj-1471-fix-stackov-relimit.test.lua b/test/tarantool-tests/lj-1471-fix-stackov-relimit.test.lua new file mode 100644 index 00000000..6a10cc20 --- /dev/null +++ b/test/tarantool-tests/lj-1471-fix-stackov-relimit.test.lua @@ -0,0 +1,48 @@ +local tap = require('tap') + +-- The test file demonstrates a heap-overflow due to Lua stack +-- out-of-bounds access after an incorrect stack shrinking after +-- unwinding. +-- See also https://github.com/LuaJIT/LuaJIT/issues/1471. + +local test = tap.test('lj-1471-fix-stackov-relimit') + +test:plan(1) + +local function recursive_add(v1, v2) + -- Slot to eat the stack. + -- luacheck: no unused + local _ + recursive_add(v1, v2) +end + +local table_mt = { + __add = recursive_add, +} + +local t1 = setmetatable({}, table_mt) +local t2 = setmetatable({}, table_mt) + +coroutine.wrap(function() + xpcall(error, function() + pcall(error) -- Shrink stack back after unwinding. + -- XXX: Empirical amount of stack slots to observe the issue. + -- After the stack overflow, the `xpcall()` handler is invoked + -- again (see https://github.com/LuaJIT/LuaJIT/issues/1382). + -- The stack is overallocated beyond its normal limit to + -- handle the error. After the `pcall(error)`, stack is + -- shrinking back to its normal size, leaving no space for + -- metamethod invocation. It is leaving to heap-overflow and a + -- crash. + -- luacheck: no unused + local _, _, _, _, _, _, _, _, _, _ + local _, _, _, _, _, _, _, _, _, _ + local _, _, _, _, _, _, _, _, _, _ + local _ + local _ = t1 + t2 + end) +end)() + +test:ok(true, 'no heap overflow after stack relimiting') + +test:done(true) -- 2.55.0