From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 57D741721A0D; Wed, 11 Feb 2026 11:30:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 57D741721A0D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1770798611; bh=i8p0pjZ+ne0PWKj2Jt7B7hdx6hJRgpnlw6kE/H6MbTE=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=EwLIpu8xtxniApYxOjzEtyzyZl8X2xG0ywAvEaRnaEMaU3CWVXRsxqgGH6PAR3OQw uGKQ3Kts4XFV2rOuLJr4UcSZr4q6xi6jc2pmbpFcsCYMLhZ+GEw3uBgkOE0VGbCE3I 1r82P+jtH9S+cQkEvseoTck0juaEmjj17U8cfzXY= Received: from send220.i.mail.ru (send220.i.mail.ru [95.163.59.59]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 1B3991721A0D for ; Wed, 11 Feb 2026 11:30:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1B3991721A0D Received: by exim-smtp-5d446c7b8f-rblcx with esmtpa (envelope-from ) id 1vq5bs-000000005NQ-3pUX; Wed, 11 Feb 2026 11:30:09 +0300 Date: Wed, 11 Feb 2026 11:30:11 +0300 To: Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org Message-ID: References: <51e75e7052824de65036abd2f5807a1224f438aa.1765350224.git.sergeyb@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51e75e7052824de65036abd2f5807a1224f438aa.1765350224.git.sergeyb@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9A6EFB945BF0DEAB281A686AE7F8143CBF73497657E1D6C21182A05F538085040BEDB1AE077E3E4AF3DE06ABAFEAF670515257392B11B0CD1D1D9CE7452FC158C5118E5AA908773F9 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE73B2A9F8A35432468EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB553375664C71A6A4846DE5218F92CE2F9064EC489C34BA40C0478D138108E040DDF11F89389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C07734D68A6916D8318941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6A50BD5087FBFCDAACC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C22496126586F7E381E2E76E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8BA7E986E6EB348C303AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735473DB2713789FF2EC4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5A504B2A011C4A1735002B1117B3ED69652C48BF5B8DBC5EA886DC9BC01168B20823CB91A9FED034534781492E4B8EEADA3A806F356AF31D6 X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34CAB70F9BE574AE9C625B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D659EC1907A02700AF94DBE3B059F96E2F8230070E555E6243020179ECCD14A44465D753BF5C1990E03BB8341EE9D5BE9A0AE5D1A153BE83DC020DBC12CD1361A8AEBCE4F236951ADF338CD93680B12512CF4C41F94D744909CECFA6C6B0C050A61A8CAF69B82BA93681CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVemV+tRo3/+h/ICeDc/Wb8s= X-Mailru-Sender: 520A125C2F17F0B1A9638AD358559B59B7BEB2F915E81E4F3DE06ABAFEAF670515257392B11B0CD1B7CBEF92542CD7C88B0A2698F12F5C9EC77752E0C033A69E86920BD37369036789A8C6A0E60D2BB63A5DB60FBEB33A8A0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 2/3][v2] LJ_FR2: Fix stack checks in vararg calls. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" On 10.12.25, Sergey Bronnikov wrote: > From: Mike Pall > > Thanks to Peter Cawley. > > (cherry picked from commit d1a2fef8a8f53b0055ee041f7f63d83a27444ffa) > > Stack overflow can cause a segmentation fault in vararg Typo: s/vararg/a vararg/ > function on ARM64 and MIPS64 in LJ_FR2 mode. This happen Typo: s/happen/happens/ > because stack check in BC_IFUNCV is off by one on these Typo: s/stack/the stack/ > platforms without the patch. The original stack check > for ARM64 and MIPS64 was incorrect: > > | RA == BASE + (RD=NARGS)*8 + framesize * 8 >= maxstack > > while stack check on x86_64 is correct and therefore is Typo: s/stack/the stack/ > not affected by the problem: > > | RA == BASE + (RD=NARGS+1)*8 + framesize * 8 +8 > maxstack > > The patch partially fixes aforementioned issue by Typo: s/aforementioned/the aforementioned/ > bumping LJ_STACK_EXTRA by 1 to give a space to write > the entire frame link and fixing a number of last Typo: s/a number/the number/ I'm not get this part. I suggest rephrasing it like the following: | The patch partially fixes the aforementioned issue by bumping | LJ_STACK_EXTRA by 1 to give a space to the entire frame link for a | vararg function as the __newindex metamethod. > free slot in the stack (LJ_FR2 summand adjustment). > > A fixup for a number of required slots in `call_init()` was added > for consistency with non-gc64 flavor. Typo: s/gc64/GC64/ I would also add: "The check is too strict, so this can't lead to any crash." > > Sergey Bronnikov: > * added the description and the test for the problem > > Part of tarantool/tarantool#12134 > --- > src/lj_def.h | 2 +- > src/lj_dispatch.c | 2 +- > src/vm_arm64.dasc | 1 + > src/vm_mips64.dasc | 1 + > ...048-fix-stack-checks-vararg-calls.test.lua | 53 +++++++++++++++++++ > 5 files changed, 57 insertions(+), 2 deletions(-) > create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > > diff --git a/src/lj_def.h b/src/lj_def.h > index a5bca6b0..7e4f251e 100644 > --- a/src/lj_def.h > +++ b/src/lj_def.h > @@ -69,7 +69,7 @@ typedef unsigned int uintptr_t; > #define LJ_MAX_UPVAL 60 /* Max. # of upvalues. */ > > #define LJ_MAX_IDXCHAIN 100 /* __index/__newindex chain limit. */ > -#define LJ_STACK_EXTRA (5+2*LJ_FR2) /* Extra stack space (metamethods). */ > +#define LJ_STACK_EXTRA (5+3*LJ_FR2) /* Extra stack space (metamethods). */ > > #define LJ_NUM_CBPAGE 1 /* Number of FFI callback pages. */ > > diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c > index a44a5adf..431cb3c2 100644 > --- a/src/lj_dispatch.c > +++ b/src/lj_dispatch.c > @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn) > int numparams = pt->numparams; > int gotparams = (int)(L->top - L->base); > int need = pt->framesize; > - if ((pt->flags & PROTO_VARARG)) need += 1+gotparams; > + if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams; > lj_state_checkstack(L, (MSize)need); > numparams -= gotparams; > return numparams >= 0 ? numparams : 0; Let's add an additional test for this part of code (since we don't have any). It may be taken from [1]. It doesn't fail now, but we may cover this branch more precise. > diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc > index c5f0a7a7..cf8e575a 100644 > --- a/src/vm_arm64.dasc > +++ b/src/vm_arm64.dasc > diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc > index da187a7a..6c2975b4 100644 > --- a/src/vm_mips64.dasc > +++ b/src/vm_mips64.dasc > diff --git a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > new file mode 100644 > index 00000000..d471d41e > --- /dev/null > +++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > @@ -0,0 +1,53 @@ > +local tap = require('tap') > + > +-- A test file to demonstrate a stack overflow in `pcall()` in I would rephrase it like "demonstrate a crash due to Lua stack out-of-bounds access". > +-- some cases, see below testcase descriptions. > +-- See also https://github.com/LuaJIT/LuaJIT/issues/1048. > +local test = tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({ > + ['Test requires JIT enabled'] = not jit.status(), > +}) I suppose there is no need in JIT here. This skipcond may be removed. > + > +test:plan(2) > + > +-- The testcase demonstrate a segmentation fault due to stack Typo: s/testcase demonstrate/test case demonstrates/ > +-- overflow by recursive calling `pcall()`. The functions are > +-- vararg because stack check in BC_IFUNCV is off by one on ARM64 Typo: s/stack/the stack/ > +-- and MIPS64 without the patch. > +local function prober_1(...) -- luacheck: no unused > + -- Any fast function can be used as metamethod, but `type` is > + -- convenient here because it works fast and can be used with > + -- any data type. Lua function cannot be used since it > + -- will check the stack on each invocation. Please add the comment, that we need to check using of the correct value LJ_STACK_EXTRA slots (5+3*LJ_FR2) = 8 for GC64 mode. > + pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, type, 0) > +end > + > +local function looper(prober, n, ...) > + prober(...) > + return looper(prober, n + 1, n, ...) > +end > + > +pcall(coroutine.wrap(looper), prober_1, 0) > + > +test:ok(true, 'no stack overflow with recursive pcall') > + > +-- The testcase demonstrate a segmentation fault due to stack Typo: s/testcase demonstrate/test case demonstrates/ > +-- overflow when `pcall()` is used as `__newindex` metamethod. > +-- The function is vararg because stack check in BC_IFUNCV is off Typo: s/stack/the stack/ > +-- by one on ARM64 and MIPS64 without the patch. > + > +-- Any fast function can be used as metamethod, but `type` is Typo: s/metamethod/a metamethod/ > +-- convenient here because it works fast and can be used with > +-- any data type. Lua function cannot be used since it Typo: s/Lua/The Lua/ > +-- will check the stack on each invocation. > +local t = setmetatable({}, { __newindex = pcall, __call = type }) > + > +local function prober_2(...) -- luacheck: no unused > + -- Invokes `pcall(t, t, t)`. > + t[t] = t > +end > + > +pcall(coroutine.wrap(looper), prober_2, 0) > + > +test:ok(true, 'no stack overflow with metamethod') > + > +test:done(true) > -- > 2.43.0 > [1]: https://github.com/LuaJIT/LuaJIT/issues/1402#issue-3569942423 -- Best regards, Sergey Kaplun