From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 84F2214CEE2F; Mon, 1 Sep 2025 16:07:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 84F2214CEE2F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1756732032; bh=CxmwgApW4Mm4F8bED+ubMaDnZgWbCRq1lLsrQfv+W4g=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=JuBfa201qqmK+2tPTyIsL76fzskElGgkJN2F8npF6bvFi7Isuxf+nDCHLcsWD55Yt 1cKl/CacAhTX2ooyHjyUyqwLJ/1l7WWnl1iVChuwSFQwe350voJIfOU5vvKwSk8jtX DeUf8dFaae1+hOqGQf/jmJa0FEbO6hL3v/u6l83k= Received: from send152.i.mail.ru (send152.i.mail.ru [89.221.237.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id B2E1E51A422 for ; Mon, 1 Sep 2025 16:07:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B2E1E51A422 Received: by exim-smtp-6944cbb85b-wdm6f with esmtpa (envelope-from ) id 1ut4FZ-00000000EGr-1fDW; Mon, 01 Sep 2025 16:07:10 +0300 Date: Mon, 1 Sep 2025 16:07:54 +0300 To: Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org Message-ID: References: <43f2870a9d46587fde4b3dd31c46af0563dac455.1756287598.git.sergeyb@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <43f2870a9d46587fde4b3dd31c46af0563dac455.1756287598.git.sergeyb@tarantool.org> X-Mailru-Src: smtp X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD908D9D1BDED0E0DF7229D069B02B55B281324E037A5184CB2182A05F5380850404C228DA9ACA6FE27A3D73056E7BB727D3DE06ABAFEAF67052A0B350491904603B05F67ACD2BF370C888ABC412A473640 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7059B0D8AC717918AEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB55337566D328D16D1FE2D8A756F477A1C2CF10AE85CE0744AE1EDE5754F046CAA0F80FD9389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0A29E2F051442AF778941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6D52CD31C43BF465FCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB86D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE7F669735A3130CE6243847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A582176688903EA7C15002B1117B3ED6968F7DEA888EEE0BE81A1B8FE1FED62FE8823CB91A9FED034534781492E4B8EEADC24E78AA85F86F6CBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC7177DD89D51EBB7742DC8270968E61249B1004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D34D8C933888226C8414D640B45DFC2D641C669CC4F6B461F0C78AAD67EEA991453E0346248D838C6C31D7E09C32AA3244CDCECF0B1ED909D1077DD89D51EBB7742CAFADCE9931B98DDEA455F16B58544A2E30DDF7C44BCB90DA5AE236DF995FB59829709634694AABAED6A17656DB59BCAD427812AF56FC65B X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVXNcwk+fZooKIjWJONHvTrU= X-DA7885C5: AA74F49CD310EDF4F255D290C0D534F907162B1BC4093F3A5A2A9E8150A1551D179D61EBE9B37AFA5B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393FE9E42A757851DB6819BD94CFEA8041313351CEB0694EFF51E68C1C05EA60FB7E49D44BB4BD9522A059A1ED8796F048DB274557F927329BE89D5A3BC2B10C37545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] LJ_FR2: Fix stack checks in vararg calls. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey! Thanks for the patch! Please consider my comments below. On 27.08.25, Sergey Bronnikov wrote: > Thanks to Peter Cawley. > > (cherry picked from commit d1a2fef8a8f53b0055ee041f7f63d83a27444ffa) > > The builtin `pcall()` has two separate ways by which it can > grow the stack by one slot: > > 1. Resolving the `__call` metamethod of its first argument. This is unrelated to this patch, so it can be omitted. > 2. Growing the stack by one slot in LJ_FR2 mode. > > The first case leads to a stack smash if `pcall()` is used as > `__call`. Setting a metatable with this metamethod will cause > an infinite loop which fills up the stack with `pcall`-frames > and then keeps going beyond the end of the stack until it segfaults. This issue is not related to this patch. > Either of these points can cause an issue if `pcall()` is used as > `__newindex`. Looks like the metamethods are not required for issue reproducing. > The patch partially fixes aforementioned issues. By how? > > Sergey Bronnikov: > * added the description and the test for the problem > > Part of tarantool/tarantool#11691 > --- > src/lj_def.h | 2 +- > src/lj_dispatch.c | 2 +- > src/vm_arm64.dasc | 1 + > src/vm_mips64.dasc | 1 + > ...048-fix-stack-checks-vararg-calls.test.lua | 56 +++++++++++++++++++ > 5 files changed, 60 insertions(+), 2 deletions(-) > create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > > diff --git a/src/lj_def.h b/src/lj_def.h > diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c > index a44a5adf..431cb3c2 100644 > --- a/src/lj_dispatch.c > +++ b/src/lj_dispatch.c > @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn) > int numparams = pt->numparams; > int gotparams = (int)(L->top - L->base); > int need = pt->framesize; > - if ((pt->flags & PROTO_VARARG)) need += 1+gotparams; > + if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams; I can't see the test related to this change. Not `prober_1()` nor `prober_2()` lead to the assertion failure for x86_64 or aarch64 without it. > lj_state_checkstack(L, (MSize)need); > numparams -= gotparams; > return numparams >= 0 ? numparams : 0; > diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc > index c5f0a7a7..cf8e575a 100644 > --- a/src/vm_arm64.dasc > +++ b/src/vm_arm64.dasc > @@ -3779,6 +3779,7 @@ static void build_ins(BuildCtx *ctx, BCOp op, int defop) > | add TMP2, BASE, RC > | add LFUNC:CARG3, CARG3, TMP0, lsl #47 > | add RA, RA, RC > + | sub CARG1, CARG1, #8 Please mention in the commit message why the original stack check was incorrect (for aarch64 and mips64). Also, mention why the x64 isn't affected: x64: | RA == BASE + (RD=NARGS+1)*8 + framesize * 8 +8 > maxstack The last summand here is the `LJ_FR2` adjustment. arm64|mips64 -- incorrect check: | RA == BASE + (RD=NARGS)*8 + framesize * 8 >= maxstack > | add TMP0, RC, #16+FRAME_VARG > | str LFUNC:CARG3, [TMP2], #8 // Store (tagged) copy of LFUNC. > | ldr KBASE, [PC, #-4+PC2PROTO(k)] > diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc > index 44fba36c..7f49df5b 100644 > --- a/src/vm_mips64.dasc > +++ b/src/vm_mips64.dasc > diff --git a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > new file mode 100644 > index 00000000..e300d5c1 > --- /dev/null > +++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > @@ -0,0 +1,56 @@ > +local tap = require('tap') > + > +-- A test file to demonstrate a stack overflow in `pcall()` in > +-- some cases, see below testcase descriptions. > +-- See also https://github.com/LuaJIT/LuaJIT/issues/1048. > +local test = tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({ > + ['Test requires JIT enabled'] = not jit.status(), > +}) > + > +test:plan(2) > + > +-- The first testcase demonstrate a stack overflow in `pcall()` > +-- by recursive calling `pcall()`. The functions are vararg > +-- because stack check in BC_IFUNCV is off by one without the Minor: by one for the arm64, mips64 architectures. > +-- patch. > +local function prober_1(...) -- luacheck: no unused > + pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, pairs, {}) > +end Why do we want to use probber_1 here? Why is this different from the second example? Only because of the metamethods? If we want to keep it, please describe why we need at least 9 pcall-s. Also, there is no need for `pairs()` here. Let's use another simpler fast function (like `type()`). Also, please add a comment about fast function usage, see the example below. > + > +local function looper_1(n, ...) > + prober_1(...) > + prober_1(nil, ...) Why do we need `nil` here? I suppose this line is excess, see the comment with the example below. > + return looper_1(n + 1, n, ...) > +end > + > +pcall(coroutine.wrap(looper_1), 0) > + > +test:ok(true, 'no stack overflow with recursive pcall') > + > +-- The second testcase demonstrate a stack overflow in `pcall()` > +-- with using metamethods. A stack overflow is triggered when > +-- `pcall()` is used as `__call` metamethod, setting metatable > +-- will cause an infinite loop which fills up the stack with > +-- `pcall`-frames and then keeps going beyond the end of the > +-- stack until it segfaults. This comment is unrelated to this test. > Also, a stack overflow can be > +-- triggered when `pcall()` is used as `__newindex` metamethod. > +-- The functions are vararg because stack check in BC_IFUNCV is > +-- off by one without the patch. > + > +local mt = setmetatable({}, { __newindex = pcall, __call = pairs }) > + > +local function prober_2(...) -- luacheck: no unused > + mt[mt] = mt > +end > + > +local function looper_2(n, ...) > + prober_2(...) > + prober_2(nil, ...) > + return looper_2(n + 1, n, ...) > +end > + > +pcall(coroutine.wrap(looper_2), 0) This can be simplified to the following: | src/luajit -e ' | -- Do not use a Lua function as metamethod -- since it will check | -- the stack on each invocation. Use simple `type()` built-in | -- instead. | local t = setmetatable({}, {__newindex = pcall, __call = type}) | local function prober(...) | -- Invokes `pcall(t, t, t)`. | t[t] = t | end | local function looper(n, ...) | prober(...) | return looper(n+1, n, ...) | end | pcall(coroutine.wrap(looper), 0) | ' > + > +test:ok(true, 'no stack overflow with using metamethod') > + > +test:done(true) > -- > 2.43.0 > -- Best regards, Sergey Kaplun