From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 2EEE91B54D27; Thu, 12 Mar 2026 13:15:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2EEE91B54D27 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1773310556; bh=cwj1uIjrn/ipNxbi11tFkKAQ7PHbN/yPVIwUyRHn6Z8=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=QUOBkNcNTOjHnKqoWhhUiZ/04S5hdbbb2BPvNBv3eB+w1+57NO7SP/XKu3UXESt61 M5qyyFcoMHKfj/7xOfpRyIqwUVzQaoeKJHjqFPZDOby2eWtac0NccACsCAGuFHuGuk fKoqwbLNII8u1jwuwJ+KC55kZg7r0wumC0dxTgKo= Received: from send194.i.mail.ru (send194.i.mail.ru [95.163.59.33]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 0086F1B54D29 for ; Thu, 12 Mar 2026 13:15:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0086F1B54D29 Received: by exim-smtp-64cdfc6c8d-kl2k5 with esmtpa (envelope-from ) id 1w0d57-00000000351-3YxR; Thu, 12 Mar 2026 13:15:54 +0300 Date: Thu, 12 Mar 2026 13:16:48 +0300 To: Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD91ABAE9865AC7DC889514E9E256EAD50689274261A8C811C4182A05F538085040B03B0981CF8C4A003DE06ABAFEAF6705DB463DADB86F4F2A56C393CEE0DED704B323337538F1B82C X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F16C4DE526EFCC04EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB5533756627194F95AE6E7C2DAC2F94FFA64854074ADB2CFACFFF3196596BDD079B6D7355389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0A3E989B1926288338941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6D52CD31C43BF465FCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C22497936DB32DC19D13A76E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8B2F0B2B5439D2DBFC3AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E7356C9A9530EBF72002C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5EBBBBDF98FF910A25002B1117B3ED6962F821D16B86634BEBFF4097FFC9E796F823CB91A9FED034534781492E4B8EEAD220496FFA5CD4785BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34CAB70F9BE574AE9C625B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D6598F640C5933EE53659307027F550BD0126C69D86F90ECD47F641404443A5116A8F39AC18FD0F932F0B8341EE9D5BE9A0A6514B0A39692E6CB701E0537804F8D2CB9F98B46A9BE9C348CD93680B12512CF4C41F94D744909CECFA6C6B0C050A61A8CAF69B82BA93681CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVdbVVJCphTR/TA+8CqlesYU= X-Mailru-Sender: 583F1D7ACE8F49BDD951BA70C165859EF209F60E14AB1026601A8E04A89F3FFC5BC61775C30F6309C0D290E587E451AEF2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A84198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 3/3][v3] Add stack check to pcall/xpcall. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey! Thanks for the patch! Please, fix my comments below. Don't forget to add the corresponding iterative changes. On 12.03.26, Sergey Bronnikov wrote: > From: Mike Pall > > Analyzed by Peter Cawley. > > (cherry picked from commit a4c1640432a9d8a60624cdc8065b15078c228e36) > > The patch adds the stack check to fast functions `pcall()` and > `xpcall()`. Please add more verbose description: | (cherry picked from commit a4c1640432a9d8a60624cdc8065b15078c228e36) | | The `pcall()` and `xpcall()` calls in GC64 mode require 2 slots. This | means that all arguments should be moved up during emitting of the frame | link to the stack. Hence, this may cause stack overflow without the | corresponding check. | | This patch adds the corresponding checks to the VM. Non-GC64 VMs are | updated as well for the consistency. > > Sergey Bronnikov: > * added the description and the test for the problem > > Part of tarantool/tarantool#12134 > --- > src/vm_arm.dasc | 7 ++++ > src/vm_arm64.dasc | 8 +++++ > src/vm_mips.dasc | 10 +++++- > src/vm_mips64.dasc | 14 ++++++-- > src/vm_ppc.dasc | 9 +++++ > src/vm_x64.dasc | 6 ++++ > src/vm_x86.dasc | 6 ++++ > ...048-fix-stack-checks-vararg-calls.test.lua | 35 ++++++++++++++++++- > 8 files changed, 90 insertions(+), 5 deletions(-) > > diff --git a/src/vm_arm.dasc b/src/vm_arm.dasc > index 7095e660..efe9dcb2 100644 > --- a/src/vm_arm.dasc > +++ b/src/vm_arm.dasc > diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc > index 5ef37243..074c1f31 100644 > --- a/src/vm_arm64.dasc > +++ b/src/vm_arm64.dasc > diff --git a/src/vm_mips.dasc b/src/vm_mips.dasc > index 32caabf7..69d09d52 100644 > --- a/src/vm_mips.dasc > +++ b/src/vm_mips.dasc > diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc > index 6c2975b4..4e60ee07 100644 > --- a/src/vm_mips64.dasc > +++ b/src/vm_mips64.dasc > @@ -1418,8 +1418,12 @@ static void build_subroutines(BuildCtx *ctx) > |//-- Base library: catch errors ---------------------------------------- > | > |.ffunc pcall > + | ld TMP1, L->maxstack > + | daddu TMP2, BASE, NARGS8:RC > + | sltu AT, TMP1, TMP2 > + | bnez AT, ->fff_fallback > + |. lbu TMP3, DISPATCH_GL(hookmask)(DISPATCH) > | daddiu NARGS8:RC, NARGS8:RC, -8 > - | lbu TMP3, DISPATCH_GL(hookmask)(DISPATCH) > | bltz NARGS8:RC, ->fff_fallback > |. move TMP2, BASE > | daddiu BASE, BASE, 16 > @@ -1440,8 +1444,12 @@ static void build_subroutines(BuildCtx *ctx) > |. nop > | > |.ffunc xpcall > - | daddiu NARGS8:TMP0, NARGS8:RC, -16 This neglets the first patch in the series. See the comment below. > - | ld CARG1, 0(BASE) > + | ld TMP1, L->maxstack > + | daddu TMP2, BASE, NARGS8:RC > + | sltu AT, TMP1, TMP2 > + | bnez AT, ->fff_fallback > + |. ld CARG1, 0(BASE) > + | daddiu NARGS8:RC, NARGS8:RC, -16 This line is incorrect. This neglets the 1st patch in the series. It should be | | daddiu NARGS8:TMP0, NARGS8:RC, -16 > | ld CARG2, 8(BASE) > | bltz NARGS8:TMP0, ->fff_fallback > |. lbu TMP1, DISPATCH_GL(hookmask)(DISPATCH) > diff --git a/src/vm_ppc.dasc b/src/vm_ppc.dasc > index 980ad897..f2ea933b 100644 > --- a/src/vm_ppc.dasc > +++ b/src/vm_ppc.dasc > diff --git a/src/vm_x64.dasc b/src/vm_x64.dasc > index 8b6781a6..c57b76b7 100644 > --- a/src/vm_x64.dasc > +++ b/src/vm_x64.dasc > diff --git a/src/vm_x86.dasc b/src/vm_x86.dasc > index 7c11c78e..36804d11 100644 > --- a/src/vm_x86.dasc > +++ b/src/vm_x86.dasc > diff --git a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > index 3a8ad63d..ad8b151b 100644 > --- a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > +++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > @@ -5,7 +5,7 @@ local tap = require('tap') > -- See also https://github.com/LuaJIT/LuaJIT/issues/1048. > local test = tap.test('lj-1048-fix-stack-checks-vararg-calls') > > -test:plan(2) > +test:plan(5) > > -- The test case demonstrates a segmentation fault due to stack > -- overflow by recursive calling `pcall()`. The functions are > @@ -50,4 +50,37 @@ pcall(coroutine.wrap(looper), prober_2, 0) > > test:ok(true, 'no stack overflow with metamethod') > > +-- The testcases demonstrates a stack overflow in > +-- `pcall()`/xpcall()` triggered using metamethod `__call`. > + > +t = coroutine.wrap(setmetatable)({}, { __call = pcall }) I've meant the following: | t = setmetatable({}, { __call = pcall }) | coroutine.wrap(function() t() end)() > + > +test:ok(true, 'no stack overflow with metamethod __call with pcall()') > + > +t = coroutine.wrap(setmetatable)({}, { __call = xpcall }) I've meant the following: | t = setmetatable({}, { __call = xpcall }) | coroutine.wrap(function() t() end)() But this won't work since the second amount of xpcall must be the function. So, this test case is invalid. We must to duplicate the second approach with `xpcall()` This works fine. | LUA_PATH="src/?.lua;;" gdb --args src/luajit -e ' | local t = {} | local function xpcall_wrapper() | return xpcall(unpack(t)) | end | | local N_ITERATIONS = 200 | | for i = 1, N_ITERATIONS do | t[i], t[i + 1], t[i + 2] = xpcall, type, {} | coroutine.wrap(xpcall_wrapper)() | end | ' > + > +test:ok(true, 'no stack overflow with metamethod __call with xpcall()') > + > +-- The testcase demonstrates a stack overflow in > +-- `pcall()`/`xpcall()` similar to the first testcase, but it is > +-- triggered using `unpack()`. > + > +t = {} > +local function f() Lets name it `pcall_wrapper()` > + return pcall(unpack(t)) > +end > + > +-- The problem is only reproduced on LuaJIT GC64 and is best Typo: s/best/better/ > +-- reproduced under Valgrind than AddressSanitizer. The chosen > +-- value was found experimentally and always results in an attempt > +-- to write beyond the allocated memory. > +local N_ITERATIONS = 200 > + > +for i = 1, N_ITERATIONS do > + t[i], t[i + 1], t[i + 2] = pcall, type, {} > + coroutine.wrap(f)() > +end > + > +test:ok(true, 'no stack overflow with unpacked pcalls') > + > test:done(true) > -- > 2.43.0 > -- Best regards, Sergey Kaplun