From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id D4C1A1713DBC; Mon, 16 Feb 2026 10:20:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D4C1A1713DBC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1771226447; bh=cXZv4tQhg0WS/2tVzQB8kNbCZO8eqE2m1hubbD2Id98=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=n519/UFajIdgVa99TZkuVV90ixXPqXES4zT09j+5jUfMp4V8ZbQeYLlUVbkRtkSYj cUSE2wQjuzL/ayss4cxXT1TdI9xtx2MkkaQTppaqUpTbmokrhfoTxPghVC9A7H+L7s P0d43Ym6gPq8bRe8TURBar9/NBol/RAamBt/C3HE= Received: from send34.i.mail.ru (send34.i.mail.ru [89.221.237.129]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 7020A1713DB9 for ; Mon, 16 Feb 2026 10:20:45 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7020A1713DB9 Received: by exim-smtp-5d446c7b8f-w96m4 with esmtpa (envelope-from ) id 1vrsuS-000000008dM-1DBv; Mon, 16 Feb 2026 10:20:44 +0300 Content-Type: multipart/alternative; boundary="------------Tmw9lT0KIhMrg0AptBsdo5Ny" Message-ID: <2c57321f-06c6-4a31-bb69-118a7dd09cce@tarantool.org> Date: Mon, 16 Feb 2026 10:20:43 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun , Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org References: <51e75e7052824de65036abd2f5807a1224f438aa.1765350224.git.sergeyb@tarantool.org> In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9A6EFB945BF0DEAB2F96CBC83A3B9494A238C435315A85C5F182A05F53808504058B68B13BA0C9F4D3DE06ABAFEAF670516BD09800F27D0DBE6A3A489B22A17A82D7423A3730B9AE4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7A179494B5629353BEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB553375662E783F4C49E27FAE0DF11E0DDE7CA9B4DFD876ACC607A737A737A09C5D10E3AF389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C045A75973B56231AD8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6A0EE70D6C4970CA7A471835C12D1D977C4224003CC8364762BB6847A3DEAEFB0F43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A39223F8577A6DFFEA7CE52E467C461EF0E943847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A5895FAF1E88C85AEC5002B1117B3ED696935B7A4DC3B9128BCA7E60A991436CA2823CB91A9FED034534781492E4B8EEAD47A3109F1ACFD409BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34CAB70F9BE574AE9C625B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D659861FF2F8EC24C542FFE780A1E6E114CD8A7D3635B65BF8D3B1756E74111DE2BD803094D9EF972510B8341EE9D5BE9A0A6343BF519F0A72D8CB7823B376064C2F433C3100753A1ADC6536EB022892E5344C41F94D744909CE2512F26BEC029E55448553D2254B8D95CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVReXxXgcs1IG8pLY3egiUlI= X-DA7885C5: EA8E3FAD63DAB8FCF255D290C0D534F9C1B9FD076F3DDD27F87EEDC8B34D57B9F6E791C4634936535B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393DDD5FD59B456EAD2FBCCD67E22F584014F1519DB1C84BE3658868B07F1D14D74EF86D5F70DA33880E41E8EF7A07863ECB274557F927329BE2DDF8182D28ACDB545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 2/3][v2] LJ_FR2: Fix stack checks in vararg calls. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------Tmw9lT0KIhMrg0AptBsdo5Ny Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, Sergey, thanks for review! On 2/11/26 11:30, Sergey Kaplun via Tarantool-patches wrote: > On 10.12.25, Sergey Bronnikov wrote: >> From: Mike Pall >> >> Thanks to Peter Cawley. >> >> (cherry picked from commit d1a2fef8a8f53b0055ee041f7f63d83a27444ffa) >> >> Stack overflow can cause a segmentation fault in vararg > Typo: s/vararg/a vararg/ Fixed. > >> function on ARM64 and MIPS64 in LJ_FR2 mode. This happen > Typo: s/happen/happens/ Fixed. > >> because stack check in BC_IFUNCV is off by one on these > Typo: s/stack/the stack/ Fixed. > >> platforms without the patch. The original stack check >> for ARM64 and MIPS64 was incorrect: >> >> | RA == BASE + (RD=NARGS)*8 + framesize * 8 >= maxstack >> >> while stack check on x86_64 is correct and therefore is > Typo: s/stack/the stack/ Fixed. > >> not affected by the problem: >> >> | RA == BASE + (RD=NARGS+1)*8 + framesize * 8 +8 > maxstack >> >> The patch partially fixes aforementioned issue by > Typo: s/aforementioned/the aforementioned/ Fixed. > >> bumping LJ_STACK_EXTRA by 1 to give a space to write >> the entire frame link and fixing a number of last > Typo: s/a number/the number/ Fixed. > > I'm not get this part. I suggest rephrasing it like the following: > > | The patch partially fixes the aforementioned issue by bumping > | LJ_STACK_EXTRA by 1 to give a space to the entire frame link for a > | vararg function as the __newindex metamethod. Updated. > > >> free slot in the stack (LJ_FR2 summand adjustment). >> >> A fixup for a number of required slots in `call_init()` was added >> for consistency with non-gc64 flavor. > Typo: s/gc64/GC64/ Fixed. > I would also add: "The check is too strict, so this can't lead to any > crash." Updated. >> Sergey Bronnikov: >> * added the description and the test for the problem >> >> Part of tarantool/tarantool#12134 >> --- >> src/lj_def.h | 2 +- >> src/lj_dispatch.c | 2 +- >> src/vm_arm64.dasc | 1 + >> src/vm_mips64.dasc | 1 + >> ...048-fix-stack-checks-vararg-calls.test.lua | 53 +++++++++++++++++++ >> 5 files changed, 57 insertions(+), 2 deletions(-) >> create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua >> >> diff --git a/src/lj_def.h b/src/lj_def.h >> index a5bca6b0..7e4f251e 100644 >> --- a/src/lj_def.h >> +++ b/src/lj_def.h >> @@ -69,7 +69,7 @@ typedef unsigned int uintptr_t; >> #define LJ_MAX_UPVAL 60 /* Max. # of upvalues. */ >> >> #define LJ_MAX_IDXCHAIN 100 /* __index/__newindex chain limit. */ >> -#define LJ_STACK_EXTRA (5+2*LJ_FR2) /* Extra stack space (metamethods). */ >> +#define LJ_STACK_EXTRA (5+3*LJ_FR2) /* Extra stack space (metamethods). */ >> >> #define LJ_NUM_CBPAGE 1 /* Number of FFI callback pages. */ >> >> diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c >> index a44a5adf..431cb3c2 100644 >> --- a/src/lj_dispatch.c >> +++ b/src/lj_dispatch.c >> @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn) >> int numparams = pt->numparams; >> int gotparams = (int)(L->top - L->base); >> int need = pt->framesize; >> - if ((pt->flags & PROTO_VARARG)) need += 1+gotparams; >> + if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams; >> lj_state_checkstack(L, (MSize)need); >> numparams -= gotparams; >> return numparams >= 0 ? numparams : 0; > Let's add an additional test for this part of code (since we don't have > any). It may be taken from [1]. It doesn't fail now, but we may cover > this branch more precise. Don't get what do you mean. true branch in gc32 is covered by the following tests: test/LuaJIT-tests test/PUC-Rio-Lua-5.1-tests test/tarantool-c-tests/lj-1087-vm-handler-call.c_test test/tarantool-tests/fix-ff-select-recording.test.lua test/tarantool-tests/fix-mips64-spare-side-exit-patching.test.lua test/tarantool-tests/fix-slot-check-for-mm-record.test.lua test/tarantool-tests/fix-slots-overflow-for-varg-record.test.lua test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua test/tarantool-tests/lj-1024-varg-maxslot.test.lua test/tarantool-tests/lj-1025-tsetm-maxslot.test.lua test/tarantool-tests/lj-1026-arm64-invalid-hrefk-offset-check.test.lua test/tarantool-tests/lj-1046-fix-bc-varg-recording.test.lua test/tarantool-tests/lj-1164-record-meta-concat-varg-pcall.test.lua test/tarantool-tests/lj-1295-bad-renames-for-sunk-values.test.lua test/tarantool-tests/lj-584-bad-renames-for-sunk-values.test.lua test/tarantool-tests/lj-704-bc-varg-use-def.test.lua >> diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc >> index c5f0a7a7..cf8e575a 100644 >> --- a/src/vm_arm64.dasc >> +++ b/src/vm_arm64.dasc > > >> diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc >> index da187a7a..6c2975b4 100644 >> --- a/src/vm_mips64.dasc >> +++ b/src/vm_mips64.dasc > > >> diff --git a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua >> new file mode 100644 >> index 00000000..d471d41e >> --- /dev/null >> +++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua >> @@ -0,0 +1,53 @@ >> +local tap = require('tap') >> + >> +-- A test file to demonstrate a stack overflow in `pcall()` in > I would rephrase it like "demonstrate a crash due to Lua stack > out-of-bounds access". Updated: --- a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua +++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua @@ -1,7 +1,7 @@  local tap = require('tap') --- A test file to demonstrate a stack overflow in `pcall()` in --- some cases, see below testcase descriptions. +-- A test file to demonstrate a crash due to Lua stack +-- out-of-bounds access, see below testcase descriptions.  -- See also https://github.com/LuaJIT/LuaJIT/issues/1048.  local test = tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({    ['Test requires JIT enabled'] = not jit.status(), > >> +-- some cases, see below testcase descriptions. >> +-- See alsohttps://github.com/LuaJIT/LuaJIT/issues/1048. >> +local test = tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({ >> + ['Test requires JIT enabled'] = not jit.status(), >> +}) > I suppose there is no need in JIT here. This skipcond may be removed. Removed. >> + >> +test:plan(2) >> + >> +-- The testcase demonstrate a segmentation fault due to stack > Typo: s/testcase demonstrate/test case demonstrates/ Updated. > >> +-- overflow by recursive calling `pcall()`. The functions are >> +-- vararg because stack check in BC_IFUNCV is off by one on ARM64 > Typo: s/stack/the stack/ Updated. > >> +-- and MIPS64 without the patch. >> +local function prober_1(...) -- luacheck: no unused >> + -- Any fast function can be used as metamethod, but `type` is >> + -- convenient here because it works fast and can be used with >> + -- any data type. Lua function cannot be used since it >> + -- will check the stack on each invocation. > Please add the comment, that we need to check using of the correct > value LJ_STACK_EXTRA slots (5+3*LJ_FR2) = 8 for GC64 mode. Updated:  local function prober_1(...) -- luacheck: no unused    -- Any fast function can be used as metamethod, but `type` is    -- convenient here because it works fast and can be used with    -- any data type. Lua function cannot be used since it -  -- will check the stack on each invocation. +  -- will check the stack on each invocation. We need to check +  -- using of the correct value LJ_STACK_EXTRA slots +  -- (5+3*LJ_FR2) = 8 for GC64 mode.    pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, type, 0)  end > >> + pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, type, 0) >> +end >> + >> +local function looper(prober, n, ...) >> + prober(...) >> + return looper(prober, n + 1, n, ...) >> +end >> + >> +pcall(coroutine.wrap(looper), prober_1, 0) >> + >> +test:ok(true, 'no stack overflow with recursive pcall') >> + >> +-- The testcase demonstrate a segmentation fault due to stack > Typo: s/testcase demonstrate/test case demonstrates/ Updated. > >> +-- overflow when `pcall()` is used as `__newindex` metamethod. >> +-- The function is vararg because stack check in BC_IFUNCV is off > Typo: s/stack/the stack/ Updated. > >> +-- by one on ARM64 and MIPS64 without the patch. >> + >> +-- Any fast function can be used as metamethod, but `type` is > Typo: s/metamethod/a metamethod/ Fixed. > >> +-- convenient here because it works fast and can be used with >> +-- any data type. Lua function cannot be used since it > Typo: s/Lua/The Lua/ Fixed. > >> +-- will check the stack on each invocation. >> +local t = setmetatable({}, { __newindex = pcall, __call = type }) >> + >> +local function prober_2(...) -- luacheck: no unused >> + -- Invokes `pcall(t, t, t)`. >> + t[t] = t >> +end >> + >> +pcall(coroutine.wrap(looper), prober_2, 0) >> + >> +test:ok(true, 'no stack overflow with metamethod') >> + >> +test:done(true) >> -- >> 2.43.0 >> > [1]:https://github.com/LuaJIT/LuaJIT/issues/1402#issue-3569942423 > --------------Tmw9lT0KIhMrg0AptBsdo5Ny Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi, Sergey,

thanks for review!

On 2/11/26 11:30, Sergey Kaplun via Tarantool-patches wrote:
On 10.12.25, Sergey Bronnikov wrote:

From: Mike Pall <mike>

Thanks to Peter Cawley.

(cherry picked from commit d1a2fef8a8f53b0055ee041f7f63d83a27444ffa)

Stack overflow can cause a segmentation fault in vararg

Typo: s/vararg/a vararg/
Fixed. 


function on ARM64 and MIPS64 in LJ_FR2 mode. This happen

Typo: s/happen/happens/
Fixed. 


because stack check in BC_IFUNCV is off by one on these

Typo: s/stack/the stack/
Fixed. 


platforms without the patch. The original stack check
for ARM64 and MIPS64 was incorrect:

| RA == BASE + (RD=NARGS)*8 + framesize * 8 >= maxstack

while stack check on x86_64 is correct and therefore is

Typo: s/stack/the stack/
Fixed. 


not affected by the problem:

| RA == BASE + (RD=NARGS+1)*8 + framesize * 8 +8 > maxstack

The patch partially fixes aforementioned issue by

Typo: s/aforementioned/the aforementioned/
Fixed. 


bumping LJ_STACK_EXTRA by 1 to give a space to write
the entire frame link and fixing a number of last

Typo: s/a number/the number/
Fixed. 

I'm not get this part. I suggest rephrasing it like the following:

| The patch partially fixes the aforementioned issue by bumping
| LJ_STACK_EXTRA by 1 to give a space to the entire frame link for a
| vararg function as the __newindex metamethod.
Updated. 



free slot in the stack (LJ_FR2 summand adjustment).

A fixup for a number of required slots in `call_init()` was added
for consistency with non-gc64 flavor.

Typo: s/gc64/GC64/
Fixed. 
I would also add: "The check is too strict, so this can't lead to any
crash."
Updated. 

      

        
Sergey Bronnikov:
* added the description and the test for the problem

Part of tarantool/tarantool#12134
---
 src/lj_def.h                                  |  2 +-
 src/lj_dispatch.c                             |  2 +-
 src/vm_arm64.dasc                             |  1 +
 src/vm_mips64.dasc                            |  1 +
 ...048-fix-stack-checks-vararg-calls.test.lua | 53 +++++++++++++++++++
 5 files changed, 57 insertions(+), 2 deletions(-)
 create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua

diff --git a/src/lj_def.h b/src/lj_def.h
index a5bca6b0..7e4f251e 100644
--- a/src/lj_def.h
+++ b/src/lj_def.h
@@ -69,7 +69,7 @@ typedef unsigned int uintptr_t;
 #define LJ_MAX_UPVAL	60		/* Max. # of upvalues. */
 
 #define LJ_MAX_IDXCHAIN	100		/* __index/__newindex chain limit. */
-#define LJ_STACK_EXTRA	(5+2*LJ_FR2)	/* Extra stack space (metamethods). */
+#define LJ_STACK_EXTRA	(5+3*LJ_FR2)	/* Extra stack space (metamethods). */
 
 #define LJ_NUM_CBPAGE	1		/* Number of FFI callback pages. */
 
diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c
index a44a5adf..431cb3c2 100644
--- a/src/lj_dispatch.c
+++ b/src/lj_dispatch.c
@@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn)
     int numparams = pt->numparams;
     int gotparams = (int)(L->top - L->base);
     int need = pt->framesize;
-    if ((pt->flags & PROTO_VARARG)) need += 1+gotparams;
+    if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams;
     lj_state_checkstack(L, (MSize)need);
     numparams -= gotparams;
     return numparams >= 0 ? numparams : 0;


      

      
Let's add an additional test for this part of code (since we don't have
any). It may be taken from [1]. It doesn't fail now, but we may cover
this branch more precise.


    

    
Don't get what do you mean. 

    
true branch in gc32 is covered by the following tests:

    
test/LuaJIT-tests

      test/PUC-Rio-Lua-5.1-tests

      test/tarantool-c-tests/lj-1087-vm-handler-call.c_test

      test/tarantool-tests/fix-ff-select-recording.test.lua

      test/tarantool-tests/fix-mips64-spare-side-exit-patching.test.lua

      test/tarantool-tests/fix-slot-check-for-mm-record.test.lua

      test/tarantool-tests/fix-slots-overflow-for-varg-record.test.lua

test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua

      test/tarantool-tests/lj-1024-varg-maxslot.test.lua

      test/tarantool-tests/lj-1025-tsetm-maxslot.test.lua

test/tarantool-tests/lj-1026-arm64-invalid-hrefk-offset-check.test.lua

      test/tarantool-tests/lj-1046-fix-bc-varg-recording.test.lua

test/tarantool-tests/lj-1164-record-meta-concat-varg-pcall.test.lua

      test/tarantool-tests/lj-1295-bad-renames-for-sunk-values.test.lua

      test/tarantool-tests/lj-584-bad-renames-for-sunk-values.test.lua

      test/tarantool-tests/lj-704-bc-varg-use-def.test.lua

      

    

    

      

      

        
diff --git a/src/vm_arm64.dasc b/src/vm_arm64.dasc
index c5f0a7a7..cf8e575a 100644
--- a/src/vm_arm64.dasc
+++ b/src/vm_arm64.dasc


      

      
<snipped>



      

        
diff --git a/src/vm_mips64.dasc b/src/vm_mips64.dasc
index da187a7a..6c2975b4 100644
--- a/src/vm_mips64.dasc
+++ b/src/vm_mips64.dasc


      

      
<snipped>



      

        
diff --git a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua
new file mode 100644
index 00000000..d471d41e
--- /dev/null
+++ b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua
@@ -0,0 +1,53 @@
+local tap = require('tap')
+
+-- A test file to demonstrate a stack overflow in `pcall()` in


      

      
I would rephrase it like "demonstrate a crash due to Lua stack
out-of-bounds access".

    

    
Updated:

    
---
      a/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua

      +++
      b/test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua

      @@ -1,7 +1,7 @@

       local tap = require('tap')

       

      --- A test file to demonstrate a stack overflow in `pcall()` in

      --- some cases, see below testcase descriptions.

      +-- A test file to demonstrate a crash due to Lua stack

      +-- out-of-bounds access, see below testcase descriptions.

       -- See also https://github.com/LuaJIT/LuaJIT/issues/1048.

       local test =
      tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({

         ['Test requires JIT enabled'] = not jit.status(),

      

    

    

      



      

        
+-- some cases, see below testcase descriptions.
+-- See also https://github.com/LuaJIT/LuaJIT/issues/1048.
+local test = tap.test('lj-1048-fix-stack-checks-vararg-calls'):skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})


      

      
I suppose there is no need in JIT here. This skipcond may be removed.


    

    Removed.
    

      

      

        
+
+test:plan(2)
+
+-- The testcase demonstrate a segmentation fault due to stack


      

      
Typo: s/testcase demonstrate/test case demonstrates/

    

    Updated.
    

      



      

        
+-- overflow by recursive calling `pcall()`. The functions are
+-- vararg because stack check in BC_IFUNCV is off by one on ARM64


      

      
Typo: s/stack/the stack/

    

    Updated.
    

      



      

        
+-- and MIPS64 without the patch.
+local function prober_1(...) -- luacheck: no unused
+  -- Any fast function can be used as metamethod, but `type` is
+  -- convenient here because it works fast and can be used with
+  -- any data type. Lua function cannot be used since it
+  -- will check the stack on each invocation.


      

      
Please add the comment, that we need to check using of the correct
value LJ_STACK_EXTRA slots (5+3*LJ_FR2) = 8 for GC64 mode.

    

    
Updated:

    
 local function prober_1(...) -- luacheck: no unused

         -- Any fast function can be used as metamethod, but `type` is

         -- convenient here because it works fast and can be used with

         -- any data type. Lua function cannot be used since it

      -  -- will check the stack on each invocation.

      +  -- will check the stack on each invocation. We need to check

      +  -- using of the correct value LJ_STACK_EXTRA slots

      +  -- (5+3*LJ_FR2) = 8 for GC64 mode.

         pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall,
      type, 0)

       end

       

      

    

    

      



      

        
+  pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, type, 0)
+end
+
+local function looper(prober, n, ...)
+  prober(...)
+  return looper(prober, n + 1, n, ...)
+end
+
+pcall(coroutine.wrap(looper), prober_1, 0)
+
+test:ok(true, 'no stack overflow with recursive pcall')
+
+-- The testcase demonstrate a segmentation fault due to stack


      

      
Typo: s/testcase demonstrate/test case demonstrates/

    

    Updated.
    

      



      

        
+-- overflow when `pcall()` is used as `__newindex` metamethod.
+-- The function is vararg because stack check in BC_IFUNCV is off


      

      
Typo: s/stack/the stack/

    

    Updated.
    

      



      

        
+-- by one on ARM64 and MIPS64 without the patch.
+
+-- Any fast function can be used as metamethod, but `type` is


      

      
Typo: s/metamethod/a metamethod/

    

    Fixed.
    

      



      

        
+-- convenient here because it works fast and can be used with
+-- any data type. Lua function cannot be used since it


      

      
Typo: s/Lua/The Lua/

    

    Fixed.
    

      



      

        
+-- will check the stack on each invocation.
+local t = setmetatable({}, { __newindex = pcall, __call = type })
+
+local function prober_2(...) -- luacheck: no unused
+  -- Invokes `pcall(t, t, t)`.
+  t[t] = t
+end
+
+pcall(coroutine.wrap(looper), prober_2, 0)
+
+test:ok(true, 'no stack overflow with metamethod')
+
+test:done(true)
-- 
2.43.0



      

      
[1]: https://github.com/LuaJIT/LuaJIT/issues/1402#issue-3569942423



    

  
  


--------------Tmw9lT0KIhMrg0AptBsdo5Ny--