From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id C970629EACD; Tue, 9 Dec 2025 17:37:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C970629EACD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1765291069; bh=aDFJPxATK2GTNvnKl6Zz4MHe9WAV4Opc+cB4Y29dyAQ=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Be8fkmWSjuPi1YG9xnSSUCx947oedU+rYKGDJGxZDLcpNbc+FAz/WjE5wcxPmZGg0 cfu27qyCXcxkYDOXgq6Gbk54Y7Y9MYL7E2kRryerUImq2KPndYT9TgBbVHhIzipfMN BpnB8QFDLzYRj1Qi/FjQpb8Yj2aj59Il6ZddQz34= Received: from send57.i.mail.ru (send57.i.mail.ru [89.221.237.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id B2C6329EACD for ; Tue, 9 Dec 2025 17:37:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B2C6329EACD Received: by exim-smtp-9954f69f5-9gbgn with esmtpa (envelope-from ) id 1vSyqX-00000000BRc-33Z7; Tue, 09 Dec 2025 17:37:46 +0300 Content-Type: multipart/alternative; boundary="------------xdPmHPjFopPUaIeVlHxUZkKF" Message-ID: <87dfb27d-f4d1-4e5e-8255-046b1c5d7326@tarantool.org> Date: Tue, 9 Dec 2025 17:37:45 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Sergey Kaplun Cc: Sergey Bronnikov , tarantool-patches@dev.tarantool.org References: <43f2870a9d46587fde4b3dd31c46af0563dac455.1756287598.git.sergeyb@tarantool.org> Content-Language: en-US In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD92D4E0D2F4C74F7252CF1D6412CDD79E3931BEC57BDDD8119182A05F5380850402DDBF8E245CCF2A83DE06ABAFEAF670565FC482C72ED4454A6D9B3D7BF56588CAFCDCBC6C9480999 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE760302A529BCAAAFCEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB55337566FF53983331E576C755A397DF4DCD9662EF6B43153C3EE01C4DA0E99E3F08ECBF389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0A3E989B1926288338941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B652D31B9D28593E51CC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C224958C1606C78F2434E76E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8B6A4E49BB0F3BA1413AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735B344165809136645C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5B895ACA6F0239DC15002B1117B3ED696C2E3B9B51065AE11C66B2B37046EC955823CB91A9FED034534781492E4B8EEADEEA082C9A12FE455BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE1918E10F71CB4DF9F96AB70F9BE574AE9C625B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D659AD952C8FB1A8BBC7B21473E0CF87E42A9434684AA63F6627A294DBBC0740533D7EACD0302499EDA9B8341EE9D5BE9A0A48F808DE8AFF9C041BD0AB25387FB12C3E96B3D23A3B50676536EB022892E5344C41F94D744909CE2512F26BEC029E55448553D2254B8D95CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVZ7tsNvok/TzhsLc4KqiYik= X-Mailru-Sender: 811C44EDE0507D1FF7A5115BD94F8393D43F921361A83259A80142CFDE86685D35A241172CFFA611128A989C49FA59F4645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] LJ_FR2: Fix stack checks in vararg calls. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------xdPmHPjFopPUaIeVlHxUZkKF Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, Sergey, thanks for review! Please consider my three comments below. Sergey On 10/27/25 11:16, Sergey Kaplun wrote: > Hi, Sergey! > Thanks for the fixes! > Please consider my comments below. > Also, please send the next version via v2 series to simplify the > review. > > On 23.09.25, Sergey Bronnikov wrote: >> Hi, Sergey, >> >> thanks for review! Please see my comments below. >> >> Sergey >> >> On 9/1/25 16:07, Sergey Kaplun via Tarantool-patches wrote: >>> Hi, Sergey! >>> Thanks for the patch! >>> Please consider my comments below. >>> >>> On 27.08.25, Sergey Bronnikov wrote: > > >>>> Sergey Bronnikov: >>>> * added the description and the test for the problem >>>> >>>> Part of tarantool/tarantool#11691 >>>> --- >>>> src/lj_def.h | 2 +- >>>> src/lj_dispatch.c | 2 +- >>>> src/vm_arm64.dasc | 1 + >>>> src/vm_mips64.dasc | 1 + >>>> ...048-fix-stack-checks-vararg-calls.test.lua | 56 +++++++++++++++++++ >>>> 5 files changed, 60 insertions(+), 2 deletions(-) >>>> create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua >>>> >>>> diff --git a/src/lj_def.h b/src/lj_def.h >>> >>> >>>> diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c >>>> index a44a5adf..431cb3c2 100644 >>>> --- a/src/lj_dispatch.c >>>> +++ b/src/lj_dispatch.c >>>> @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn) >>>> int numparams = pt->numparams; >>>> int gotparams = (int)(L->top - L->base); >>>> int need = pt->framesize; >>>> - if ((pt->flags & PROTO_VARARG)) need += 1+gotparams; >>>> + if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams; >>> I can't see the test related to this change. Not `prober_1()` nor >>> `prober_2()` lead to the assertion failure for x86_64 or aarch64 without >>> it. The LJ_FR2 check was added for consistency with non-gc64 flavor, the commit message was updated >    A fixup for a number of required slots in `call_init()` was added >   for consistency with non-gc64 flavor. Also, there is an issue [1] about inconsistencies in stack checks. 1. https://github.com/LuaJIT/LuaJIT/issues/1402 >> Please check again. Both testcases trigger segfault on AArch64 (odroid). > Double checked: > | root@odroid:/home/skaplun/lj-1048-review# git diff > | diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c > | index 431cb3c2..a44a5adf 100644 > | --- a/src/lj_dispatch.c > | +++ b/src/lj_dispatch.c > | @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn) > | int numparams = pt->numparams; > | int gotparams = (int)(L->top - L->base); > | int need = pt->framesize; > | - if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams; > | + if ((pt->flags & PROTO_VARARG)) need += 1+gotparams; > | lj_state_checkstack(L, (MSize)need); > | numparams -= gotparams; > | return numparams >= 0 ? numparams : 0; > | Test project /home/skaplun/lj-1048-review > | Start 118: test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua > | 1/1 Test #118: test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua ... Passed 3.38 sec > | > | 100% tests passed, 0 tests failed out of 1 > | > | Label Time Summary: > | tarantool-tests = 3.38 sec*proc (1 test) > | > | Total Test time (real) = 3.42 sec > > > >>>> +-- patch. >>>> +local function prober_1(...) -- luacheck: no unused >>>> + pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, pairs, {}) >>>> +end >>> Why do we want to use probber_1 here? Why is this different from the >>> second example? Only because of the metamethods? `prober_1` triggers the issue by using recursive (p)call > Still need an explanation. > >>> If we want to keep it, please describe why we need at least 9 pcall-s. >> As I got right, exactly this number of pcall's is needed to trigger a >> stack overflow. > Yes, but why 9 is minimum number of pcall's when the issue is reproduced? > > The number depends on a previous value of LJ_STACK_EXTRA. LJ_STACK_EXTRA, is an "overlay" on top of the stack, and for a buffer overflow you need at least 8 + 1 frames to write slots above this "overlay". These pcalls generates additional frames. --------------xdPmHPjFopPUaIeVlHxUZkKF Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi, Sergey,

thanks for review! Please consider my three comments below.

Sergey

On 10/27/25 11:16, Sergey Kaplun wrote:
Hi, Sergey!
Thanks for the fixes!
Please consider my comments below.
Also, please send the next version via v2 series to simplify the
review.

On 23.09.25, Sergey Bronnikov wrote:

Hi, Sergey,

thanks for review! Please see my comments below.

Sergey

On 9/1/25 16:07, Sergey Kaplun via Tarantool-patches wrote:

Hi, Sergey!
Thanks for the patch!
Please consider my comments below.

On 27.08.25, Sergey Bronnikov wrote:

<snipped>


Sergey Bronnikov:
* added the description and the test for the problem

Part of tarantool/tarantool#11691
---
  src/lj_def.h                                  |  2 +-
  src/lj_dispatch.c                             |  2 +-
  src/vm_arm64.dasc                             |  1 +
  src/vm_mips64.dasc                            |  1 +
  ...048-fix-stack-checks-vararg-calls.test.lua | 56 +++++++++++++++++++
  5 files changed, 60 insertions(+), 2 deletions(-)
  create mode 100644 test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua

diff --git a/src/lj_def.h b/src/lj_def.h

<snipped>


diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c
index a44a5adf..431cb3c2 100644
--- a/src/lj_dispatch.c
+++ b/src/lj_dispatch.c
@@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn)
      int numparams = pt->numparams;
      int gotparams = (int)(L->top - L->base);
      int need = pt->framesize;
-    if ((pt->flags & PROTO_VARARG)) need += 1+gotparams;
+    if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams;

I can't see the test related to this change. Not `prober_1()` nor
`prober_2()` lead to the assertion failure for x86_64 or aarch64 without
it.

The LJ_FR2 check was added for consistency with non-gc64 flavor, the commit message was updated

>    A fixup for a number of required slots in `call_init()` was added
>   for consistency with non-gc64 flavor.

Also, there is an issue [1] about inconsistencies in stack checks.

1. https://github.com/LuaJIT/LuaJIT/issues/1402

Please check again. Both testcases trigger segfault on AArch64 (odroid).


Double checked:
| root@odroid:/home/skaplun/lj-1048-review# git diff
| diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c
| index 431cb3c2..a44a5adf 100644
| --- a/src/lj_dispatch.c
| +++ b/src/lj_dispatch.c
| @@ -453,7 +453,7 @@ static int call_init(lua_State *L, GCfunc *fn)
|      int numparams = pt->numparams;
|      int gotparams = (int)(L->top - L->base);
|      int need = pt->framesize;
| -    if ((pt->flags & PROTO_VARARG)) need += 1+LJ_FR2+gotparams;
| +    if ((pt->flags & PROTO_VARARG)) need += 1+gotparams;
|      lj_state_checkstack(L, (MSize)need);
|      numparams -= gotparams;
|      return numparams >= 0 ? numparams : 0;
| Test project /home/skaplun/lj-1048-review
|     Start 118: test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua
| 1/1 Test #118: test/tarantool-tests/lj-1048-fix-stack-checks-vararg-calls.test.lua ...   Passed    3.38 sec
|
| 100% tests passed, 0 tests failed out of 1
|
| Label Time Summary:
| tarantool-tests    =   3.38 sec*proc (1 test)
|
| Total Test time (real) =   3.42 sec

<snipped>


+-- patch.
+local function prober_1(...) -- luacheck: no unused
+  pcall(pcall, pcall, pcall, pcall, pcall, pcall, pcall, pcall, pairs, {})
+end

Why do we want to use probber_1 here? Why is this different from the
second example? Only because of the metamethods?
`prober_1` triggers the issue by using recursive (p)call 
Still need an explanation.


If we want to keep it, please describe why we need at least 9 pcall-s.

As I got right, exactly this number of pcall's is needed to trigger a 
stack overflow.

Yes, but why 9 is minimum number of pcall's when the issue is reproduced?

<snipped>

The number depends on a previous value of LJ_STACK_EXTRA.

LJ_STACK_EXTRA, is an "overlay" on top of the stack, and for a buffer overflow
you need at least 8 + 1 frames to write slots above this "overlay". These pcalls generates additional frames. 

    

  
  


--------------xdPmHPjFopPUaIeVlHxUZkKF--