[Tarantool-patches] [PATCH luajit] Avoid out-of-range number of results when compiling select(k, ...).

[Sergey Kaplun]

Hi, Sergey!
Thanks for the review!

On 08.02.24, Sergey Bronnikov wrote:
> Hi, Sergey
> 
> thanks for the patch
> 
> couldn't reproduce a problem by provided test.

As you can see in the test comment, the test failed under ASAN with a
heap buffer overflow. Unfortunately, I didn't come up with an idea of
how to observe the misbehaviour (too long IRs for the trace) without
ASAN.

> 
> What compilation options I've used:
> 
> 
> 1st attempt:
> 
> cmake -S . -B build -DLUA_USE_ASSERT=ON -DLUA_USE_APICHECK=ON
> 
> 
> 2nd attempt:
> 
> CMAKE_BUILD_TYPE="Debug"
> CMAKE_C_COMPILER="clang"
> CMAKE_EXPORT_COMPILE_COMMANDS:BOOL="TRUE"
> LUAJIT_ENABLE_COVERAGE:BOOL="FALSE"
> LUAJIT_ENABLE_GC64:BOOL="TRUE"
> LUAJIT_USE_ASAN:BOOL="FALSE"
> LUAJIT_USE_SYSMALLOC:BOOL="FALSE"
> LUA_USE_APICHECK:BOOL="TRUE"
>    LUA_USE_ASSERT:BOOL="TRUE"
> 
> 
> Sergey
> 
> On 2/7/24 15:06, Sergey Kaplun wrote:

<snipped>

> > +local function varg_frame_wp()
> > +  -- XXX: Need some constant negative value as the first argument
> > +  -- of `select()` when recording the trace.
> > +  -- Also, it should be huge enough to be greater than
> > +  -- `J->maxslot`. The value on the first iteration is ignored.
> > +  -- This will fail under ASAN due to a heap buffer overflow.
> > +  varg_frame(recording and -(LJ_MAX_JSLOTS + 1) or 1)
> > +end

<snipped>

> > +
> > +test:done(true)

-- 
Best regards,
Sergey Kaplun