[Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Sergey Kaplun via Tarantool-patches]

From: Mike Pall <mike>

Reported by Changochen.

(cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)

Additional stack traversal is needed to find an error function set for
handling runtime errors. cframe unwinding is missed for a C protected
frame during this stack traversal. It leads to undefined behaviour or
crash, when raising a runtime error on stack with the CP frame before an
error function handler (for example, an error in __gc handler).

This patch adds missing unwinding for CP frame.

Sergey Kaplun:
* added the description and the test for the problem
---

Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601

 src/lj_err.c                                  |  1 +
 .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
 2 files changed, 26 insertions(+)
 create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua

diff --git a/src/lj_err.c b/src/lj_err.c
index b6be357e..b520b3d3 100644
--- a/src/lj_err.c
+++ b/src/lj_err.c
@@ -585,6 +585,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
       if (cframe_canyield(cf)) return 0;
       if (cframe_errfunc(cf) >= 0)
 	return cframe_errfunc(cf);
+      cf = cframe_prev(cf);
       frame = frame_prevd(frame);
       break;
     case FRAME_PCALL:
diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
new file mode 100644
index 00000000..d8d79100
--- /dev/null
+++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
@@ -0,0 +1,25 @@
+local tap = require('tap')
+
+local test = tap.test('lj-601-fix-gc-finderrfunc')
+test:plan(1)
+
+-- Test file to demonstrate LuaJIT incorrect behaviour, when
+-- throwing error in __gc finalizer.
+-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
+
+collectgarbage()
+
+local a = newproxy(true)
+getmetatable(a).__gc = function()
+  -- Function to raise error via `lj_err_run()` inside __gc.
+  local _ = load(function() collectgarbage()() end)
+end
+
+-- XXX: Generate a small bunch of proxies. Need several to call
+-- `collectgarbage()` on another proxy inside __gc. N cycles is
+-- empirical number.
+for _ = 1, 4 do newproxy(a) end
+collectgarbage('collect')
+
+test:ok(true, 'successfully collectgarbage with error')
+os.exit(test:check() and 0 or 1)
-- 
2.31.0

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[sergos via Tarantool-patches]

Hi!
Thanks for the patch! See my 2 comments below.

Sergos

> On 5 Oct 2021, at 13:28, Sergey Kaplun <skaplun@tarantool.org> wrote:
> 
> From: Mike Pall <mike>
> 
> Reported by Changochen.
> 
> (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
> 
> Additional stack traversal is needed to find an error function set for
   ^^^                          ^^^
Additional to what?            
 				
> handling runtime errors. cframe unwinding is missed for a C protected
> frame during this stack traversal.

I would rephrase - 
A cframe unwinding is miseed for a C protected frame during a serach for 
an error function to handle a runtime error. 

> It leads to undefined behaviour or
> crash, when raising a runtime error on stack with the CP frame before an
> error function handler (for example, an error in __gc handler).
> 
> This patch adds missing unwinding for CP frame.
> 
> Sergey Kaplun:
> * added the description and the test for the problem
> ---
> 
> Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
> Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
> LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601
> 
> src/lj_err.c                                  |  1 +
> .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
> 2 files changed, 26 insertions(+)
> create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> 
> diff --git a/src/lj_err.c b/src/lj_err.c
> index b6be357e..b520b3d3 100644
> --- a/src/lj_err.c
> +++ b/src/lj_err.c
> @@ -585,6 +585,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
>       if (cframe_canyield(cf)) return 0;
>       if (cframe_errfunc(cf) >= 0)
> 	return cframe_errfunc(cf);
> +      cf = cframe_prev(cf);
>       frame = frame_prevd(frame);
>       break;
>     case FRAME_PCALL:
> diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> new file mode 100644
> index 00000000..d8d79100
> --- /dev/null
> +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua

Unfortunately the test passes on the ’tarantool’ branch 

s-ostanevich:luajit s.ostanevich$ git checkout tarantool
Switched to branch 'tarantool'
s-ostanevich:luajit s.ostanevich$ git clean -xdff
[…]
s-ostanevich:luajit s.ostanevich$ cmake .
[…]
s-ostanevich:luajit s.ostanevich$ make
[…]
[100%] Built target libluajit_shared
[100%] Built target libluajit
[100%] Built target luajit
s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
TAP version 13
1..1
ok - successfully collectgarbage with error

> @@ -0,0 +1,25 @@
> +local tap = require('tap')
> +
> +local test = tap.test('lj-601-fix-gc-finderrfunc')
> +test:plan(1)
> +
> +-- Test file to demonstrate LuaJIT incorrect behaviour, when
> +-- throwing error in __gc finalizer.
> +-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
> +
> +collectgarbage()
> +
> +local a = newproxy(true)
> +getmetatable(a).__gc = function()
> +  -- Function to raise error via `lj_err_run()` inside __gc.
> +  local _ = load(function() collectgarbage()() end)
> +end
> +
> +-- XXX: Generate a small bunch of proxies. Need several to call
> +-- `collectgarbage()` on another proxy inside __gc. N cycles is
> +-- empirical number.
> +for _ = 1, 4 do newproxy(a) end
> +collectgarbage('collect')
> +
> +test:ok(true, 'successfully collectgarbage with error')
> +os.exit(test:check() and 0 or 1)
> -- 
> 2.31.0
> 

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Sergey Kaplun via Tarantool-patches]

Hi, Sergos!

Thanks for the review!

On 07.10.21, sergos wrote:
> Hi!
> Thanks for the patch! See my 2 comments below.
> 
> Sergos
> 
> > On 5 Oct 2021, at 13:28, Sergey Kaplun <skaplun@tarantool.org> wrote:
> > 
> > From: Mike Pall <mike>
> > 
> > Reported by Changochen.
> > 
> > (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
> > 
> > Additional stack traversal is needed to find an error function set for
>    ^^^                          ^^^
> Additional to what?            
>  				

I mean additional to stack traversal during error raising to find
protected frame, but it is confusing, so I rephrase it as you suggested.

> > handling runtime errors. cframe unwinding is missed for a C protected
> > frame during this stack traversal.
> 
> I would rephrase - 
> A cframe unwinding is miseed for a C protected frame during a serach for 
> an error function to handle a runtime error. 
> 
> > It leads to undefined behaviour or
> > crash, when raising a runtime error on stack with the CP frame before an
> > error function handler (for example, an error in __gc handler).
> > 
> > This patch adds missing unwinding for CP frame.
> > 
> > Sergey Kaplun:
> > * added the description and the test for the problem
> > ---

The new commit message is the following:

| Fix frame traversal for __gc handler frames.
|
| Reported by Changochen.
|
| (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
|
| A cframe unwinding is missed for a C protected frame during a search for
| an error function to handle a runtime error. It leads to undefined
| behaviour or crash, when raising a runtime error on stack with the CP
| frame before an error function handler (for example, an error in __gc
| handler).
|
| This patch adds missing unwinding for CP frame.
|
| Sergey Kaplun:
| * added the description and the test for the problem

Branch is force-pushed.

> > 
> > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
> > Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
> > LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601
> > 
> > src/lj_err.c                                  |  1 +
> > .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
> > 2 files changed, 26 insertions(+)
> > create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > 
> > diff --git a/src/lj_err.c b/src/lj_err.c
> > index b6be357e..b520b3d3 100644
> > --- a/src/lj_err.c
> > +++ b/src/lj_err.c
> > @@ -585,6 +585,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
> >       if (cframe_canyield(cf)) return 0;
> >       if (cframe_errfunc(cf) >= 0)
> > 	return cframe_errfunc(cf);
> > +      cf = cframe_prev(cf);
> >       frame = frame_prevd(frame);
> >       break;
> >     case FRAME_PCALL:
> > diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > new file mode 100644
> > index 00000000..d8d79100
> > --- /dev/null
> > +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> 
> Unfortunately the test passes on the ’tarantool’ branch 
> 
> s-ostanevich:luajit s.ostanevich$ git checkout tarantool
> Switched to branch 'tarantool'
> s-ostanevich:luajit s.ostanevich$ git clean -xdff
> […]
> s-ostanevich:luajit s.ostanevich$ cmake .
> […]
> s-ostanevich:luajit s.ostanevich$ make
> […]
> [100%] Built target libluajit_shared
> [100%] Built target libluajit
> [100%] Built target luajit
> s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
> s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
> s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
> TAP version 13
> 1..1
> ok - successfully collectgarbage with error

Wild guess: it doesn't fail on Mac due to GC64 ;).
See CI [1] to check my hypothesis.

> 
> > @@ -0,0 +1,25 @@
> > +local tap = require('tap')
> > +
> > +local test = tap.test('lj-601-fix-gc-finderrfunc')
> > +test:plan(1)
> > +
> > +-- Test file to demonstrate LuaJIT incorrect behaviour, when
> > +-- throwing error in __gc finalizer.
> > +-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
> > +
> > +collectgarbage()
> > +
> > +local a = newproxy(true)
> > +getmetatable(a).__gc = function()
> > +  -- Function to raise error via `lj_err_run()` inside __gc.
> > +  local _ = load(function() collectgarbage()() end)
> > +end
> > +
> > +-- XXX: Generate a small bunch of proxies. Need several to call
> > +-- `collectgarbage()` on another proxy inside __gc. N cycles is
> > +-- empirical number.
> > +for _ = 1, 4 do newproxy(a) end
> > +collectgarbage('collect')
> > +
> > +test:ok(true, 'successfully collectgarbage with error')
> > +os.exit(test:check() and 0 or 1)
> > -- 
> > 2.31.0
> > 
> 

[1]: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc-no-fix

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[sergos via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 5494 bytes --]

Hi!

You were absolutely right on my MacOS testing.
With your fixes applied - LGTM.

Sergos


> On 8 Oct 2021, at 11:39, Sergey Kaplun <skaplun@tarantool.org> wrote:
> 
> Hi, Sergos!
> 
> Thanks for the review!
> 
> On 07.10.21, sergos wrote:
>> Hi!
>> Thanks for the patch! See my 2 comments below.
>> 
>> Sergos
>> 
>>> On 5 Oct 2021, at 13:28, Sergey Kaplun <skaplun@tarantool.org> wrote:
>>> 
>>> From: Mike Pall <mike>
>>> 
>>> Reported by Changochen.
>>> 
>>> (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
>>> 
>>> Additional stack traversal is needed to find an error function set for
>>   ^^^                          ^^^
>> Additional to what?            
>> 				
> 
> I mean additional to stack traversal during error raising to find
> protected frame, but it is confusing, so I rephrase it as you suggested.
> 
>>> handling runtime errors. cframe unwinding is missed for a C protected
>>> frame during this stack traversal.
>> 
>> I would rephrase - 
>> A cframe unwinding is miseed for a C protected frame during a serach for 
>> an error function to handle a runtime error. 
>> 
>>> It leads to undefined behaviour or
>>> crash, when raising a runtime error on stack with the CP frame before an
>>> error function handler (for example, an error in __gc handler).
>>> 
>>> This patch adds missing unwinding for CP frame.
>>> 
>>> Sergey Kaplun:
>>> * added the description and the test for the problem
>>> ---
> 
> The new commit message is the following:
> 
> | Fix frame traversal for __gc handler frames.
> |
> | Reported by Changochen.
> |
> | (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
> |
> | A cframe unwinding is missed for a C protected frame during a search for
> | an error function to handle a runtime error. It leads to undefined
> | behaviour or crash, when raising a runtime error on stack with the CP
> | frame before an error function handler (for example, an error in __gc
> | handler).
> |
> | This patch adds missing unwinding for CP frame.
> |
> | Sergey Kaplun:
> | * added the description and the test for the problem
> 
> Branch is force-pushed.
> 
>>> 
>>> Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
>>> Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
>>> LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601
>>> 
>>> src/lj_err.c                                  |  1 +
>>> .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
>>> 2 files changed, 26 insertions(+)
>>> create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
>>> 
>>> diff --git a/src/lj_err.c b/src/lj_err.c
>>> index b6be357e..b520b3d3 100644
>>> --- a/src/lj_err.c
>>> +++ b/src/lj_err.c
>>> @@ -585,6 +585,7 @@ static ptrdiff_t finderrfunc(lua_State *L)
>>>      if (cframe_canyield(cf)) return 0;
>>>      if (cframe_errfunc(cf) >= 0)
>>> 	return cframe_errfunc(cf);
>>> +      cf = cframe_prev(cf);
>>>      frame = frame_prevd(frame);
>>>      break;
>>>    case FRAME_PCALL:
>>> diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
>>> new file mode 100644
>>> index 00000000..d8d79100
>>> --- /dev/null
>>> +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
>> 
>> Unfortunately the test passes on the ’tarantool’ branch 
>> 
>> s-ostanevich:luajit s.ostanevich$ git checkout tarantool
>> Switched to branch 'tarantool'
>> s-ostanevich:luajit s.ostanevich$ git clean -xdff
>> […]
>> s-ostanevich:luajit s.ostanevich$ cmake .
>> […]
>> s-ostanevich:luajit s.ostanevich$ make
>> […]
>> [100%] Built target libluajit_shared
>> [100%] Built target libluajit
>> [100%] Built target luajit
>> s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
>> s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
>> s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
>> TAP version 13
>> 1..1
>> ok - successfully collectgarbage with error
> 
> Wild guess: it doesn't fail on Mac due to GC64 ;).
> See CI [1] to check my hypothesis.
> 
>> 
>>> @@ -0,0 +1,25 @@
>>> +local tap = require('tap')
>>> +
>>> +local test = tap.test('lj-601-fix-gc-finderrfunc')
>>> +test:plan(1)
>>> +
>>> +-- Test file to demonstrate LuaJIT incorrect behaviour, when
>>> +-- throwing error in __gc finalizer.
>>> +-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
>>> +
>>> +collectgarbage()
>>> +
>>> +local a = newproxy(true)
>>> +getmetatable(a).__gc = function()
>>> +  -- Function to raise error via `lj_err_run()` inside __gc.
>>> +  local _ = load(function() collectgarbage()() end)
>>> +end
>>> +
>>> +-- XXX: Generate a small bunch of proxies. Need several to call
>>> +-- `collectgarbage()` on another proxy inside __gc. N cycles is
>>> +-- empirical number.
>>> +for _ = 1, 4 do newproxy(a) end
>>> +collectgarbage('collect')
>>> +
>>> +test:ok(true, 'successfully collectgarbage with error')
>>> +os.exit(test:check() and 0 or 1)
>>> -- 
>>> 2.31.0
>>> 
>> 
> 
> [1]: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc-no-fix <https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc-no-fix>
> 
> -- 
> Best regards,
> Sergey Kaplun


[-- Attachment #2: Type: text/html, Size: 35435 bytes --]

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Igor Munkin via Tarantool-patches]

Sergey,

Thanks for the patch! Please, consider my comments below.

On 08.10.21, Sergey Kaplun wrote:
> Hi, Sergos!
> 
> Thanks for the review!
> 

<snipped>

> 
> The new commit message is the following:
> 
> | Fix frame traversal for __gc handler frames.
> |
> | Reported by Changochen.
> |
> | (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
> |
> | A cframe unwinding is missed for a C protected frame during a search for
> | an error function to handle a runtime error. It leads to undefined
> | behaviour or crash, when raising a runtime error on stack with the CP
> | frame before an error function handler (for example, an error in __gc
> | handler).
> |
> | This patch adds missing unwinding for CP frame.
> |
> | Sergey Kaplun:
> | * added the description and the test for the problem
> 
> Branch is force-pushed.
> 
> > > 
> > > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
> > > Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
> > > LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601
> > > 
> > > src/lj_err.c                                  |  1 +
> > > .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
> > > 2 files changed, 26 insertions(+)
> > > create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > 

<snipped>

> > > diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > new file mode 100644
> > > index 00000000..d8d79100
> > > --- /dev/null
> > > +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > 
> > Unfortunately the test passes on the ’tarantool’ branch 
> > 
> > s-ostanevich:luajit s.ostanevich$ git checkout tarantool
> > Switched to branch 'tarantool'
> > s-ostanevich:luajit s.ostanevich$ git clean -xdff
> > […]
> > s-ostanevich:luajit s.ostanevich$ cmake .
> > […]
> > s-ostanevich:luajit s.ostanevich$ make
> > […]
> > [100%] Built target libluajit_shared
> > [100%] Built target libluajit
> > [100%] Built target luajit
> > s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
> > s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
> > s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
> > TAP version 13
> > 1..1
> > ok - successfully collectgarbage with error
> 
> Wild guess: it doesn't fail on Mac due to GC64 ;).
> See CI [1] to check my hypothesis.

Is it possible to fix the test chunk making it check the error even with
GC64 enabled?

> 
> > 
> > > @@ -0,0 +1,25 @@
> > > +local tap = require('tap')
> > > +
> > > +local test = tap.test('lj-601-fix-gc-finderrfunc')
> > > +test:plan(1)
> > > +
> > > +-- Test file to demonstrate LuaJIT incorrect behaviour, when
> > > +-- throwing error in __gc finalizer.
> > > +-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
> > > +
> > > +collectgarbage()
> > > +
> > > +local a = newproxy(true)
> > > +getmetatable(a).__gc = function()
> > > +  -- Function to raise error via `lj_err_run()` inside __gc.

What does exactly raise an error in this function?

> > > +  local _ = load(function() collectgarbage()() end)
> > > +end
> > > +
> > > +-- XXX: Generate a small bunch of proxies. Need several to call
> > > +-- `collectgarbage()` on another proxy inside __gc. N cycles is
> > > +-- empirical number.

Why do you even need this loop? Why can't you just assign nil to <a>?

> > > +for _ = 1, 4 do newproxy(a) end
> > > +collectgarbage('collect')
> > > +
> > > +test:ok(true, 'successfully collectgarbage with error')

Minor: I propose to reword the following way:
| error in __gc is successfully handled

> > > +os.exit(test:check() and 0 or 1)
> > > -- 
> > > 2.31.0
> > > 
> > 
> 
> [1]: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc-no-fix
> 
> -- 
> Best regards,
> Sergey Kaplun

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Sergey Kaplun via Tarantool-patches]

Igor,

Thanks for the review!

On 02.11.21, Igor Munkin wrote:
> Sergey,
> 
> Thanks for the patch! Please, consider my comments below.
> 
> On 08.10.21, Sergey Kaplun wrote:
> > Hi, Sergos!
> > 
> > Thanks for the review!
> > 
> 
> <snipped>
> 
> > 
> > The new commit message is the following:
> > 
> > | Fix frame traversal for __gc handler frames.
> > |
> > | Reported by Changochen.
> > |
> > | (cherry picked from 53f82e6e2e858a0a62fd1a2ff47e9866693382e6)
> > |
> > | A cframe unwinding is missed for a C protected frame during a search for
> > | an error function to handle a runtime error. It leads to undefined
> > | behaviour or crash, when raising a runtime error on stack with the CP
> > | frame before an error function handler (for example, an error in __gc
> > | handler).
> > |
> > | This patch adds missing unwinding for CP frame.
> > |
> > | Sergey Kaplun:
> > | * added the description and the test for the problem
> > 
> > Branch is force-pushed.
> > 
> > > > 
> > > > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-601-fix-gc-finderrfunc
> > > > Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc
> > > > LuaJIT issue: https://github.com/LuaJIT/LuaJIT/issues/601
> > > > 
> > > > src/lj_err.c                                  |  1 +
> > > > .../lj-601-fix-gc-finderrfunc.test.lua        | 25 +++++++++++++++++++
> > > > 2 files changed, 26 insertions(+)
> > > > create mode 100644 test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > > 
> 
> <snipped>
> 
> > > > diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > > new file mode 100644
> > > > index 00000000..d8d79100
> > > > --- /dev/null
> > > > +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > 
> > > Unfortunately the test passes on the ’tarantool’ branch 
> > > 
> > > s-ostanevich:luajit s.ostanevich$ git checkout tarantool
> > > Switched to branch 'tarantool'
> > > s-ostanevich:luajit s.ostanevich$ git clean -xdff
> > > […]
> > > s-ostanevich:luajit s.ostanevich$ cmake .
> > > […]
> > > s-ostanevich:luajit s.ostanevich$ make
> > > […]
> > > [100%] Built target libluajit_shared
> > > [100%] Built target libluajit
> > > [100%] Built target luajit
> > > s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
> > > s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
> > > s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
> > > TAP version 13
> > > 1..1
> > > ok - successfully collectgarbage with error
> > 
> > Wild guess: it doesn't fail on Mac due to GC64 ;).
> > See CI [1] to check my hypothesis.
> 
> Is it possible to fix the test chunk making it check the error even with
> GC64 enabled?

Yes, may be, but it is hard to forecast its behaviour:
We need to generate some garbage on the C stack to be used as an errfunc
value from the unprotected C frame suggested as C protected frame (since
`cframe_prev()` unwinding is missing for C protected frame (*).

(*) I.e. C frames are the following CP|C|CP and during handling the second
CP frame in finderrfunc we return the errfunc for the unprotected C
frame.

> 
> > 
> > > 
> > > > @@ -0,0 +1,25 @@
> > > > +local tap = require('tap')
> > > > +
> > > > +local test = tap.test('lj-601-fix-gc-finderrfunc')
> > > > +test:plan(1)
> > > > +
> > > > +-- Test file to demonstrate LuaJIT incorrect behaviour, when
> > > > +-- throwing error in __gc finalizer.
> > > > +-- See also, https://github.com/LuaJIT/LuaJIT/issues/601.
> > > > +
> > > > +collectgarbage()
> > > > +
> > > > +local a = newproxy(true)
> > > > +getmetatable(a).__gc = function()
> > > > +  -- Function to raise error via `lj_err_run()` inside __gc.
> 
> What does exactly raise an error in this function?

`collectgarbage()` returns number and attemt to call this number raises
an error.

> 
> > > > +  local _ = load(function() collectgarbage()() end)
> > > > +end
> > > > +
> > > > +-- XXX: Generate a small bunch of proxies. Need several to call
> > > > +-- `collectgarbage()` on another proxy inside __gc. N cycles is
> > > > +-- empirical number.
> 
> Why do you even need this loop? Why can't you just assign nil to <a>?

We need to create a chain with C|CP frames to use errfunction value
(that is garbage) from C frame as it was from C protected frame.
Add the corresponding comment.

See the iterative patch below. Branch is force-pushed.

> 
> > > > +for _ = 1, 4 do newproxy(a) end
> > > > +collectgarbage('collect')
> > > > +
> > > > +test:ok(true, 'successfully collectgarbage with error')
> 
> Minor: I propose to reword the following way:
> | error in __gc is successfully handled

Fixed.

Side note: 4 cycles is not enough sometimes (test is flaky) -- change
them to 6. See no false positives oks after the changes without fix.

===================================================================
diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
index d8d79100..38248c5b 100644
--- a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
+++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
@@ -16,10 +16,12 @@ getmetatable(a).__gc = function()
 end
 
 -- XXX: Generate a small bunch of proxies. Need several to call
--- `collectgarbage()` on another proxy inside __gc. N cycles is
--- empirical number.
-for _ = 1, 4 do newproxy(a) end
+-- `collectgarbage()` on another proxy inside __gc to get mix from
+-- C and C protected frames (we call `collectgarbage()` that
+-- starts finalizer in __gc metamethod which calls
+-- `collectgarbage()` again). N cycles is empirical number.
+for _ = 1, 6 do newproxy(a) end
 collectgarbage('collect')
 
-test:ok(true, 'successfully collectgarbage with error')
+test:ok(true, 'error in __gc is successfully handled')
 os.exit(test:check() and 0 or 1)
===================================================================

> 
> > > > +os.exit(test:check() and 0 or 1)
> > > > -- 
> > > > 2.31.0
> > > > 
> > > 
> > 
> > [1]: https://github.com/tarantool/tarantool/tree/skaplun/gh-noticket-fix-gc-finderrfunc-no-fix
> > 
> > -- 
> > Best regards,
> > Sergey Kaplun
> 
> -- 
> Best regards,
> IM

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit] Fix frame traversal for __gc handler frames.

[Igor Munkin via Tarantool-patches]

Sergey has already sent v2[1], but I decided to leave a few notes for the
history here.

On 08.11.21, Sergey Kaplun wrote:
> Igor,
> 
> Thanks for the review!
> 
> On 02.11.21, Igor Munkin wrote:

<snipped>

> > > > > diff --git a/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > > > new file mode 100644
> > > > > index 00000000..d8d79100
> > > > > --- /dev/null
> > > > > +++ b/test/tarantool-tests/lj-601-fix-gc-finderrfunc.test.lua
> > > > 
> > > > Unfortunately the test passes on the ’tarantool’ branch 
> > > > 
> > > > s-ostanevich:luajit s.ostanevich$ git checkout tarantool
> > > > Switched to branch 'tarantool'
> > > > s-ostanevich:luajit s.ostanevich$ git clean -xdff
> > > > […]
> > > > s-ostanevich:luajit s.ostanevich$ cmake .
> > > > […]
> > > > s-ostanevich:luajit s.ostanevich$ make
> > > > […]
> > > > [100%] Built target libluajit_shared
> > > > [100%] Built target libluajit
> > > > [100%] Built target luajit
> > > > s-ostanevich:luajit s.ostanevich$ git checkout   skaplun/lj-601-fix-gc-finderrfunc
> > > > s-ostanevich:luajit s.ostanevich$ cd test/tarantool-tests
> > > > s-ostanevich:tarantool-tests s.ostanevich$ ../../src/luajit lj-601-fix-gc-finderrfunc.test.lua 
> > > > TAP version 13
> > > > 1..1
> > > > ok - successfully collectgarbage with error
> > > 
> > > Wild guess: it doesn't fail on Mac due to GC64 ;).
> > > See CI [1] to check my hypothesis.
> > 
> > Is it possible to fix the test chunk making it check the error even with
> > GC64 enabled?
> 
> Yes, may be, but it is hard to forecast its behaviour:
> We need to generate some garbage on the C stack to be used as an errfunc
> value from the unprotected C frame suggested as C protected frame (since
> `cframe_prev()` unwinding is missing for C protected frame (*).
> 
> (*) I.e. C frames are the following CP|C|CP and during handling the second
> CP frame in finderrfunc we return the errfunc for the unprotected C
> frame.

In v2[1] Sergey provided a test where the desired stack layout is made
via Lua C API and garbage is generated with the area allocated on the
host stack. As a result the test also unconditionally fails with GC64
mode enabled (when the patch is not applied, obviously).

> 

<snipped>

> > > > > +getmetatable(a).__gc = function()
> > > > > +  -- Function to raise error via `lj_err_run()` inside __gc.
> > 
> > What does exactly raise an error in this function?
> 
> `collectgarbage()` returns number and attemt to call this number raises
> an error.

PEBKAC, I totally forgot what <load> function does :)

> 
> > 
> > > > > +  local _ = load(function() collectgarbage()() end)
> > > > > +end
> > > > > +
> > > > > +-- XXX: Generate a small bunch of proxies. Need several to call
> > > > > +-- `collectgarbage()` on another proxy inside __gc. N cycles is
> > > > > +-- empirical number.
> > 
> > Why do you even need this loop? Why can't you just assign nil to <a>?
> 
> We need to create a chain with C|CP frames to use errfunction value
> (that is garbage) from C frame as it was from C protected frame.
> Add the corresponding comment.

Again, this chain is created with no loop at all but via Lua C in v2[1].

> 

<snipped>

> 
> Side note: 4 cycles is not enough sometimes (test is flaky) -- change
> them to 6. See no false positives oks after the changes without fix.

AFAICS this value can vary from 4 to 10000 depending on the number of
garbage to be collected. This was the main reason to reimplement the
test, since it was more a reproducer rather than a good test.

> 

<snipped>

> 
> -- 
> Best regards,
> Sergey Kaplun

[1]: https://lists.tarantool.org/tarantool-patches/20211119164157.18344-1-skaplun@tarantool.org/

-- 
Best regards,
IM