[Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Sergey Kaplun via Tarantool-patches]

From: Mike Pall <mike>

Reported by Sergey Kaplun.

(cherry-picked from commit 0ef51b495f9497aac77b41eb3d837c9c38b9424b)

Table `NEWREF` storage for non-constant keys also emits `FREF` IR with
`IRFL_TAB_NOMM` to invalidate the metamethod cache. When table creation
and `NEWREF` are sinked, the corresponding `FSTORE` is sinked too and
should be restored on trace exit. However, `snap_unsink()` doesn't
expect anything except `IRFL_TAB_META` as the second operand of `FREF`,
so the corresponding assertion fails.

This patch adds a switch-case statement to handle the `IRFL_TAB_NOMM`
case. Since `FREF` with `IRFL_TAB_NOMM` always follows some hash store,
we can avoid a duplication of the cache invalidation, so this case just
does nothing.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#8825
---
Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1052-unsink-with-irfl-tab-nomm
Tarantool PR: https://github.com/tarantool/tarantool/pull/9055
Related issues:
* https://github.com/LuaJIT/LuaJIT/issues/1052
* https://github.com/tarantool/tarantool/issues/8825

 src/lj_snap.c                                 | 18 +++++++---
 ...lj-1052-unsink-with-irfl-tab-nomm.test.lua | 36 +++++++++++++++++++
 2 files changed, 49 insertions(+), 5 deletions(-)
 create mode 100644 test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua

diff --git a/src/lj_snap.c b/src/lj_snap.c
index 2dc281cb..6c5e5e53 100644
--- a/src/lj_snap.c
+++ b/src/lj_snap.c
@@ -836,11 +836,19 @@ static void snap_unsink(jit_State *J, GCtrace *T, ExitState *ex,
 		   irs->o == IR_FSTORE,
 		   "sunk store with bad op %d", irs->o);
 	if (irk->o == IR_FREF) {
-	  lj_assertJ(irk->op2 == IRFL_TAB_META,
-		     "sunk store with bad field %d", irk->op2);
-	  snap_restoreval(J, T, ex, snapno, rfilt, irs->op2, &tmp);
-	  /* NOBARRIER: The table is new (marked white). */
-	  setgcref(t->metatable, obj2gco(tabV(&tmp)));
+	  switch (irk->op2) {
+	  case IRFL_TAB_META:
+	    snap_restoreval(J, T, ex, snapno, rfilt, irs->op2, &tmp);
+	    /* NOBARRIER: The table is new (marked white). */
+	    setgcref(t->metatable, obj2gco(tabV(&tmp)));
+	    break;
+	  case IRFL_TAB_NOMM:
+	    /* Negative metamethod cache invalidated by lj_tab_set() below. */
+	    break;
+	  default:
+	    lj_assertJ(0, "sunk store with bad field %d", irk->op2);
+	    break;
+	  }
 	} else {
 	  irk = &T->ir[irk->op2];
 	  if (irk->o == IR_KSLOT) irk = &T->ir[irk->op1];
diff --git a/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
new file mode 100644
index 00000000..aaf5349f
--- /dev/null
+++ b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
@@ -0,0 +1,36 @@
+local tap = require('tap')
+
+-- Test file to demonstrate LuaJIT's incorrect restoration of a
+-- table from a snapshot with the presence of `IRFL_TAB_NOMM`.
+-- See also: https://github.com/LuaJIT/LuaJIT/issues/1052.
+
+local test = tap.test('lj-1052-unsink-with-irfl-tab-nomm'):skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})
+
+test:plan(2)
+
+local TEST_VALUE = 'test'
+
+jit.opt.start('hotloop=1')
+
+local counter = 0
+local slot = 'slot'
+while true do
+  counter = counter + 1
+  -- Use a non-constant slot to emit `FREF` with `IRFL_TAB_NOMM`.
+  -- After re-emitting the variant part of the loop, NEWREF will
+  -- contain a constant key (see below).
+  slot = {[slot] = TEST_VALUE}
+  -- Emit exit here to be sure that the table will be restored
+  -- from the snapshot.
+  if counter > 2 then break end
+  -- We need a constant reference for NEWREF. Just use the old
+  -- value.
+  slot = 'slot'
+end
+
+test:is(slot.slot, TEST_VALUE, 'correct table content')
+test:ok(debug.getmetatable(slot) == nil, 'no metatable on the restored table')
+
+test:done(true)
-- 
2.42.0

Re: [Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Maxim Kokryashkin via Tarantool-patches]

Hi, Sergey!
Thanks for the patch!
LGTM.

Re: [Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Sergey Bronnikov via Tarantool-patches]

Hi, Sergey


Thanks for the patch! LGTM

See a minor comment below.


On 8/29/23 15:36, Sergey Kaplun wrote:

<snipped>

> diff --git a/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
> new file mode 100644
> index 00000000..aaf5349f
> --- /dev/null
> +++ b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
> @@ -0,0 +1,36 @@
> +local tap = require('tap')
> +
> +-- Test file to demonstrate LuaJIT's incorrect restoration of a
> +-- table from a snapshot with the presence of `IRFL_TAB_NOMM`.
> +-- See also: https://github.com/LuaJIT/LuaJIT/issues/1052.
> +
> +local test = tap.test('lj-1052-unsink-with-irfl-tab-nomm'):skipcond({
> +  ['Test requires JIT enabled'] = not jit.status(),
> +})
> +
> +test:plan(2)
> +
> +local TEST_VALUE = 'test'
> +
> +jit.opt.start('hotloop=1')
> +
> +local counter = 0
> +local slot = 'slot'
> +while true do

You are using endless loop here, but loop is limited by 3 iterations 
(break after counter > 2).

Why "while true do end" is used here? Should we left a comment here?

> +  counter = counter + 1
> +  -- Use a non-constant slot to emit `FREF` with `IRFL_TAB_NOMM`.
> +  -- After re-emitting the variant part of the loop, NEWREF will
> +  -- contain a constant key (see below).
> +  slot = {[slot] = TEST_VALUE}
> +  -- Emit exit here to be sure that the table will be restored
> +  -- from the snapshot.
> +  if counter > 2 then break end
> +  -- We need a constant reference for NEWREF. Just use the old
> +  -- value.
> +  slot = 'slot'
> +end
<snipped>

Re: [Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Sergey Kaplun via Tarantool-patches]

Hi, Sergey!
Thanks for the review!
Fixed your comment below.

On 16.09.23, Sergey Bronnikov wrote:
> Hi, Sergey
> 
> 
> Thanks for the patch! LGTM
> 
> See a minor comment below.

<snipped>

> > +jit.opt.start('hotloop=1')
> > +
> > +local counter = 0
> > +local slot = 'slot'
> > +while true do
> 
> You are using endless loop here, but loop is limited by 3 iterations 
> (break after counter > 2).
> 
> Why "while true do end" is used here? Should we left a comment here?
> 

Added the following comment, as you suggested. Branch is force-pushed.

===================================================================
diff --git a/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
index aaf5349f..eaf89501 100644
--- a/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
+++ b/test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
@@ -16,6 +16,9 @@ jit.opt.start('hotloop=1')
 
 local counter = 0
 local slot = 'slot'
+-- XXX: The loop is limited to 3 iterations to compile a trace and
+-- start to execute it. Use the `while true do` loop format to see
+-- the side effects on the restoration from the snapshot.
 while true do
   counter = counter + 1
   -- Use a non-constant slot to emit `FREF` with `IRFL_TAB_NOMM`.
===================================================================

<snipped>

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Sergey Bronnikov via Tarantool-patches]

On 9/18/23 10:56, Sergey Kaplun wrote:
> Hi, Sergey!
> Thanks for the review!
> Fixed your comment below.
>
>
Nice! LGTM

<snipped>

Re: [Tarantool-patches] [PATCH luajit] Handle table unsinking in the presence of IRFL_TAB_NOMM.

[Igor Munkin via Tarantool-patches]

Sergey,

I've checked the patchset into all long-term branches in
tarantool/luajit and bumped a new version in master, release/2.11 and
release/2.10.

On 29.08.23, Sergey Kaplun via Tarantool-patches wrote:
> From: Mike Pall <mike>
> 
> Reported by Sergey Kaplun.
> 
> (cherry-picked from commit 0ef51b495f9497aac77b41eb3d837c9c38b9424b)
> 
> Table `NEWREF` storage for non-constant keys also emits `FREF` IR with
> `IRFL_TAB_NOMM` to invalidate the metamethod cache. When table creation
> and `NEWREF` are sinked, the corresponding `FSTORE` is sinked too and
> should be restored on trace exit. However, `snap_unsink()` doesn't
> expect anything except `IRFL_TAB_META` as the second operand of `FREF`,
> so the corresponding assertion fails.
> 
> This patch adds a switch-case statement to handle the `IRFL_TAB_NOMM`
> case. Since `FREF` with `IRFL_TAB_NOMM` always follows some hash store,
> we can avoid a duplication of the cache invalidation, so this case just
> does nothing.
> 
> Sergey Kaplun:
> * added the description and the test for the problem
> 
> Part of tarantool/tarantool#8825
> ---
> Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1052-unsink-with-irfl-tab-nomm
> Tarantool PR: https://github.com/tarantool/tarantool/pull/9055
> Related issues:
> * https://github.com/LuaJIT/LuaJIT/issues/1052
> * https://github.com/tarantool/tarantool/issues/8825
> 
>  src/lj_snap.c                                 | 18 +++++++---
>  ...lj-1052-unsink-with-irfl-tab-nomm.test.lua | 36 +++++++++++++++++++
>  2 files changed, 49 insertions(+), 5 deletions(-)
>  create mode 100644 test/tarantool-tests/lj-1052-unsink-with-irfl-tab-nomm.test.lua
> 

<snipped>

> -- 
> 2.42.0
> 

-- 
Best regards,
IM