[Tarantool-patches] [PATCH luajit] FFI: Fix pointer difference operation on 64 bit platforms.

[Tarantool-patches] [PATCH luajit] FFI: Fix pointer difference operation on 64 bit platforms.

[Sergey Kaplun via Tarantool-patches]

From: Mike Pall <mike>

Thanks to cuiweixie.

(cherry picked from commit b58b07189521e82c7a8e8bc43fc3c271f89832fd)

The metamethod for FFI pointer subtraction casts the result value to a
32-bit value, which leads to invalid results.

This patch removes the excess cast.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#12880
---

Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1449-fix-ptr-diff-64-bit
Related issues:
* https://github.com/LuaJIT/LuaJIT/issues/1449
* https://github.com/tarantool/tarantool/issues/12880

 src/lj_carith.c                               |  2 +-
 .../lj-1449-fix-ptr-diff-64-bit.test.lua      | 39 +++++++++++++++++++
 2 files changed, 40 insertions(+), 1 deletion(-)
 create mode 100644 test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua

diff --git a/src/lj_carith.c b/src/lj_carith.c
index eb56d552..3384c2cd 100644
--- a/src/lj_carith.c
+++ b/src/lj_carith.c
@@ -120,7 +120,7 @@ static int carith_ptr(lua_State *L, CTState *cts, CDArith *ca, MMS mm)
 	/* All valid pointer differences on x64 are in (-2^47, +2^47),
 	** which fits into a double without loss of precision.
 	*/
-	setintptrV(L->top-1, (int32_t)diff);
+	setintptrV(L->top-1, diff);
 	return 1;
       } else if (mm == MM_lt) {  /* Pointer comparison (unsigned). */
 	setboolV(L->top-1, ((uintptr_t)pp < (uintptr_t)pp2));
diff --git a/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
new file mode 100644
index 00000000..032b9b4d
--- /dev/null
+++ b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
@@ -0,0 +1,39 @@
+local tap = require('tap')
+
+-- Test file to demonstrate LuaJIT's incorrect 64-bit pointer
+-- subtraction.
+-- See also: https://github.com/LuaJIT/LuaJIT/issues/1449.
+
+local test = tap.test('lj-1449-fix-ptr-diff-64-bit')
+
+local ffi = require('ffi')
+
+test:plan(2)
+
+local diff = 0x80000001ULL
+local base = 0x700000000000ULL
+local p0 = ffi.cast('char *', base)
+local p1 = ffi.cast('char *', base + diff)
+
+test:is(p1 - p0, diff, 'correct pointer difference between 64-bit pointers')
+
+test:skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})
+
+local results = {}
+
+jit.opt.start('hotloop=1')
+
+for i = 1, 4 do
+  -- Use constants on trace.
+  local delta = 0x80000001ULL
+  local b = 0x700000000000ULL
+  local pt0 = ffi.cast('char *', b)
+  local pt1 = ffi.cast('char *', b + delta)
+  results[i] = pt1 - pt0
+end
+
+test:samevalues(results, 'consistent JIT and VM behaviour for ptr subtraction')
+
+test:done(true)
-- 
2.55.0

Re: [Tarantool-patches] [PATCH luajit] FFI: Fix pointer difference operation on 64 bit platforms.

[Sergey Bronnikov via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 2862 bytes --]

Hi, Sergey,

thanks for the patch! LGTM

Sergey

On 7/20/26 17:24, Sergey Kaplun wrote:
> From: Mike Pall <mike>
>
> Thanks to cuiweixie.
>
> (cherry picked from commit b58b07189521e82c7a8e8bc43fc3c271f89832fd)
>
> The metamethod for FFI pointer subtraction casts the result value to a
> 32-bit value, which leads to invalid results.
>
> This patch removes the excess cast.
>
> Sergey Kaplun:
> * added the description and the test for the problem
>
> Part of tarantool/tarantool#12880
> ---
>
> Branch:https://github.com/tarantool/luajit/tree/skaplun/lj-1449-fix-ptr-diff-64-bit
> Related issues:
> *https://github.com/LuaJIT/LuaJIT/issues/1449
> *https://github.com/tarantool/tarantool/issues/12880
>
>   src/lj_carith.c                               |  2 +-
>   .../lj-1449-fix-ptr-diff-64-bit.test.lua      | 39 +++++++++++++++++++
>   2 files changed, 40 insertions(+), 1 deletion(-)
>   create mode 100644 test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
>
> diff --git a/src/lj_carith.c b/src/lj_carith.c
> index eb56d552..3384c2cd 100644
> --- a/src/lj_carith.c
> +++ b/src/lj_carith.c
> @@ -120,7 +120,7 @@ static int carith_ptr(lua_State *L, CTState *cts, CDArith *ca, MMS mm)
>   	/* All valid pointer differences on x64 are in (-2^47, +2^47),
>   	** which fits into a double without loss of precision.
>   	*/
> -	setintptrV(L->top-1, (int32_t)diff);
> +	setintptrV(L->top-1, diff);
>   	return 1;
>         } else if (mm == MM_lt) {  /* Pointer comparison (unsigned). */
>   	setboolV(L->top-1, ((uintptr_t)pp < (uintptr_t)pp2));
> diff --git a/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> new file mode 100644
> index 00000000..032b9b4d
> --- /dev/null
> +++ b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> @@ -0,0 +1,39 @@
> +local tap = require('tap')
> +
> +-- Test file to demonstrate LuaJIT's incorrect 64-bit pointer
> +-- subtraction.
> +-- See also:https://github.com/LuaJIT/LuaJIT/issues/1449.
> +
> +local test = tap.test('lj-1449-fix-ptr-diff-64-bit')
> +
> +local ffi = require('ffi')
> +
> +test:plan(2)
> +
> +local diff = 0x80000001ULL
> +local base = 0x700000000000ULL
> +local p0 = ffi.cast('char *', base)
> +local p1 = ffi.cast('char *', base + diff)
> +
> +test:is(p1 - p0, diff, 'correct pointer difference between 64-bit pointers')
> +
> +test:skipcond({
> +  ['Test requires JIT enabled'] = not jit.status(),
> +})
> +
> +local results = {}
> +
> +jit.opt.start('hotloop=1')
> +
> +for i = 1, 4 do
> +  -- Use constants on trace.
> +  local delta = 0x80000001ULL
> +  local b = 0x700000000000ULL
> +  local pt0 = ffi.cast('char *', b)
> +  local pt1 = ffi.cast('char *', b + delta)
> +  results[i] = pt1 - pt0
> +end
> +
> +test:samevalues(results, 'consistent JIT and VM behaviour for ptr subtraction')
> +
> +test:done(true)

[-- Attachment #2: Type: text/html, Size: 3573 bytes --]

Re: [Tarantool-patches] [PATCH luajit] FFI: Fix pointer difference operation on 64 bit platforms.

[Evgeniy Temirgaleev via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 3241 bytes --]

Hi, Sergey! Thanks for the patch!

LGTM with a note on a test, please see.
--
Best regards,
Evgeniy Temirgaleev

> 
> From: Sergey Kaplun <skaplun@tarantool.org>
> To: Sergey Bronnikov <sergeyb@tarantool.org>, Evgeniy Temirgaleev <e.temirgaleev@tarantool.org
> >
> Cc: tarantool-patches@dev.tarantool.org, Sergey Kaplun <skaplun@tarantool.org
> >
> Date: Monday, July 20, 2026 5:24 PM +03:00
> From: Mike Pall <mike>
> 
> Thanks to cuiweixie.
> 
> (cherry picked from commit b58b07189521e82c7a8e8bc43fc3c271f89832fd)
> 
> The metamethod for FFI pointer subtraction casts the result value to a
> 32-bit value, which leads to invalid results.
> 
> This patch removes the excess cast.
> 
> Sergey Kaplun:
> * added the description and the test for the problem
> 
> Part of tarantool/tarantool#12880
> ---
> 
> Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1449-fix-ptr-diff-64-bit
> 
> Related issues:
> * https://github.com/LuaJIT/LuaJIT/issues/1449
> * https://github.com/tarantool/tarantool/issues/12880
> 
> src/lj_carith.c | 2 +-
> .../lj-1449-fix-ptr-diff-64-bit.test.lua | 39 +++++++++++++++++++
> 2 files changed, 40 insertions(+), 1 deletion(-)
> create mode 100644
> test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> 
> diff --git a/src/lj_carith.c b/src/lj_carith.c
> index eb56d552..3384c2cd 100644
> --- a/src/lj_carith.c
> +++ b/src/lj_carith.c
> @@ -120,7 +120,7 @@ static int carith_ptr(lua_State *L, CTState *cts,
> CDArith *ca, MMS mm)
> /* All valid pointer differences on x64 are in (-2^47, +2^47),
> ** which fits into a double without loss of precision.
> */
> - setintptrV(L->top-1, (int32_t)diff);
> + setintptrV(L->top-1, diff);
> return 1;
> } else if (mm == MM_lt) { /* Pointer comparison (unsigned). */
> setboolV(L->top-1, ((uintptr_t)pp < (uintptr_t)pp2));
> diff --git a/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> new file mode 100644
> index 00000000..032b9b4d
> --- /dev/null
> +++ b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> @@ -0,0 +1,39 @@
> +local tap = require('tap')
> +
> +-- Test file to demonstrate LuaJIT's incorrect 64-bit pointer
> +-- subtraction.
> +-- See also: https://github.com/LuaJIT/LuaJIT/issues/1449.
> +
> +local test = tap.test('lj-1449-fix-ptr-diff-64-bit')
> +
> +local ffi = require('ffi')
> +
> +test:plan(2)
> +
> +local diff = 0x80000001ULL
> +local base = 0x700000000000ULL
> +local p0 = ffi.cast('char *', base)
> +local p1 = ffi.cast('char *', base + diff)
> +
> +test:is(p1 - p0, diff, 'correct pointer difference between 64-bit
> pointers')
> +
> +test:skipcond({
> + ['Test requires JIT enabled'] = not jit.status(),
> +})
> +
> +local results = {}
> 

> 
> +
> +jit.opt.start('hotloop=1')
> +
> +for i = 1, 4 do
> + -- Use constants on trace.
> + local delta = 0x80000001ULL
> + local b = 0x700000000000ULL
> + local pt0 = ffi.cast('char *', b)
> + local pt1 = ffi.cast('char *', b + delta)
> + results[i] = pt1 - pt0
> +end
> +
> 

It seems, we need to add a correct sample to the results, e.g. table.insert(results, diff)
> 
> 
> +test:samevalues(results, 'consistent JIT and VM behaviour for ptr
> subtraction')
> +
> +test:done(true)
> --
> 2.55.0
>

[-- Attachment #2: Type: text/html, Size: 4910 bytes --]

Re: [Tarantool-patches] [PATCH luajit] FFI: Fix pointer difference operation on 64 bit platforms.

[Sergey Kaplun via Tarantool-patches]

Hi, Evgeniy!
Thanks for the review!
Please consider my answer below.

On 28.07.26, Evgeniy Temirgaleev wrote:
> Hi, Sergey! Thanks for the patch!
> 
> LGTM with a note on a test, please see.
> --
> Best regards,
> Evgeniy Temirgaleev
> 
> > 
> > From: Sergey Kaplun <skaplun@tarantool.org>
> > To: Sergey Bronnikov <sergeyb@tarantool.org>, Evgeniy Temirgaleev <e.temirgaleev@tarantool.org
> > >
> > Cc: tarantool-patches@dev.tarantool.org, Sergey Kaplun <skaplun@tarantool.org
> > >
> > Date: Monday, July 20, 2026 5:24 PM +03:00
> > From: Mike Pall <mike>
> > 
> > Thanks to cuiweixie.
> > 
> > (cherry picked from commit b58b07189521e82c7a8e8bc43fc3c271f89832fd)
> > 
> > The metamethod for FFI pointer subtraction casts the result value to a
> > 32-bit value, which leads to invalid results.
> > 
> > This patch removes the excess cast.
> > 
> > Sergey Kaplun:
> > * added the description and the test for the problem
> > 
> > Part of tarantool/tarantool#12880
> > ---
> > 
> > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1449-fix-ptr-diff-64-bit
> > 
> > Related issues:
> > * https://github.com/LuaJIT/LuaJIT/issues/1449
> > * https://github.com/tarantool/tarantool/issues/12880
> > 
> > src/lj_carith.c | 2 +-
> > .../lj-1449-fix-ptr-diff-64-bit.test.lua | 39 +++++++++++++++++++
> > 2 files changed, 40 insertions(+), 1 deletion(-)
> > create mode 100644
> > test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> > 
> > diff --git a/src/lj_carith.c b/src/lj_carith.c
> > index eb56d552..3384c2cd 100644
> > --- a/src/lj_carith.c
> > +++ b/src/lj_carith.c

<snipped>

> > diff --git a/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> > b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua
> > new file mode 100644
> > index 00000000..032b9b4d
> > --- /dev/null
> > +++ b/test/tarantool-tests/lj-1449-fix-ptr-diff-64-bit.test.lua

<snipped>

> > +jit.opt.start('hotloop=1')
> > +
> > +for i = 1, 4 do
> > + -- Use constants on trace.
> > + local delta = 0x80000001ULL
> > + local b = 0x700000000000ULL
> > + local pt0 = ffi.cast('char *', b)
> > + local pt1 = ffi.cast('char *', b + delta)
> > + results[i] = pt1 - pt0
> > +end
> > +
> > 
> 
> It seems, we need to add a correct sample to the results, e.g. table.insert(results, diff)

It is not necessary. The correctness of the VM is checked in the
previous test case. Here we check only consistency between the VM and
JIT. So, if JIT behaviour is different the test will fail (also, it
means that the JIT result is incorrect, since it doesn't match the
correct VM result tested before).

Also, if one runs this test without the patch applied, the assertion in
`record_check_slots()` is raised, since the resulting value after
folding is not the same as given by the VM.

> > +test:samevalues(results, 'consistent JIT and VM behaviour for ptr
> > subtraction')
> > +
> > +test:done(true)
> > --
> > 2.55.0
> >

-- 
Best regards,
Sergey Kaplun