[Tarantool-patches] [PATCH luajit] FFI: Add tonumber() specialization for failed conversions.

[Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

From: Mike Pall <mike>

Contributed by Javier Guerra Giraldez.

(cherry picked from commit 02b521981a1ab919ff2cd4d9bcaee80baf77dce2)

When `tonumber()` is recorded (as a part of a trace) for cdata argument
can't be converted to number the `nil` value is recorded as the yielded
result. But without special check on trace for cdata type this nil will
be returned for another type of cdata that can be converted.

This patch adds the corresponding check for recoding of failed cdata
conversions.

Sergey Kaplun:
* added the description and the test for the problem

Resolves tarantool/tarantool#7655
Part of tarantool/tarantool#7230
---

Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-408-tonumber-cdata-record-full-ci
Issues and PRs:
* https://github.com/tarantool/tarantool/issues/7655
* https://github.com/tarantool/tarantool/issues/7230
* https://github.com/LuaJIT/LuaJIT/issues/408
* https://github.com/LuaJIT/LuaJIT/pull/412
Tarantool PR: https://github.com/tarantool/tarantool/pull/7668

 src/lj_crecord.c                              |  2 +
 .../lj-408-tonumber-cdata-record.test.lua     | 44 +++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 test/tarantool-tests/lj-408-tonumber-cdata-record.test.lua

diff --git a/src/lj_crecord.c b/src/lj_crecord.c
index 0d7b71f0..32c767e3 100644
--- a/src/lj_crecord.c
+++ b/src/lj_crecord.c
@@ -1895,6 +1895,8 @@ void LJ_FASTCALL lj_crecord_tonumber(jit_State *J, RecordFFData *rd)
       d = ctype_get(cts, CTID_DOUBLE);
     J->base[0] = crec_ct_tv(J, d, 0, J->base[0], &rd->argv[0]);
   } else {
+    /* Specialize to the ctype that couldn't be converted. */
+    argv2cdata(J, J->base[0], &rd->argv[0]);
     J->base[0] = TREF_NIL;
   }
 }
diff --git a/test/tarantool-tests/lj-408-tonumber-cdata-record.test.lua b/test/tarantool-tests/lj-408-tonumber-cdata-record.test.lua
new file mode 100644
index 00000000..1c175de1
--- /dev/null
+++ b/test/tarantool-tests/lj-408-tonumber-cdata-record.test.lua
@@ -0,0 +1,44 @@
+local ffi = require('ffi')
+local tap = require('tap')
+
+-- Test file to demonstrate the incorrect JIT recording for
+-- `tonumber()` function with cdata argument for failed
+-- conversions.
+-- See also https://github.com/LuaJIT/LuaJIT/issues/408,
+-- https://github.com/LuaJIT/LuaJIT/pull/412,
+-- https://github.com/LuaJIT/LuaJIT/pull/412,
+-- https://github.com/tarantool/tarantool/issues/7655.
+local test = tap.test('lj-408-tonumber-cdata-record')
+
+local NULL = ffi.cast('void *', 0)
+
+test:plan(4)
+
+local function check(x)
+  -- Don't use a tail call to avoid "leaving loop in root trace"
+  -- error, so the trace will be compiled.
+  local res = tonumber(x)
+  return res
+end
+
+jit.opt.start('hotloop=1')
+-- Record `check()` with `tonumber(NULL)` -- not converted.
+check(NULL)
+check(NULL)
+
+test:ok(not check(NULL), 'recorded with NULL and not converted for NULL')
+test:ok(check(0LL), 'recorded with NULL and converted for 0LL')
+
+-- Reset JIT.
+jit.off()
+jit.flush()
+jit.on()
+
+-- Record `check()` with `tonumber(0LL)` -- converted.
+check(0LL)
+check(0LL)
+
+test:ok(check(0LL), 'recorded with 0LL and converted for 0LL')
+test:ok(not check(NULL), 'recorded with 0LL and not converted for NULL')
+
+os.exit(test:check() and 0 or 1)
-- 
2.34.1