[Tarantool-patches] [PATCH luajit 2/2] x64/!LJ_GC64: The allocation limit is required for a no-JIT build, too.

[Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

From: Mike Pall <mike>

Thanks to Sergey Kaplun.

(cherry picked from commit eff4006837792b6105e0a1743283ddde3548fc09)

For the non-GC64 build with disabled JIT, LuaJIT's internal allocator
may return 32-bit addresses. This may lead to the assertion failure
during the reallocation of the array part of the table or to the crash
(if assertions are disabled) due to an incorrect arithmetic in the x86
VM. For example, the addition with a 32-bit wide address may overflow
in TSETV or TGETV and cause the crash.

This patch sets the allocation limit for the build without JIT.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#12134
---
 src/lj_alloc.c                                |  4 +-
 .../lj-1430-internal-alloc-limit.test.lua     | 39 +++++++++++++++++++
 2 files changed, 41 insertions(+), 2 deletions(-)
 create mode 100644 test/tarantool-tests/lj-1430-internal-alloc-limit.test.lua

diff --git a/src/lj_alloc.c b/src/lj_alloc.c
index f82c9854..97eb94d2 100644
--- a/src/lj_alloc.c
+++ b/src/lj_alloc.c
@@ -99,8 +99,8 @@
 
 #if LJ_GC64
 #define LJ_ALLOC_MBITS		47	/* 128 TB in LJ_GC64 mode. */
-#elif LJ_TARGET_X64 && LJ_HASJIT
-/* Due to limitations in the x64 compiler backend. */
+#elif LJ_TARGET_X64
+/* Due to limitations in the x64 non-GC64 VM. */
 #define LJ_ALLOC_MBITS		31	/* 2 GB on x64 with !LJ_GC64. */
 #else
 #define LJ_ALLOC_MBITS		32	/* 4 GB on other archs with !LJ_GC64. */
diff --git a/test/tarantool-tests/lj-1430-internal-alloc-limit.test.lua b/test/tarantool-tests/lj-1430-internal-alloc-limit.test.lua
new file mode 100644
index 00000000..969d26d6
--- /dev/null
+++ b/test/tarantool-tests/lj-1430-internal-alloc-limit.test.lua
@@ -0,0 +1,39 @@
+local tap = require('tap')
+
+-- Test file to demonstrate incorrect allocation limit for the
+-- non-GC64 build with disabled JIT.
+-- See also: https://github.com/LuaJIT/LuaJIT/issues/1430.
+
+local test = tap.test('lj-1430-internal-alloc-limit')
+
+test:plan(1)
+
+-- This function creates a bunch of long array-like tables.
+-- Eventually for one of the tables the address of the array
+-- element will not fit in the 31-bit range, causing the incorrect
+-- arithmetic inside the VM and a crash or assertion failure
+-- during the reallocation.
+local function test_payload()
+  local POOL_SZ = 8
+  -- luacheck: no unused
+  local pools = {}
+  for i = 1, POOL_SZ do
+    pools[i] = {}
+  end
+
+  local v = 1
+  for j = 1, POOL_SZ do
+    for i = 1, 0x2000000 do
+      pools[j][i] = v
+    end
+  end
+end
+
+-- Protect the call to avoid the OOM.
+pcall(test_payload)
+
+-- Free memory for the TAP tests.
+collectgarbage()
+
+test:ok(true, 'no crash or assertion failure')
+test:done(true)
-- 
2.53.0