[Tarantool-patches] [PATCH luajit v2] x64: Fix 64 bit shift code generation.

[Maxim Kokryashkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

From: Mike Pall <mike>

Reported by Philipp Kutin.
Fix contributed by Peter Cawley.

(cherry-picked from commit 03a7ebca4f6819658cdaa12ba3af54a17b8035e9)

In a situation where a variable shift left bitwise rotation that
has a 64-bit result is recorded on an x86 64-bit processor and
the result is supposed to end up in the `rcx` register, that value
could be written into the `ecx` instead, thus being truncated into
32 bits. This patch fixes the described behavior, so now that
value is written into the `rcx`.

Resulting assembly changes from the following before the patch:
| rol rsi, cl
| mov ecx, esi

to the following after the patch:
| rol rsi, cl
| mov rcx, rsi

Importantly, the same behavior is impossible with the right
rotation on machines with BMI2 support because there is a
BMI2 instruction for it, so it is handled differently.

Maxim Kokryashkin:
* added the description and the test for the problem

Part of tarantool/tarantool#8516
---
Changes in v2:
- Fixed comments as per review by Sergey Kaplun
PR: https://github.com/tarantool/tarantool/pull/8727
Branch: https://github.com/tarantool/luajit/tree/fckxorg/fix-bit-shift-generation

 src/lj_asm_x86.h                              |  2 +-
 .../fix-bit-shift-generation.test.lua         | 50 +++++++++++++++++++
 2 files changed, 51 insertions(+), 1 deletion(-)
 create mode 100644 test/tarantool-tests/fix-bit-shift-generation.test.lua

diff --git a/src/lj_asm_x86.h b/src/lj_asm_x86.h
index e6c42c6d..63d332ca 100644
--- a/src/lj_asm_x86.h
+++ b/src/lj_asm_x86.h
@@ -2328,7 +2328,7 @@ static void asm_bitshift(ASMState *as, IRIns *ir, x86Shift xs, x86Op xv)
     dest = ra_dest(as, ir, rset_exclude(RSET_GPR, RID_ECX));
     if (dest == RID_ECX) {
       dest = ra_scratch(as, rset_exclude(RSET_GPR, RID_ECX));
-      emit_rr(as, XO_MOV, RID_ECX, dest);
+      emit_rr(as, XO_MOV, REX_64IR(ir, RID_ECX), dest);
     }
     right = irr->r;
     if (ra_noreg(right))
diff --git a/test/tarantool-tests/fix-bit-shift-generation.test.lua b/test/tarantool-tests/fix-bit-shift-generation.test.lua
new file mode 100644
index 00000000..9f14a9e3
--- /dev/null
+++ b/test/tarantool-tests/fix-bit-shift-generation.test.lua
@@ -0,0 +1,50 @@
+local tap = require('tap')
+local test = tap.test('fix-bit-shift-generation'):skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})
+
+local NITERATIONS = 4
+local NTESTS = NITERATIONS * 2
+
+test:plan(NTESTS)
+
+local ffi = require('ffi')
+local bit = require('bit')
+local rol = bit.rol
+local shl = bit.lshift
+
+ffi.cdef('int snprintf(char *str, size_t n, const char *format, ...);')
+
+-- Buffer size is adjsuted to fit `(1 << 36)`,
+-- which has exactly 11 digits.
+local BUFFER_SIZE = 12
+local bufs = {}
+for i = 1, NTESTS do
+  bufs[i] = ffi.new('char[?]', BUFFER_SIZE)
+end
+
+local result = {}
+jit.opt.start('hotloop=1')
+
+for i = 1, NITERATIONS do
+  -- Rotation is performed beyond the 32-bit size, for truncation
+  -- to become noticeable. Sprintf is used to ensure that the
+  -- result of rotation goes into the `rcx`, corresponing to
+  -- the x86_64 ABI.
+  result[i] = ffi.C.snprintf(bufs[i], BUFFER_SIZE, '%llu', rol(1ULL, i + 32))
+  -- Resulting assembly for the `rol` instruction above changes
+  -- from the following before the patch:
+  -- | rol rsi, cl
+  -- | mov ecx, esi
+  --
+  -- to the following after the patch:
+  -- | rol rsi, cl
+  -- | mov rcx, rsi
+end
+
+for i = 1, NITERATIONS do
+  test:ok(result[i] > 1, '64-bit value was not truncated')
+  test:ok(tonumber(ffi.string(bufs[i])) == shl(1ULL, i + 32), 'valid rol')
+end
+
+os.exit(test:check() and 0 or 1)
-- 
2.40.1