From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 07C73CE4101; Tue, 8 Oct 2024 15:07:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 07C73CE4101 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1728389242; bh=BE6kiu3u5xr1HAUtrRuZVFI//+R85FMryycnN3wbwUk=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=ktguAGHuiaeuXjuXu7Mm/ITPzKfj90ha+hjMGMgGxuJkav1STEaMmShNsNdHAGLMk Mc/Ei14TbuTi1NDbSo0ZbUeVffX1xHVhzvJpK0mjzD3OM9rUfhU2CNoi4HBXa88ZmS DCPLtvx1PBAvTJNljQR93Ge1+7H08z+G/M/iQc5w= Received: from smtp54.i.mail.ru (smtp54.i.mail.ru [95.163.41.89]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 539FBCE4106 for ; Tue, 8 Oct 2024 15:07:21 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 539FBCE4106 Received: by exim-smtp-57dbb65494-qnzxs with esmtpa (envelope-from ) id 1sy8zo-00000000BqJ-1eLF; Tue, 08 Oct 2024 15:07:20 +0300 Content-Type: multipart/alternative; boundary="------------s941DAkiKxkdRq2BYB67q5Uo" Message-ID: Date: Tue, 8 Oct 2024 15:07:20 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun , Maxim Kokryashkin Cc: tarantool-patches@dev.tarantool.org References: <7531b1a6a3f39f8f2d83a54befdc67af987cebaf.1727855711.git.skaplun@tarantool.org> In-Reply-To: <7531b1a6a3f39f8f2d83a54befdc67af987cebaf.1727855711.git.skaplun@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9B01871A0ED523BBF0EA4F0057A5B4EB36890D9CCEC1C8281182A05F538085040B28DB84650E2734A3DE06ABAFEAF67056F41B11335A07CB9E273160033E4C1248DC7380D2C131FBD X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE71BDE6A359BD5B800EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006371750936FC250F8708638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D81BEE04E4272BCFCAFE9744FB21F87EB71DC9693E2904A0CDCC7F00164DA146DAFE8445B8C89999728AA50765F790063783E00425F71A4181389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8989FD0BDF65E50FBF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C0A9EC8C3488E7643C0837EA9F3D197644AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C385B415260A3EE7FDBA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CFED8438A78DFE0A9E1DD303D21008E298D5E8D9A59859A8B6957A4DEDD2346B4275ECD9A6C639B01B78DA827A17800CE7C13763150CD373CE731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: 0D63561A33F958A56DD5BD8BA96AB7B95002B1117B3ED696E01CC2F6229E0CF9ED71F038FC046993823CB91A9FED034534781492E4B8EEADDFC043C56F70D752BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF3265850114E669D530FFFDD95E4F5C74302C9E8F6E35D8D491D7A713BC5394F2686167004DDA272FAF66815EE7BDF79007421DF4AF5FE11105D530DFA8714B048BA0B4E66509FEB25F4332CA8FE04980913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj/pBY5inp/EaTpWns38YgIQ== X-Mailru-Sender: 520A125C2F17F0B1E52FEF5D219D6140081F98C5A235B8ED33594132A326AF8B989732A58D7E837C0152A3D17938EB451EB5A0BCEC6A560B3DDE9B364B0DF289BE2DA36745F2EEB5CEBA01FB949A1F1EEAB4BC95F72C04283CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 2/2] FFI: Fix 64 bit shift fold rules. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------s941DAkiKxkdRq2BYB67q5Uo Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello, Sergey, thanks for the patch! Please see my comments below. On 02.10.2024 11:09, Sergey Kaplun wrote: > From: Mike Pall > > Thanks to Peter Cawley. > > (cherry picked from commit 9e0437240f1fb4bfa7248f6ec8be0e3181016119) > > For `IR_BSHR`, `IR_BROL`, `IR_BROR` during `kfold_int64arith()` the left > argument is truncated down to 32 bits, which leads to incorrect results > if the right argument is >= 32. typo: is >= 2,147,483,647 > > Also, `IR_BSAR` does an unsigned shift rather than a signed shift, but > since this case branch is unreachable, it is harmless for now. > > This patch fixes all misbehaviours (including possible for `IR_BSAR`) to > preserve IR semantics. > > Sergey Kaplun: > * added the description and the test for the problem > > Part of tarantool/tarantool#10199 > --- > src/lj_opt_fold.c | 8 +- > .../lj-1079-fix-64-bitshift-folds.test.lua | 74 +++++++++++++++++++ > 2 files changed, 78 insertions(+), 4 deletions(-) > create mode 100644 test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua > > diff --git a/src/lj_opt_fold.c b/src/lj_opt_fold.c > index e2171e1b..2702f79f 100644 > --- a/src/lj_opt_fold.c > +++ b/src/lj_opt_fold.c > @@ -382,10 +382,10 @@ static uint64_t kfold_int64arith(jit_State *J, uint64_t k1, uint64_t k2, > case IR_BOR: k1 |= k2; break; > case IR_BXOR: k1 ^= k2; break; > case IR_BSHL: k1 <<= (k2 & 63); break; > - case IR_BSHR: k1 = (int32_t)((uint32_t)k1 >> (k2 & 63)); break; > - case IR_BSAR: k1 >>= (k2 & 63); break; > - case IR_BROL: k1 = (int32_t)lj_rol((uint32_t)k1, (k2 & 63)); break; > - case IR_BROR: k1 = (int32_t)lj_ror((uint32_t)k1, (k2 & 63)); break; > + case IR_BSHR: k1 >>= (k2 & 63); break; > + case IR_BSAR: k1 = (uint64_t)((int64_t)k1 >> (k2 & 63)); break; > + case IR_BROL: k1 = lj_rol(k1, (k2 & 63)); break; > + case IR_BROR: k1 = lj_ror(k1, (k2 & 63)); break; > default: lj_assertJ(0, "bad IR op %d", op); break; > } > #else > diff --git a/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua b/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua > new file mode 100644 > index 00000000..6cc0b319 > --- /dev/null > +++ b/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua > @@ -0,0 +1,74 @@ > +local tap = require('tap') > + > +-- Test file to demonstrate LuaJIT misbehaviour on folding > +-- for bitshift operations. > +-- See also,https://github.com/LuaJIT/LuaJIT/issues/1079. > + > +local test = tap.test('lj-1079-fix-64-bitshift-folds'):skipcond({ > + ['Test requires JIT enabled'] = not jit.status(), > +}) > + > +local bit = require('bit') > + > +test:plan(4) > + > +-- Generic function for `bit.ror()`, `bit.rol()`. > +local function bitop_rotation(bitop) I would rename arg `bitop` to `bitop_func` to highlight the type of the value. > + local r = {} > + for i = 1, 4 do > + -- (i & k1) o k2 ==> (i o k2) & (k1 o k2) > + local int64 = bit.band(i, 7LL) > + r[i] = tonumber(bitop(int64, 32)) please add comments about magic constants here and below > + end > + return r > +end > + > +-- Similar function for `bit.rshift()`. > +local function bitop_rshift_signed() > + local r = {} > + for i = 1, 4 do > + -- (i & k1) o k2 ==> (i o k2) & (k1 o k2) > + -- XXX: Use `-i` instead of `i` to prevent other folding due > + -- to IR difference so the IRs don't match fold rule mask. > + -- (-i & 7LL) < 1 << 32 => result == 0. > + local int64 = bit.band(-i, 7LL) > + r[i] = tonumber(bit.rshift(int64, 32)) > + end > + return r > +end > + > +-- A little bit different example, which leads to the assertion > +-- failure due to the incorrect recording. > +local function bitop_rshift_huge() > + local r = {} > + for i = 1, 4 do > + -- (i & k1) o k2 ==> (i o k2) & (k1 o k2) > + -- XXX: Need to use cast to the int64_t via `+ 0LL`, see the > + -- documentation [1] for the details. > + -- [1]:https://bitop.luajit.org/semantics.html > + local int64 = bit.band(2 ^ 33 + i, 2 ^ 33 + 0LL) > + r[i] = tonumber(bit.rshift(int64, 32)) > + end > + return r > +end > + > +local function test_64bitness(subtest, payload_func, bitop) > + subtest:plan(1) > + > + jit.off() > + jit.flush() > + local results_joff = payload_func(bitop) > + jit.on() > + -- Reset hotcounters. > + jit.opt.start('hotloop=1') > + local results_jon = payload_func(bitop) > + subtest:is_deeply(results_jon, results_joff, > + 'same results for VM and JIT for ' .. subtest.name) > +end > + > +test:test('rol', test_64bitness, bitop_rotation, bit.rol) > +test:test('ror', test_64bitness, bitop_rotation, bit.ror) > +test:test('rshift signed', test_64bitness, bitop_rshift_signed) > +test:test('rshift huge', test_64bitness, bitop_rshift_huge) have you added additional whitespaces intentionally? > + > +test:done(true) --------------s941DAkiKxkdRq2BYB67q5Uo Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hello, Sergey,

thanks for the patch! Please see my comments below.

On 02.10.2024 11:09, Sergey Kaplun wrote:
From: Mike Pall <mike>

Thanks to Peter Cawley.

(cherry picked from commit 9e0437240f1fb4bfa7248f6ec8be0e3181016119)

For `IR_BSHR`, `IR_BROL`, `IR_BROR` during `kfold_int64arith()` the left
argument is truncated down to 32 bits, which leads to incorrect results
if the right argument is >= 32.
typo: is >= 2,147,483,647

Also, `IR_BSAR` does an unsigned shift rather than a signed shift, but
since this case branch is unreachable, it is harmless for now.

This patch fixes all misbehaviours (including possible for `IR_BSAR`) to
preserve IR semantics.

Sergey Kaplun:
* added the description and the test for the problem

Part of tarantool/tarantool#10199
---
 src/lj_opt_fold.c                             |  8 +-
 .../lj-1079-fix-64-bitshift-folds.test.lua    | 74 +++++++++++++++++++
 2 files changed, 78 insertions(+), 4 deletions(-)
 create mode 100644 test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua

diff --git a/src/lj_opt_fold.c b/src/lj_opt_fold.c
index e2171e1b..2702f79f 100644
--- a/src/lj_opt_fold.c
+++ b/src/lj_opt_fold.c
@@ -382,10 +382,10 @@ static uint64_t kfold_int64arith(jit_State *J, uint64_t k1, uint64_t k2,
   case IR_BOR: k1 |= k2; break;
   case IR_BXOR: k1 ^= k2; break;
   case IR_BSHL: k1 <<= (k2 & 63); break;
-  case IR_BSHR: k1 = (int32_t)((uint32_t)k1 >> (k2 & 63)); break;
-  case IR_BSAR: k1 >>= (k2 & 63); break;
-  case IR_BROL: k1 = (int32_t)lj_rol((uint32_t)k1, (k2 & 63)); break;
-  case IR_BROR: k1 = (int32_t)lj_ror((uint32_t)k1, (k2 & 63)); break;
+  case IR_BSHR: k1 >>= (k2 & 63); break;
+  case IR_BSAR: k1 = (uint64_t)((int64_t)k1 >> (k2 & 63)); break;
+  case IR_BROL: k1 = lj_rol(k1, (k2 & 63)); break;
+  case IR_BROR: k1 = lj_ror(k1, (k2 & 63)); break;
   default: lj_assertJ(0, "bad IR op %d", op); break;
   }
 #else
diff --git a/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua b/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua
new file mode 100644
index 00000000..6cc0b319
--- /dev/null
+++ b/test/tarantool-tests/lj-1079-fix-64-bitshift-folds.test.lua
@@ -0,0 +1,74 @@
+local tap = require('tap')
+
+-- Test file to demonstrate LuaJIT misbehaviour on folding
+-- for bitshift operations.
+-- See also, https://github.com/LuaJIT/LuaJIT/issues/1079.
+
+local test = tap.test('lj-1079-fix-64-bitshift-folds'):skipcond({
+  ['Test requires JIT enabled'] = not jit.status(),
+})
+
+local bit = require('bit')
+
+test:plan(4)
+
+-- Generic function for `bit.ror()`, `bit.rol()`.
+local function bitop_rotation(bitop)

I would rename arg `bitop` to `bitop_func` to highlight the type

of the value.

+  local r = {}
+  for i = 1, 4 do
+    -- (i & k1) o k2 ==> (i o k2) & (k1 o k2)
+    local int64 = bit.band(i, 7LL)
+    r[i] = tonumber(bitop(int64, 32))
please add comments about magic constants here and below
+  end
+  return r
+end
+
+-- Similar function for `bit.rshift()`.
+local function bitop_rshift_signed()
+  local r = {}
+  for i = 1, 4 do
+    -- (i & k1) o k2 ==> (i o k2) & (k1 o k2)
+    -- XXX: Use `-i` instead of `i` to prevent other folding due
+    -- to IR difference so the IRs don't match fold rule mask.
+    -- (-i & 7LL) < 1 << 32 => result == 0.
+    local int64 = bit.band(-i, 7LL)
+    r[i] = tonumber(bit.rshift(int64, 32))
+  end
+  return r
+end
+
+-- A little bit different example, which leads to the assertion
+-- failure due to the incorrect recording.
+local function bitop_rshift_huge()
+  local r = {}
+  for i = 1, 4 do
+    -- (i & k1) o k2 ==> (i o k2) & (k1 o k2)
+    -- XXX: Need to use cast to the int64_t via `+ 0LL`, see the
+    -- documentation [1] for the details.
+    -- [1]: https://bitop.luajit.org/semantics.html
+    local int64 = bit.band(2 ^ 33 + i, 2 ^ 33 + 0LL)
+    r[i] = tonumber(bit.rshift(int64, 32))
+  end
+  return r
+end
+
+local function test_64bitness(subtest, payload_func, bitop)
+  subtest:plan(1)
+
+  jit.off()
+  jit.flush()
+  local results_joff = payload_func(bitop)
+  jit.on()
+  -- Reset hotcounters.
+  jit.opt.start('hotloop=1')
+  local results_jon = payload_func(bitop)
+  subtest:is_deeply(results_jon, results_joff,
+                    'same results for VM and JIT for ' .. subtest.name)
+end
+
+test:test('rol', test_64bitness, bitop_rotation, bit.rol)
+test:test('ror', test_64bitness, bitop_rotation, bit.ror)
+test:test('rshift signed', test_64bitness, bitop_rshift_signed)
+test:test('rshift huge',   test_64bitness, bitop_rshift_huge)
have you added additional whitespaces intentionally?
+
+test:done(true)
--------------s941DAkiKxkdRq2BYB67q5Uo--