From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6656D170DA6; Thu, 15 Dec 2022 13:16:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6656D170DA6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1671099414; bh=5gGpn/k3dGvQ6Yc0F3hpNOfcdzlo7A43fU6nokUi1II=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=fmC03FbxC0oKlyryNr+maLEj+L/ystviVO3uQFgFNBo4UiHl6xe9QXoPTej12Zx9+ bSOJDGmPe+FJ2QmFjKVxbNDRJQ3cLCmESpQODCEaWXjXRpvH2IVI7TU4zzI/p/hEFQ xOMmY+2DPRKxSijb6VIT2piLTBW5EEayLmW5zbUI= Received: from smtp50.i.mail.ru (smtp50.i.mail.ru [94.100.177.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C2094170DA6 for ; Thu, 15 Dec 2022 13:16:53 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C2094170DA6 Received: by smtp50.i.mail.ru with esmtpa (envelope-from ) id 1p5lIK-0004h5-Pz; Thu, 15 Dec 2022 13:16:53 +0300 Date: Thu, 15 Dec 2022 13:13:35 +0300 To: sergos Message-ID: References: <20221208054618.9104-1-skaplun@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD90D1502B3BE350FE420345CBC5A180DD43DEA314B30AECAB400894C459B0CD1B91F0D15041ED0A28792084749794A37640DFD564ECD3794F0BE3CD06344D71D5D X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE70043D879A87EF1BCEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637575C0790A70C4B158638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D87844BD1E93A53983FAE8A3F3C9F192AE117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC1F8789D36234D406A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520CCD848CCB6FE560C6FD1C55BDD38FC3FD2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE4B6963042765DA4B1AB2475877E8919AD8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE364E7220B7C5505926136E347CC761E07C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F7900637B8F435DEDE9E76EBEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34AC632F0BE69382F39302DB9C911CBC36767EB897ABE2860AD3FB896250BF41DFFC802834C7C2F3DC1D7E09C32AA3244CF8FBECA8D680508D416352A49ADD78BA8580396430872480FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioja/mV86HvUK8ACr5kMixAWw== X-Mailru-Sender: F16D9CAFEEA6770E7B6EAD4ADB3BCAF0E573B9E0BB61BB330B655D935F444F79A0A75FDC5059D49DF2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A84198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit] LJ_GC64: Fix ir_khash for non-string GCobj. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergos! Thanks for the review! On 14.12.22, sergos wrote: > Hi! > > Thanks for the patch! > > Some addition to Max’s comments. And a question on the test. > > Sergos > > > On 8 Dec 2022, at 08:46, Sergey Kaplun wrote: > > > > From: Mike Pall > > > > Contributed by Peter Cawley. > > > > (cherry picked from commit b4ed3219a1a98dd9fe7d1e3eeea3b82f5a780948) > > > > When emitting `IR_HREF` for constant value to lookup the `ir_khash()` > an ^^^ > perhaps just ‘for a constant value lokup’? > > > function is used to calculate hash for the corresponding object. > > This calculation must be the same as in the corresponding `hashkey()` > > function from . > > > > Hash calculating via passing two arguments `lo`, and `hi` to `hashrot()` > the > > > routine. For non-string GC objects the first `lo` argument is the same > > for GC64 and not GC64 mode -- lower 32 bits of the object address. For > > GC64 mode `hi` argument is upper 32 bits of the object address, > > including specific type NaN-tag. This `hi` argument in `ir_khash()` > a > > > function is miscalculated in GC64 using non-GC64 value (`lo` + > mode a > > > `HASH_BIAS`). As a result, the hash for the GC object is miscalculated > > on trace and we exit from trace due to assertion guard on the type or > the an > > value check. > > > > This patch fixes calculation of hash value on trace for GC64 mode by > > making it consistent with `hashkey()`. > the > > Fixed your comments. The new commit message is the following: | LJ_GC64: Fix ir_khash for non-string GCobj. | | Contributed by Peter Cawley. | | (cherry picked from commit b4ed3219a1a98dd9fe7d1e3eeea3b82f5a780948) | | When emitting the `IR_HREF` for a constant value lookup the `ir_khash()` | function is used to calculate the hash for the corresponding object. | This calculation must be the same as in the corresponding `hashkey()` | function from . | | Hash is calculated by passing two arguments `lo`, and `hi` to the | `hashrot()` routine. For non-string GC objects the first `lo` argument | is the same for GC64 and not GC64 mode -- lower 32 bits of the object | address. For GC64 mode `hi` argument is upper 32 bits of the object | address, including a specific type NaN-tag. This `hi` argument in | `ir_khash()` function is miscalculated in GC64 mode using a non-GC64 | value (`lo` + `HASH_BIAS`). As a result, the hash for the GC object is | miscalculated on trace and we exit from the trace due to an assertion | guard on the type or value check. | | This patch fixes calculation of the hash value on trace for GC64 mode by | making it consistent with the `hashkey()`. | | Sergey Kaplun: | * added the description and the test for the problem | | Part of tarantool/tarantool#7230 > > Sergey Kaplun: > > * added the description and the test for the problem > > > > Part of tarantool/tarantool#7230 > > --- > > > > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-356-ir-khash-non-string-obj-full-ci > > Issue/PR: > > * https://github.com/tarantool/tarantool/issues/7230 > > * https://github.com/LuaJIT/LuaJIT/pull/356 > > Tarantool PR: https://github.com/tarantool/tarantool/pull/8020 > > > > Side note: Problems with red fuzzer jobs look irrelevant to the patch. > > diff --git a/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua b/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua > > new file mode 100644 > > index 00000000..fff0b1a5 > > --- /dev/null > > +++ b/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua > > @@ -0,0 +1,90 @@ > > +local tap = require('tap') > > +local traceinfo = require('jit.util').traceinfo > > +local table_new = require('table.new') > > + > > +-- Test file to demonstrate the incorrect GC64 JIT behaviour > > +-- for `IR_HREF` for on-trace-constant key lookup. > of an an > > +-- See also https://github.com/LuaJIT/LuaJIT/pull/356. > > +local test = tap.test('lj-356-ir-khash-non-string-obj') > > +local N_ITERATIONS = 4 > > + > > +-- Amount of iteration for trace compilation and execution and > > +-- additional check, that there is no new trace compiled. > > +test:plan(N_ITERATIONS + 1) > > + > > +-- To reproduce the issue we need to compile a trace with > > +-- `IR_HREF`, with a lookup of constant hash key GC value. To > > +-- prevent `IR_HREFK` to be emitted instead, we need a table with > an `IR_HREFK` emission Side note: I'm not sure about "emission" corectness here, so ignoring this part. I've fixed the rest of your comments, see the iterative patch below. =================================================================== diff --git a/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua b/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua index fff0b1a5..7f304183 100644 --- a/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua +++ b/test/tarantool-tests/lj-356-ir-khash-non-string-obj.test.lua @@ -3,7 +3,7 @@ local traceinfo = require('jit.util').traceinfo local table_new = require('table.new') -- Test file to demonstrate the incorrect GC64 JIT behaviour --- for `IR_HREF` for on-trace-constant key lookup. +-- of an `IR_HREF` for the on-trace-constant key lookup. -- See also https://github.com/LuaJIT/LuaJIT/pull/356. local test = tap.test('lj-356-ir-khash-non-string-obj') local N_ITERATIONS = 4 @@ -14,10 +14,10 @@ test:plan(N_ITERATIONS + 1) -- To reproduce the issue we need to compile a trace with -- `IR_HREF`, with a lookup of constant hash key GC value. To --- prevent `IR_HREFK` to be emitted instead, we need a table with --- a huge hash part. Delta of address between the start of the --- hash part of the table and the current node to lookup must be --- more than `(1024 * 64 - 1) * sizeof(Node)`. +-- prevent an `IR_HREFK` to be emitted instead, we need a table +-- with a huge hash part. Delta of address between the start of +-- the hash part of the table and the current node to lookup must +-- be more than `(1024 * 64 - 1) * sizeof(Node)`. -- See , for details. -- XXX: This constant is well suited to prevent test to be flaky, -- because the aforementioned delta is always large enough. @@ -36,8 +36,8 @@ end -- exiting the main test cycle. jit.opt.start('hotloop=1') --- Prevent `get_const_cdata()` become hot and be compiled before --- the main test cycle. +-- Prevent `get_const_cdata()` from becoming hot and being +-- compiled before the main test cycle. jit.off() filled_tab[get_const_cdata()] = MAGIC @@ -46,10 +46,10 @@ filled_tab[get_const_cdata()] = MAGIC jit.on() -- Filling-up the table with GC values to minimize the amount of --- hash collisions and increases delta between the start of the +-- hash collisions and increase delta between the start of the -- hash part of the table and currently stored node. -for i = 1, N_HASH_FIELDS do - filled_tab[1LL] = i +for _ = 1, N_HASH_FIELDS do + filled_tab[1LL] = 1 end -- Prevent JIT misbehaviour before the main test chunk. =================================================================== > > > +-- a huge hash part. Delta of address between the start of the > > +-- hash part of the table and the current node to lookup must be > > +-- more than `(1024 * 64 - 1) * sizeof(Node)`. > > +-- See , for details. > > +-- XXX: This constant is well suited to prevent test to be flaky, > > +-- because the aforementioned delta is always large enough. > > +local N_HASH_FIELDS = 1024 * 1024 * 8 > > +local MAGIC = 42 > > + > > +test:ok(not traceinfo(2), 'the second trace should not be compiled') > > That’s not quite clear to me: a second trace generation is a side-effect > of the incorrect hash calculation. Is it always leads to the trace > generation? How I see this for now. There are two possibilities, when the aforementioned hash is miscalculated: 1) We got `nil` value on a trace to lookup and we exit from the trace by assertion guard on the field type (the most possible one, AFAIKS). 2) We got a value for some existing cdata after hash lookup, so we don't exit from a trace, but got an incorrect value by the given key. NB: I've updated the generation of the table content to avoid clashing with `MAGIC` value on the 42nd iteration :). So this test should cover both cases. > > > + > > +-- No more need to prevent trace compilation. > > +jit.on() > > + > > +for i = 1, N_ITERATIONS do > > + -- Check that that all lookups are correct and there is no > > + -- value from other cdata stored in the table. > > + test:ok(result_tab[i] == MAGIC, 'correct hash lookup from the table') > > And this one checks what then? The hash is calculated correctly, but the value > read from the `filled_tab` is incorrect - what can lead to this? > > > +end > > + > > +os.exit(test:check() and 0 or 1) > > -- > > 2.34.1 > > > -- Best regards, Sergey Kaplun