From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <tarantool-patches-bounces@dev.tarantool.org>
Received: from [87.239.111.99] (localhost [127.0.0.1])
by dev.tarantool.org (Postfix) with ESMTP id 520C26EC40;
Sun, 8 Aug 2021 22:52:27 +0300 (MSK)
DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 520C26EC40
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev;
t=1628452347; bh=cO33R0ab5iAnctWWQKzE9H6GLLcrnjrV0QcDe9CcERE=;
h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:
List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:
From;
b=fxVdYA+8JJlzWkiBa6u6Er72S51ZJht34MAMKumNREUCXZn0oKNAqQ+XQSLcXUvkB
yzMi5eO4Sh1nqu24wXPofw/YHr3auvymrwsk9l4333L1hD/BKNTKh3vDv+WwIuJd6W
AnBnAPJZA8zje1d3vOzc4ihVTyVq7ZKHJOo+g5C4=
Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by dev.tarantool.org (Postfix) with ESMTPS id 604396EC40
for <tarantool-patches@dev.tarantool.org>;
Sun, 8 Aug 2021 22:52:26 +0300 (MSK)
DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 604396EC40
Received: by smtpng1.m.smailru.net with esmtpa (envelope-from
<imun@tarantool.org>)
id 1mCoqP-00031x-5K; Sun, 08 Aug 2021 22:52:25 +0300
Date: Sun, 8 Aug 2021 22:28:46 +0300
To: Sergey Kaplun <skaplun@tarantool.org>
Message-ID: <20210808192846.GH27855@tarantool.org>
References: <20210707143606.3499-1-skaplun@tarantool.org>
<20210801103955.GY27855@tarantool.org> <YQbTMVH8bm7pBPyL@root>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <YQbTMVH8bm7pBPyL@root>
X-Clacks-Overhead: GNU Terry Pratchett
User-Agent: Mutt/1.10.1 (2018-07-13)
X-4EC0790: 10
X-7564579A: B8F34718100C35BD
X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9D5AC6413C25DCF08CC98B8FCC5CD86F3182A05F53808504087103EC15C0A0B48FB209195FF006A8B0CD902C7A2B878D687E9E0E27295AF6F
X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE736C75B72B9FDC350B287FD4696A6DC2FA8DF7F3B2552694A4E2F5AFA99E116B42401471946AA11AF23F8577A6DFFEA7C4B44AB1D52BB6B9B8F08D7030A58E5AD1A62830130A00468AEEEE3FBA3A834EE7353EFBB5533756672045CD04223AD15373F9EE645B5486164C31D90A07A9334A471835C12D1D9774AD6D5ED66289B5278DA827A17800CE7ABB305BD10C6E5099FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C3FD59DAE6580CC3C3117882F4460429728AD0CFFFB425014E868A13BD56FB6657E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F79006377870F476E0DB9443EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A
X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B505EC52DA00B274C2B4D11C9A876B4E905C
X-C1DE0DAB: 0D63561A33F958A5B0F58623B224CED0ECC3467C3BE15175C69A71CABA7E4063D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75AF0B556A5A327A45410CA545F18667F91A7EA1CDA0B5A7A0
X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D348E5EF936B2E46EBA3D477A6FB29728C8CCD7AA72A7422932A97E33CC7648F7DAA776C1486F49196C1D7E09C32AA3244C2FFDA36CB2B6EDBD05A19A684DF6989B55E75C8D0ED9F6EE927AC6DF5659F194
X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojMTMPlNJj3Si8NRneRjEyjA==
X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D109A84986CA3E09267C8C2EEBD882108A7C8D0F45F857DBFE9F1EFEE2F478337FB559BB5D741EB964C8C2C849690F8E70A04DAD6CC59E33667EA787935ED9F1B
X-Mras: Ok
Subject: Re: [Tarantool-patches] [PATCH luajit] ARM64: Fix write barrier in
BC_USETS.
X-BeenThere: tarantool-patches@dev.tarantool.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Tarantool development patches <tarantool-patches.dev.tarantool.org>
List-Unsubscribe: <https://lists.tarantool.org/mailman/options/tarantool-patches>,
<mailto:tarantool-patches-request@dev.tarantool.org?subject=unsubscribe>
List-Archive: <https://lists.tarantool.org/pipermail/tarantool-patches/>
List-Post: <mailto:tarantool-patches@dev.tarantool.org>
List-Help: <mailto:tarantool-patches-request@dev.tarantool.org?subject=help>
List-Subscribe: <https://lists.tarantool.org/mailman/listinfo/tarantool-patches>,
<mailto:tarantool-patches-request@dev.tarantool.org?subject=subscribe>
From: Igor Munkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>
Reply-To: Igor Munkin <imun@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Errors-To: tarantool-patches-bounces@dev.tarantool.org
Sender: "Tarantool-patches" <tarantool-patches-bounces@dev.tarantool.org>
Sergey,
Thanks for the fixes! See some new comments below.
On 01.08.21, Sergey Kaplun wrote:
> Igor,
>
> Thanks for the review!
> Update commit message on the branch, considering you comments.
Got it, but I still have some more comments regarding it.
>
> See answers to you questions below.
>
<snipped>
>
> >
> > > | ccmp TMP0w, #0, #0, ne
> > > | beq <1 // branch out from barrier movement
> > > `TMP0w` contains `upvalue->closed` field. If it equals NULL (the first
> > > `#0`). The second zero is the value of NZCV condition flags set if the
> > > condition (`ne`) is FALSE [1][2]. If the set value is not white, then
> > > flags are set to zero and branch is not taken (no Zero flag). If it
> > > happens at propagate or atomic GC State and the `lj_gc_barrieruv()`
> > > function is called then the gray value to set is marked as white. That
> > > leads to the assertion failure in the `gc_mark()` function.
> >
> > OK, I understand almost nothing from the part above. Here are the
> > comments:
> > 1. "If it equals NULL (the first `#0`)", then what?
>
> My bad:
> I mean here:
> If it equals NULL (the first `#0`), then the upvalue is open.
So why do you use NULL instead of 0? The field is uint8_t type, so 0 is
much clearer.
> Added this.
>
> > 2. Just to check we are on the same page: the second "immediate"
> > mentioned in docs[1] is NZCV?
>
> Yes.
>
> > Then beq <1 branch is not taken since
> > (TMP0w != 0) is FALSE (i.e. upvalue is not closed), but zero flag in
> > NZCV value is not set?
>
> Yes.
>
> > So how does the color of the value to be stored
> > relate to this control flow?
>
> This NZCV value isn't set if the upvalue is white, because condition is
> of the following instruction
>
> | tst TMP1w, #LJ_GC_WHITES // iswhite(str)
>
> is TRUE. So the <1 branch is taken, because the upvalue is closed.
Well... I can't imagine how I needed to find this... This relates mostly
to ARM docs you've mentioned, but it would be nice to describe this
behaviour in the commit message (since you're writing a verbose one).
>
> > 3. AFAICS, if the branch is not taken and <lj_gc_barrieruv> is called at
> > propagate or atomic phase, the value is colored either to gray or black.
>
> Yes, that leads to the assertion failure mentioned in the ticket in the
> LuaJIT upstream.
>
> >
> > >
> > > This patch changes yielded NZCV condition flag to 4 (Zero flag is up) to
> > > take the correct branch after `ccmp` instruction.
> > >
> > > Sergey Kaplun:
> > > * added the description and the test for the problem
> > >
> > > [1]: https://developer.arm.com/documentation/dui0801/g/pge1427897656225
> > > [2]: https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/condition-codes-1-condition-flags-and-codes
> >
> > Minor: Why #5629 is not mentioned?
>
> Added.
Considering everything above, I propose the following wording:
| Contributed by Javier Guerra Giraldez.
|
| (cherry picked from commit c785131ca5a6d24adc519e5e0bf1b69b671d912f)
|
|
| Closed upvalues are never gray. Hence when closed upvalue is marked, it
| is marked as black. Black objects can't refer white objects, so for
| storing a white value in a closed upvalue, we need to move the barrier
| forward and color our value to gray by using `lj_gc_barrieruv()`. This
| function can't be called on closed upvalues with non-white values since
| there is no need to mark it again.
|
| USETS bytecode for arm64 architecture has the incorrect NZCV condition
| flag value in the instruction that checks the upvalue is closed:
| | tst TMP1w, #LJ_GC_WHITES
| | ccmp TMP0w, #0, #0, ne
| | beq <1 // branch out from barrier movement
| `TMP0w` contains `upvalue->closed` field, so the upvalue is open if this
| field equals to zero (the first one in `ccmp`). The second zero is the
| value of NZCV condition flags[1] yielded if the specified condition
| (`ne`) is met for the current values of the condition flags[2]. Hence,
| if the value to be stored is not white (`TMP1w` holds its color), then
| the condition is FALSE and all flags bits are set to zero so branch is
| not taken (Zero flag is not set). If this happens at propagate or atomic
| GC phase, the `lj_gc_barrieruv()` function is called and the gray value
| to be set is marked like if it is white. That leads to the assertion
| failure in the `gc_mark()` function.
|
| This patch changes NZCV condition flag to 4 (Zero flag is set) to take
| the correct branch after `ccmp` instruction.
|
| Sergey Kaplun:
| * added the description and the test for the problem
|
| [1]: https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/condition-codes-1-condition-flags-and-codes
| [2]: https://developer.arm.com/documentation/dui0801/g/pge1427897656225
|
| Part of tarantool/tarantool#5629
>
<snipped>
> >
> > >
> > > src/vm_arm64.dasc | 2 +-
> > > ...6-arm64-incorrect-check-closed-uv.test.lua | 38 +++++++++++++++++++
> > > 2 files changed, 39 insertions(+), 1 deletion(-)
> > > create mode 100644 test/tarantool-tests/lj-426-arm64-incorrect-check-closed-uv.test.lua
> > >
> >
> > <snipped>
> >
> > > diff --git a/test/tarantool-tests/lj-426-arm64-incorrect-check-closed-uv.test.lua b/test/tarantool-tests/lj-426-arm64-incorrect-check-closed-uv.test.lua
> > > new file mode 100644
> > > index 00000000..b757133f
> > > --- /dev/null
> > > +++ b/test/tarantool-tests/lj-426-arm64-incorrect-check-closed-uv.test.lua
> > > @@ -0,0 +1,38 @@
> > > +local tap = require('tap')
> > > +
> > > +local test = tap.test('lj-426-arm64-incorrect-check-closed-uv')
> > > +test:plan(1)
> > > +
> > > +-- Test file to demonstrate LuaJIT USETS bytecode incorrect
> > > +-- behaviour on arm64 in case when non-white object is set to
> > > +-- closed upvalue.
> > > +-- See also, https://github.com/LuaJIT/LuaJIT/issues/426.
> > > +
> > > +-- First, create a closed upvalue.
> > > +do
> >
> > Minor: I'm not sure, we need a separate lexical block here. Could you
> > please clarify the reason in the comment?
>
> We need a closed upvalue. I suppose that it is the simpiest way to
> create one. Please, provide a simplier example if you know one.
My bad. Yes, the easiest way to emit UCLO bytecode is using a separate
lexical block.
>
> >
> > > + local uv -- luacheck: no unused
> > > + -- The function's prototype is created with the following
> > > + -- constants at chunk parsing. After adding this constant to
> > > + -- the function's prototype it will be marked as gray during
> > > + -- propogate phase.
> >
> > Then what does it test, if the constant is marked as gray? Will this
> > string be white later?
>
> It shouldn't be white, it should be gray, otherwise the aforementioned
> condition is TRUE (remember, we need FALSE).
Again, PEBKAC, thanks for the explanation.
>
> >
> > > + local function usets() uv = '' end
> > > + _G.usets = usets
> > > +end
> > > +
> > > +-- Set GC state to GCpause.
> > > +collectgarbage()
> > > +-- Do GC step as often as possible.
> > > +collectgarbage('setstepmul', 100)
> >
> > Minor: Don't get, why you need to make GC less aggressive for the test.
> > The test is run, until propagate phase is finished.
>
> More likely, that it is run, until the upvalue is marked as black
> during traversing (with the bug). I can remove this line if you insist.
Drop it, please. I can't even *feel* its effect ;)
>
> >
> > > +
> > > +-- We don't know on what exactly step our upvalue is marked as
> > > +-- black and USETS become dangerous, so just check it at each
> > > +-- step.
> > > +-- Don't need to do the full GC cycle step by step.
Minor: It would be nice to drop a few words about string and upvalue
colours during this loop, but it's up to you.
> > > +local old_steps_atomic = misc.getmetrics().gc_steps_atomic
> > > +while (misc.getmetrics().gc_steps_atomic == old_steps_atomic) do
> > > + collectgarbage('step')
> > > + usets() -- luacheck: no global
> > > +end
> > > +
> > > +test:ok(true)
> > > +os.exit(test:check() and 0 or 1)
> > > --
> > > 2.31.0
> > >
> >
> > [1]: https://lists.tarantool.org/tarantool-patches/20210719073632.12008-1-skaplun@tarantool.org/T/#u
> >
> > --
> > Best regards,
> > IM
>
> --
> Best regards,
> Sergey Kaplun
--
Best regards,
IM