From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E60B240E76C; Thu, 8 Feb 2024 14:11:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E60B240E76C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1707390683; bh=LPfZogPHof1Ld1kMdibZvJ2Pog4/I8s8O/I4lEnTHfU=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=b/D+7+oDKLP2+JoikMBUb/cjoaijSzPMWGL9ZnPPQ91TDt5ntdLSzm1DT3tYDzaL+ C2DRoo2Mh241r1vLUnSH83KApbExaiLVEaQwzuTAw1ur+SvqqfXQ2P/H8bku16Uz3U c2F3vFESjoRXjJDQ4WSM+oigRMrKPKgDDN01GzRw= Received: from smtp56.i.mail.ru (smtp56.i.mail.ru [95.163.41.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2157740E76C for ; Thu, 8 Feb 2024 14:11:21 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2157740E76C Received: by smtp56.i.mail.ru with esmtpa (envelope-from ) id 1rY2JK-00000004Mlr-29Pq; Thu, 08 Feb 2024 14:11:19 +0300 Message-ID: Date: Thu, 8 Feb 2024 14:11:18 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun , Maxim Kokryashkin Cc: tarantool-patches@dev.tarantool.org References: <20240207145108.29786-1-skaplun@tarantool.org> In-Reply-To: <20240207145108.29786-1-skaplun@tarantool.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD91FEFD63CE1B099168EBFA235A9DCEE77838A3BB561EABAC000894C459B0CD1B9CE36E94D07AB7995A108A89D615787B3778D079A48A72421D21B02BA560489C752B81C99D67B1670 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE751DD1FEBB966604DEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006379448E89E2A57838D8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D83D14972EB4D73087EA604E5054B3491A5A6A5314A46C7BA9CC7F00164DA146DAFE8445B8C89999728AA50765F7900637CAEE156C82D3D7D9389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8062BEEFFB5F8EA3EF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C13BDA61BF53F5E1D9735652A29929C6C4AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C355C31AA2AB4E8F64BA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CFED8438A78DFE0A9E1DD303D21008E298D5E8D9A59859A8B6B372FE9A2E580EFC725E5C173C3A84C37727919777A35F2B35872C767BF85DA2F004C90652538430E4A6367B16DE6309 X-C1DE0DAB: 0D63561A33F958A50D71BD2524DB181D5002B1117B3ED696ECF7E5BEED97A4C047A99E6294EE8661823CB91A9FED034534781492E4B8EEAD1E02C13459908652BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CFEFCF195B0C94D8F6AFA3E133583C8096D6B50081F467EAB293DBDD405C6B3AF8ACCA692FBCDC96C2766A7423F6D95CBE7C32FE9ECC52B5A33679C2A2F9C1B99027860954C436FA845F4332CA8FE04980913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojlN/n4z3iVuT/nekXXTF2/A== X-Mailru-Sender: C4F68CFF4024C8867DFDF7C7F2588458145F34A82DB8DDA3FD3DE253F98C97F8E59DA70317CAC7C87FADC677C6C102E0645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit] Consider slots used by upvalues in use-def analysis. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey thanks for the patch! LGTM with a minor comment On 2/7/24 17:51, Sergey Kaplun wrote: > From: Mike Pall > > Reported by XmiliaH. > > (cherry picked from commit 3a654999c6f00de4cb9e61232d23579442e544a0) > > `snap_usedef()` analysis doesn't check slots for child upvalues of the > currentlly recorded function in use-def analysis. Hence, such slots may typo: currentlly > be considered unused and not stored in the snapshot. So on snapshot > restoration, values from these slots may be lost. > > This patch adds a marking for all such local upvalues. > > Sergey Kaplun: > * added the description and the test for the problem > > Part of tarantool/tarantool#9595 > --- > > Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-737-snap-usedef-upvalues > Tarantool PR: https://github.com/tarantool/tarantool/pull/9662 > Related issues: > * https://github.com/tarantool/tarantool/issues/9595 > * https://github.com/LuaJIT/LuaJIT/issues/737 > > src/lj_snap.c | 35 +++++++++++- > .../lj-737-snap-use-def-upvalues.test.lua | 55 +++++++++++++++++++ > 2 files changed, 87 insertions(+), 3 deletions(-) > create mode 100644 test/tarantool-tests/lj-737-snap-use-def-upvalues.test.lua > > diff --git a/src/lj_snap.c b/src/lj_snap.c > index 26352080..1ac9296a 100644 > --- a/src/lj_snap.c > +++ b/src/lj_snap.c > @@ -302,6 +302,31 @@ static BCReg snap_usedef(jit_State *J, uint8_t *udf, > return 0; /* unreachable */ > } > > +/* Mark slots used by upvalues of child prototypes as used. */ > +void snap_useuv(GCproto *pt, uint8_t *udf) > +{ > + /* This is a coarse check, because it's difficult to correlate the lifetime > + ** of slots and closures. But the number of false positives is quite low. > + ** A false positive may cause a slot not to be purged, which is just > + ** a missed optimization. > + */ > + if ((pt->flags & PROTO_CHILD)) { > + ptrdiff_t i, j, n = pt->sizekgc; > + GCRef *kr = mref(pt->k, GCRef) - 1; > + for (i = 0; i < n; i++, kr--) { > + GCobj *o = gcref(*kr); > + if (o->gch.gct == ~LJ_TPROTO) { > + for (j = 0; j < gco2pt(o)->sizeuv; j++) { > + uint32_t v = proto_uv(gco2pt(o))[j]; > + if ((v & PROTO_UV_LOCAL)) { > + udf[(v & 0xff)] = 0; > + } > + } > + } > + } > + } > +} > + > /* Purge dead slots before the next snapshot. */ > void lj_snap_purge(jit_State *J) > { > @@ -310,9 +335,12 @@ void lj_snap_purge(jit_State *J) > if (bc_op(*J->pc) == BC_FUNCV && maxslot > J->pt->numparams) > maxslot = J->pt->numparams; > s = snap_usedef(J, udf, J->pc, maxslot); > - for (; s < maxslot; s++) > - if (udf[s] != 0) > - J->base[s] = 0; /* Purge dead slots. */ > + if (s < maxslot) { > + snap_useuv(J->pt, udf); > + for (; s < maxslot; s++) > + if (udf[s] != 0) > + J->base[s] = 0; /* Purge dead slots. */ > + } > } > > /* Shrink last snapshot. */ > @@ -325,6 +353,7 @@ void lj_snap_shrink(jit_State *J) > BCReg maxslot = J->maxslot; > BCReg baseslot = J->baseslot; > BCReg minslot = snap_usedef(J, udf, snap_pc(&map[nent]), maxslot); > + if (minslot < maxslot) snap_useuv(J->pt, udf); > maxslot += baseslot; > minslot += baseslot; > snap->nslots = (uint8_t)maxslot; > diff --git a/test/tarantool-tests/lj-737-snap-use-def-upvalues.test.lua b/test/tarantool-tests/lj-737-snap-use-def-upvalues.test.lua > new file mode 100644 > index 00000000..8535f9f6 > --- /dev/null > +++ b/test/tarantool-tests/lj-737-snap-use-def-upvalues.test.lua > @@ -0,0 +1,55 @@ > +local tap = require('tap') > + > +-- Test file to demonstrate LuaJIT misbehaviour in use-def > +-- snapshot analysis for local upvalues. > +-- See also https://github.com/LuaJIT/LuaJIT/issues/737. > + > +local test = tap.test('lj-737-snap-use-def-upvalues'):skipcond({ > + ['Test requires JIT enabled'] = not jit.status(), > +}) > + > +test:plan(1) > + > +-- XXX: simplify `jit.dump()` output. > +local fmod = math.fmod > + > +local EXPECTED = 'expected' > + > +jit.opt.start('hotloop=1') > + > +local function wrapped_trace(create_closure) > + local local_upvalue, closure > + if create_closure then > + closure = function() return local_upvalue end > + end > + for i = 1, 4 do > + -- On the second iteration, the trace is recorded. > + if i == 2 then > + -- Before the patch, this slot was considered unused by > + -- use-def analysis in the `snap_usedef()` since there are > + -- no open unpvalues for `closure()` on recording (1st call). > + local_upvalue = EXPECTED > + -- luacheck: ignore > + -- Emit an additional snapshot after setting the > + -- upvalue. > + if i == 0 then end > + -- Stitching ends the trace here. > + fmod(1,1) > + return closure > + end > + end > +end > + > +-- Compile the trace. > +local func_with_uv = wrapped_trace(false) > +assert(func_with_uv == nil, 'no function is returned on the first call') > + > +-- Now run this trace when `closure()` is defined and has an open > +-- local upvalue. > +func_with_uv = wrapped_trace(true) > +assert(type(func_with_uv) == 'function', > + 'function is returned after the second call') > + > +test:is(func_with_uv(), EXPECTED, 'correct result of the closure call') > + > +test:done(true)