From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 8EA465B18C5; Wed, 16 Aug 2023 16:00:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 8EA465B18C5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1692190811; bh=Ks0aV+Zxn17M9UHacQ5002jW60V72fviXqYSPRcQBZ8=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=EXVPvT7SobOdaQCfEOrYtzZLeUiKW4noJdl9soQrkId193BXQJv4Ha1Q2EmHJIYMo jpYf9E8gEpu24TApVZqcbAEKuQvSiao1WGydIhJ+4d4ciDFAb5oWOze8MvIOEqRgsO m1NL8/dBcC+YCQHBCc7599VUzYo1AoaCKiCWBfTI= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 02FE654EB90 for ; Wed, 16 Aug 2023 16:00:09 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 02FE654EB90 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1qWG88-00057d-Tu; Wed, 16 Aug 2023 16:00:09 +0300 Date: Wed, 16 Aug 2023 15:55:21 +0300 To: Maxim Kokryashkin Message-ID: References: <67642b3989e440fb554bf60db140828653c59659.1691592488.git.skaplun@tarantool.org> <5djx5fttrkbvnik22jgf37dhxku5ibqtqwrq4tdvbgljq7kctf@pchufp4vvegt> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5djx5fttrkbvnik22jgf37dhxku5ibqtqwrq4tdvbgljq7kctf@pchufp4vvegt> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD93AA62145E837FE287BD17FB31885916E472656A32A0A88AB182A05F5380850404C228DA9ACA6FE27512898847DDA8E5BA8F54CD6D5EBA31B742B85EA4683296BAA0EC1D356B06CC8 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE721B3E54BB37EA0B4EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637F898CA578D17CA188638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8706B0E3913D842463B21658C824071D4117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCB816BE3345416868389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8062BEEFFB5F8EA3EF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F7900637B6C899C330ACEAEA22CA9DD8327EE4930A3850AC1BE2E735458315FA5AE40766C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A51502A18FFF0E04C023579C015113BA6A8072219AF6C649E7F87CCE6106E1FC07E67D4AC08A07B9B0034D30FDF2F620DBCB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC7177DD89D51EBB7742424CF958EAFF5D571004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D3473457D764E1CDE77988658A4C0571C3F7811C2B20CA7D0E76771426DE13CCB0034D5F9CDCB0B363D1D7E09C32AA3244CBDB57AFF68183C4FA7B061647D853CDA8A6D4CC6FBFAC251BAD658CF5C8AB4025DA084F8E80FEBD3202CD0F03380D9577A83BD0C44CE203720ABEDE4BBDD9CDD X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojHVl7ekwB6hh6nz2pPBq7LA== X-DA7885C5: E1F40516B259B37B0ABBE2D4FC9E4D957107B1D0A38DCE6CC4C55567BDE60C6C262E2D401490A4A0DB037EFA58388B346E8BC1A9835FDE71 X-Mailru-Sender: 689FA8AB762F73930F533AC2B33E986BBB062625C7CAD1DCEC42C4D1DE564B560FBE9A32752B8C9C2AA642CC12EC09F1FB559BB5D741EB962F61BD320559CF1EFD657A8799238ED55FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 02/19] test: introduce mcode generator for tests X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Maxim! Thanks for the review! Please, see my replies below. On 15.08.23, Maxim Kokryashkin wrote: > Hi, Sergey! > Thanks for the patch! > Please consider my comments below. > > On Wed, Aug 09, 2023 at 06:35:51PM +0300, Sergey Kaplun via Tarantool-patches wrote: > > The test > > depends on particular offset of mcode for side trace regarding the > > parent trace. Before this commit just run some amount of functions to > > generate traces to fill the required mcode range. Unfortunately, this > > approach is not robust, since sometimes trace is not recorded due to > > errors "leaving loop in root trace" observed because of hotcount > > collisions. > > > > This patch introduces the following helpers: > > * `frontend.gettraceno(func)` -- returns the traceno for the given > > function, assumming that there is compiled trace for its prototype > > (i.e. the 0th bytecode is JFUNC). > > * `jit.generators.fillmcode(traceno, size)` fills mcode area of the > > given size from the given trace. It is useful to generate some mcode > > to test jumps to side traces remote enough from the parent. > > --- > > ...8-fix-side-exit-patching-on-arm64.test.lua | 78 ++---------- > > test/tarantool-tests/utils/frontend.lua | 24 ++++ > > test/tarantool-tests/utils/jit/generators.lua | 115 ++++++++++++++++++ > > 3 files changed, 150 insertions(+), 67 deletions(-) > > create mode 100644 test/tarantool-tests/utils/jit/generators.lua > > > > diff --git a/test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua b/test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua > > index 93db3041..678ac914 100644 > > --- a/test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua > > +++ b/test/tarantool-tests/gh-6098-fix-side-exit-patching-on-arm64.test.lua > > diff --git a/test/tarantool-tests/utils/frontend.lua b/test/tarantool-tests/utils/frontend.lua > > index 2afebbb2..414257fd 100644 > > --- a/test/tarantool-tests/utils/frontend.lua > > +++ b/test/tarantool-tests/utils/frontend.lua > > diff --git a/test/tarantool-tests/utils/jit/generators.lua b/test/tarantool-tests/utils/jit/generators.lua > > new file mode 100644 > > index 00000000..62b6e0ef > > --- /dev/null > > +++ b/test/tarantool-tests/utils/jit/generators.lua > > @@ -0,0 +1,115 @@ > > +local M = {} > > + > > +local jutil = require('jit.util') > > + > > +local function getlast_traceno() > > + return misc.getmetrics().jit_trace_num > > +end > > + > > +-- Convert addr to positive value if needed. > > +local function canonize_address(addr) > Nit: most of the time, the `canonize` variant is used in theological materials, > while the `canonicalize` is more common in the sphere of software development. > Feel free to ignore. Fixed, thanks. > > + if addr < 0 then addr = addr + 2 ^ 32 end > > + return addr > > +end > > + > > +-- Need some storage to avoid functions and traces to be > > +-- collected. > Typo: s/Need/We need/ or s/Need some storage/Some storage is needed/ > Typo: s/to be collected/being collected/ Fixed. > > +local recfuncs = {} > > +local last_i = 0 > > +-- This function generates a table of functions with heavy mcode > > +-- payload with tab arithmetics to fill the mcode area from the > > +-- one trace mcode by the some given size. This size is usually > Typo: s/by the some/by some/ Fixed, thanks! > > +-- big enough, because we want to check long jump side exits from > > +-- some traces. > > +-- Assumes, that maxmcode and maxtrace options are set to be sure, > Typo: s/that/that the/ Fixed. > > +-- that we can produce such amount of mcode. > > +function M.fillmcode(trace_from, size) > > + local mcode, addr_from = jutil.tracemc(trace_from) > > + assert(mcode, 'the #1 argument should be an existed trace number') > Typo: s/existed/existing/ Fixed, thanks! > > + addr_from = canonize_address(addr_from) > > + local required_diff = size + #mcode > > + > > + -- Marker to check that traces are not flushed. > > + local maxtraceno = getlast_traceno() > > + local FLUSH_ERR = 'Traces are flushed, check your maxtrace, maxmcode options' > > + > > + local _, last_addr = jutil.tracemc(maxtraceno) > > + last_addr = canonize_address(last_addr) > > + > > + -- Addresses of traces may increase or decrease depending on OS, > > + -- so use absolute diff. > > + while math.abs(last_addr - addr_from) > required_diff do > > + last_i = last_i + 1 > > + -- This is a quite heavy workload (though it doesn't look like > Typo: s/This is a quite/This is quite a/ Fixed. > > + -- one at first). Each load from a table is type guarded. Each > > + -- table lookup (for both stores and loads) is guarded for > > + -- table value and presence of the metatable. The code > Typo: s/and presence/and the presence/ Fixed. > > + -- below results to ~8Kb of mcode for ARM64 and MIPS64 in > Typo: s/results to/results in/ Fixed. > > + -- practice. > > + local fname = ('fillmcode[%d]'):format(last_i) > > + recfuncs[last_i] = assert(loadstring(([[ > > + return function(src) > > + local p = %d > Nit: Poor naming, a more descriptive name is preferred. It has no much sense, because we really don't care about of the function's content. Since it's just moved part of the code, I prefer to leave it as is. Ignoring for now. > > + local tmp = { } > > + local dst = { } > > + -- XXX: use 5 as stop index to reduce LLEAVE (leaving loop > Typo: s/as stop/as a stop/ Fixed, thanks! > > + -- in root trace) errors due to hotcount collisions. > > + for i = 1, 5 do > > + local function tnew(p) > Nit: same issue with naming. Ditto. > > + return { See the iterative patch below: =================================================================== diff --git a/test/tarantool-tests/utils/jit/generators.lua b/test/tarantool-tests/utils/jit/generators.lua index 62b6e0ef..65abfdaa 100644 --- a/test/tarantool-tests/utils/jit/generators.lua +++ b/test/tarantool-tests/utils/jit/generators.lua @@ -7,26 +7,26 @@ local function getlast_traceno() end -- Convert addr to positive value if needed. -local function canonize_address(addr) +local function canonicalize_address(addr) if addr < 0 then addr = addr + 2 ^ 32 end return addr end --- Need some storage to avoid functions and traces to be +-- Some storage is needed to avoid functions and traces being -- collected. local recfuncs = {} local last_i = 0 -- This function generates a table of functions with heavy mcode -- payload with tab arithmetics to fill the mcode area from the --- one trace mcode by the some given size. This size is usually --- big enough, because we want to check long jump side exits from --- some traces. --- Assumes, that maxmcode and maxtrace options are set to be sure, --- that we can produce such amount of mcode. +-- one trace mcode by some given size. This size is usually big +-- enough, because we want to check long jump side exits from some +-- traces. +-- Assumes, that the maxmcode and maxtrace options are set to be +-- sure, that we can produce such amount of mcode. function M.fillmcode(trace_from, size) local mcode, addr_from = jutil.tracemc(trace_from) - assert(mcode, 'the #1 argument should be an existed trace number') - addr_from = canonize_address(addr_from) + assert(mcode, 'the #1 argument should be an existing trace number') + addr_from = canonicalize_address(addr_from) local required_diff = size + #mcode -- Marker to check that traces are not flushed. @@ -34,17 +34,17 @@ function M.fillmcode(trace_from, size) local FLUSH_ERR = 'Traces are flushed, check your maxtrace, maxmcode options' local _, last_addr = jutil.tracemc(maxtraceno) - last_addr = canonize_address(last_addr) + last_addr = canonicalize_address(last_addr) -- Addresses of traces may increase or decrease depending on OS, -- so use absolute diff. while math.abs(last_addr - addr_from) > required_diff do last_i = last_i + 1 - -- This is a quite heavy workload (though it doesn't look like + -- This is quite a heavy workload (though it doesn't look like -- one at first). Each load from a table is type guarded. Each -- table lookup (for both stores and loads) is guarded for - -- table value and presence of the metatable. The code - -- below results to ~8Kb of mcode for ARM64 and MIPS64 in + -- table value and the presence of the metatable. The + -- code below results in ~8Kb of mcode for ARM64 and MIPS64 in -- practice. local fname = ('fillmcode[%d]'):format(last_i) recfuncs[last_i] = assert(loadstring(([[ @@ -52,8 +52,8 @@ function M.fillmcode(trace_from, size) local p = %d local tmp = { } local dst = { } - -- XXX: use 5 as stop index to reduce LLEAVE (leaving loop - -- in root trace) errors due to hotcount collisions. + -- XXX: use 5 as a stop index to reduce LLEAVE (leaving + -- loop in root trace) errors due to hotcount collisions. for i = 1, 5 do tmp.a = src.a * p tmp.j = src.j * p tmp.s = src.s * p tmp.b = src.b * p tmp.k = src.k * p tmp.t = src.t * p @@ -108,7 +108,7 @@ function M.fillmcode(trace_from, size) if not last_addr then error(FLUSH_ERR) end - last_addr = canonize_address(last_addr) + last_addr = canonicalize_address(last_addr) end end =================================================================== > > +end > > + > > +return M > > -- > > 2.41.0 > Best regards, > Maxim Kokryashkin > > -- Best regards, Sergey Kaplun