From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9854A454EA4; Tue, 16 May 2023 13:55:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9854A454EA4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1684234550; bh=ZUjT1/6WQng0u9KoIpoUpUP5QXM3Vz7/UQiEvcZGY0k=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=owAS/jQ1XWZ5usg7pzE2s53bapM0KGjXLI42dnxbScT6KtOacsT1xnDUOrhy3Nuv+ KmrpWufHZGsHsxkSj+n40siHLEDah/IFYwwwDWyLaPp5EXzfvk5qslxH24fySuWr8O mC/Gr39M8AYq65GuJREnqT5BXMPKCVKHsn//AwO4= Received: from smtp35.i.mail.ru (smtp35.i.mail.ru [95.163.41.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 6E4182B8A01 for ; Tue, 16 May 2023 13:55:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6E4182B8A01 Received: by smtp35.i.mail.ru with esmtpa (envelope-from ) id 1pysLL-003omf-Nd; Tue, 16 May 2023 13:55:48 +0300 Message-ID: <4959c9e3-e0e3-7d26-1a7f-836849cecf4c@tarantool.org> Date: Tue, 16 May 2023 13:55:47 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 To: Sergey Kaplun , Maxim Kokryashkin Cc: tarantool-patches@dev.tarantool.org References: <44354d682d3cabf5a3ab1a268f900d7e6f981020.1683720396.git.skaplun@tarantool.org> Content-Language: en-US In-Reply-To: <44354d682d3cabf5a3ab1a268f900d7e6f981020.1683720396.git.skaplun@tarantool.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailru-Src: smtp X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92B2351A05AA37E8C85389922DA3DCC4DFF6E0EBA22BA430F00894C459B0CD1B9C61D92DE246C2EC2CF94691D3B2F14FB1E429EF47AE4D89F4ED6240D679448A3 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE745C0EDBD94D46193EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637CC84CAE328DCD2648638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8D419CA3736B5495CFB405CF84482F3A2117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC406C66621D3021AFA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520437C869540D2AB0FBDFBBEFFF4125B51D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE652FD71AFB96DC7D28F6BDBBAB179F4ED8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE3A85A14DF5F041C992D242C3BD2E3F4C6C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F79006372DAFD17ABDE6187BEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C1DE0DAB: 0D63561A33F958A558AFBF6AC03D42D856E9CE33045975E1E5E046773473FECEF87CCE6106E1FC07E67D4AC08A07B9B0DB8A315C1FF4794DBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF77DD89D51EBB7742D3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF97DCD9C8BB91AB412FF1139D480A2C27C6BED2EB427CBCF74D124A8A6C2EE864A5CD7B9A905003229416E56A45473B18937D84595BAD20A238A2500CA0B01DABA8CAA110EC39F0D1 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbsH3UG36S2Yp26FQsBGb4g== X-Mailru-Sender: 11C2EC085EDE56FAC07928AF2646A769A0033A896A7B0B00873C70AE669127BAC9F871340C5A6C40EBA65886582A37BD66FEC6BF5C9C28D98A98C1125256619760D574B6FC815AB872D6B4FCE48DF648AE208404248635DF X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] test: add utility for parsing `jit.dump` X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hello, Sergey Thanks for the patch! See my three comments inline. On 5/10/23 15:34, Sergey Kaplun wrote: > This commit adds simple parser for traces to be analyzed in the test. > For now nothing too fancy at all -- just start `jit.dump` to temporary > file and parse and remove this file when dump is stopped. The array > with resulting traces is returned. > > For now, just find a single IR by pattern and return ref number and IR, > if exists. More functionality may be added if it will be necessary for > tests. > --- > test/tarantool-tests/utils/jit_parse.lua | 156 +++++++++++++++++++++++ > 1 file changed, 156 insertions(+) > create mode 100644 test/tarantool-tests/utils/jit_parse.lua > > diff --git a/test/tarantool-tests/utils/jit_parse.lua b/test/tarantool-tests/utils/jit_parse.lua > new file mode 100644 > index 00000000..fe8e0e08 > --- /dev/null > +++ b/test/tarantool-tests/utils/jit_parse.lua The main purpose of this script is using in tests. It could be dangerous to leave it without tests at all, breakage in it could require time for debugging someday. From other side I believe no one wants writing tests for test infrastructure. I suggest add at least a couple of unit tests that will parse trace in plain text format and check that results are equal to expected. > @@ -0,0 +1,156 @@ > +local jdump = require('jit.dump') > + > +local M = {} > + > +local trace_mt = {} > +trace_mt.__index = trace_mt > + > +-- Find and return the first IR ref and value matched the pattern > +-- for the trace. > +trace_mt.has_ir = function(trace, ir_pattern) > + for i = 1, #trace.ir do > + local ir = trace.ir[i] > + if ir:match(ir_pattern) then > + return i, ir > + end > + end > +end > + > +local function trace_new(n) > + return setmetatable({ > + number = n, > + parent = nil, > + parent_exitno = nil, > + start_loc = nil, > + bc = {}, > + ir = {}, > + mcode = {}, > + snaps = {}, > + }, trace_mt) > +end > + > +local function parse_bc(trace, line) > + trace.bc[#trace.bc + 1] = line > +end > + > +local function parse_snap(trace, n_snap, line) > + assert(trace.snaps[n_snap] == nil) > + trace[n_snap] = {ref = #trace.ir + 1, slots = line:match('%[(.*)%]$'),} > +end > + > +local function parse_ir(trace, line) > + local n_snap = line:match('SNAP%s+#(%d+)') > + if n_snap then > + parse_snap(trace, n_snap, line) > + return > + end > + > + local current_ins, ir = line:match('^(%d+)%s+(.*)$') > + current_ins = tonumber(current_ins) > + -- Insert NOP instruction hidden in IR dump. > + if current_ins ~= #trace.ir + 1 then > + trace.ir[#trace.ir + 1] = 'nil NOP' > + end > + assert(current_ins == #trace.ir + 1) > + trace.ir[current_ins] = ir > +end > + > +local function parse_mcode(trace, line) > + -- Skip loop label. > + if line == '-> LOOP:' then > + return > + end > + local addr, instruction = line:match('^(%w+)%s+(.*)$') > + trace.mcode[#trace.mcode + 1] = {addr = addr, instruction = instruction,} > +end > + > +local function parse_trace_hdr(ctx, line, trace_num, status) > + local traces = ctx.traces > + if status == 'start' then > + local trace = trace_new(trace_num) > + trace.parent, trace.parent_exitno = line:match('start (%d+)/(%d+)') > + -- XXX: Assume, that source line can't contain spaces. > + -- For example, it's not "(command line)". > + trace.start_loc = line:match(' ([^%s]+)$') > + traces[trace_num] = trace > + ctx.parsing_trace = trace_num > + ctx.parsing = 'bc' > + elseif status == 'stop' then > + assert(ctx.parsing_trace == trace_num) > + ctx.parsing_trace = nil > + ctx.parsing = nil > + elseif status == 'abort' then > + assert(ctx.parsing_trace == trace_num) > + ctx.parsing_trace = nil > + ctx.parsing = nil > + traces[trace_num] = nil > + elseif status == 'IR' then > + ctx.parsing = 'IR' > + elseif status == 'mcode' then > + ctx.parsing = 'mcode' > + else > + error('Unknown trace status: ' .. status) > + end > +end > + > +local function parse_line(ctx, line) > + if line == '' then > + return > + end > + > + if line:match('TRACE flush') then > + ctx.traces = {} > + return > + end > + > + local trace_num, status = line:match('TRACE (%d+) (%w+)') > + if trace_num then > + parse_trace_hdr(ctx, line, tonumber(trace_num), status) > + return > + end > + > + assert(ctx.parsing_trace) > + > + local trace = ctx.traces[ctx.parsing_trace] > + if ctx.parsing == 'bc' then > + parse_bc(trace, line) > + elseif ctx.parsing == 'IR' then > + parse_ir(trace, line) > + elseif ctx.parsing == 'mcode' then > + parse_mcode(trace, line) > + end > +end > + > +local JDUMP_FILE > + > +local function parse_jit_dump() > + local ctx = {traces = {}} > + for line in io.lines(JDUMP_FILE) do > + parse_line(ctx, line) > + end > + return ctx.traces > +end > + > +M.start = function(flags) Comment with description of function's arguments will be useful here and M.finish(). For internal functions too, but it is up to you. > + assert(JDUMP_FILE == nil, 'jit_parse is already running') > + -- Always use plain text output. > + flags = flags .. 'T' > + JDUMP_FILE = os.tmpname() > + jdump.start(flags, JDUMP_FILE) > +end > + > +M.finish = function() > + assert(JDUMP_FILE ~= nil, 'jit_parse is not running') > + jdump.off() > + local traces = parse_jit_dump() > + os.remove(JDUMP_FILE) > + JDUMP_FILE = nil > + return traces > +end > + > +-- Turn off compilation for these modules to avoid side effects. > +jit.off(true, true) > +jit.off(jdump.on, true) > +jit.off(jdump.off, true) I would put these three lines on top of file or even disable/enable compilation in M.start() and M.finish(). Without enabling compilation back you are mutating environment on using this script and this can be unexpected for those who will use scripts. > + > +return M