From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 802D56F3E5; Thu, 11 Nov 2021 18:34:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 802D56F3E5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1636644886; bh=yAYKAGM4QpwiCFAEtFplfA+huBOZIopOz5F2rHcMB0g=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=eANIK319nFZSF5Bq8y8VcuWUao9Xdc3I+Ew0vwpQJF7Kv6F7aKdb6SHR8oYXpiOMu Ml7YYJjnXsgeBIMjPmIWkmUBvptdURzePkf339NR3TYVRQLc6rxOZvW4RFfKe0LYmo 3RNYXj8MvK+zn6mdcdyIHM9l2/6EecuNPWsdTvoE= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 57EC46F3E5 for ; Thu, 11 Nov 2021 18:34:45 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 57EC46F3E5 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1mlC68-0004cL-Ch; Thu, 11 Nov 2021 18:34:44 +0300 Date: Thu, 11 Nov 2021 18:34:37 +0300 To: Mikhail Shishatskiy Message-ID: References: <20210929200758.149446-1-m.shishatskiy@tarantool.org> <20210929200758.149446-4-m.shishatskiy@tarantool.org> <20211027135655.GD8831@tarantool.org> <20211104130156.f2botlihlfhwd3yh@surf.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20211104130156.f2botlihlfhwd3yh@surf.localdomain> X-Clacks-Overhead: GNU Terry Pratchett X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9731B3922EC063979F8160E2FE2FDCF123B0537AAB149205200894C459B0CD1B97AB71B848C6159924A14462C1879910191562ECE1F08447F906DE0852CB233E4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7C6068CE86C2B75F5EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063770F3384661E7DC4D8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8AD979E3A6012FE0AF4E8BA98492B29C9117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735209ECD01F8117BC8BEA471835C12D1D977C4224003CC8364762BB6847A3DEAEFB0F43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A39223F8577A6DFFEA7CC415C329B279CF9D43847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A5F2A7118B93780FC3FB51573C36F4944BD09F5EA31FDD65A6D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75438CC92D4039F4E2410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D349C39CCCDAABCB54B95B846D1A5DBCC4783872D045427B896F3EE20CA5D383DB1F25A4E814ABC23AB1D7E09C32AA3244CD37CE795EDE26719F2C3FE70986EC214C86C126E7119A0FE927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj4t8MBgWr8bLDvVwh9kBO8g== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D7F841DF3F91E849D7C7E3ED43BEFEBD6A7C8D0F45F857DBFE9F1EFEE2F478337FB559BB5D741EB964C8C2C849690F8E70A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit v4 3/4] memprof: group allocations on traces by traceno X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Misha, Thanks for the fixes! LGTM now. On 04.11.21, Mikhail Shishatskiy wrote: > Hi, Igor! > Thank you for the review! > > New commit message: > ============================================================ > memprof: group allocations on traces by traceno > > When LuaJIT executes a trace, the trace number is stored as > the virtual machine state. So, we can treat this number as > an allocation event source in memprof and report allocation events > from traces as well. > > Previously, all the allocations from traces were marked as INTERNAL. > > This patch introduces the functionality described above by adding > a new allocation source type named ASOURCE_TRACE. If allocation event > occurs when trace is executed, trace number and trace's mcode starting > address are streamed to a binary file: > > | loc-trace := trace-no trace-addr > | trace-no := > | trace-addr := > > Also, the memory profiler parser is adjusted to recognize entries > mentioned above. The structure is extended with field , > representing trace number. Trace locations are demangled as > > | TRACE [] > > Resolves tarantool/tarantool#5814 > ============================================================ > > Diff: > ============================================================ > diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua > index 0328c06d..4b7140e9 100644 > --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua > +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua > @@ -113,14 +113,15 @@ local function form_source_line(line) > return string.format("@%s:%d", arg[0], line) > end > > -local function check_alloc_report(alloc, traceno, line, function_line, nevents) > +local function check_alloc_report(alloc, location, nevents) > local expected_name, event > - if traceno ~= 0 then > + local traceno = location.traceno > + if traceno then > expected_name = string.format("TRACE [%d]", traceno) > event = alloc.trace[traceno] > else > - expected_name = form_source_line(function_line) > - event = alloc.line[line] > + expected_name = form_source_line(location.linedefined) > + event = alloc.line[location.line] > end > assert(expected_name == event.name, ("got='%s', expected='%s'"):format( > event.name, > @@ -173,9 +174,9 @@ test:test("output", function(subtest) > -- one is the number of allocations. 1 event - alocation of > -- table by itself + 1 allocation of array part as far it is > -- bigger than LJ_MAX_COLOSIZE (16). > - subtest:ok(check_alloc_report(alloc, 0, 34, 32, 2)) > + subtest:ok(check_alloc_report(alloc, { line = 34, linedefined = 32 }, 2)) > -- 20 strings allocations. > - subtest:ok(check_alloc_report(alloc, 0, 39, 32, 20)) > + subtest:ok(check_alloc_report(alloc, { line = 39, linedefined = 32 }, 20)) > > -- Collect all previous allocated objects. > subtest:ok(free.INTERNAL.num == 22) > @@ -238,9 +239,9 @@ test:test("jit-output", function(subtest) > local free = fill_ev_type(events, symbols, "free") > > -- We expect, that loop will be compiled into a trace. > - subtest:ok(check_alloc_report(alloc, 1, 37, 32, 20)) > + subtest:ok(check_alloc_report(alloc, { traceno = 1 }, 20)) > -- See same checks with jit.off(). > - subtest:ok(check_alloc_report(alloc, 0, 34, 32, 2)) > + subtest:ok(check_alloc_report(alloc, { line = 34, linedefined = 32 }, 2)) > subtest:ok(free.INTERNAL.num == 22) > > -- Restore default JIT settings. > diff --git a/tools/utils/symtab.lua b/tools/utils/symtab.lua > index 85945fb2..c4acc4cb 100644 > --- a/tools/utils/symtab.lua > +++ b/tools/utils/symtab.lua > @@ -77,26 +77,22 @@ function M.id(loc) > return string_format("f%#xl%dt%d", loc.addr, loc.line, loc.traceno) > end > > -local function demangle_lfunc(symtab, loc) > +function M.demangle(symtab, loc) > + if loc.traceno ~= 0 then > + return string_format("TRACE [%d] %#x", loc.traceno, loc.addr) > + end > + > local addr = loc.addr > > if addr == 0 then > return "INTERNAL" > - elseif symtab[addr] then > - return string_format("%s:%d", symtab[addr].source, loc.line) > end > - return string_format("CFUNC %#x", addr) > -end > > -local function demangle_trace(loc) > - return string_format("TRACE [%d] %#x", loc.traceno, loc.addr) > -end > - > -function M.demangle(symtab, loc) > - if loc.traceno ~= 0 then > - return demangle_trace(loc) > + if symtab[addr] then > + return string_format("%s:%d", symtab[addr].source, loc.line) > end > - return demangle_lfunc(symtab, loc) > + > + return string_format("CFUNC %#x", addr) > end > > return M > ============================================================ > > Issue: https://github.com/tarantool/tarantool/issues/5814 > Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number > CI: https://github.com/tarantool/tarantool/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number > > On 27.10.2021 16:56, Igor Munkin wrote: > >Misha, > > > >Thanks for the patch! Please, consider the comments below. > > > >On 29.09.21, Mikhail Shishatskiy wrote: > > > > >Typo: s/streamed/is streamed/. > > I assume, there should be s/streamed/are streamed/. Precisely. > > >> > >> Resolves tarantool/tarantool#5814 > > > >Does this patch "resolve" the issue or only being a "part of" it? > > IMO, this patch _resolves_ the issue, as we are now able to get > information about allocations on particular traces: > > | TRACE [] > > And this is exactly what was requested in the issue. Got it, thanks. > > > > >-- > >Best regards, > >IM > > -- > Best regards, > Mikhail Shishatskiy -- Best regards, IM