From: Igor Munkin via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: Mikhail Shishatskiy <m.shishatskiy@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: Re: [Tarantool-patches] [PATCH luajit v4 4/4] memprof: add info about trace start to symtab Date: Mon, 1 Nov 2021 19:31:14 +0300 [thread overview] Message-ID: <20211101163113.GE8831@tarantool.org> (raw) In-Reply-To: <20210929200758.149446-5-m.shishatskiy@tarantool.org> Misha, Thanks for the patch! Please consider my comments below. On 29.09.21, Mikhail Shishatskiy wrote: > Trace allocation sources, recorded by the memory profiler, > were reported as Minor: As for me, it's better to say it the following way: | Allocation events occurred on traces are recorded by the memory | profiler the following way now > > | TRACE [<trace-no>] <trace-addr> > > This approach is not descriptive enough to understand, where > exactly allocation took place, as we do not know the code > chunk, associated with the trace. > > This patch fixes the problem described above by extending the > symbol table with <sym-trace> entries, consisting of a trace's > mcode starting address, trace number, address of function proto, > and line, where trace recording started: > > | sym-trace := sym-header trace-no trace-addr sym-addr sym-line > | trace-no := <ULEB128> > | trace-addr := <ULEB128> > > The memory profiler parser is adjusted to recognize the entries > mentioned above. On top of that, the API of <utils/symtab.lua> changed: > now table with symbols contains two tables: `lfunc` for Lua functions > symbols and `trace` for trace entries. > > The demangler module has not changed, but the function > `describe_location` is added to the <memprof/humanize.lua> module, > which allows one to get a description of the trace location in the > format described below: > > | TRACE [<trace-no>] <trace-addr> started at @<sym-chunk>:<sym-line> > > Follows up tarantool/tarantool#5814 > --- > > Issue: https://github.com/tarantool/tarantool/issues/5814 > Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number > CI: https://github.com/tarantool/tarantool/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number > > src/lj_memprof.c | 43 +++++++++++++++++++ > src/lj_memprof.h | 8 +++- > .../misclib-memprof-lapi.test.lua | 15 ++++--- > tools/memprof.lua | 4 +- > tools/memprof/humanize.lua | 30 ++++++++++--- > tools/memprof/process.lua | 9 ++-- > tools/utils/symtab.lua | 31 ++++++++++--- > 7 files changed, 118 insertions(+), 22 deletions(-) > > diff --git a/src/lj_memprof.c b/src/lj_memprof.c > index 8702557f..e8b2ebbc 100644 > --- a/src/lj_memprof.c > +++ b/src/lj_memprof.c > @@ -28,6 +28,45 @@ > static const unsigned char ljs_header[] = {'l', 'j', 's', LJS_CURRENT_VERSION, > 0x0, 0x0, 0x0}; > > +#if LJ_HASJIT > + > +static void dump_symtab_trace(struct lj_wbuf *out, const GCtrace *trace) > +{ > + GCproto *pt = &gcref(trace->startpt)->pt; > + BCLine lineno = 0; > + > + const BCIns *startpc = mref(trace->startpc, const BCIns); > + lua_assert(startpc >= proto_bc(pt) && > + startpc < proto_bc(pt) + pt->sizebc); > + > + lineno = lj_debug_line(pt, proto_bcpos(pt, startpc)); > + lua_assert(lineno >= 0); I doubt whether this assertion is not always true. If it is, then what does it check? > + <snipped> > diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua > index 3f4ffea0..b9edb80d 100644 > --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua > +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua > @@ -87,9 +87,13 @@ local function fill_ev_type(events, symbols, event_type) > local addr = event.loc.addr > local traceno = event.loc.traceno > > - if traceno ~= 0 then > + if traceno ~= 0 and symbols.trace[traceno] then > + local trace_loc = symbols.trace[traceno].start > + addr = trace_loc.addr > ev_type.trace[traceno] = { > - name = string.format("TRACE [%d]", traceno), > + name = string.format("TRACE [%d] %s:%d", > + traceno, symbols.lfunc[addr].source, symbols.lfunc[addr].linedefined <trace_loc.line> need to be used as a last argument in <string.format> instead of <symbols.lfunc[addr].linedefined>, don't you? > + ), > num = event.num, > } > elseif addr == 0 then <snipped> > @@ -116,7 +120,8 @@ end > local function check_alloc_report(alloc, traceno, line, function_line, nevents) > local expected_name, event > if traceno ~= 0 then > - expected_name = string.format("TRACE [%d]", traceno) > + expected_name = string.format("TRACE [%d] ", traceno).. > + form_source_line(function_line) The output format differs from the one produced by memprof parser, doesn't it? > event = alloc.trace[traceno] > else > expected_name = form_source_line(function_line) <snipped> > diff --git a/tools/memprof/humanize.lua b/tools/memprof/humanize.lua > index 7771005d..7d30f976 100644 > --- a/tools/memprof/humanize.lua > +++ b/tools/memprof/humanize.lua > @@ -7,6 +7,23 @@ local symtab = require "utils.symtab" > > local M = {} > > +function M.describe_location(symbols, loc) There is no need to export <describe_location> from humanize.lua, so it can be just a local function within this Lua chunk. > + if loc.traceno == 0 then > + return symtab.demangle(symbols, loc) > + end > + > + local trace = symbols.trace[loc.traceno] > + > + -- If trace, which was remembered in the symtab, has not > + -- been flushed, assotiate it with a proto, where trace Typo: s/assotiate/associate/. > + -- recording started. > + if trace and trace.addr == loc.addr then > + return symtab.demangle(symbols, loc).." started at ".. > + symtab.demangle(symbols, trace.start) Finally, I got the thing that bothers me the most. Why do you make <describe_location> so complex? It looks that you can move all these if-else branching to <symtab.demangle> and concatenation to <demangle_trace> function, doesn't it? AFAICS, you can remove <describe_location> as a result and trace demangling will be encapsulated in scope of <demangle_trace> function. Feel free to correct me if I'm wrong. > + end > + return symtab.demangle(symbols, loc) > +end > + <snipped> > diff --git a/tools/utils/symtab.lua b/tools/utils/symtab.lua > index 85945fb2..496d8480 100644 > --- a/tools/utils/symtab.lua > +++ b/tools/utils/symtab.lua <snipped> > @@ -24,18 +25,38 @@ local function parse_sym_lfunc(reader, symtab) > local sym_chunk = reader:read_string() > local sym_line = reader:read_uleb128() > > - symtab[sym_addr] = { > + symtab.lfunc[sym_addr] = { > source = sym_chunk, > linedefined = sym_line, > } > end > > +local function parse_sym_trace(reader, symtab) > + local traceno = reader:read_uleb128() > + local trace_addr = reader:read_uleb128() > + local sym_addr = reader:read_uleb128() > + local sym_line = reader:read_uleb128() > + > + symtab.trace[traceno] = { > + addr = trace_addr, > + start = { > + addr = sym_addr, > + line = sym_line, > + traceno = 0, Please, leave a comment regarding the fact the structure is the same as the one yielded from <parse_location> in memprof/parse.lua. > + }, > + } > +end > + <snipped> > -- > 2.33.0 > -- Best regards, IM
next prev parent reply other threads:[~2021-11-01 16:31 UTC|newest] Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-20 7:05 [Tarantool-patches] [PATCH luajit v3 0/4] memprof: group allocations on traces by trace number Mikhail Shishatskiy via Tarantool-patches 2021-08-20 7:05 ` [Tarantool-patches] [PATCH luajit v3 1/5] core: add const to lj_debug_line proto parameter Mikhail Shishatskiy via Tarantool-patches 2021-09-16 15:29 ` Igor Munkin via Tarantool-patches 2021-08-20 7:05 ` [Tarantool-patches] [PATCH luajit v3 2/5] test: separate memprof Lua API tests into subtests Mikhail Shishatskiy via Tarantool-patches 2021-09-16 15:29 ` Igor Munkin via Tarantool-patches 2021-08-20 7:05 ` [Tarantool-patches] [PATCH luajit v3 3/5] memprof: dump traceno if allocate from trace Mikhail Shishatskiy via Tarantool-patches 2021-09-16 15:32 ` Igor Munkin via Tarantool-patches 2021-09-29 19:21 ` Mikhail Shishatskiy via Tarantool-patches 2021-08-20 7:05 ` [Tarantool-patches] [PATCH luajit v3 4/5] memprof: extend symtab with info about traces Mikhail Shishatskiy via Tarantool-patches 2021-09-16 15:32 ` Igor Munkin via Tarantool-patches 2021-09-29 19:21 ` Mikhail Shishatskiy via Tarantool-patches 2021-08-20 7:05 ` [Tarantool-patches] [PATCH luajit v3 5/5] luajit: change order of modules Mikhail Shishatskiy via Tarantool-patches 2021-09-16 15:32 ` Igor Munkin via Tarantool-patches 2021-09-29 20:07 ` [Tarantool-patches] [PATCH luajit v4 0/4] memprof: group allocations on traces by traceno Mikhail Shishatskiy via Tarantool-patches 2021-09-29 20:07 ` [Tarantool-patches] [PATCH luajit v4 1/4] test: separate memprof Lua API tests into subtests Mikhail Shishatskiy via Tarantool-patches 2021-10-27 13:56 ` Igor Munkin via Tarantool-patches 2021-10-27 15:07 ` Sergey Kaplun via Tarantool-patches 2021-09-29 20:07 ` [Tarantool-patches] [PATCH luajit v4 2/4] memprof: refactor location parsing Mikhail Shishatskiy via Tarantool-patches 2021-10-27 13:56 ` Igor Munkin via Tarantool-patches [not found] ` <20211104130010.mcvnra6e4yl5moo2@surf.localdomain> 2021-11-10 15:38 ` Igor Munkin via Tarantool-patches 2021-09-29 20:07 ` [Tarantool-patches] [PATCH luajit v4 3/4] memprof: group allocations on traces by traceno Mikhail Shishatskiy via Tarantool-patches 2021-10-27 13:56 ` Igor Munkin via Tarantool-patches [not found] ` <20211104130156.f2botlihlfhwd3yh@surf.localdomain> 2021-11-11 15:34 ` Igor Munkin via Tarantool-patches 2021-09-29 20:07 ` [Tarantool-patches] [PATCH luajit v4 4/4] memprof: add info about trace start to symtab Mikhail Shishatskiy via Tarantool-patches 2021-11-01 16:31 ` Igor Munkin via Tarantool-patches [this message] [not found] ` <20211104130228.x6qcne5xeh544hm7@surf.localdomain> 2021-11-12 13:34 ` Igor Munkin via Tarantool-patches 2021-11-17 8:17 ` Sergey Kaplun via Tarantool-patches 2021-11-22 15:11 ` Mikhail Shishatskiy via Tarantool-patches 2021-11-24 12:42 ` Mikhail Shishatskiy via Tarantool-patches 2021-11-24 16:44 ` Igor Munkin via Tarantool-patches 2022-01-27 23:29 ` [Tarantool-patches] [PATCH luajit v4 0/4] memprof: group allocations on traces by traceno Igor Munkin via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20211101163113.GE8831@tarantool.org \ --to=tarantool-patches@dev.tarantool.org \ --cc=imun@tarantool.org \ --cc=m.shishatskiy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH luajit v4 4/4] memprof: add info about trace start to symtab' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox