From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 314C06EC55; Sat, 21 Aug 2021 15:51:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 314C06EC55 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1629550282; bh=D5Mmaqd4y1hJO7nWlDjxukRiyHll2K9yFlvJ2VE9ktI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=viyIZ8HI2TU7/P9fAKaBNrg1K3+G1XbA+aEdxK5mPrgARo+BIRtwbU8zbaC+1pYR1 9O95UiuQ7sWCEGcq3VmVFIO+lGz5bRAh4XvhIav/cKgammveXgQytm8/kZR6Fk8nUm F1XswS3iiw0X3Pq5MbiTLGNVkixMpcPUIMpQhCyk= Received: from smtp57.i.mail.ru (smtp57.i.mail.ru [217.69.128.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 56E416EC57 for ; Sat, 21 Aug 2021 15:50:39 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 56E416EC57 Received: by smtp57.i.mail.ru with esmtpa (envelope-from ) id 1mHQSM-0007xl-5U; Sat, 21 Aug 2021 15:50:38 +0300 To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Sat, 21 Aug 2021 19:50:00 +0700 Message-Id: <20210821125002.408132-3-m.shishatskiy@tarantool.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210821125002.408132-1-m.shishatskiy@tarantool.org> References: <20210821125002.408132-1-m.shishatskiy@tarantool.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD91BCCB18F2C129F87F36E61E9E4584E9D182A05F53808504019B3F2D1821C8B762544690EA1993C066EF8C80228D8C15117408F313C42A00E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7364F8074C6DFACE2C2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE7747FD4AFDBA6C67AEA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F291493D544CDAC22646DCB95780A45B68CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F6B57BC7E64490618DEB871D839B7333395957E7521B51C2DFABB839C843B9C08941B15DA834481F8AA50765F7900637F6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947CCE135D2742255B359735652A29929C6C4AD6D5ED66289B52698AB9A7B718F8C46E0066C2D8992A16725E5C173C3A84C3C6B67B73CDEA4D6EBA3038C0950A5D36B5C8C57E37DE458B0BC6067A898B09E46D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE78DD9044B304389D4731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C8DC2057B3FE6961B3D1FB0FD35D30A47F348EE59802F33689C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EFFE5EE7C534F234F5699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34087368DEEB028F17A968451E827A153C9AA7BEBF52A08634484BB7D87976321C904E275D8E3845DC1D7E09C32AA3244C469687355FEEE73B47FCB876A33975BB408A6A02710B7304927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojzcVJReGZsqOGMkt8yIpyCg== X-Mailru-Sender: 2317F5BEA8D613097CB8A225516EF02745036F1BE5D654E3052F84546A83495F3B8F6AE1EE37632FFF6B26DEAE20951894E739346DD9ABB838AE394B99C5394F99323991B04ED3930F27244EEAA5B9A5AE208404248635DF X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit v1 2/4] memprof: demangle symbols on the spot X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mikhail Shishatskiy via Tarantool-patches Reply-To: Mikhail Shishatskiy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" There may be situations, when new prototype has the same address than the previously dumped one. In this case, the new symtab entry will shadow the preceeding, and if we demangle symbols when all the events are parsed, we can get irrelevant information: | Dump symtab on the start: | addr name | * 0x001 proto_1 | * 0x002 proto_2 | * 0x003 proto_3 | | Memprof running: | ... | * alloc: 0x001 proto_1 | // 0x001 proto_1 collected by the GC | // new proto loaded on the 0x001 address | * alloc: 0x001 proto_NEW | + symtab enriching -> [0x001] = proto_NEW If we demangle symbols in the end, we will get 2 allocations by proto_NEW because of the collision. But since symtab enriching is implemented, we have enough information to avoid collision while parsing events, demangling now happens on the spot, while symtab entries are not shadowed by new ones. Follows up tarantool/tarantool#5815 --- Issue: https://github.com/tarantool/tarantool/issues/5815 Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5815-enrich-symtab-when-prototype-is-allocated Tarantool branch: https://github.com/tarantool/tarantool/tree/shishqa/gh-5815-enrich-symtab-when-prototype-is-allocated tools/memprof.lua | 4 ++-- tools/memprof/humanize.lua | 16 ++++++++-------- tools/memprof/parse.lua | 26 +++++++++++++------------- tools/memprof/process.lua | 12 +++++------- 4 files changed, 28 insertions(+), 30 deletions(-) diff --git a/tools/memprof.lua b/tools/memprof.lua index 18b44fdd..496a0d78 100644 --- a/tools/memprof.lua +++ b/tools/memprof.lua @@ -102,9 +102,9 @@ local function dump(inputfile) local symbols = symtab.parse(reader) local events = memprof.parse(reader, symbols) if not leak_only then - view.profile_info(events, symbols) + view.profile_info(events) end - local dheap = process.form_heap_delta(events, symbols) + local dheap = process.form_heap_delta(events) view.leak_info(dheap) os.exit(0) end diff --git a/tools/memprof/humanize.lua b/tools/memprof/humanize.lua index 800a465e..0360362d 100644 --- a/tools/memprof/humanize.lua +++ b/tools/memprof/humanize.lua @@ -7,7 +7,7 @@ local symtab = require "utils.symtab" local M = {} -function M.render(events, symbols) +function M.render(events) local ids = {} for id, _ in pairs(events) do @@ -21,15 +21,15 @@ function M.render(events, symbols) for i = 1, #ids do local event = events[ids[i]] print(string.format("%s: %d events\t+%d bytes\t-%d bytes", - M.describe_location(symbols, event.loc), + ids[i], event.num, event.alloc, event.free )) local prim_loc = {} - for _, heap_chunk in pairs(event.primary) do - table.insert(prim_loc, M.describe_location(symbols, heap_chunk.loc)) + for id, _ in pairs(event.primary) do + table.insert(prim_loc, id) end if #prim_loc ~= 0 then table.sort(prim_loc) @@ -42,17 +42,17 @@ function M.render(events, symbols) end end -function M.profile_info(events, symbols) +function M.profile_info(events) print("ALLOCATIONS") - M.render(events.alloc, symbols) + M.render(events.alloc) print("") print("REALLOCATIONS") - M.render(events.realloc, symbols) + M.render(events.realloc) print("") print("DEALLOCATIONS") - M.render(events.free, symbols) + M.render(events.free) print("") end diff --git a/tools/memprof/parse.lua b/tools/memprof/parse.lua index 888c6636..a540360b 100644 --- a/tools/memprof/parse.lua +++ b/tools/memprof/parse.lua @@ -9,6 +9,7 @@ local band = bit.band local lshift = bit.lshift local symtab = require "utils.symtab" +local humanize = require "memprof.humanize" local string_format = string.format @@ -66,12 +67,13 @@ local function link_to_previous(heap_chunk, e, nsize) end end -local function id_location(addr, line, traceno) - return string_format("f%#xl%dt%d", addr, line, traceno), { +local function id_location(addr, line, traceno, symbols) + local loc = { addr = addr, line = line, traceno = traceno, } + return humanize.describe_location(symbols, loc), loc end local function parse_location_symbols(reader, asource, symbols) @@ -79,21 +81,22 @@ local function parse_location_symbols(reader, asource, symbols) return id_location( symtab.parse_sym_lfunc(reader, symbols), reader:read_uleb128(), - 0 + 0, + symbols ) end error("Unknown asource "..asource) end -local function parse_location_common(reader, asource) +local function parse_location_common(reader, asource, symbols) if asource == ASOURCE_INT then - return id_location(0, 0, 0) + return id_location(0, 0, 0, symbols) elseif asource == ASOURCE_CFUNC then - return id_location(reader:read_uleb128(), 0, 0) + return id_location(reader:read_uleb128(), 0, 0, symbols) elseif asource == ASOURCE_LFUNC then - return id_location(reader:read_uleb128(), reader:read_uleb128(), 0) + return id_location(reader:read_uleb128(), reader:read_uleb128(), 0, symbols) elseif asource == ASOURCE_TRACE then - return id_location(reader:read_uleb128(), 0, reader:read_uleb128()) + return id_location(reader:read_uleb128(), 0, reader:read_uleb128(), symbols) end error("Unknown asource "..asource) end @@ -101,13 +104,10 @@ end local function parse_location(reader, asource, symbols) local has_symbols = band(asource, LJM_SYMTAB) local asource_type = band(asource, ASOURCE_TYPE_MASK) - local id, loc if has_symbols == LJM_SYMTAB then - id, loc = parse_location_symbols(reader, asource_type, symbols) - else - id, loc = parse_location_common(reader, asource_type) + return parse_location_symbols(reader, asource_type, symbols) end - return id, loc + return parse_location_common(reader, asource_type, symbols) end local function parse_alloc(reader, asource, events, heap, symbols) diff --git a/tools/memprof/process.lua b/tools/memprof/process.lua index f277ed84..0a39d333 100644 --- a/tools/memprof/process.lua +++ b/tools/memprof/process.lua @@ -2,9 +2,7 @@ local M = {} -local humanize = require "memprof.humanize" - -function M.form_heap_delta(events, symbols) +function M.form_heap_delta(events) -- Auto resurrects source event lines for counting/reporting. local dheap = setmetatable({}, {__index = function(t, line) rawset(t, line, { @@ -15,9 +13,9 @@ function M.form_heap_delta(events, symbols) return t[line] end}) - for _, event in pairs(events.alloc) do + for id, event in pairs(events.alloc) do if event.loc then - local ev_line = humanize.describe_location(symbols, event.loc) + local ev_line = id if (event.alloc > 0) then dheap[ev_line].dbytes = dheap[ev_line].dbytes + event.alloc @@ -36,8 +34,8 @@ function M.form_heap_delta(events, symbols) -- Realloc and free events always have key named "primary" -- that references the table with memory changed -- (may be empty). - for _, heap_chunk in pairs(event.primary) do - local ev_line = humanize.describe_location(symbols, heap_chunk.loc) + for id, heap_chunk in pairs(event.primary) do + local ev_line = id if (heap_chunk.alloced > 0) then dheap[ev_line].dbytes = dheap[ev_line].dbytes + heap_chunk.alloced -- 2.32.0