From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 284A158AE3D; Tue, 15 Aug 2023 22:07:21 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 284A158AE3D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1692126441; bh=aXbYZpvRe8LAJF0DaQGJ0z0EOUOLwschXvNq3DH1rtI=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ejkHM8KIRZKnPl/qJlrVlRdXXC/xb2xwNB/HeyRtPGQzavfqqgabSEujXtpsxQKgf U0evVxVHgoA4KkHGevWezb9uktY0jfPNEJngsbCRZX3yWXDFjtZPFkmMLBxv1MLfdV 9w4QoXHLO4BJ0jhnrdaa43zMhE0ciBZBQGyvu3Es= Received: from smtp33.i.mail.ru (smtp33.i.mail.ru [95.163.41.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 3DF5958AE3F for ; Tue, 15 Aug 2023 22:07:04 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3DF5958AE3F Received: by smtp33.i.mail.ru with esmtpa (envelope-from ) id 1qVzNf-000Sn7-1L; Tue, 15 Aug 2023 22:07:03 +0300 Date: Tue, 15 Aug 2023 18:52:00 +0000 To: Maxim Kokryashkin Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD969E04B5EED670DC83148C852D424A0A0119F8DAA2A6AAA9D182A05F538085040648EEFD06B88A9794ED2BB092006B0D20E0B05B49C83AA16E5E2CC6504696EBF X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7AB524098FB2F2222EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637BEEB0D1ADA650026EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BE5CCB53A13BC8DBAD1E55497DEC0FCEEEEA3CB427EA9E474CC7F00164DA146DAFE8445B8C89999728AA50765F7900637876557C5A8D58DD0389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8FA486DC37A503D0BF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C457EE4B4996FC54603F1AB874ED890284AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C3CD582FA7F3E0DDE5BA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE76D0F27F7E6A6C418731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: 1B70FBA5C9BEEE72C9761FC34675ADEB871C96603B655635EE9D5CB6078CC77C9B1735672830C97ED896CDA25F6F05AD X-C1DE0DAB: 0D63561A33F958A5D0B3DD75B6E69832BF91208797EF1A85A5BB2383CD4DD81EF87CCE6106E1FC07E67D4AC08A07B9B0B355ED1E20F5346ACB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC71106E36FF2641B7B8424CF958EAFF5D571004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D34D8FD11F74BDD6E8D644F2EF755408D324B00B9B952868D6D142EE766DA251FE171B4042278FD23C71D7E09C32AA3244C9D8C3556A5A2DFB757FAC6A148EF5C5C24AF4FAF06DA24FDED9A463B7D0AAFF65DA084F8E80FEBD376A4ED3E9341DC2FCD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojHVl7ekwB6hhat6WhFdP0WQ== X-Mailru-Sender: 2FEBA92C8E508479FE7B9A1DF348D5313B602CCADF384B675E1FB71C590D77D03893FEE2500F63042326FE6F2A341ACE0FB9F97486540B4CD9E8847AB8CFED4D9ABF8A61C016C2CFB0DAF586E7D11B3E67EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit v3 4/4] sysprof: improve parser's memory footprint X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Max, Thanks for the patch! Everything is OK in general, but please consider my comments below. On 31.07.23, Maxim Kokryashkin wrote: > This patch reduces sysprof's parser memory footprint, > by avoiding reading all callchains before collapsing them. > Instead of it, parser merges stacks immediately after > reading them and stores counts in a lua table. > > The `collapse.lua` module is purged as a result of the > patch, but it is left as a stub to keep the integrational > testing intact. This stub should be removed in the next > series. > > Resolves tarantool/tarantool#8700 > --- > tools/CMakeLists.txt | 4 ++ > tools/sysprof.lua | 21 +------ > tools/sysprof/collapse.lua | 123 +----------------------------------- > tools/sysprof/parse.lua | 126 ++++++++++++++++++++++++++----------- > 4 files changed, 101 insertions(+), 173 deletions(-) > > diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt > index dd7ec6bd..1ae559ee 100644 > --- a/tools/CMakeLists.txt > +++ b/tools/CMakeLists.txt > @@ -112,6 +112,8 @@ else() > add_custom_target(tools-parse-sysprof EXCLUDE_FROM_ALL DEPENDS > luajit-parse-sysprof > sysprof/parse.lua > + # FIXME: This line is not deleted only for the sake of integrational > + # testing. It should be deleted in the next series. Minor: I'd rather left TODO instead of FIXME, but this is not a big deal, so feel free to ignore. > sysprof/collapse.lua > sysprof.lua > utils/bufread.lua > @@ -121,6 +123,8 @@ else() > > install(FILES > ${CMAKE_CURRENT_SOURCE_DIR}/sysprof/parse.lua > + # FIXME: This line is not deleted only for the sake of integrational > + # testing. It should be deleted in the next series. Ditto. > ${CMAKE_CURRENT_SOURCE_DIR}/sysprof/collapse.lua > DESTINATION ${LUAJIT_DATAROOTDIR}/sysprof > PERMISSIONS > diff --git a/tools/sysprof/collapse.lua b/tools/sysprof/collapse.lua > index ac5269ea..9e815e0d 100755 > --- a/tools/sysprof/collapse.lua > +++ b/tools/sysprof/collapse.lua > @@ -1,120 +1,3 @@ > +-- FIXME: This line is not deleted only for the sake of > +-- integrational testing. It should be deleted in the > +-- next series. Honestly, I would literally "purge" collapse.lua the following way: replace all of its contents with the only assert call to check that nobody will use it. However, if it breaks Tarantool, I agree to left it intact until the file is removed completely from the source tree. > diff --git a/tools/sysprof/parse.lua b/tools/sysprof/parse.lua > index 5b52f104..19add4f3 100755 > --- a/tools/sysprof/parse.lua > +++ b/tools/sysprof/parse.lua > @@ -143,18 +153,63 @@ local function parse_symtab(reader, symbols, vmstate) > > +local function insert_lua_callchain(chain, lua) > + local ins_cnt = 0 > + local name_lua > + for _, fr in ipairs(lua.callchain) do > + ins_cnt = ins_cnt + 1 > + if fr.type == FRAME.CFUNC then > + -- C function encountered, the next chunk > + -- of frames is located on the C stack. > + break > + end > + name_lua = fr.name > + > + if fr.type == FRAME.LFUNC > + and lua.trace.traceno ~= nil > + and lua.trace.addr == fr.addr > + and lua.trace.line == fr.line > + then > + name_lua = lua.trace.name > + end Something bad with indentation. I guess there should be something similar to this: | if fr.type == FRAME.LFUNC | and lua.trace.traceno ~= nil | and lua.trace.addr == fr.addr | and lua.trace.line == fr.line | then | name_lua = lua.trace.name | end > + > + table.insert(chain, name_lua) > + end > + table.remove(lua.callchain, ins_cnt) > +end > @@ -171,8 +226,9 @@ local function parse_event(reader, events, symbols) > event.lua.vmstate = vmstate > > event_parsers[vmstate](reader, event, symbols) > - > - table.insert(events, event) > + local callchain = merge(event) > + local cur_cnt = events[callchain] > + events[callchain] = (cur_cnt or 0) + 1 Minor: The following line looks better (IMHO), but feel free to ignore. | events[callchain] = (events[callchain] or 0) + 1 > return true > end > > -- > 2.41.0 > -- Best regards, IM