From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id AC87C6ECE3; Wed, 24 Nov 2021 16:01:16 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org AC87C6ECE3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1637758876; bh=yvw1KcKuRYm+p/Z3MlPBeDcLBsLGEbcqhF5tMHOpYFs=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ui7BffG+efgBFfPTFG1yK/b3BJBGNscJyT6Eb5wqzR+3THA8PtdHYuW/6vUevmYm3 drYYkxblINoZ27sQbYy1Ty5uv40t1yHauPLRBcDQefx4isVh1HPoXt0exUzwHCVJ7J xdqV/AWKkb34TDEBpWOb09Jl2VscT4bKfOPc8tsA= Received: from smtp44.i.mail.ru (smtp44.i.mail.ru [94.100.177.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 45A476ECE3 for ; Wed, 24 Nov 2021 16:01:15 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 45A476ECE3 Received: by smtp44.i.mail.ru with esmtpa (envelope-from ) id 1mprti-0007OE-Ai; Wed, 24 Nov 2021 16:01:14 +0300 Date: Wed, 24 Nov 2021 16:01:11 +0300 To: Mikhail Shishatskiy Message-ID: <20211124130111.pav42lt2zlve7n2r@surf.localdomain> References: <20210821044353.kbbgcgiapat2q4ve@surf.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline In-Reply-To: <20210821044353.kbbgcgiapat2q4ve@surf.localdomain> X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD948575D5A18866828B13414D782013ADDCA2EBBB45C0F1B3D182A05F5380850404C228DA9ACA6FE274C905F60BACC435F90430462CAF265284BCF234018DF9B38C40DDAF728574921 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE77545ECFDF1E157EBEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637EA9DEEAA3ECF8E948638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8E646B087BC1FFA48B48C41FEF266DC1C117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735209ECD01F8117BC8BEA471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FCD821F6303F073B1F3AA81AA40904B5D9CF19DD082D7633A078D18283394535A93AA81AA40904B5D98AA50765F79006379C295477DF02BC6CD81D268191BDAD3D698AB9A7B718F8C4D1B931868CE1C5783A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E73535872C767BF85DA2F004C90652538430E4A6367B16DE6309 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B50584825F93A5258A464250998EDE830631 X-C1DE0DAB: 0D63561A33F958A58CE9478B49A22E70761CA454E9A184D79D97736D1AF87C02D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7506FE1F977233B9BB410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34E69981C39E7B068A89E3FD7A9024B436E1BEBB357D1857E10E0C3172E42707DC6C962E075D215E6F1D7E09C32AA3244C414CB5C683F36A1C101919227E984F0A39C99C45E8D137E9927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojFw1tprMwPO3/A8wBDMNwag== X-Mailru-Sender: EFA0F3A8419EF2168CD9BA0E6A2E45C538522BAE672F528590430462CAF26528F43CCAED39DA92F12376072A51849BFFE66B5C1DBFD5D09D5E022D45988A037B448E0EA96F20AB367402F9BA4338D657ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit v2] memprof: report JIT-side allocations as internal X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mikhail Shishatskiy via Tarantool-patches Reply-To: Mikhail Shishatskiy Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Rebased to the updated gh-5814 [1] branch. @ChangeLog ====================================================================== ##feature/luajit * Now memory profiler reports JIT-related allocations, which are not connected to a particular trace, as INTERNAL. ====================================================================== [1]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number On 21.08.2021 11:43, Mikhail Shishatskiy via Tarantool-patches wrote: >Hi, Sergey! >Thanks for the review! > >On 14.08.2021 14:24, Sergey Kaplun wrote: >>Hi, thanks for the fixes! >> >>Please, move test adjustment mentioned below to the patch for the symtab >>with group allocations by trace's number. >>Otherwise, LGTM. >> >>On 29.07.21, Mikhail Shishatskiy wrote: >>>There are cases when the memory profiler attempts to attribute >>>allocations triggered by the JIT engine recording phase >>>with a Lua function to be recorded. In this case, >>>lj_debug_frameline() may return BC_NOPOS (i.e. a negative value). >>> >>>Previously, these situations were ignored and the profiler >>>reported, that the source line was equal to zero. >>> >>>This patch adjusts profiler behavior to treat allocations >>>described above as internal by dumping ASOURCE_INT if >>>lj_debug_frameline() returns a negative value. >>> >>>Resolves tarantool/tarantool#5679 >>>--- >>> >>>Issue: https://github.com/tarantool/tarantool/issues/5679 >>>Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5679-report-jit-allocations-as-internal >>>CI: https://github.com/tarantool/tarantool/tree/shishqa/gh-5679-report-jit-allocations-as-internal >>> >>>Changes in v2: >>>- Fixed commit title to fit in width of 50 symbols; >>>- Rebased to the branch [1]; >>>- Reused default_payload() as a payload to test patch behavior. >>> >>>[1]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number >>> >>>src/lj_memprof.c | 28 ++++++++++++------- >>>.../misclib-memprof-lapi.test.lua | 16 ++++++++--- >>>2 files changed, 30 insertions(+), 14 deletions(-) >>> >>>diff --git a/src/lj_memprof.c b/src/lj_memprof.c >>>index 87512c3a..e4d819a0 100644 >>>--- a/src/lj_memprof.c >>>+++ b/src/lj_memprof.c >> >> >> >>>diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua >>>index b7e456e1..96864c6a 100644 >>>--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua >>>+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua >>>@@ -12,7 +12,7 @@ utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819') >>>local tap = require("tap") >>> >>>local test = tap.test("misc-memprof-lapi") >>>-test:plan(16) >>>+test:plan(17) >>> >>>jit.off() >>>jit.flush() >>>@@ -81,7 +81,7 @@ local function fill_ev_type(events, symbols, event_type) >>>for _, event in pairs(events[event_type]) do >>>local addr = event.loc.addr >>>local traceno = event.loc.traceno >>>- if traceno ~= 0 then >>>+ if traceno ~= 0 and symbols.trace[traceno] then >> >>This should be moved to the commit, that changes symtab, this change >>looks unrelated to this patch, but relates to previous one [1]. > >Fixed in the patchset [1] and rebased to the branch [2] > >> >>>local trace_loc = symbols.trace[traceno] >>>addr = trace_loc.addr >>>ev_type[trace_loc.line] = { >>>@@ -214,9 +214,17 @@ misc.memprof.stop() >> >> >> >>>-- >>>2.32.0 >>> >> >>[1]: https://lists.tarantool.org/pipermail/tarantool-patches/2021-July/024944.html >> >>-- >>Best regards, >>Sergey Kaplun > >Also, I slightly changed the test by deleting the check for deallocations >number because now trace generation happens for memprof payload, too, and >we can meet untrivial JIT-related deallocations in the second run. > >--- >diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua >index f84b6df0..17bcb2f1 100644 >--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua >+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua >@@ -231,19 +231,25 @@ test:test("jit-output", function(subtest) > jit.opt.start(3, "hotloop=10") > jit.flush() >- -- Pregenerate traces to fill symtab entries in the next run. >- default_payload() >- >+ -- On this run traces are generated, JIT-related allocations >+ -- will be recorded as well. > local symbols, events = generate_parsed_output(default_payload) > local alloc = fill_ev_type(events, symbols, "alloc") >- local free = fill_ev_type(events, symbols, "free") >+ >+ -- Test for marking JIT-related allocations as internal. >+ -- See also https://github.com/tarantool/tarantool/issues/5679. >+ subtest:ok(alloc[0] == nil) >+ >+ -- Run already generated traces. >+ symbols, events = generate_parsed_output(default_payload) >+ >+ alloc = fill_ev_type(events, symbols, "alloc") > -- We expect, that loop will be compiled into a trace. > subtest:ok(check_alloc_report(alloc, 1, 37, 32, 20)) > -- See same checks with jit.off(). > subtest:ok(check_alloc_report(alloc, 0, 34, 32, 2)) >- subtest:ok(free.INTERNAL.num == 22) > -- Restore default JIT settings. > jit.opt.start(unpack(jit_opt_default)) >-- >2.32.0 > >[1]: https://lists.tarantool.org/pipermail/tarantool-patches/2021-August/025680.html >[2]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number > >Issue: https://github.com/tarantool/tarantool/issues/5679 >Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5679-report-jit-allocations-as-internal >Tarantool branch: https://github.com/tarantool/tarantool/tree/shishqa/gh-5679-report-jit-allocations-as-internal > >Best regards, >Mikhail Shishatskiy