From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E35076DB00; Sat, 21 Aug 2021 07:43:59 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E35076DB00 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1629521040; bh=3JZaNrm7W+t7fhkJgMb5/f4c77q6srJvS77p2I4qOm0=; h=Date:To:In-Reply-To:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=OY95AlKRjA0li8JETXYwh/rNqL81gewJ2jzn5975ixD4CzuQlFc8lmhvPEME34+qi HT30fZ6zsmDSdDZxUjJsYgLNTaNSa8t0Kh/nG4B1QCm2hanzBBr860j7/41DsdNqzV sGI7NubeXIIvFIPDFKwrmolDeoe5R6EEOuy/JPKM= Received: from smtp55.i.mail.ru (smtp55.i.mail.ru [217.69.128.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 29B8F6DB00 for ; Sat, 21 Aug 2021 07:43:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 29B8F6DB00 Received: by smtp55.i.mail.ru with esmtpa (envelope-from ) id 1mHIrM-00048T-VA; Sat, 21 Aug 2021 07:43:57 +0300 Date: Sat, 21 Aug 2021 11:43:53 +0700 To: Sergey Kaplun , tarantool-patches@dev.tarantool.org, imun@tarantool.org Message-ID: <20210821044353.kbbgcgiapat2q4ve@surf.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline In-Reply-To: X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD91BCCB18F2C129F87F36E61E9E4584E9D182A05F538085040C87089CDB6602CACCA03499650DB28D802B553FA119F4339AD46C359CDFEB437 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7818460412E3A2163EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006371758572763D318798638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D878514D2DA7296C12EAB565E7A6325C27117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18C26CFBAC0749D213D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EEC65AC60A1F0286FE3D5BA627BF9F2FCFD8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE36A1CB4668A9CA5FA040F9FF01DFDA4A8C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F790063747074E5DE1517656EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79D0B18DC6AC13D9A1C1BA386BE5FA2F7BE X-C1DE0DAB: 0D63561A33F958A5ECE3CE6644C8233D09F94D58A41FB44BD6AA05E6C01B88B7D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75601EFBC916B793DF410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D349379E7F8541B6C9AE71216AA9E1A9A3212B89D85FA286B758675D79B4050A82F9CD2D295C221D8B11D7E09C32AA3244CFC144C778CFBF2E04F42305AE320A4D0A90944CA99CF22E3FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojzcVJReGZsqOoS0aVXIaDHw== X-Mailru-Sender: 2317F5BEA8D613097CB8A225516EF027D0AF16F724834DEB87571A3F0C5EBC016A6420687F745674FF6B26DEAE20951894E739346DD9ABB838AE394B99C5394F99323991B04ED3930F27244EEAA5B9A5AE208404248635DF X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit v2] memprof: report JIT-side allocations as internal X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mikhail Shishatskiy via Tarantool-patches Reply-To: Mikhail Shishatskiy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey! Thanks for the review! On 14.08.2021 14:24, Sergey Kaplun wrote: >Hi, thanks for the fixes! > >Please, move test adjustment mentioned below to the patch for the symtab >with group allocations by trace's number. >Otherwise, LGTM. > >On 29.07.21, Mikhail Shishatskiy wrote: >> There are cases when the memory profiler attempts to attribute >> allocations triggered by the JIT engine recording phase >> with a Lua function to be recorded. In this case, >> lj_debug_frameline() may return BC_NOPOS (i.e. a negative value). >> >> Previously, these situations were ignored and the profiler >> reported, that the source line was equal to zero. >> >> This patch adjusts profiler behavior to treat allocations >> described above as internal by dumping ASOURCE_INT if >> lj_debug_frameline() returns a negative value. >> >> Resolves tarantool/tarantool#5679 >> --- >> >> Issue: https://github.com/tarantool/tarantool/issues/5679 >> Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5679-report-jit-allocations-as-internal >> CI: https://github.com/tarantool/tarantool/tree/shishqa/gh-5679-report-jit-allocations-as-internal >> >> Changes in v2: >> - Fixed commit title to fit in width of 50 symbols; >> - Rebased to the branch [1]; >> - Reused default_payload() as a payload to test patch behavior. >> >> [1]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number >> >> src/lj_memprof.c | 28 ++++++++++++------- >> .../misclib-memprof-lapi.test.lua | 16 ++++++++--- >> 2 files changed, 30 insertions(+), 14 deletions(-) >> >> diff --git a/src/lj_memprof.c b/src/lj_memprof.c >> index 87512c3a..e4d819a0 100644 >> --- a/src/lj_memprof.c >> +++ b/src/lj_memprof.c > > > >> diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua >> index b7e456e1..96864c6a 100644 >> --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua >> +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua >> @@ -12,7 +12,7 @@ utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819') >> local tap = require("tap") >> >> local test = tap.test("misc-memprof-lapi") >> -test:plan(16) >> +test:plan(17) >> >> jit.off() >> jit.flush() >> @@ -81,7 +81,7 @@ local function fill_ev_type(events, symbols, event_type) >> for _, event in pairs(events[event_type]) do >> local addr = event.loc.addr >> local traceno = event.loc.traceno >> - if traceno ~= 0 then >> + if traceno ~= 0 and symbols.trace[traceno] then > >This should be moved to the commit, that changes symtab, this change >looks unrelated to this patch, but relates to previous one [1]. Fixed in the patchset [1] and rebased to the branch [2] > >> local trace_loc = symbols.trace[traceno] >> addr = trace_loc.addr >> ev_type[trace_loc.line] = { >> @@ -214,9 +214,17 @@ misc.memprof.stop() > > > >> -- >> 2.32.0 >> > >[1]: https://lists.tarantool.org/pipermail/tarantool-patches/2021-July/024944.html > >-- >Best regards, >Sergey Kaplun Also, I slightly changed the test by deleting the check for deallocations number because now trace generation happens for memprof payload, too, and we can meet untrivial JIT-related deallocations in the second run. --- diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua index f84b6df0..17bcb2f1 100644 --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua @@ -231,19 +231,25 @@ test:test("jit-output", function(subtest) jit.opt.start(3, "hotloop=10") jit.flush() - -- Pregenerate traces to fill symtab entries in the next run. - default_payload() - + -- On this run traces are generated, JIT-related allocations + -- will be recorded as well. local symbols, events = generate_parsed_output(default_payload) local alloc = fill_ev_type(events, symbols, "alloc") - local free = fill_ev_type(events, symbols, "free") + + -- Test for marking JIT-related allocations as internal. + -- See also https://github.com/tarantool/tarantool/issues/5679. + subtest:ok(alloc[0] == nil) + + -- Run already generated traces. + symbols, events = generate_parsed_output(default_payload) + + alloc = fill_ev_type(events, symbols, "alloc") -- We expect, that loop will be compiled into a trace. subtest:ok(check_alloc_report(alloc, 1, 37, 32, 20)) -- See same checks with jit.off(). subtest:ok(check_alloc_report(alloc, 0, 34, 32, 2)) - subtest:ok(free.INTERNAL.num == 22) -- Restore default JIT settings. jit.opt.start(unpack(jit_opt_default)) -- 2.32.0 [1]: https://lists.tarantool.org/pipermail/tarantool-patches/2021-August/025680.html [2]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number Issue: https://github.com/tarantool/tarantool/issues/5679 Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5679-report-jit-allocations-as-internal Tarantool branch: https://github.com/tarantool/tarantool/tree/shishqa/gh-5679-report-jit-allocations-as-internal Best regards, Mikhail Shishatskiy