From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id B89BA6EC55; Thu, 29 Jul 2021 12:34:38 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B89BA6EC55 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627551278; bh=xbrGbWqu2UfgWOflvRJMU8QZmPmXRtL7pzLVtqKi3Wo=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=vJMYN/Chdql1KJ1gLZ7+6K9qU4kF15Xialz2mQbwGpdbDpPP9MlBTNvwPAUhZ/rkN JQEokG6r0uSWTBQN8ZtG8CmYPUp+DtmBsnfrLYejUkQFpQ5I52k85jN3Qs72+6bSWK aU0/BG6cWgA4RLjHiQYaL0GfrTVxYqR1wwRqHGbk= Received: from smtp32.i.mail.ru (smtp32.i.mail.ru [94.100.177.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 3480A6EC55 for ; Thu, 29 Jul 2021 12:34:37 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3480A6EC55 Received: by smtp32.i.mail.ru with esmtpa (envelope-from ) id 1m92R2-0004Or-3i; Thu, 29 Jul 2021 12:34:36 +0300 To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Thu, 29 Jul 2021 16:34:25 +0700 Message-Id: <20210729093425.179158-1-m.shishatskiy@tarantool.org> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD941C43E597735A9C351B198F4576AC7B21928AAE70459C21B182A05F5380850409CFC398917E2D1729641362C4649345DDA4AA781A351DE3A25BF82367592C476 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7A0175C48BD57B26BC2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6778DA827A17800CE7952C4D7BD0BF3359EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F2316855959AA85489B8221973264E8E0DCC7F00164DA146DAFE8445B8C89999728AA50765F790063793270F7220657A0A389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8ED96AA85C75E140D117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CF53CD89B44251D82CBA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B63FDD67786A3DFEF576E601842F6C81A1F004C906525384307823802FF610243DF43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A39223F8577A6DFFEA7CE0F3BA37685B2B9043847C11F186F3C59DAA53EE0834AAEE X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B50551A814B78B329958B464E56AE7839C53 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C82A4B7DEF7839881545EFFB4D9DAD63949C2A87A0F3A7C399C2B6934AE262D3EE7EAB7254005DCED255AF2367009C741C581CE5390BD6510078749EA292E75700EA540613ABC294F04EBA3D8E7E5B87ABF8C51168CD8EBDB77172F5E35264BF615E2725BA614EAEA1EF972C1F679AE1C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D347324AA9FA07FF01EADB7427DAA3A7DC4E7A8C3D6728CFBF37BCCFF01524957E0FF9002FBBF2BA24E1D7E09C32AA3244C38DF167A4F164BAF9DA531E076BD50F039C99C45E8D137E9927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojPp/mPgZxawE61OeSOqEYVw== X-Mailru-Sender: 2317F5BEA8D613097CB8A225516EF027B75D12C696C8FF3EE9583683FFF36AFD97C066105E11400DFF6B26DEAE20951894E739346DD9ABB838AE394B99C5394F99323991B04ED3930F27244EEAA5B9A5AE208404248635DF X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit v2] memprof: report JIT-side allocations as internal X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mikhail Shishatskiy via Tarantool-patches Reply-To: Mikhail Shishatskiy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" There are cases when the memory profiler attempts to attribute allocations triggered by the JIT engine recording phase with a Lua function to be recorded. In this case, lj_debug_frameline() may return BC_NOPOS (i.e. a negative value). Previously, these situations were ignored and the profiler reported, that the source line was equal to zero. This patch adjusts profiler behavior to treat allocations described above as internal by dumping ASOURCE_INT if lj_debug_frameline() returns a negative value. Resolves tarantool/tarantool#5679 --- Issue: https://github.com/tarantool/tarantool/issues/5679 Branch: https://github.com/tarantool/luajit/tree/shishqa/gh-5679-report-jit-allocations-as-internal CI: https://github.com/tarantool/tarantool/tree/shishqa/gh-5679-report-jit-allocations-as-internal Changes in v2: - Fixed commit title to fit in width of 50 symbols; - Rebased to the branch [1]; - Reused default_payload() as a payload to test patch behavior. [1]: https://github.com/tarantool/luajit/tree/shishqa/gh-5814-group-allocations-on-trace-by-trace-number src/lj_memprof.c | 28 ++++++++++++------- .../misclib-memprof-lapi.test.lua | 16 ++++++++--- 2 files changed, 30 insertions(+), 14 deletions(-) diff --git a/src/lj_memprof.c b/src/lj_memprof.c index 87512c3a..e4d819a0 100644 --- a/src/lj_memprof.c +++ b/src/lj_memprof.c @@ -108,19 +108,27 @@ static void memprof_write_lfunc(struct lj_wbuf *out, uint8_t aevent, GCfunc *fn, struct lua_State *L, cTValue *nextframe) { - const BCLine line = lj_debug_frameline(L, fn, nextframe); - lj_wbuf_addbyte(out, aevent | ASOURCE_LFUNC); - lj_wbuf_addu64(out, (uintptr_t)funcproto(fn)); /* - ** Line is >= 0 if we are inside a Lua function. - ** There are cases when the memory profiler attempts - ** to attribute allocations triggered by JIT engine recording - ** phase with a Lua function to be recorded. At this case - ** lj_debug_frameline() may return BC_NOPOS (i.e. a negative value). - ** Equals to zero when LuaJIT is built with the + ** Line equals to zero when LuaJIT is built with the ** -DLUAJIT_DISABLE_DEBUGINFO flag. */ - lj_wbuf_addu64(out, line >= 0 ? (uint64_t)line : 0); + const BCLine line = lj_debug_frameline(L, fn, nextframe); + + if (line < 0) { + /* + ** Line is >= 0 if we are inside a Lua function. + ** There are cases when the memory profiler attempts + ** to attribute allocations triggered by JIT engine recording + ** phase with a Lua function to be recorded. It this case, + ** lj_debug_frameline() may return BC_NOPOS (i.e. a negative value). + ** We report such allocations as internal in order not to confuse users. + */ + lj_wbuf_addbyte(out, aevent | ASOURCE_INT); + } else { + lj_wbuf_addbyte(out, aevent | ASOURCE_LFUNC); + lj_wbuf_addu64(out, (uintptr_t)funcproto(fn)); + lj_wbuf_addu64(out, (uint64_t)line); + } } static void memprof_write_cfunc(struct lj_wbuf *out, uint8_t aevent, diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua index b7e456e1..96864c6a 100644 --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua @@ -12,7 +12,7 @@ utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819') local tap = require("tap") local test = tap.test("misc-memprof-lapi") -test:plan(16) +test:plan(17) jit.off() jit.flush() @@ -81,7 +81,7 @@ local function fill_ev_type(events, symbols, event_type) for _, event in pairs(events[event_type]) do local addr = event.loc.addr local traceno = event.loc.traceno - if traceno ~= 0 then + if traceno ~= 0 and symbols.trace[traceno] then local trace_loc = symbols.trace[traceno] addr = trace_loc.addr ev_type[trace_loc.line] = { @@ -214,9 +214,17 @@ misc.memprof.stop() -- Test profiler with enabled JIT. jit.on() --- Pregenerate traces to get expected output: -default_payload() +-- On this run traces are generated, JIT-related allocations +-- will be recorded as well. +symbols, events = generate_parsed_output(default_payload) + +alloc = fill_ev_type(events, symbols, "alloc") + +-- Test for marking JIT-related allocations as internal. +-- See also https://github.com/tarantool/tarantool/issues/5679. +test:ok(alloc[0] == nil) +-- Run already generated traces. symbols, events = generate_parsed_output(default_payload) alloc = fill_ev_type(events, symbols, "alloc") -- 2.32.0