From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 437476FC8F; Thu, 25 Mar 2021 23:15:09 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 437476FC8F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1616703309; bh=Aajs9YrTYTt/iBkGG1AGw63MIWP7sGtT4zoTywjlzyA=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=c+TyF8Mj14me2rHghVUFtGD2yM0vpkPQMhfbAFBArmGR3lF8IJi3A5whivtRg9BhP aow1zoJEKs4oaWSnhLgSzBl5qxcL0o5x0LvAdlb87z5vKXD7Z0JBDXkx22etqEE08K nhn7T97u0rC1zTpx3AgBIWDTixA6fSPF/q0jdl8g= Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 924A76FC8F for ; Thu, 25 Mar 2021 23:15:08 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 924A76FC8F Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1lPWNn-0003YZ-JV; Thu, 25 Mar 2021 23:15:07 +0300 Date: Thu, 25 Mar 2021 23:14:59 +0300 To: Sergey Kaplun Message-ID: <20210325201459.GG29703@tarantool.org> References: <20210309175422.25432-1-skaplun@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20210309175422.25432-1-skaplun@tarantool.org> X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.10.1 (2018-07-13) X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9064ADF4728AA0EE9AECA9F3C9C9885BEE78E91CF33279E24182A05F53808504014EB131DC7F166B2A550C94FC3ABBC08A267F9515B60882DB24B862F3D1DFD47 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7CEC4F91A22C5F5FEEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637C904E3CF4B5CD3198638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C5DD32608FC869F5D5660C3ADF0421F939BD81697194E2804A471835C12D1D9774AD6D5ED66289B5278DA827A17800CE7328B01A8D746D8839FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C3E478A468B35FE767117882F4460429728AD0CFFFB425014E868A13BD56FB6657E2021AF6380DFAD1A18204E546F3947C0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E73528A6D463EDFD0DBBC4224003CC83647689D4C264860C145E X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C97831D62ED4694B1BEA52A31E5B19C74FC7CE X-C1DE0DAB: 0D63561A33F958A5460FECDE3BDF81E7036CBB0DAC8DEFACB9666A5E8E579077D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75F04B387B5D7535DE410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34047E322BFAFD69BD0E5286C9CFF03009F09AEF111253FDBA7AA0F5CF8B96E50A8814EEF5AF6947801D7E09C32AA3244C162140B7A1AC2C2DB6C84CA4D4B0248BF2F5F14F68F1805B927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojapPp7P/VpAjmxUMTM2yx9A== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822C579337C8BB1084AFD0C06BE23DFD989A7C8D0F45F857DBFE9F1EFEE2F478337FB559BB5D741EB964C8C2C849690F8E70A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit] memprof: report stack resizing as internal event X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Sergey, Thanks for the patch! Please consider my comments below. On 09.03.21, Sergey Kaplun wrote: > Resizing of the Lua stack is not reported as internal allocation. Was it done intentionally? You claim this as a plain fact with no rationale. If this is a bug (by design) then mention it explicitly here. Otherwise, you need to provide a reason why you change the semantics of Lua stack reallocations in terms of memprof. For me it looks like a bug we missed in the previous release: user has nothing to do with guest stack reallocations (like with JIT related ones), hence these memory manipulations are totally internal. Maybe you have another vision regarding this. > Moreover, it may lead to crash inside Lua or FF frames. This is a symptom, not another reason. So "As a result" fits better than "Moreover" here. > > Profiler performs reallocation first and after reports corresponding Profiler makes no allocations. I believe you would like to say that reallocation occurs at first and after memprof reports the event. Typo: s/reports corresponding/reports the corresponding/. > event. When the stack is resized for local function arguments, the link > to previous frame is invalid in the cause of reallocation. Therefore, > assertion in `debug_framepc()` failes, because of invalid function > reference at previous frame. Typo: s/at previous/at the previous/. > > Resolves tarantool/tarantool#5842 > Follows up tarantool/tarantool#5442 > --- > Branch: https://github.com/tarantool/luajit/tree/skaplun/gh-5842-memprof-core-on-resizestack > Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5842-memprof-core-on-resizestack > Issue: https://github.com/tarantool/tarantool/issues/5842 > > > src/lj_state.c | 6 ++++++ > .../misclib-memprof-lapi.test.lua | 18 ++++++++++++++++++ > 2 files changed, 24 insertions(+) > > diff --git a/src/lj_state.c b/src/lj_state.c > index 1ed79a5..ea9abd4 100644 > --- a/src/lj_state.c > +++ b/src/lj_state.c > @@ -64,7 +64,11 @@ static void resizestack(lua_State *L, MSize n) > MSize oldsize = L->stacksize; > MSize realsize = n + 1 + LJ_STACK_EXTRA; > GCobj *up; > + int32_t old_vmstate = G(L)->vmstate; Please consider the naming and the workflow in lj_gc.c for such situations: G(L) is stored into a separate variable and is named . It makes grep for such spots much easier, doesn't it? > + > lua_assert((MSize)(tvref(L->maxstack)-oldst)==L->stacksize-LJ_STACK_EXTRA-1); > + > + setvmstate(G(L), INTERP); We didn't notice this before. Now you leave not a single word regarding this hack. How come? > st = (TValue *)lj_mem_realloc(L, tvref(L->stack), > (MSize)(oldsize*sizeof(TValue)), > (MSize)(realsize*sizeof(TValue))); > @@ -80,6 +84,8 @@ static void resizestack(lua_State *L, MSize n) > L->top = (TValue *)((char *)L->top + delta); > for (up = gcref(L->openupval); up != NULL; up = gcnext(up)) > setmref(gco2uv(up)->v, (TValue *)((char *)uvval(gco2uv(up)) + delta)); > + > + G(L)->vmstate = old_vmstate; > } > > /* Relimit stack after error, in case the limit was overdrawn. */ > diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua > index 1c36c8a..93cc348 100644 > --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua > +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua > @@ -125,5 +125,23 @@ test:ok(check_alloc_report(alloc, 25, 18, 100)) > -- Collect all previous allocated objects. > test:ok(free.INTERNAL.num == 102) > > +-- Test for https://github.com/tarantool/tarantool/issues/5842. > +-- We do not interested in report itself. > +misc.memprof.start("/dev/null") > +-- We need to cause stack resize for local variables at function > +-- call. Let's create a new coroutine (all slots are free). > +-- It has 1 slot for dummy frame + 39 free slots + 5 extra slots > +-- (so-called red zone) + 2 * LJ_FR2 slots. So 50 local variables > +-- is enough. > +local payload_str = "" > +for i = 1, 50 do > + payload_str = payload_str..("local v%d = %d\n"):format(i, i) > +end > +local f, errmsg = loadstring(payload_str) > +assert(f, errmsg) Minor: you can use a TAP check here, but feel free to leave a plain assert, since you expect works fine always. > +local co = coroutine.create(f) > +coroutine.resume(co) > +misc.memprof.stop() > + > jit.on() > os.exit(test:check() and 0 or 1) > -- > 2.28.0 > -- Best regards, IM