Tarantool development patches archive
 help / color / mirror / Atom feed
From: Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Sergey Ostanevich <sergos@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH luajit] memprof: report stack resizing as internal event
Date: Wed, 10 Mar 2021 20:37:59 +0300	[thread overview]
Message-ID: <20210310173759.GI6842@root> (raw)
In-Reply-To: <51AE3320-136E-422E-9BCC-AFA4A338B77A@tarantool.org>

Hi, Sergos!

Thanks for the review!

On 10.03.21, Sergey Ostanevich wrote:
> Hi!
> 
> Thanks for the patch!
> 
> > On 9 Mar 2021, at 20:54, Sergey Kaplun <skaplun@tarantool.org> wrote:
> > 
> > Resizing of the Lua stack is not reported as internal allocation.
> > Moreover, it may lead to crash inside Lua or FF frames.
> > 
> > Profiler performs reallocation first and after reports corresponding
> > event. When the stack is resized for local function arguments, the link
> > to previous frame is invalid in the cause of reallocation. Therefore,
> 				  ^^^^^^^^ case
> > assertion in `debug_framepc()` failes, because of invalid function
> 				fails.   ^^^ is it just dubbing the link 
> 					     is invalid? Then just remove.

Fixed and force-pushed to the branch. The commit message now is the
following:

===================================================================
memprof: report stack resizing as internal event

Resizing of the Lua stack is not reported as internal allocation.
Moreover, it may lead to crash inside Lua or FF frames.

Profiler performs reallocation first and after reports corresponding
event. When the stack is resized for local function arguments, the link
to previous frame is invalid in case of reallocation. Therefore,
assertion in `debug_framepc()` fails.

Resolves tarantool/tarantool#5842
Follows up tarantool/tarantool#5442
===================================================================

> 
> > reference at previous frame.
> > 
> > Resolves tarantool/tarantool#5842
> > Follows up tarantool/tarantool#5442
> > ---
> > Branch: https://github.com/tarantool/luajit/tree/skaplun/gh-5842-memprof-core-on-resizestack
> > Tarantool branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5842-memprof-core-on-resizestack
> > Issue: https://github.com/tarantool/tarantool/issues/5842
> > 
> > 
> > src/lj_state.c                                 |  6 ++++++
> > .../misclib-memprof-lapi.test.lua              | 18 ++++++++++++++++++
> > 2 files changed, 24 insertions(+)
> > 
> > diff --git a/src/lj_state.c b/src/lj_state.c
> > index 1ed79a5..ea9abd4 100644
> > --- a/src/lj_state.c
> > +++ b/src/lj_state.c
> > @@ -64,7 +64,11 @@ static void resizestack(lua_State *L, MSize n)
> >   MSize oldsize = L->stacksize;
> >   MSize realsize = n + 1 + LJ_STACK_EXTRA;
> >   GCobj *up;
> > +  int32_t old_vmstate = G(L)->vmstate;
> > +
> >   lua_assert((MSize)(tvref(L->maxstack)-oldst)==L->stacksize-LJ_STACK_EXTRA-1);
> > +
> > +  setvmstate(G(L), INTERP);
> 
> This looks like a hack. But even so - why not to return it back right after the
> realloc, where you supposedly use lua_getinfo? 

The Lua state L is inconsistent during stack resizing, so it can't be
interpreted in any way, until reallocation and corresponding converting
are finished.

> 
> The righteous way would be to fix the debug machinery anyhow.

Don't get your point here. Lua state is invalid, debug machinery works
correct, as I understand.

> 
> >   st = (TValue *)lj_mem_realloc(L, tvref(L->stack),
> > 				(MSize)(oldsize*sizeof(TValue)),
> > 				(MSize)(realsize*sizeof(TValue)));
> > @@ -80,6 +84,8 @@ static void resizestack(lua_State *L, MSize n)
> >   L->top = (TValue *)((char *)L->top + delta);
> >   for (up = gcref(L->openupval); up != NULL; up = gcnext(up))
> >     setmref(gco2uv(up)->v, (TValue *)((char *)uvval(gco2uv(up)) + delta));
> > +
> > +  G(L)->vmstate = old_vmstate;
> > }
> > 
> > /* Relimit stack after error, in case the limit was overdrawn. */
> > diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
> > index 1c36c8a..93cc348 100644
> > --- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
> > +++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
> > @@ -125,5 +125,23 @@ test:ok(check_alloc_report(alloc, 25, 18, 100))
> > -- Collect all previous allocated objects.
> > test:ok(free.INTERNAL.num == 102)
> > 
> > +-- Test for https://github.com/tarantool/tarantool/issues/5842.
> > +-- We do not interested in report itself.
>      We ->			     ourselves?

I've removed confusing "itself". See the iterative patch below,
branch is force-pushed.

===================================================================
diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
index 93cc348..9b9fb76 100644
--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
@@ -126,7 +126,7 @@ test:ok(check_alloc_report(alloc, 25, 18, 100))
 test:ok(free.INTERNAL.num == 102)
 
 -- Test for https://github.com/tarantool/tarantool/issues/5842.
--- We do not interested in report itself.
+-- We do not interested in this report.
 misc.memprof.start("/dev/null")
 -- We need to cause stack resize for local variables at function
 -- call. Let's create a new coroutine (all slots are free).
===================================================================

> > +misc.memprof.start("/dev/null")
> > +-- We need to cause stack resize for local variables at function
> > +-- call. Let's create a new coroutine (all slots are free).
> > +-- It has 1 slot for dummy frame + 39 free slots + 5 extra slots
> > +-- (so-called red zone) + 2 * LJ_FR2 slots. So 50 local variables
> > +-- is enough.
> > +local payload_str = ""
> > +for i = 1, 50 do
> > +  payload_str = payload_str..("local v%d = %d\n"):format(i, i)
> > +end
> > +local f, errmsg = loadstring(payload_str)
> > +assert(f, errmsg)
> > +local co = coroutine.create(f)
> > +coroutine.resume(co)
> > +misc.memprof.stop()
> > +
> > jit.on()
> > os.exit(test:check() and 0 or 1)
> > — 
> > 2.28.0
> > 
> 
> 
> Regards,
> Sergos
> 
> 

-- 
Best regards,
Sergey Kaplun

  reply	other threads:[~2021-03-10 17:38 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-09 17:54 Sergey Kaplun via Tarantool-patches
2021-03-10 14:59 ` Sergey Ostanevich via Tarantool-patches
2021-03-10 17:37   ` Sergey Kaplun via Tarantool-patches [this message]
2021-03-25 20:14 ` Igor Munkin via Tarantool-patches
2021-03-26  8:54   ` Sergey Kaplun via Tarantool-patches
2021-03-26 10:10     ` Igor Munkin via Tarantool-patches
2021-03-26 14:09       ` Sergey Kaplun via Tarantool-patches
2021-03-26 18:38         ` Igor Munkin via Tarantool-patches
2021-03-29  8:09           ` Sergey Kaplun via Tarantool-patches
2021-03-29  8:32             ` Igor Munkin via Tarantool-patches
2021-03-29  8:35               ` Igor Munkin via Tarantool-patches
2021-03-29 16:01                 ` Sergey Ostanevich via Tarantool-patches
2021-03-29 20:48 ` Igor Munkin via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210310173759.GI6842@root \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=sergos@tarantool.org \
    --cc=skaplun@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH luajit] memprof: report stack resizing as internal event' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox