Re: [Tarantool-patches] [PATCH luajit v2 6/7] misc: add Lua API for memory profiler

[Sergey Kaplun <skaplun@tarantool.org>]

Igor,

Thanks for the review!

On 27.12.20, Igor Munkin wrote:
> Sergey,
> 
> Thanks for the patch! Please consider the comments below.
> 
> On 25.12.20, Sergey Kaplun wrote:
> > This patch introduces Lua API for LuaJIT memory profiler implemented in
> > the scope of the previous patch.
> > 
> > Profiler returns some true value if started/stopped successfully,
> 
> There is no "some" true value there is a sole one. Please fix it in both
> the patch and the RFC.

Fixed.

> 
> > returns nil on failure (plus an error message as a second result and a
> > system-dependent error code as a third result).
> > If LuaJIT build without memory profiler both return `false`.
> > 
> > <lj_errmsg.h> have adjusted with two new errors
> > PROF_ISRUNNING/PROF_NOTRUNNING returned in case when profiler has
> > started/stopped already correspondingly.
> > 
> > Part of tarantool/tarantool#5442
> > ---
> > 
> > Changes in v2:
> >   - Added pushing of errno for ERR_PROF* and ERRMEM
> >   - Added forgotten assert.
> > 
> >  src/Makefile.dep |   5 +-
> >  src/lib_misc.c   | 167 +++++++++++++++++++++++++++++++++++++++++++++++
> >  src/lj_errmsg.h  |   6 ++
> >  3 files changed, 176 insertions(+), 2 deletions(-)
> > 
> 
> <snipped>
> 
> > diff --git a/src/lib_misc.c b/src/lib_misc.c
> > index 6f7b9a9..36fe29f 100644
> > --- a/src/lib_misc.c
> > +++ b/src/lib_misc.c
> 
> <snipped>
> 
> > @@ -67,8 +75,167 @@ LJLIB_CF(misc_getmetrics)
> 
> <snipped>
> 
> > +static LJ_AINLINE void memprof_ctx_free(struct memprof_ctx *ctx, uint8_t *buf)
> > +{
> > +  lj_mem_free(ctx->g, buf, STREAM_BUFFER_SIZE);
> 
> Side note: This is odd that you free the buffer here, but the buffer
> itself is not a part of the memprof context. Let's return to this later.

I'll rewrite this part considering the fact that lj_mem_new raises an
error in case of OOM.

> 
> > +  lj_mem_free(ctx->g, ctx, sizeof(*ctx));
> > +}
> > +
> > +/* Default buffer writer function. Just call fwrite to corresponding FILE. */
> > +static size_t buffer_writer_default(const void **buf_addr, size_t len,
> > +				    void *opt)
> > +{
> 
> <snipped>
> 
> > +    if (LJ_UNLIKELY(written == 0)) {
> > +      /* Re-tries write in case of EINTR. */
> > +      if (errno == EINTR) {
> 
> Minor: It's better to use early return here. Feel free to ignore.

Applied.

> 
> > +	errno = 0;
> > +	continue;
> > +      }
> > +      break;
> 
> If other error occurs, you need to pass the NULL to buf_addr, right?
> Otherwise, there is no guarantee everything is written to the file and
> profiling proceeds.

Yes, lets stopped profiling immedeately.

> 
> > +    }
> 
> <snipped>
> 
> > +
> > +/* Default on stop callback. Just close corresponding stream. */
> 
> Typo: s/close corresponding/close the corresponding/.

Thanks fixed.

> 
> > +static int on_stop_cb_default(void *opt, uint8_t *buf)
> 
> <snipped>
> 
> > +/* local started, err, errno = misc.memprof.start(fname) */
> > +LJLIB_CF(misc_memprof_start)
> > +{
> 
> <snipped>
> 
> > +  fname = strdata(lj_lib_checkstr(L, 1));
> 
> Minor: You can make this assignment alongside with the declaration.

Fixed.

> 
> > +
> > +  ctx = lj_mem_new(L, sizeof(*ctx));
> > +  if (ctx == NULL)
> 
> This is a dead code: <lj_mem_new> raises a LUA_ERRMEM.

Fixed in the new version.

> 
> > +    goto errmem;
> > +
> 
> <snipped>
> 
> > +  if (NULL == opt.buf) {
> 
> This is a dead code: <lj_mem_new> raises a LUA_ERRMEM.

Fixed in the new version.

> 
> > +    lj_mem_free(G(L), ctx, sizeof(*ctx));
> > +    goto errmem;
> > +  }
> 
> <snipped>
> 
> > +  memprof_status = lj_memprof_start(L, &opt);
> > +  started = memprof_status == PROFILE_SUCCESS;
> 
> Trust me, you don't need this variable.
> */me making Jedi mind tricks here*

I don't need this variable.

> 
> > +
> > +  if (LJ_UNLIKELY(!started)) {
> > +    fclose(ctx->stream);
> > +    remove(fname);
> 
> Minor: I doubt we need to remove a file even if LuaJIT failed to start
> profiling. Leave the comment if this makes sense. Feel free to ignore.

Dropped the corresponding comment.

> 
> > +    memprof_ctx_free(ctx, opt.buf);
> > +    switch (memprof_status) {
> 
> <snipped>
> 
> > +    }
> > +  }
> > +  lua_pushboolean(L, started);
> 
> Please, s/started/1/ since there is no another value here.

Fixed. Added return 0 after lua_assert(0).

> 
> > +
> > +  return 1;
> 
> <snipped>
> 
> > +}
> > +
> > +/* local stopped, err, errno = misc.memprof.stop() */
> > +LJLIB_CF(misc_memprof_stop)
> > +{
> > +  int status = lj_memprof_stop();
> > +  int stopped_successfully = status == PROFILE_SUCCESS;
> 
> Trust me, you don't need this variable.
> */me making Jedi mind tricks here*

I don't need this variable.

> 
> > +  if (!stopped_successfully) {
> 
> <snipped>
> 
> > +  lua_pushboolean(L, stopped_successfully);
> 
> Please, s/stopped_succesfully/1/ since there is no another value here.

Fixed. Added return 0 after lua_assert(0).

> 
> > +  return 1;
> > +}
> > +
> 
> <snipped>
> 
> > diff --git a/src/lj_errmsg.h b/src/lj_errmsg.h
> > index de7b867..6816da2 100644
> > --- a/src/lj_errmsg.h
> > +++ b/src/lj_errmsg.h
> > @@ -185,6 +185,12 @@ ERRDEF(FFI_NYIPACKBIT,	"NYI: packed bit fields")
> 
> <snipped>
> 
> > +#if LJ_HASPROFILE || LJ_HASMEMPROF
> 
> Why did you also initialize them for <jit.p> profiler?

I've supposed that it can be used there too. Let's drop it until the
refactoring.

> 
> > +/* Profiler errors. */
> 
> <snipped>
> 
> > -- 
> > 2.28.0
> > 
> 
> -- 
> Best regards,
> IM

-- 
Best regards,
Sergey Kaplun