Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools
@ 2020-12-28  7:04 Sergey Kaplun
  2020-12-28  7:09 ` Igor Munkin
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Sergey Kaplun @ 2020-12-28  7:04 UTC (permalink / raw)
  To: Igor Munkin, Sergey Ostanevich; +Cc: tarantool-patches

This patch embeds a parser for binary data dumped via the memory
profiler to Tarantool binary.

It is a set of the following Lua modules:
* utils/bufread.lua: read binary data from a file.
* utils/symtab.lua: symbol table decode functions
* memprof/parse.lua: decode the memory profiler event stream
* memprof/humanize.lua: display decoded data in human readable format
* memprof.lua: Lua script and module to display data

It launch with the following command:
$ tarantool -e 'require("memprof")(arg[1])' - filename.bin

Closed #5490
---

Issue: https://github.com/tarantool/tarantool/issues/5490
Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5442-luajit-memory-profiler
CI: https://gitlab.com/tarantool/tarantool/-/commits/skaplun/gh-5442-luajit-memory-profiler

 src/CMakeLists.txt |  6 ++++++
 src/lua/init.c     | 15 ++++++++++++++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index b95688c1a..9a712bc29 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -61,6 +61,12 @@ lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/vmdef.lua
 lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/v.lua")
 lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/p.lua")
 lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/zone.lua")
+# LuaJIT tools.* library
+lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof.lua")
+lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof/humanize.lua")
+lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof/parse.lua")
+lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/utils/bufread.lua")
+lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/utils/symtab.lua")
 
 add_custom_target(generate_lua_sources
     WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/src/box
diff --git a/src/lua/init.c b/src/lua/init.c
index a0b2fc775..82075c595 100644
--- a/src/lua/init.c
+++ b/src/lua/init.c
@@ -121,7 +121,14 @@ extern char strict_lua[],
 	string_lua[],
 	swim_lua[],
 	p_lua[], /* LuaJIT 2.1 profiler */
-	zone_lua[] /* LuaJIT 2.1 profiler */;
+	zone_lua[], /* LuaJIT 2.1 profiler */
+	/* tools.* libraries. */
+	bufread_lua[],
+	symtab_lua[],
+	parse_lua[],
+	humanize_lua[],
+	memprof_lua[]
+;
 
 static const char *lua_modules[] = {
 	/* Make it first to affect load of all other modules */
@@ -167,6 +174,12 @@ static const char *lua_modules[] = {
 	/* Profiler */
 	"jit.p", p_lua,
 	"jit.zone", zone_lua,
+	/* tools.* libraries. Order is important. */
+	"utils.bufread", bufread_lua,
+	"utils.symtab", symtab_lua,
+	"memprof.parse", parse_lua,
+	"memprof.humanize", humanize_lua,
+	"memprof", memprof_lua,
 	NULL
 };
 
-- 
2.28.0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools
  2020-12-28  7:04 [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools Sergey Kaplun
@ 2020-12-28  7:09 ` Igor Munkin
  2020-12-28  7:10   ` Sergey Kaplun
  2020-12-28  7:55 ` Alexander V. Tikhonov
  2020-12-28  8:17 ` Igor Munkin
  2 siblings, 1 reply; 5+ messages in thread
From: Igor Munkin @ 2020-12-28  7:09 UTC (permalink / raw)
  To: Sergey Kaplun; +Cc: tarantool-patches

Sergey,

Thanks for the patch! LGTM except a single nit.

On 28.12.20, Sergey Kaplun wrote:
> This patch embeds a parser for binary data dumped via the memory
> profiler to Tarantool binary.
> 
> It is a set of the following Lua modules:
> * utils/bufread.lua: read binary data from a file.
> * utils/symtab.lua: symbol table decode functions
> * memprof/parse.lua: decode the memory profiler event stream
> * memprof/humanize.lua: display decoded data in human readable format
> * memprof.lua: Lua script and module to display data
> 
> It launch with the following command:
> $ tarantool -e 'require("memprof")(arg[1])' - filename.bin
> 
> Closed #5490

Typo: s/Closed/Closes/.

> ---
> 
> Issue: https://github.com/tarantool/tarantool/issues/5490
> Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5442-luajit-memory-profiler
> CI: https://gitlab.com/tarantool/tarantool/-/commits/skaplun/gh-5442-luajit-memory-profiler
> 
>  src/CMakeLists.txt |  6 ++++++
>  src/lua/init.c     | 15 ++++++++++++++-
>  2 files changed, 20 insertions(+), 1 deletion(-)
> 

<snipped>

> -- 
> 2.28.0
> 

-- 
Best regards,
IM

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools
  2020-12-28  7:09 ` Igor Munkin
@ 2020-12-28  7:10   ` Sergey Kaplun
  0 siblings, 0 replies; 5+ messages in thread
From: Sergey Kaplun @ 2020-12-28  7:10 UTC (permalink / raw)
  To: Igor Munkin; +Cc: tarantool-patches

Igor,

On 28.12.20, Igor Munkin wrote:
> Sergey,
> 
> Thanks for the patch! LGTM except a single nit.
> 
> On 28.12.20, Sergey Kaplun wrote:
> > This patch embeds a parser for binary data dumped via the memory
> > profiler to Tarantool binary.
> > 
> > It is a set of the following Lua modules:
> > * utils/bufread.lua: read binary data from a file.
> > * utils/symtab.lua: symbol table decode functions
> > * memprof/parse.lua: decode the memory profiler event stream
> > * memprof/humanize.lua: display decoded data in human readable format
> > * memprof.lua: Lua script and module to display data
> > 
> > It launch with the following command:
> > $ tarantool -e 'require("memprof")(arg[1])' - filename.bin
> > 
> > Closed #5490
> 
> Typo: s/Closed/Closes/.

Fixed and repushed branch with --force.

> 
> > ---
> > 
> > Issue: https://github.com/tarantool/tarantool/issues/5490
> > Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5442-luajit-memory-profiler
> > CI: https://gitlab.com/tarantool/tarantool/-/commits/skaplun/gh-5442-luajit-memory-profiler
> > 
> >  src/CMakeLists.txt |  6 ++++++
> >  src/lua/init.c     | 15 ++++++++++++++-
> >  2 files changed, 20 insertions(+), 1 deletion(-)
> > 
> 
> <snipped>
> 
> > -- 
> > 2.28.0
> > 
> 
> -- 
> Best regards,
> IM

-- 
Best regards,
Sergey Kaplun

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools
  2020-12-28  7:04 [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools Sergey Kaplun
  2020-12-28  7:09 ` Igor Munkin
@ 2020-12-28  7:55 ` Alexander V. Tikhonov
  2020-12-28  8:17 ` Igor Munkin
  2 siblings, 0 replies; 5+ messages in thread
From: Alexander V. Tikhonov @ 2020-12-28  7:55 UTC (permalink / raw)
  To: Sergey Kaplun; +Cc: tarantool-patches

Hi Sergey, thanks for the patch, as I see no new degradation found in
gitlab-ci testing commit criteria pipeline [1], patch LGTM.

[1] - https://gitlab.com/tarantool/tarantool/-/pipelines/234950657

On Mon, Dec 28, 2020 at 10:04:31AM +0300, Sergey Kaplun via Tarantool-patches wrote:
> This patch embeds a parser for binary data dumped via the memory
> profiler to Tarantool binary.
> 
> It is a set of the following Lua modules:
> * utils/bufread.lua: read binary data from a file.
> * utils/symtab.lua: symbol table decode functions
> * memprof/parse.lua: decode the memory profiler event stream
> * memprof/humanize.lua: display decoded data in human readable format
> * memprof.lua: Lua script and module to display data
> 
> It launch with the following command:
> $ tarantool -e 'require("memprof")(arg[1])' - filename.bin
> 
> Closed #5490
> ---
> 
> Issue: https://github.com/tarantool/tarantool/issues/5490
> Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5442-luajit-memory-profiler
> CI: https://gitlab.com/tarantool/tarantool/-/commits/skaplun/gh-5442-luajit-memory-profiler
> 
>  src/CMakeLists.txt |  6 ++++++
>  src/lua/init.c     | 15 ++++++++++++++-
>  2 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
> index b95688c1a..9a712bc29 100644
> --- a/src/CMakeLists.txt
> +++ b/src/CMakeLists.txt
> @@ -61,6 +61,12 @@ lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/vmdef.lua
>  lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/v.lua")
>  lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/p.lua")
>  lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/src/jit/zone.lua")
> +# LuaJIT tools.* library
> +lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof.lua")
> +lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof/humanize.lua")
> +lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/memprof/parse.lua")
> +lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/utils/bufread.lua")
> +lua_source(lua_sources "${CMAKE_BINARY_DIR}/third_party/luajit/tools/utils/symtab.lua")
>  
>  add_custom_target(generate_lua_sources
>      WORKING_DIRECTORY ${CMAKE_BINARY_DIR}/src/box
> diff --git a/src/lua/init.c b/src/lua/init.c
> index a0b2fc775..82075c595 100644
> --- a/src/lua/init.c
> +++ b/src/lua/init.c
> @@ -121,7 +121,14 @@ extern char strict_lua[],
>  	string_lua[],
>  	swim_lua[],
>  	p_lua[], /* LuaJIT 2.1 profiler */
> -	zone_lua[] /* LuaJIT 2.1 profiler */;
> +	zone_lua[], /* LuaJIT 2.1 profiler */
> +	/* tools.* libraries. */
> +	bufread_lua[],
> +	symtab_lua[],
> +	parse_lua[],
> +	humanize_lua[],
> +	memprof_lua[]
> +;
>  
>  static const char *lua_modules[] = {
>  	/* Make it first to affect load of all other modules */
> @@ -167,6 +174,12 @@ static const char *lua_modules[] = {
>  	/* Profiler */
>  	"jit.p", p_lua,
>  	"jit.zone", zone_lua,
> +	/* tools.* libraries. Order is important. */
> +	"utils.bufread", bufread_lua,
> +	"utils.symtab", symtab_lua,
> +	"memprof.parse", parse_lua,
> +	"memprof.humanize", humanize_lua,
> +	"memprof", memprof_lua,
>  	NULL
>  };
>  
> -- 
> 2.28.0
> 

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools
  2020-12-28  7:04 [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools Sergey Kaplun
  2020-12-28  7:09 ` Igor Munkin
  2020-12-28  7:55 ` Alexander V. Tikhonov
@ 2020-12-28  8:17 ` Igor Munkin
  2 siblings, 0 replies; 5+ messages in thread
From: Igor Munkin @ 2020-12-28  8:17 UTC (permalink / raw)
  To: Sergey Kaplun; +Cc: tarantool-patches

Sergey,

On 28.12.20, Sergey Kaplun wrote:
> This patch embeds a parser for binary data dumped via the memory
> profiler to Tarantool binary.
> 
> It is a set of the following Lua modules:
> * utils/bufread.lua: read binary data from a file.
> * utils/symtab.lua: symbol table decode functions
> * memprof/parse.lua: decode the memory profiler event stream
> * memprof/humanize.lua: display decoded data in human readable format
> * memprof.lua: Lua script and module to display data
> 
> It launch with the following command:
> $ tarantool -e 'require("memprof")(arg[1])' - filename.bin
> 
> Closed #5490
> ---
> 
> Issue: https://github.com/tarantool/tarantool/issues/5490
> Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5442-luajit-memory-profiler
> CI: https://gitlab.com/tarantool/tarantool/-/commits/skaplun/gh-5442-luajit-memory-profiler

I've checked your patch master.

> 

<snipped>

>  
> -- 
> 2.28.0
> 

-- 
Best regards,
IM

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-12-28  8:17 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-28  7:04 [Tarantool-patches] [PATCH] build: embed LuaJIT memory profiler tools Sergey Kaplun
2020-12-28  7:09 ` Igor Munkin
2020-12-28  7:10   ` Sergey Kaplun
2020-12-28  7:55 ` Alexander V. Tikhonov
2020-12-28  8:17 ` Igor Munkin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox