Hi, Max

Hi, Sergey!

Sure, consider this script:

| jit.off()

| misc.sysprof.start{mode = 'C', interval=10}

| for i = 1, 1e7 do tostring(i) end

| misc.sysprof.stop()

I failed with step:

$ cat prof.lua
jit.off()
misc.sysprof.start{mode = 'C', interval=10}
for i = 1, 1e7 do tostring(i) end
misc.sysprof.stop()

$ ./build/src/luajit prof.lua
Segmentation fault (core dumped)

119       lj_wbuf_addu64(buf, (uint64_t)pt->firstline);
(gdb) bt
#0  0x0000563092387308 in stream_lfunc (buf=0x563092449458 <sysprof+24>, func=0x7fdec2bda0a0)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:119
#1  0x00005630923874a9 in stream_frame_lua (buf=0x563092449458 <sysprof+24>, frame=0x7fdec48a2cd8)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:141
#2  0x00005630923876b1 in stream_backtrace_lua (sp=0x563092449440 <sysprof>)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:169
#3  0x0000563092387ab1 in stream_guest (sp=0x563092449440 <sysprof>, vmstate=2)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:237
#4  0x0000563092387c3f in stream_event (sp=0x563092449440 <sysprof>, vmstate=2)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:274
#5  0x0000563092387d26 in sysprof_record_sample (sp=0x563092449440 <sysprof>, info=0x7ffe5b59b730)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:296
#6  0x0000563092387ddc in sysprof_signal_handler (sig=27, info=0x7ffe5b59b730, ctx=0x7ffe5b59b600)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:312
#7  <signal handler called>
#8  0x00005630923ca3c5 in lj_fff_res1 () at buildvm_x86.dasc:1890
#9  0x000056309232c8e2 in lua_pcall (L=0x7fdec48a1378, nargs=0, nresults=-1, errfunc=2)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1172
#10 0x000056309231ec29 in docall (L=0x7fdec48a1378, narg=0, clear=0) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:133
#11 0x000056309231f63d in handle_script (L=0x7fdec48a1378, argx=0x7ffe5b59c0b0)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:303
#12 0x000056309232059b in pmain (L=0x7fdec48a1378) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:603
#13 0x00005630923c9029 in lj_BC_FUNCC () at buildvm_x86.dasc:852
#14 0x000056309232cf4a in lua_cpcall (L=0x7fdec48a1378, func=0x5630923203a3 <pmain>, ud=0x0)
    at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1207
#15 0x00005630923206b7 in main (argc=2, argv=0x7ffe5b59c0a8) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:632
(gdb)

 

After running it with luajit you can parse the output

like this:

| $ time -v luajit-parse-sysprof sysprof.bin

 

So, before the patch:

| Maximum resident set size (kbytes): 224928

 

And after the patch:

| Maximum resident set size (kbytes): 32780

 

Which is 85% reduction in memory consumption.

I propose to add these numbers to commit message.

--
Best regards,

Maxim Kokryashkin

 

 

Вторник, 5 сентября 2023, 14:53 +03:00 от Sergey Bronnikov <sergeyb@tarantool.org>:
 

Hi, Max

thanks for the patch. See my comment below.


On 8/28/23 18:23, Maxim Kokryashkin wrote:

> Since both of the profiler parsers are now processing
> the events in a stream-like fashion, the generation
> mechanism is excessive and can be purged. This results
> in a significant memory consumption drop, especially
> for the AVL-tree part.

Would be interesting to see a numbers with memory consumption

before and after the patch.