Re: [Tarantool-patches] [PATCH luajit 2/2] profilers: purge generation mechanism

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 3881 bytes --]

Hi, Max


On 9/12/23 13:26, Maxim Kokryashkin wrote:
> Hi, Sergey!
> Sure, consider this script:
> | jit.off()
> | misc.sysprof.start{mode = 'C', interval=10}
> | for i = 1, 1e7 do tostring(i) end
> | misc.sysprof.stop()

I failed with step:

$ cat prof.lua
jit.off()
misc.sysprof.start{mode = 'C', interval=10}
for i = 1, 1e7 do tostring(i) end
misc.sysprof.stop()

$ ./build/src/luajit prof.lua
Segmentation fault (core dumped)

119       lj_wbuf_addu64(buf, (uint64_t)pt->firstline);
(gdb) bt
#0  0x0000563092387308 in stream_lfunc (buf=0x563092449458 <sysprof+24>, 
func=0x7fdec2bda0a0)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:119
#1  0x00005630923874a9 in stream_frame_lua (buf=0x563092449458 
<sysprof+24>, frame=0x7fdec48a2cd8)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:141
#2  0x00005630923876b1 in stream_backtrace_lua (sp=0x563092449440 <sysprof>)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:169
#3  0x0000563092387ab1 in stream_guest (sp=0x563092449440 <sysprof>, 
vmstate=2)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:237
#4  0x0000563092387c3f in stream_event (sp=0x563092449440 <sysprof>, 
vmstate=2)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:274
#5  0x0000563092387d26 in sysprof_record_sample (sp=0x563092449440 
<sysprof>, info=0x7ffe5b59b730)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:296
#6  0x0000563092387ddc in sysprof_signal_handler (sig=27, 
info=0x7ffe5b59b730, ctx=0x7ffe5b59b600)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:312
#7  <signal handler called>
#8  0x00005630923ca3c5 in lj_fff_res1 () at buildvm_x86.dasc:1890
#9  0x000056309232c8e2 in lua_pcall (L=0x7fdec48a1378, nargs=0, 
nresults=-1, errfunc=2)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1172
#10 0x000056309231ec29 in docall (L=0x7fdec48a1378, narg=0, clear=0) at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:133
#11 0x000056309231f63d in handle_script (L=0x7fdec48a1378, 
argx=0x7ffe5b59c0b0)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:303
#12 0x000056309232059b in pmain (L=0x7fdec48a1378) at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:603
#13 0x00005630923c9029 in lj_BC_FUNCC () at buildvm_x86.dasc:852
#14 0x000056309232cf4a in lua_cpcall (L=0x7fdec48a1378, 
func=0x5630923203a3 <pmain>, ud=0x0)
     at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1207
#15 0x00005630923206b7 in main (argc=2, argv=0x7ffe5b59c0a8) at 
/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:632
(gdb)


> After running it with luajit you can parse the output
> like this:
> | $ time -v luajit-parse-sysprof sysprof.bin
> So, before the patch:
> | Maximum resident set size (kbytes): 224928
> And after the patch:
> | Maximum resident set size (kbytes): 32780
> Which is 85% reduction in memory consumption.


I propose to add these numbers to commit message.


> --
> Best regards,
> Maxim Kokryashkin
>
>     Вторник, 5 сентября 2023, 14:53 +03:00 от Sergey Bronnikov
>     <sergeyb@tarantool.org>:
>     Hi, Max
>
>     thanks for the patch. See my comment below.
>
>
>     On 8/28/23 18:23, Maxim Kokryashkin wrote:
>     > Since both of the profiler parsers are now processing
>     > the events in a stream-like fashion, the generation
>     > mechanism is excessive and can be purged. This results
>     > in a significant memory consumption drop, especially
>     > for the AVL-tree part.
>
>     Would be interesting to see a numbers with memory consumption
>
>     before and after the patch.
>

[-- Attachment #2: Type: text/html, Size: 6612 bytes --]