<HTML><BODY><div>Hi, Sergey!</div><div>Thanks for the comments!</div><div>Here is the new message:</div><div>=======</div><div><div><div>profilers: purge generation mechanism</div></div><div> </div><div><div>Since both of the profiler parsers are now processing</div><div>the events in a stream-like fashion, the generation</div><div>mechanism is excessive and can be purged. This results</div><div>in a significant memory consumption drop, especially</div><div>for the AVL-tree part.</div></div><div> </div><div><div>Consider this script:</div><div>| jit.off()</div><div>| misc.sysprof.start{mode = 'C', interval=10}</div><div>| for i = 1, 1e7 do tostring(i) end</div><div>| misc.sysprof.stop()</div></div><div> </div><div><div>After executing it with LuaJIT, you can parse it like this:</div><div>| $ time -v luajit-parse-sysprof sysprof.bin</div></div><div> </div><div><div>So, before the patch:</div><div>| Maximum resident set size (kbytes): 224928</div></div><div> </div><div><div>And after the patch:</div><div>| Maximum resident set size (kbytes): 32780</div></div><div> </div><div><div>That is the 85% reduction in memory consumption.</div></div><div> </div><div><div>Follows up tarantool/tarantool#8700</div><div>=======</div></div></div><div> </div><blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;"><div> <blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;"><div id=""><div class="js-helper js-readmsg-msg"><div><div id="style_16946848220666326337_BODY"><div class="cl_060196"><p>Hi, Max</p><p> </p><div class="moz-cite-prefix_mr_css_attr">On 9/12/23 13:26, Maxim Kokryashkin wrote:</div><blockquote type="cite"><div>Hi, Sergey!</div><div>Sure, consider this script:</div><div><div><div>| jit.off()</div><div>| misc.sysprof.start{mode = 'C', interval=10}</div><div>| for i = 1, 1e7 do tostring(i) end</div><div>| misc.sysprof.stop()</div></div></div></blockquote><p>I failed with step:</p><p>$ cat prof.lua<br>jit.off()<br>misc.sysprof.start{mode = 'C', interval=10}<br>for i = 1, 1e7 do tostring(i) end<br>misc.sysprof.stop()<br> </p><p>$ ./build/src/luajit prof.lua<br>Segmentation fault (core dumped)</p></div></div></div></div></div></blockquote><div>That may happen sometimes, since we still haven’t merged my patch for ffuncs.</div><blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;"><div><div class="js-helper js-readmsg-msg"><div><div><div class="cl_060196"><p> </p><p>119 lj_wbuf_addu64(buf, (uint64_t)pt->firstline);<br>(gdb) bt<br>#0 0x0000563092387308 in stream_lfunc (buf=0x563092449458 <sysprof+24>, func=0x7fdec2bda0a0)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:119<br>#1 0x00005630923874a9 in stream_frame_lua (buf=0x563092449458 <sysprof+24>, frame=0x7fdec48a2cd8)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:141<br>#2 0x00005630923876b1 in stream_backtrace_lua (sp=0x563092449440 <sysprof>)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:169<br>#3 0x0000563092387ab1 in stream_guest (sp=0x563092449440 <sysprof>, vmstate=2)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:237<br>#4 0x0000563092387c3f in stream_event (sp=0x563092449440 <sysprof>, vmstate=2)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:274<br>#5 0x0000563092387d26 in sysprof_record_sample (sp=0x563092449440 <sysprof>, info=0x7ffe5b59b730)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:296<br>#6 0x0000563092387ddc in sysprof_signal_handler (sig=27, info=0x7ffe5b59b730, ctx=0x7ffe5b59b600)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:312<br>#7 <signal handler called><br>#8 0x00005630923ca3c5 in lj_fff_res1 () at buildvm_x86.dasc:1890<br>#9 0x000056309232c8e2 in lua_pcall (L=0x7fdec48a1378, nargs=0, nresults=-1, errfunc=2)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1172<br>#10 0x000056309231ec29 in docall (L=0x7fdec48a1378, narg=0, clear=0) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:133<br>#11 0x000056309231f63d in handle_script (L=0x7fdec48a1378, argx=0x7ffe5b59c0b0)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:303<br>#12 0x000056309232059b in pmain (L=0x7fdec48a1378) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:603<br>#13 0x00005630923c9029 in lj_BC_FUNCC () at buildvm_x86.dasc:852<br>#14 0x000056309232cf4a in lua_cpcall (L=0x7fdec48a1378, func=0x5630923203a3 <pmain>, ud=0x0)<br> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1207<br>#15 0x00005630923206b7 in main (argc=2, argv=0x7ffe5b59c0a8) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:632<br>(gdb)<br> </p><p> </p><blockquote type="cite"><div> </div><div>After running it with luajit you can parse the output</div><div>like this:</div><div>| $ time -v luajit-parse-sysprof sysprof.bin</div><div> </div><div>So, before the patch:</div><div>| Maximum resident set size (kbytes): 224928</div><div> </div><div>And after the patch:</div><div><div><div>| Maximum resident set size (kbytes): 32780</div><div> </div><div>Which is 85% reduction in memory consumption.</div></div></div></blockquote><p> </p><p>I propose to add these numbers to commit message.</p><p> </p><blockquote type="cite"><div data-signature-widget="container"><div data-signature-widget="content"><div>--<br>Best regards,</div><div>Maxim Kokryashkin</div></div></div><div> </div><div> </div><blockquote style="border-left:1px solid #0857A6;margin:10px;padding:0 0 0 10px;">Вторник, 5 сентября 2023, 14:53 +03:00 от Sergey Bronnikov <a href="//e.mail.ru/compose/?mailto=mailto%3asergeyb@tarantool.org" class="moz-txt-link-rfc2396E_mr_css_attr"><sergeyb@tarantool.org></a>:<br> <div id=""><div class="js-helper_mr_css_attr js-readmsg-msg_mr_css_attr"><div><div id="style_16939148370749566697_BODY_mr_css_attr">Hi, Max<br><br>thanks for the patch. See my comment below.<br><br><br>On <span class="js-phone-number_mr_css_attr"><span class="js-phone-number">8/28/23 18</span></span>:23, Maxim Kokryashkin wrote:<div class="mail-quote-collapse">> Since both of the profiler parsers are now processing<br>> the events in a stream-like fashion, the generation<br>> mechanism is excessive and can be purged. This results<br>> in a significant memory consumption drop, especially<br>> for the AVL-tree part.</div><br>Would be interesting to see a numbers with memory consumption<br><br>before and after the patch.<br><br> </div></div></div></div></blockquote><div> </div></blockquote></div></div></div></div></div></blockquote><div><div>--<br>Best regards,</div><div>Maxim Kokryashkin</div></div><div> </div></div></blockquote></BODY></HTML>