From: Maxim Kokryashkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: "Sergey Bronnikov" <sergeyb@tarantool.org>
Cc: "Maxim Kokryashkin" <max.kokryashkin@gmail.com>,
tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH luajit 2/2] profilers: purge generation mechanism
Date: Mon, 18 Sep 2023 12:18:15 +0300 [thread overview]
Message-ID: <1695028695.970460840@f316.i.mail.ru> (raw)
In-Reply-To: <370699e3-9e7d-f24b-600d-970cd5bed080@tarantool.org>
[-- Attachment #1: Type: text/plain, Size: 4978 bytes --]
Hi, Sergey!
Thanks for the comments!
Here is the new message:
=======
profilers: purge generation mechanism
Since both of the profiler parsers are now processing
the events in a stream-like fashion, the generation
mechanism is excessive and can be purged. This results
in a significant memory consumption drop, especially
for the AVL-tree part.
Consider this script:
| jit.off()
| misc.sysprof.start{mode = 'C', interval=10}
| for i = 1, 1e7 do tostring(i) end
| misc.sysprof.stop()
After executing it with LuaJIT, you can parse it like this:
| $ time -v luajit-parse-sysprof sysprof.bin
So, before the patch:
| Maximum resident set size (kbytes): 224928
And after the patch:
| Maximum resident set size (kbytes): 32780
That is the 85% reduction in memory consumption.
Follows up tarantool/tarantool#8700
=======
>
>>Hi, Max
>>
>>On 9/12/23 13:26, Maxim Kokryashkin wrote:
>>>Hi, Sergey!
>>>Sure, consider this script:
>>>| jit.off()
>>>| misc.sysprof.start{mode = 'C', interval=10}
>>>| for i = 1, 1e7 do tostring(i) end
>>>| misc.sysprof.stop()
>>I failed with step:
>>$ cat prof.lua
>>jit.off()
>>misc.sysprof.start{mode = 'C', interval=10}
>>for i = 1, 1e7 do tostring(i) end
>>misc.sysprof.stop()
>>
>>$ ./build/src/luajit prof.lua
>>Segmentation fault (core dumped)
>That may happen sometimes, since we still haven’t merged my patch for ffuncs.
>>
>>119 lj_wbuf_addu64(buf, (uint64_t)pt->firstline);
>>(gdb) bt
>>#0 0x0000563092387308 in stream_lfunc (buf=0x563092449458 <sysprof+24>, func=0x7fdec2bda0a0)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:119
>>#1 0x00005630923874a9 in stream_frame_lua (buf=0x563092449458 <sysprof+24>, frame=0x7fdec48a2cd8)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:141
>>#2 0x00005630923876b1 in stream_backtrace_lua (sp=0x563092449440 <sysprof>)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:169
>>#3 0x0000563092387ab1 in stream_guest (sp=0x563092449440 <sysprof>, vmstate=2)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:237
>>#4 0x0000563092387c3f in stream_event (sp=0x563092449440 <sysprof>, vmstate=2)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:274
>>#5 0x0000563092387d26 in sysprof_record_sample (sp=0x563092449440 <sysprof>, info=0x7ffe5b59b730)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:296
>>#6 0x0000563092387ddc in sysprof_signal_handler (sig=27, info=0x7ffe5b59b730, ctx=0x7ffe5b59b600)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_sysprof.c:312
>>#7 <signal handler called>
>>#8 0x00005630923ca3c5 in lj_fff_res1 () at buildvm_x86.dasc:1890
>>#9 0x000056309232c8e2 in lua_pcall (L=0x7fdec48a1378, nargs=0, nresults=-1, errfunc=2)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1172
>>#10 0x000056309231ec29 in docall (L=0x7fdec48a1378, narg=0, clear=0) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:133
>>#11 0x000056309231f63d in handle_script (L=0x7fdec48a1378, argx=0x7ffe5b59c0b0)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:303
>>#12 0x000056309232059b in pmain (L=0x7fdec48a1378) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:603
>>#13 0x00005630923c9029 in lj_BC_FUNCC () at buildvm_x86.dasc:852
>>#14 0x000056309232cf4a in lua_cpcall (L=0x7fdec48a1378, func=0x5630923203a3 <pmain>, ud=0x0)
>> at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/lj_api.c:1207
>>#15 0x00005630923206b7 in main (argc=2, argv=0x7ffe5b59c0a8) at /home/sergeyb/sources/MRG/tarantool/third_party/luajit/src/luajit.c:632
>>(gdb)
>>
>>
>>>
>>>After running it with luajit you can parse the output
>>>like this:
>>>| $ time -v luajit-parse-sysprof sysprof.bin
>>>
>>>So, before the patch:
>>>| Maximum resident set size (kbytes): 224928
>>>
>>>And after the patch:
>>>| Maximum resident set size (kbytes): 32780
>>>
>>>Which is 85% reduction in memory consumption.
>>
>>I propose to add these numbers to commit message.
>>
>>>--
>>>Best regards,
>>>Maxim Kokryashkin
>>>
>>>
>>>>Вторник, 5 сентября 2023, 14:53 +03:00 от Sergey Bronnikov <sergeyb@tarantool.org> :
>>>>
>>>>Hi, Max
>>>>
>>>>thanks for the patch. See my comment below.
>>>>
>>>>
>>>>On 8/28/23 18 :23, Maxim Kokryashkin wrote:
>>>>> Since both of the profiler parsers are now processing
>>>>> the events in a stream-like fashion, the generation
>>>>> mechanism is excessive and can be purged. This results
>>>>> in a significant memory consumption drop, especially
>>>>> for the AVL-tree part.
>>>>Would be interesting to see a numbers with memory consumption
>>>>
>>>>before and after the patch.
>>>>
>>>>
>>>
>--
>Best regards,
>Maxim Kokryashkin
>
[-- Attachment #2: Type: text/html, Size: 7404 bytes --]
next prev parent reply other threads:[~2023-09-18 9:18 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-28 15:23 [Tarantool-patches] [PATCH luajit 0/2] profilers: purge generations Maxim Kokryashkin via Tarantool-patches
2023-08-28 15:23 ` [Tarantool-patches] [PATCH luajit 1/2] memprof: refactor symbol resolution Maxim Kokryashkin via Tarantool-patches
2023-09-03 10:14 ` Sergey Kaplun via Tarantool-patches
2023-09-04 5:44 ` Maxim Kokryashkin via Tarantool-patches
2023-09-05 11:47 ` Sergey Bronnikov via Tarantool-patches
2023-09-08 11:04 ` Maxim Kokryashkin via Tarantool-patches
2023-10-06 14:52 ` Sergey Bronnikov via Tarantool-patches
2023-08-28 15:23 ` [Tarantool-patches] [PATCH luajit 2/2] profilers: purge generation mechanism Maxim Kokryashkin via Tarantool-patches
2023-09-03 10:21 ` Sergey Kaplun via Tarantool-patches
2023-09-04 5:47 ` Maxim Kokryashkin via Tarantool-patches
2023-09-05 11:53 ` Sergey Bronnikov via Tarantool-patches
2023-09-12 10:26 ` Maxim Kokryashkin via Tarantool-patches
2023-09-14 9:47 ` Sergey Bronnikov via Tarantool-patches
2023-09-18 9:18 ` Maxim Kokryashkin via Tarantool-patches [this message]
2023-10-06 14:43 ` Sergey Bronnikov via Tarantool-patches
2023-10-06 14:45 ` Sergey Bronnikov via Tarantool-patches
2023-10-06 14:40 ` Sergey Bronnikov via Tarantool-patches
2023-10-06 14:52 ` Sergey Bronnikov via Tarantool-patches
2023-11-13 18:48 ` Igor Munkin via Tarantool-patches
2023-11-23 6:32 ` [Tarantool-patches] [PATCH luajit 0/2] profilers: purge generations Igor Munkin via Tarantool-patches
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1695028695.970460840@f316.i.mail.ru \
--to=tarantool-patches@dev.tarantool.org \
--cc=m.kokryashkin@tarantool.org \
--cc=max.kokryashkin@gmail.com \
--cc=sergeyb@tarantool.org \
--subject='Re: [Tarantool-patches] [PATCH luajit 2/2] profilers: purge generation mechanism' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox