From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 871B1445320 for ; Sat, 25 Jul 2020 00:55:48 +0300 (MSK) Date: Sat, 25 Jul 2020 00:45:28 +0300 From: Igor Munkin Message-ID: <20200724214528.GV18920@tarantool.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH] lua/utils: improve luaT_newthread performance List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Kirill Yukhin Cc: tarantool-patches@dev.tarantool.org, Vladislav Shpilevoy Kirill, Please proceed with the patch. On 20.07.20, Igor Munkin wrote: > created a new GCfunc object for the helper invoked in a > protected frame (i.e. ) on each > call. The change introduces a static reference to a GCfunc object for > to be initialized on Tarantool startup to > reduce Lua GC memory usage. > > Furthermore, since yields nothing on guest stack, the newly > created Lua coroutine need to be pushed back to prevent its sweep. So > to reduce guest stack manipulations is replaced with > and the resulting Lua thread is obtained via guest stack. > > Signed-off-by: Igor Munkin > --- > Recently we discussed with Timur his struggling with linking his binary > Lua module against Tarantool. The reason is LuaJIT internals usage for > manipulations with the guest stack that are not provided by the binary. > I glanced the current implementation and found out two > another problems related to the platform overall performance (as it is > proved with the corresponding benchmarks). > > The first problem is the similar one had prior to the > patch[1]: creates an auxiliary GCfunc object for the > function to be called in protected frame. However this function is the > same throughout the platform uptime. It can be created on Taranool > startup and I see no reason to clobber GC that way. > > Another problem I found are excess manipulations with the guest stack: > one need the newly born coroutine on it to prevent it being collected by > GC, but purges everything left on the stack in scope of the > invoked function. As a result the obtained Lua coroutine is "pushed" > back to the guest stack via LuaJIT internal interfaces. It's a bit > ridiculous, since one can just use public API to get the same results: > Lua coroutine on the guest stack and the corresponding pointer to it. > > I tested the platform performance with the same benchmark[2] I made for > the patch and here are the numbers I obtained after > the 15 runs: > * Vanilla bleeding master (mean): > | ===== 2.5.0-267-gbf047ad44 ===== > | call by ref GC: 921877 Kb > | call by ref time: 1.340172 sec > | call GC: 476563.991667 Kb > | eval GC: 655274.935547 Kb > * Patched bleeding master (mean): > | ===== 2.5.0-268-gec0eb12f4 ===== > | call by ref GC: 859377 Kb > | call by ref time: 1.215410 sec > | call GC: 445313 Kb > | eval GC: 624024 Kb > * Relative measurements (before -> after): > | call by ref GC: -6% (-62500 Kb) > | call by ref time: -9% (-0.124762 sec) > | call GC: -6% (-31250 Kb) > | eval GC: -4% (-31250 Kb) > > There is one hot path I left unverified -- Lua-born fibers creation, but > I guess the relative numbers are quite similar to the ones I mentioned > above. However, if one wonders these results, feel free to ask me. > > Branch: https://github.com/tarantool/tarantool/tree/imun/experimental-luaT-newthread > > @ChangeLog: > * Improved safe Lua coroutine creation for the case of fiber > initialization. Prepared GCfunc object is used instead of temporary > one, resulting in 3-6% garbage collection reduction. Also excess guest > stack manipulations are removed. > > src/lua/utils.c | 32 ++++++++++++++++++++++++++++++++ > src/lua/utils.h | 38 ++++++++++---------------------------- > 2 files changed, 42 insertions(+), 28 deletions(-) > > [1]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-June/017834.html > [2]: https://gist.github.com/igormunkin/c941074fa9fdf0f7a4f068498fb5e24c > > -- Best regards, IM