From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp40.i.mail.ru (smtp40.i.mail.ru [94.100.177.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2C7FD469719 for ; Wed, 30 Sep 2020 13:09:16 +0300 (MSK) Date: Wed, 30 Sep 2020 13:09:28 +0300 From: Alexander Turenko Message-ID: <20200930100928.ybprk7zniggipe45@tkn_work_nb> References: <6fc62e3a3935978b650fd6d1dcbdef9a5353a1fb.1600955781.git.tsafin@tarantool.org> <40b631b2-c660-32f7-c421-9770d5d06de4@tarantool.org> <20200929151002.GY18920@tarantool.org> <20200929210345.jseljymlmkgdrp3c@tkn_work_nb> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH 2.X 5/7] module api: luaT_temp_luastate & luaT_release_temp_luastate List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org On Wed, Sep 30, 2020 at 01:23:17AM +0200, Vladislav Shpilevoy wrote: > On 29.09.2020 23:03, Alexander Turenko wrote: > > On Tue, Sep 29, 2020 at 06:10:02PM +0300, Igor Munkin wrote: > >> Vlad, > >> > >> I guess there should be a question to be asked in scope of the original > >> series: what performance enhancement can we obtain with such hacky > >> optimization? Do we have any benchmarks? If the performance impact is > >> negligible I believe these interfaces should be left internal. > > > > Why hacky? I'm a module developer and I ask Tarantool: Hey, I know you > > have a cache of Lua states. Give me one? Can I yield while the state is > > in use? I can, nice. Can I share it between fibers? No, okay. > > It is not really a cache of lua states. Maybe of a single lua state, and > working only in Lua fibers (in C fibers you will create/delete the state > constantly), but not of states. I was about to say that background fibers also can reuse a Lua state after ec9b544a123f3e0614ab94b83a7ac4b395326f34 ('lua: expose temporary Lua state for iproto calls'), but it is not so in 1.10. And you're about C fibers, so, yep, I agree: it is not as simple as the cache pattern. The near goal of exposing those functions would be offer an easy way to save some allocations in the external merger module (and maybe others in future). Since the module is mainly needed for 1.10 and the functions will not save allocations on 1.10 -- so the practial reason is negligible. The performance difference also looks small (I shared benchmarking results in this mailing thread). Since there are questions about design of those functions, I would use lua_newthread() (or wrapper around it to handle OOM -- written in the module). We can return back to this API later. > > A proper Lua state cache/pool can be implemented easily without fiber at > all. Just have a pool with these states, where you either take a free one, > or allocate a new when the pool is empty. And when you don't need a state, > put it back into the pool. Then you will have at most the same number of > states in this pool, as the number of fibers, using this pool. Does not > seem to be a complex thing, will fit in 50-70 lines on C I think, top. > Neither seem to be too heavy (I didn't check), since all these states will > have empty stack, cleared before putting them back to the pool. > > > That's all. Quite simple. > > It is not that simple. Whatever you export now, we will need to support > potentially forever. So exporting not vital things increases support costs > for us for a very long time by saving a few time to you and Timur so you > don't need to change merger code and can move it as is. I would better > one time made merger less depending on Tarantool internals, then do long > support of the new pile of internal methods exposed in a hurry. I agree, but also I consider this activity as good opportunity to improve the module API. Of course, when we have good design.