From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 777357030B; Tue, 26 Jan 2021 00:08:38 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 777357030B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1611608918; bh=allermnSmz3pP6uUiNKZhjVm4ddeOvQOp5OxXIWlFQE=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wJkWVuSZT6GdYzQbleDdHU1iDwwLuITOu21mm0jTjKzo1E+sgGvQHUKt5M2jsmcD7 Pzi+MGqrcQwqBy3q5TtO57D4uawrxsIacNl86J4I/g8SUCZDcMcX1Eeh43VGUTpcYN 4vaoHBr7A2WCKCoc0Bq9zuCHJ5cDt2y25ESqYibw= Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id B180B7030B for ; Tue, 26 Jan 2021 00:08:36 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B180B7030B Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1l496A-0004w1-Pa; Tue, 26 Jan 2021 00:08:35 +0300 To: Cyrill Gorcunov , TML References: <20210125203550.GH2174@grain> Message-ID: <9beb6862-b21a-a054-e66d-b744152c1be0@tarantool.org> Date: Mon, 25 Jan 2021 22:08:33 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <20210125203550.GH2174@grain> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92BC6D7A73B5E1EC959C1FBF906E5CB90956EF1D42741DB9200894C459B0CD1B97388F6D33E5F4A68CFC7A3EB6551C28A319CA0423476892BA9248E35AFBCB78E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7D9C4478D0B876341EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637204D59D994DFFAD78638F802B75D45FF5571747095F342E8C7A0BC55FA0FE5FC94D3DCEB11FDB5CCFC150252B7D9A6D230A8D6B98F749220389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0ECC8AC47CD0EDEFF8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6D52CD31C43BF465FCC7F00164DA146DA6F5DAA56C3B73B23C77107234E2CFBA567F23339F89546C55F5C1EE8F4F765FC47E915A23788DB9575ECD9A6C639B01BBD4B6F7A4D31EC0BC0CAF46E325F83A522CA9DD8327EE4930A3850AC1BE2E73526B9191E2D567F0EC4224003CC836476C0CAF46E325F83A50BF2EBBBDD9D6B0FECB2555BB02FD5A93B503F486389A921A5CC5B56E945C8DA X-B7AD71C0: 502860A5C27E5D50661228F9A2C48A9379311020FFC8D4AD045D7F41E4F490996CAD3FA2B197871C X-C1DE0DAB: 0D63561A33F958A512C89AB9361649B3F7729E560FB44D3B58D9131F9745240ED59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75448CF9D3A7B2C848410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3416EA6E382A5BB1762AABF0244C76B2E7565339586D47B770923D9C40240989F2A4CB31EDA436A1141D7E09C32AA3244C70D865FDAEF1E771B64A3C0746ECA0B6E3D93501275E802F927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbLwCnVtyrif7WZODvI+40A== X-Mailru-Sender: 689FA8AB762F73936BC43F508A0638221FCD1EE643DE0D36402172D6020037663841015FED1DE5223CC9A89AB576DD93FB559BB5D741EB963CF37A108A312F5C27E8A8C3839CE0E267EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [RFC] cmod user api X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Cc: Mons Anderson Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! On 25.01.2021 21:35, Cyrill Gorcunov wrote: > Hi guys! Recently I've sent a series which implements that > named "cmod" module which allows to load and run stored C > procedures similar to 'box.func' except they allows to run > such functions even on nodes in read-only mode. > > The overall picture of api was not clear from changelog and > here I provide a short description to gather comments. It is clear from the changelog. The problem was that the API was *implemented* before getting approves on it. For big public features like this it usually leads to huge waste of time and disappointment of the author when patches get rejected due to implementing something in a wrong way. Which is not the case for this version I hope. For me all looks sane in the API except reload. > Entry point of any function comes from "module" instance. > To load a module one have to > > module = require('cmod').load('path-to-module') > > Note the path to the module should be shipped without .so > extension (or dylib for macos). I suspect we should support > both variants with and without file extension. Nevermind, it is platform-dependent. I think it is fine just like it works in _func now - file name without an extension. > Once module is loaded we can associate a function from > this module with some Lua variable. > > foo = module:load('name-of-function') > > Then we can execute it > > res = foo(arguments, ...) > > Both function and module supports explicit unloading via > unload method. > > foo:unload() > module:unload() > > When function/module is unloaded any attempt to use this > variable will produce an error. If they are not explicitly > unloaded then GC will reap them. > > Module Lua object provides :reload() method which re-reads > the shared library and updates associated functions such than > any new call will be executed via new code. If there are some > already executing functions which are in yield state then such > functions gonna finish execution of old code first and only > next calls will be pacing the new code. > > Does such api looks sane or there some other ideas? I will copy-paste here what I said about reloading the existing function objects under the hood. Reason for not changing the existing function objects was that the reload is supposed to happen when the whole application reloads. It means there can be some fibers running the old code for a while after reload until they notice that they must be restarted. If you do reload like you did in this patch, the old fibers will get new functions immediately right under their feet, and this may result into unexpected behaviour for code in these fibers. For instance, you did this code: local func = mod:load('do_replace_and_yield') for i, obj in pairs(objs) do func(obj) end func:unload() Assume 'do_replace_and_yield' yields inside. Also assume that the loop is started in a separate fiber, and func is called first time. It yields, and now the whole Lua module, having this code, is reloaded. This old code still runs in the fiber, created in the beginning before reload. During Lua application reload the shared module was also reloaded, and now 'do_replace_and_yield' expects two arguments instead of 1. So on the next iteration of the loop 'func' will fail. The user didn't do anything with 'func' object, but it was changed inside, and the user couldn't do anything about it. Calling 'load()' on each iteration is not an option if the loop iterates thousands of times and more. All becomes even worse if 'do_replace_and_yield' does accept the arguments, but its behaviour is changed. For instance, its result format. Another way to get the desired behaviour - use unload() + load() - I assume in this case the old function objects will remain unchanged, right? But do we really need such 'reload' behaviour then? I suggest at least to ask Mons what to do with this.