From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 4353970310; Thu, 4 Mar 2021 16:29:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4353970310 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1614864562; bh=zIgnPpnoLsD0mCOTU1XAeXsZaJPUrMG4btA19jTGU+8=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Y0suTbTPV5tc3KnzAbM2rROI/0PJYKZsxX5TeMilXZxoAkZ1KGRnFoRHS/dWLG8/j PeegEqppKjl/QJF7/wApdCUHsBQ6yXbhT864lDiAzgZx20XcQpSKKKBcNFQX4F0y/6 kp921vTklIkTGI18Qykq36zOKVH57g41loVqYqD8= Received: from smtp29.i.mail.ru (smtp29.i.mail.ru [94.100.177.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 9559270310 for ; Thu, 4 Mar 2021 16:29:20 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9559270310 Received: by smtp29.i.mail.ru with esmtpa (envelope-from ) id 1lHo2Z-0007zZ-Jz; Thu, 04 Mar 2021 16:29:20 +0300 Date: Thu, 4 Mar 2021 16:28:32 +0300 To: Igor Munkin Message-ID: <20210304132832.GE6842@root> References: <20210216201044.20952-1-skaplun@tarantool.org> <20210226220704.GF9042@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210226220704.GF9042@tarantool.org> X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92A98208ECBDD29F5B42DD879439ED63E04973C1061E33475182A05F538085040ED0BF229BA32F2E89BF0560CF23947F2AE11FAF7AEF23CB48CCC144430E44D82 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7194E3A96294A28ABEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637BBB1CE3B4DF50705EA1F7E6F0F101C67CDEEF6D7F21E0D1D174C73DBBBFC7664CA530058E31B75AEEB006A7FC3F5754CA70388049BCF3C09389733CBF5DBD5E913377AFFFEAFD269A417C69337E82CC2CC7F00164DA146DAFE8445B8C89999729449624AB7ADAF37F6B57BC7E64490611E7FA7ABCAF51C92176DF2183F8FC7C0A3E989B1926288338941B15DA834481F9449624AB7ADAF3735872C767BF85DA29E625A9149C048EE0A3850AC1BE2E735B58781B77DE60D364AD6D5ED66289B524E70A05D1297E1BB35872C767BF85DA227C277FBC8AE2E8B8C0F931425D397B1EFF80C71ABB335746BA297DBC24807EA27F269C8F02392CD20465B3A5AADEC6827F269C8F02392CD5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: 0D63561A33F958A5873F5B03B2203674530C5D7D9AAFA913B85224A0437DF469D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75448CF9D3A7B2C848410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34D2AC226BFD455724DD7B41318B7D6674D3E655558E18E89554A7F26CD3C1189178600371430E272C1D7E09C32AA3244C931578724DD7F428F83FEEEDBFCFAE798580396430872480FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojSsmoOoMLSh05EW/P74L5LA== X-Mailru-Sender: 3B9A0136629DC91206CBC582EFEF4CB41B378F40B656906753C1653FEA2C3FCA8DE0F829CB21A9BAF2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A89437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit] core: fix cdata decrementing X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Igor! Thanks for the review! Added your tag: | Reviewed-by: Igor Munkin On 27.02.21, Igor Munkin wrote: > Sergey, > > Thanks for the patch, nice catch! I guess we missed this bug since the > tests do not cover the case for GCcdata object with a custom finalizer. > But these are frequently used in Tarantool though. The changes LGTM, > except several nits below. > > On 16.02.21, Sergey Kaplun wrote: > > When cdata has custom finalizer (and so LJ_GC_CDATA_FIN flag) it is not > > collected immediately, when lj_cdata_free() is called. Instead, it is > > resurrected and marked finalized, so it is collected at the next GC > > Minor: AFAIR, the payload is released (or finalized in LuaJIT terms) at > the current GC cycle. Hence, the GCobject is marked finalized at this > cycle too, but it is processed once more at the next one to be released. Yes, as you said below: it is made white (is resurrected) in sweep phase and finalized additional payload in GCfinalize phase. The cdata object is collected at the next GC cycle. I haven't change commit message since it means the same. Please correct me if I'm missing something. You may update this commit message in way what you want during cherry picking. > > > cycle. The reason of the bug is that gc_cdatanum is decremented when > > cdata is resurrected too (i.e. twice). > > > > This patch excludes cdata decrementing from resurrection branch and > > adds corresponding tests. > > > > Resolves tarantool/tarantool#5820 > > Follows up tarantool/tarantool#5187 > > --- > > > > Branch: https://github.com/tarantool/luajit/tree/skaplun/gh-5820-improperly-cdata-counting > > Test Branch: https://github.com/tarantool/tarantool/tree/skaplun/gh-5820-improperly-cdata-counting > > Issue: https://github.com/tarantool/tarantool/issues/5820 > > > > ChangeLog entry for bumping LuaJIT: > > =================================================================== > > ## bugfix/LuaJIT > > > > * Fix double cdata decrementing in platform metrics when finalizer is set (gh-5820). > > =================================================================== > > > > src/lj_cdata.c | 3 +- > > test/misclib-getmetrics-capi.test.lua | 15 +++++++++- > > test/misclib-getmetrics-capi/testgetmetrics.c | 28 +++++++++++++++++++ > > test/misclib-getmetrics-lapi.test.lua | 15 +++++++++- > > 4 files changed, 58 insertions(+), 3 deletions(-) > > > > > diff --git a/test/misclib-getmetrics-capi.test.lua b/test/misclib-getmetrics-capi.test.lua > > index 1ad6958..e088c48 100755 > > --- a/test/misclib-getmetrics-capi.test.lua > > +++ b/test/misclib-getmetrics-capi.test.lua > > > > > @@ -62,6 +62,19 @@ test:ok(testgetmetrics.objcount(function(iterations) > > jit.opt.start(unpack(jit_opt_default)) > > end)) > > > > +test:ok(testgetmetrics.objcount_cdata_decrement(function() > > + -- cdata decrement test. > > Minor: The metric name is 'gc_cdatanum', not 'cdata'. > > > + -- See https://github.com/tarantool/tarantool/issues/5820. > > + local ffi = require("ffi") > > + local function nop() end > > + ffi.gc(ffi.cast("void *", 0), nop) > > + -- Does not collect cdata, but removes LJ_GC_CDATA_FIN flag > > + -- and resurrects object. > > Typo: s/resurrects object/resurrects the object/. > > Minor: Strictly saying, the order is a bit different: GC resurrects the > object first at GCSsweep phase, and removes LJ_GC_CDATA_FIN right before > invoking the finalizer at GCSfinalize phase. Fixed, branch is force pushed. See the itterative patch below. =================================================================== diff --git a/test/misclib-getmetrics-capi.test.lua b/test/misclib-getmetrics-capi.test.lua index e088c48..5c0e659 100755 --- a/test/misclib-getmetrics-capi.test.lua +++ b/test/misclib-getmetrics-capi.test.lua @@ -63,15 +63,15 @@ test:ok(testgetmetrics.objcount(function(iterations) end)) test:ok(testgetmetrics.objcount_cdata_decrement(function() - -- cdata decrement test. + -- gc_cdatanum decrement test. -- See https://github.com/tarantool/tarantool/issues/5820. local ffi = require("ffi") local function nop() end ffi.gc(ffi.cast("void *", 0), nop) - -- Does not collect cdata, but removes LJ_GC_CDATA_FIN flag - -- and resurrects object. + -- Does not collect the cdata, but resurrects the object and + -- removes LJ_GC_CDATA_FIN flag. collectgarbage() - -- Collects cdata. + -- Collects the cdata. collectgarbage() end)) diff --git a/test/misclib-getmetrics-lapi.test.lua b/test/misclib-getmetrics-lapi.test.lua index 59bcea6..0d054d3 100755 --- a/test/misclib-getmetrics-lapi.test.lua +++ b/test/misclib-getmetrics-lapi.test.lua @@ -231,15 +231,15 @@ test:test("objcount", function(subtest) subtest:is(new_metrics.gc_cdatanum, old_metrics.gc_cdatanum, "cdatanum don't change") - -- cdata decrement test. + -- gc_cdatanum decrement test. -- See https://github.com/tarantool/tarantool/issues/5820. local function nop() end local cdatanum_old = misc.getmetrics().gc_cdatanum ffi.gc(ffi.cast("void *", 0), nop) - -- Does not collect cdata, but removes LJ_GC_CDATA_FIN flag - -- and resurrects object. + -- Does not collect the cdata, but resurrects the object and + -- removes LJ_GC_CDATA_FIN flag. collectgarbage() - -- Collects cdata. + -- Collects the cdata. collectgarbage() subtest:is(misc.getmetrics().gc_cdatanum, cdatanum_old, "cdatanum is decremented correctly") =================================================================== > > @@ -231,6 +231,19 @@ test:test("objcount", function(subtest) > > subtest:is(new_metrics.gc_cdatanum, old_metrics.gc_cdatanum, > > "cdatanum don't change") > > > > + -- cdata decrement test. > > + -- See https://github.com/tarantool/tarantool/issues/5820. > > + local function nop() end > > + local cdatanum_old = misc.getmetrics().gc_cdatanum > > + ffi.gc(ffi.cast("void *", 0), nop) > > + -- Does not collect cdata, but removes LJ_GC_CDATA_FIN flag > > + -- and resurrects object. > > + collectgarbage() > > Side note: It's worth to check whether Tarantool internals do not spoil > the counters the same way they do after making LuaJIT testing machinery > self-sufficient. There is call at the beginning of the > test, but I haven't tested your patch against my series. I've checked it at the new branch version: https://github.com/tarantool/tarantool/tree/skaplun/gh-5820-improperly-cdata-counting Looks OK. > > > + -- Collects cdata. > > + collectgarbage() > > + subtest:is(misc.getmetrics().gc_cdatanum, cdatanum_old, > > + "cdatanum is decremented correctly") > > Minor: The metric name is 'gc_cdatanum', not 'cdatanum'. > > > + > > -- Restore default jit settings. > > jit.opt.start(unpack(jit_opt_default)) > > end) > > -- > > 2.28.0 > > > > -- > Best regards, > IM -- Best regards, Sergey Kaplun