From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E10E014EC140; Tue, 19 Aug 2025 10:39:43 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E10E014EC140 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1755589184; bh=JpQa06Vx496WFHzqWGYK0faMu0+gU/dfalQ0DN+h0k0=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=gJx5vUbE3Tzw2WT71pe2t0O2gg2chvL+SysvRaS7aInFhuLKiDpuWFIEOjj0v4XaU Vi4KWKT+z2+QVrDLiyrfEi3BVtmv/XApme/uUSDW9G6aZ1wuSP0Fyh3HsAaCkohqKa yNiM2RaBS8xIXFZ4enlt56hDeWIYz4241uJV31lQ= Received: from send105.i.mail.ru (send105.i.mail.ru [89.221.237.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id EE8F750F2C1 for ; Tue, 19 Aug 2025 10:39:41 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org EE8F750F2C1 Received: by exim-smtp-7bdf5d4766-h76tx with esmtpa (envelope-from ) id 1uoGwW-000000000M8-3P61; Tue, 19 Aug 2025 10:39:41 +0300 To: Sergey Bronnikov Date: Tue, 19 Aug 2025 10:40:20 +0300 Message-ID: <20250819074020.12306-1-skaplun@tarantool.org> X-Mailer: git-send-email 2.50.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD98963C40F031035C8FB254B4E12889349B255E26CAE66CDC3182A05F53808504028742B13315311243DE06ABAFEAF670537BEA78315ECDA57D6E1E52FA343A84A48DB9E46051D5CA5 X-7FA49CB5: 4BFC5D8501E51E47CA8F6BE7C18995EBF17A87A288458776F08A921EB9DAD9F4A6333920E0F9D84FF01C4FAB541DE28FCE208E8FC270FF883B56EE1CD41719AD6EEED3D1F8F94EF624AD3460B7163A6507627EB66F572D0C63D3EDE7CE9F49F504D6AF0AB02B6E8A0F5A30F1B7D8EFBB7AAC33C9E76FB2E27FE99514D0EBFDA4E0021A3BF02774CD14A8176813119CCC6BCF00562456A40908BD49AA745F4EB9DB3D003A4FE8D71063A17EB2A3824495F01C4FAB541DE28FC6CB7D05AEC4C6A97259D723EF62CBF8CD57035CC7A4315E3B037027FE88D2288CC2A23726E83CCB646C2A33599DE4AAFB68450CCBE39A02AEABE2275C500D4DAC7943500F4AA200F19F5A8EFF1ED75337BE6F7B60635183E35907F946520C922962FACDDF493130953E0C1C45D6C233DE330D01E624C8436741AF5F649F9888DAB9A681A4DB395AE4A3704882F68E2A X-C1DE0DAB: B30BEEBCB9DAB3F03765BA39BC7D1A2E478DEAD3877DCF16A7CC37FB2CAB3CE3B71B37401A2A7B98BCC2DF43DD22AB26DB841BC0FD54B82633012DAE6C4C67E065C483732B0C61AE0404AC03FA789E4130AFED0B0E96279A148C4CCFEF37126E2290FBF13227320B X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF77DD89D51EBB7742D3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF5544790E8C6652F6C235676C1698E2400BDC903CBA2B9B7EB61FAF3EA390BDE986260C86DC1AF1466F1FC95A0658FFCB717312F6839375310DB927E8A4B084B73C4F3AE4745216805F4332CA8FE04980913E6812662D5F2A5EAB5682573093F7837F15F2B5E4A70B33F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVUcWHxAuDCnGe+nd5HT22lE= X-DA7885C5: A195B34566D6D6E4F255D290C0D534F935467691C7BC21281890833DFB11CC6E80CA8933C3223E8D5B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393FE9E42A757851DB6727A05D3D81F4C864AA36052FC99EBD1994F424ECCC387DBE49D44BB4BD9522A059A1ED8796F048DB274557F927329BE89D5A3BC2B10C37545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit] Correctly close VM state after early OOM during open. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mike Pall Reported by Assumeru. (cherry picked from commit 5ca25ee83ec1b0343556cd5783ade449676b4037) `lua_newstate()` sets `g->str.mask` to `~(MSize)0` before calling `cpluaopen(). If OOM happens before the `lj_str_init()` call, `lj_gc_freeall()` calls `gc_sweepstr()` in a loop with incorrect top limit `g->str.mask`, which leads to the crash. This patch changes the order of the loop iteration with the correct bottom limit. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#11691 --- Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1248-close-state-early-OOM Related issues: * https://github.com/tarantool/tarantool/issues/11691 * https://github.com/LuaJIT/LuaJIT/issues/1248 * https://github.com/LuaJIT/LuaJIT/issues/1311 Note: The test works for the GC64 build only, since we can't set a custom allocator for the non-GC64 LJ_64 build. Also, to avoid failures related to the lj-1311 the !LJ_NO_UNWIND builds are disabled. src/lj_gc.c | 5 +- .../lj-1248-close-state-early-OOM.test.c | 71 +++++++++++++++++++ 2 files changed, 73 insertions(+), 3 deletions(-) create mode 100644 test/tarantool-c-tests/lj-1248-close-state-early-OOM.test.c diff --git a/src/lj_gc.c b/src/lj_gc.c index f455b55b..3142482f 100644 --- a/src/lj_gc.c +++ b/src/lj_gc.c @@ -598,12 +598,11 @@ void lj_gc_finalize_cdata(lua_State *L) /* Free all remaining GC objects. */ void lj_gc_freeall(global_State *g) { - MSize i, strmask; + MSize i; /* Free everything, except super-fixed objects (the main thread). */ g->gc.currentwhite = LJ_GC_WHITES | LJ_GC_SFIXED; gc_fullsweep(g, &g->gc.root); - strmask = g->strmask; - for (i = 0; i <= strmask; i++) /* Free all string hash chains. */ + for (i = g->strmask; i != ~(MSize)0; i--) /* Free all string hash chains. */ gc_fullsweep(g, &g->strhash[i]); } diff --git a/test/tarantool-c-tests/lj-1248-close-state-early-OOM.test.c b/test/tarantool-c-tests/lj-1248-close-state-early-OOM.test.c new file mode 100644 index 00000000..6c9cb2ca --- /dev/null +++ b/test/tarantool-c-tests/lj-1248-close-state-early-OOM.test.c @@ -0,0 +1,71 @@ +#include "lua.h" +/* XXX: The "lj_arch.h" header is included for the skipcond. */ +#include "lj_arch.h" + +#include "test.h" + +#include + +/* + * LuaJIT requires at least 12000 something bytes for initial + * allocations. The `GG_State` requires a little bit more than + * 6000 bytes (around 3000 bytes is the `jit_State`). + */ + +/* Currently allocated Lua memory and its limit. */ +static size_t current_memory = 0; +const size_t memory_limit = 7000; + +void *limited_alloc_f(void *msp, void *ptr, size_t osize, size_t nsize) +{ + void *ret_ptr = NULL; + /* Overflow is OK here. */ + const size_t requested_diff = nsize - osize; + (void)msp; + + if (current_memory + requested_diff > memory_limit) + return NULL; + + if (nsize == 0) { + free(ptr); + current_memory -= osize; + } else if (ptr == NULL) { + ret_ptr = malloc(nsize); + current_memory += ret_ptr ? nsize : 0; + } else { + ret_ptr = realloc(ptr, nsize); + current_memory += ret_ptr ? requested_diff : 0; + } + return ret_ptr; +} + +static int limited_memory_on_lua_newstate(void *test_state) +{ + (void)test_state; +#if LJ_64 && !LJ_GC64 + (void)limited_alloc_f; + return skip("Can't use custom allocator for 64-bit host without GC64"); +#else + /* + * Check that there is no crash and the limit is small enough. + */ + lua_State *L = lua_newstate(limited_alloc_f, NULL); + assert_true(L == NULL); + return TEST_EXIT_SUCCESS; +#endif +} + +#ifndef LJ_NO_UNWIND +# define LJ_NO_UNWIND 0 +#endif + +int main(void) +{ + /* See https://github.com/LuaJIT/LuaJIT/issues/1311. */ + if (!LJ_NO_UNWIND) + return skip_all("Disabled for external unwinding build due to #1311"); + const struct test_unit tgroup[] = { + test_unit_def(limited_memory_on_lua_newstate), + }; + return test_run_group(tgroup, NULL); +} -- 2.50.1