From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E4EC8C8D4E2; Mon, 2 Sep 2024 15:54:30 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E4EC8C8D4E2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1725281671; bh=fCOGgQHkODY0I60kVrY3Q7X4YHMVAzSH5ocegd0VA30=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=aOLTg9LsajqW9UFfE+1Q8Qw6lS77ll7G/J1e74QNUcQOzbtCB1gsvkVAEDVb+UrpG XrpCvmhJvtaBMv7KSIpHEWIF0JXTghG+2lzEWgCj1SN34lEce9eIHqBcK+SSBEVlA4 W1Uj/tNfLa4VPmvAPzI1H0mVP+eIm4Cy+ormxFoI= Received: from smtp16.i.mail.ru (smtp16.i.mail.ru [95.163.41.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id B31C647F900 for ; Mon, 2 Sep 2024 15:54:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B31C647F900 Received: by smtp16.i.mail.ru with esmtpa (envelope-from ) id 1sl6Zf-000000075kq-3V01; Mon, 02 Sep 2024 15:54:28 +0300 To: Maxim Kokryashkin , Sergey Bronnikov Date: Mon, 2 Sep 2024 15:54:21 +0300 Message-ID: <20240902125421.16727-1-skaplun@tarantool.org> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9C93E46B305ECE2F087FB3F20830A0FDBAEF90185F913885E182A05F5380850408F313AFBB676FB6791417EB218679B82F8B8A95A7E20886EFB1D898CA228317D5E5D4A160AC272F4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE72E0BB8D059315229EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063769AE3176B9E099158638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8D5FB25FA212465325AEE4B83BAF871DE5354B598E92C5751CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F6B57BC7E64490618DEB871D839B7333395957E7521B51C2DFABB839C843B9C08941B15DA834481F8AA50765F79006372A3B24BF85B2E607389733CBF5DBD5E9B5C8C57E37DE458B9E9CE733340B9D5F3BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E7356C9A9530EBF72002C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A54188B3BE5F8AC1D25002B1117B3ED69630EBD36C64C30B55F09842853758E9E5823CB91A9FED034534781492E4B8EEAD81B3E0F64AD3EF57C79554A2A72441328621D336A7BC284946AD531847A6065A535571D14F44ED41 X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF77DD89D51EBB7742D3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF1CEEC62DFCF43F260AE41421D01CEAA0284FF7B6A5620FCC22A81CF8114D635F49C71475D3C809313EE79E6E40FF6950704802A9AE00CCD4F84577D503A29045AE7F0CB02949EE1DC226CC413062362A913E6812662D5F2A5EAB5682573093F7837F15F2B5E4A70B33F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojI0MmMMS1XREM2FHdrFtswg== X-DA7885C5: ABF3129A5B2BDE67F255D290C0D534F91E1C8416BD2C4D376A4DAB05DF9D44B595AB6E37D129F9A95B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393C6D0B12EA33CAA9BD5A1C9B32BB1D96D4A678655EE628AE60A967638938F6A0DE49D44BB4BD9522A059A1ED8796F048DB274557F927329BE89D5A3BC2B10C37545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit] FFI: Drop finalizer table rehash after GC cycle. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mike Pall Reported by Sergey Kaplun. (cherry picked from commit fb22d0f80f291827a4004e16bc589b54bcc4a3c7) The raising of the OOM error when rehashing the finalizer table (when we can't allocate a new hash part) leads to crashes in either `lj_trace_exit()` or `lj_trace_unwind()` due to unprotected error raising, which either has no DWARF eh_frame or loses the context of the JIT compiler. This patch drops rehashing of the finalizer table to avoid these crashes. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#10199 Resolves tarantool/tarantool#10290 --- Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1247-fin-tab-rehashing-on-trace Related Issues: * https://github.com/tarantool/tarantool/issues/10290 * https://github.com/LuaJIT/LuaJIT/issues/1247 * https://github.com/tarantool/tarantool/issues/10199 src/lj_gc.c | 7 - src/lj_obj.h | 2 +- test/tarantool-tests/CMakeLists.txt | 1 + ...j-1247-fin-tab-rehashing-on-trace.test.lua | 127 ++++++++++++++++++ .../CMakeLists.txt | 1 + .../lj_1247_allocinject.c | 49 +++++++ 6 files changed, 179 insertions(+), 8 deletions(-) create mode 100644 test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace.test.lua create mode 100644 test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/CMakeLists.txt create mode 100644 test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/lj_1247_allocinject.c diff --git a/src/lj_gc.c b/src/lj_gc.c index 4c222f21..a2fc93a0 100644 --- a/src/lj_gc.c +++ b/src/lj_gc.c @@ -548,7 +548,6 @@ static void gc_finalize(lua_State *L) setcdataV(L, &tmp, gco2cd(o)); tv = lj_tab_set(L, tabref(g->gcroot[GCROOT_FFI_FIN]), &tmp); if (!tvisnil(tv)) { - g->gc.nocdatafin = 0; copyTV(L, &tmp, tv); setnilV(tv); /* Clear entry in finalizer table. */ gc_call_finalizer(g, L, &tmp, o); @@ -694,9 +693,6 @@ static size_t gc_onestep(lua_State *L) lj_str_resize(L, g->strmask >> 1); /* Shrink string table. */ if (gcref(g->gc.mmudata)) { /* Need any finalizations? */ g->gc.state = GCSfinalize; -#if LJ_HASFFI - g->gc.nocdatafin = 1; -#endif } else { /* Otherwise skip this phase to help the JIT. */ g->gc.state = GCSpause; /* End of GC cycle. */ g->gc.debt = 0; @@ -713,9 +709,6 @@ static size_t gc_onestep(lua_State *L) g->gc.estimate -= GCFINALIZECOST; return GCFINALIZECOST; } -#if LJ_HASFFI - if (!g->gc.nocdatafin) lj_tab_rehash(L, tabref(g->gcroot[GCROOT_FFI_FIN])); -#endif g->gc.state = GCSpause; /* End of GC cycle. */ g->gc.debt = 0; return 0; diff --git a/src/lj_obj.h b/src/lj_obj.h index 06ea0cd0..ff22e5f8 100644 --- a/src/lj_obj.h +++ b/src/lj_obj.h @@ -611,7 +611,7 @@ typedef struct GCState { GCSize threshold; /* Memory threshold. */ uint8_t currentwhite; /* Current white color. */ uint8_t state; /* GC state. */ - uint8_t nocdatafin; /* No cdata finalizer called. */ + uint8_t unused0; #if LJ_64 uint8_t lightudnum; /* Number of lightuserdata segments - 1. */ #else diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt index e3750bf3..e5d5a470 100644 --- a/test/tarantool-tests/CMakeLists.txt +++ b/test/tarantool-tests/CMakeLists.txt @@ -37,6 +37,7 @@ add_subdirectory(lj-flush-on-trace) add_subdirectory(lj-1004-oom-error-frame) add_subdirectory(lj-1066-fix-cur_L-after-coroutine-resume) add_subdirectory(lj-1166-error-stitch) +add_subdirectory(lj-1247-fin-tab-rehashing-on-trace) # The part of the memory profiler toolchain is located in tools # directory, jit, profiler, and bytecode toolchains are located diff --git a/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace.test.lua b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace.test.lua new file mode 100644 index 00000000..308043a2 --- /dev/null +++ b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace.test.lua @@ -0,0 +1,127 @@ +local tap = require('tap') + +-- The test file to demonstrate the incorrect JIT behaviour during +-- OOM on the finalizer table rehashing in the context of the JIT +-- trace. +-- See also: +-- * https://github.com/LuaJIT/LuaJIT/issues/1247, +-- * https://github.com/tarantool/tarantool/issues/10290. + +local test = tap.test('lj-1247-fin-tab-rehashing-on-trace'):skipcond({ + ['Broken unwiding in tarantool_panic_handler'] = _TARANTOOL and + (jit.os == 'OSX'), + ['Disabled on MacOS due to #8652'] = jit.os == 'OSX', + ['Test requires JIT enabled'] = not jit.status(), +}) + +-- XXX: The original issue has 2 ways to crash: +-- 1) in `lj_trace_unwind()` +-- 2) in `lj_trace_exit()` +-- But, since we have an additional GC pressure due to requiring a +-- `tap` module, the second case needs an impossibly big +-- `gcstepmul` value to reproduce the issue. So, since the root +-- issue is the same and now rehashing of finalizer table is +-- omitted, we test only the first case. +test:plan(2) + +local allocinject = require('lj_1247_allocinject') + +local ffi = require('ffi') +ffi.cdef[[ + struct test {int a;}; +]] + +local N_GC_STEPS = 100 +local N_GC_FINALIZERS = 100 + +local function empty() end + +-- Create a chunk like the following: +--[[ + local tostring = tostring + local r = ... + for _ = 1, 4 do + r[1] = tostring(1) + -- ... + r[N_GCSTEPS] = tostring(N_GC_STEPS) + end +--]] +local function create_chunk(n_steps) + local chunk = 'local tostring = tostring\n' + chunk = chunk .. ('local r = ...\n') + chunk = chunk .. 'for _ = 1, 4 do\n' + for i = 1, n_steps do + chunk = chunk .. (' r[%d] = tostring(%d)\n'):format(i, i) + end + chunk = chunk .. 'end\n' + chunk = chunk .. 'return r\n' + return chunk +end + +local function add_more_garbage(size) + return ffi.new('char[?]', size) +end + +-- Helper to skip the atomic phase. +local function skip_atomic() + local first_gc_called = false + local function mark_fin() first_gc_called = true end + jit.off(mark_fin) + debug.getmetatable(newproxy(true)).__gc = mark_fin + + -- Skip the atomic phase. + jit.off() + while not first_gc_called do collectgarbage('step') end + jit.on() +end + +local function crash_on_trace_unwind_gc_setup() + skip_atomic() + collectgarbage('setstepmul', 1000) + add_more_garbage(1024 * 1024) +end + +local f = assert(loadstring(create_chunk(N_GC_STEPS))) + +-- Create a really long trace. +jit.flush() +jit.opt.start('hotloop=2', 'maxirconst=5000', 'maxrecord=10000', 'maxsnap=1000', + '-fold') + +-- luacheck: no unused +local gc_anchor = {} +local function anchor_finalizer(i) + gc_anchor[i] = ffi.gc(ffi.new('struct test', i), empty) +end + +for i = 1, N_GC_FINALIZERS do + anchor_finalizer(i) +end + +-- Record the trace first. +f({}) + +-- The table for anchoring cdata objects. +local res_tab = {} + +collectgarbage() +collectgarbage() +collectgarbage('setpause', 0) +collectgarbage('setstepmul', 1) + +gc_anchor = nil + +crash_on_trace_unwind_gc_setup() + +-- OOM on every allocation (i.e., on finalizer table rehashing +-- too). +allocinject.enable() + +local r, err = pcall(f, res_tab) + +allocinject.disable() + +test:ok(not r, 'correct status') +test:like(err, 'not enough memory', 'correct error message') + +test:done(true) diff --git a/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/CMakeLists.txt b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/CMakeLists.txt new file mode 100644 index 00000000..c3742e45 --- /dev/null +++ b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/CMakeLists.txt @@ -0,0 +1 @@ +BuildTestCLib(lj_1247_allocinject lj_1247_allocinject.c) diff --git a/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/lj_1247_allocinject.c b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/lj_1247_allocinject.c new file mode 100644 index 00000000..81aea60b --- /dev/null +++ b/test/tarantool-tests/lj-1247-fin-tab-rehashing-on-trace/lj_1247_allocinject.c @@ -0,0 +1,49 @@ +#include "lua.h" +#include "lauxlib.h" + +#undef NDEBUG +#include + +static lua_Alloc old_allocf = NULL; +static void *old_alloc_state = NULL; + +/* Function to be used instead of the default allocator. */ +static void *allocf_with_injection(void *ud, void *ptr, size_t osize, + size_t nsize) +{ + /* Always OOM on allocation (not on realloc). */ + if (ptr == NULL) + return NULL; + else + return old_allocf(ud, ptr, osize, nsize); +} + +static int enable(lua_State *L) +{ + assert(old_allocf == NULL); + old_allocf = lua_getallocf(L, &old_alloc_state); + lua_setallocf(L, allocf_with_injection, old_alloc_state); + return 0; +} + +static int disable(lua_State *L) +{ + assert(old_allocf != NULL); + assert(old_allocf != allocf_with_injection); + lua_setallocf(L, old_allocf, old_alloc_state); + old_allocf = NULL; + old_alloc_state = NULL; + return 0; +} + +static const struct luaL_Reg allocinject[] = { + {"enable", enable}, + {"disable", disable}, + {NULL, NULL} +}; + +LUA_API int luaopen_lj_1247_allocinject(lua_State *L) +{ + luaL_register(L, "lj_1247_allocinject", allocinject); + return 1; +} -- 2.46.0