From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 751A31BC9DF3; Fri, 13 Mar 2026 14:23:49 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 751A31BC9DF3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1773401029; bh=cFTCEBGx2Aq3ECvqIpLE56MXVyAHTozbjPTtktpPyOI=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=II5Ks3eJAfoCMmC5XeT3vIUDnPQ64fela0DNMc15B6+BWPTQX4Cv0V5OzDbWx6ba/ e6zbBYNUB/YFHyXgJ1D4wyaaE+eqK4y5q8VSbbq5p4vDqpA8VPuWYDYma7LwqZGoTV MHCqIYvuGQpnt5+xJF70hbw8p+eAeVU1+/vlJa9E= Received: from send173.i.mail.ru (send173.i.mail.ru [95.163.59.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 519C71BC9DF0 for ; Fri, 13 Mar 2026 14:23:47 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 519C71BC9DF0 Received: by exim-smtp-64cdfc6c8d-hxgf4 with esmtpa (envelope-from ) id 1w10cM-00000000PGy-0XPO; Fri, 13 Mar 2026 14:23:46 +0300 To: Sergey Bronnikov Date: Fri, 13 Mar 2026 14:24:40 +0300 Message-ID: <20260313112440.27237-1-skaplun@tarantool.org> X-Mailer: git-send-email 2.53.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD91ABAE9865AC7DC88B865A22E496AAB539DFA8EEAAC77DF87182A05F5380850409367D8801ACF5B813DE06ABAFEAF670513880D1C65F9B04ABBB4507A600C28AC9797BF2D09611BC0 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7CE4525FFB91B9BBCEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB553375661EAFBE79684C2D41E31D9C390FE7BBAC0C697CF7F8C8AB6628F474F52ACE0B97389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C02F11D39E7306DD338941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B64AAE2D1698E8717BCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C224927A44A7985839C7B76E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8BE5F4C85FF6F81C613AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735D2D576BCF940C736C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5D3CDE0F526EBD66B5002B1117B3ED6967E1C7E868CABF042E20DC3F561CE4150823CB91A9FED034534781492E4B8EEADD3CF082B023A3D3CC79554A2A72441328621D336A7BC284946AD531847A6065A535571D14F44ED41 X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34C77DD89D51EBB774225B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D6594A43A426D19387C11F395AC3B4AE23F03FE023D9C13FC470484F7A3E82A96C2803CAD96FF434DE49B8341EE9D5BE9A0A8782E911B5E630223B85113FB09E7F3F7B25CA35BA33EC096536EB022892E5344C41F94D744909CECFA6C6B0C050A61A8CAF69B82BA93681CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVdbVVJCphTR/18aNsqpKhDk= X-Mailru-Sender: 583F1D7ACE8F49BDD951BA70C165859E580B8BA17A3069C008AF9BFB128101EEFD99898C6C564A8D0519623D50EA6432F2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A84198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit] Handle OOM error on stack resize in coroutine.resume and lua_checkstack. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mike Pall Thanks to Peter Cawley. (cherry picked from commit a5d2f70c73e406beb617afa829a7af5b8c1d842c) If the OOM or stack overflow error is obtained during `lj_state_growstack()` in the `coroutine.resume()` it may raise an error leading to the incorrect error message and unexpected behaviour for this corner case. Also, `lua_checkstack()` may raise an OOM error leading to panic on an unprotected lua_State, for example. This patch handles these cases by protecting `lj_state_growstack()`. Sergey Kaplun: * added the description and the test for the problem Part of tarantool/tarantool#12134 --- Branch: https://github.com/tarantool/luajit/tree/skaplun/lj-1066-err-coroutine-resume Related issues: * https://github.com/LuaJIT/LuaJIT/issues/1066 * https://github.com/tarantool/tarantool/issues/12134 src/lib_base.c | 5 +- src/lj_api.c | 7 +- src/lj_state.c | 12 +++ src/lj_state.h | 1 + .../lj-1066-err-lua-checkstack.test.c | 84 +++++++++++++++++++ .../lj-1066-err-coroutine-resume.test.lua | 27 ++++++ 6 files changed, 134 insertions(+), 2 deletions(-) create mode 100644 test/tarantool-c-tests/lj-1066-err-lua-checkstack.test.c create mode 100644 test/tarantool-tests/lj-1066-err-coroutine-resume.test.lua diff --git a/src/lib_base.c b/src/lib_base.c index 23728d6c..a5b907de 100644 --- a/src/lib_base.c +++ b/src/lib_base.c @@ -606,7 +606,10 @@ static int ffh_resume(lua_State *L, lua_State *co, int wrap) setstrV(L, L->base-LJ_FR2, lj_err_str(L, em)); return FFH_RES(2); } - lj_state_growstack(co, (MSize)(L->top - L->base)); + if (lj_state_cpgrowstack(co, (MSize)(L->top - L->base)) != LUA_OK) { + cTValue *msg = --co->top; + lj_err_callermsg(L, strVdata(msg)); + } return FFH_RETRY; } diff --git a/src/lj_api.c b/src/lj_api.c index 2e915306..16a1da0e 100644 --- a/src/lj_api.c +++ b/src/lj_api.c @@ -116,7 +116,12 @@ LUA_API int lua_checkstack(lua_State *L, int size) if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) { return 0; /* Stack overflow. */ } else if (size > 0) { - lj_state_checkstack(L, (MSize)size); + int avail = (int)(mref(L->maxstack, TValue) - L->top); + if (size > avail && + lj_state_cpgrowstack(L, (MSize)(size - avail)) != LUA_OK) { + L->top--; + return 0; /* Out of memory. */ + } } return 1; } diff --git a/src/lj_state.c b/src/lj_state.c index 8a1acf87..1b698485 100644 --- a/src/lj_state.c +++ b/src/lj_state.c @@ -170,6 +170,18 @@ void LJ_FASTCALL lj_state_growstack1(lua_State *L) lj_state_growstack(L, 1); } +static TValue *cpgrowstack(lua_State *co, lua_CFunction dummy, void *ud) +{ + UNUSED(dummy); + lj_state_growstack(co, *(MSize *)ud); + return NULL; +} + +int LJ_FASTCALL lj_state_cpgrowstack(lua_State *L, MSize need) +{ + return lj_vm_cpcall(L, NULL, &need, cpgrowstack); +} + /* Allocate basic stack for new state. */ static void stack_init(lua_State *L1, lua_State *L) { diff --git a/src/lj_state.h b/src/lj_state.h index 02a0eafa..cb3581d3 100644 --- a/src/lj_state.h +++ b/src/lj_state.h @@ -18,6 +18,7 @@ LJ_FUNC void lj_state_relimitstack(lua_State *L); LJ_FUNC void lj_state_shrinkstack(lua_State *L, MSize used); LJ_FUNCA void LJ_FASTCALL lj_state_growstack(lua_State *L, MSize need); LJ_FUNC void LJ_FASTCALL lj_state_growstack1(lua_State *L); +LJ_FUNC int LJ_FASTCALL lj_state_cpgrowstack(lua_State *L, MSize need); static LJ_AINLINE void lj_state_checkstack(lua_State *L, MSize need) { diff --git a/test/tarantool-c-tests/lj-1066-err-lua-checkstack.test.c b/test/tarantool-c-tests/lj-1066-err-lua-checkstack.test.c new file mode 100644 index 00000000..d1cc8b06 --- /dev/null +++ b/test/tarantool-c-tests/lj-1066-err-lua-checkstack.test.c @@ -0,0 +1,84 @@ +#include "lua.h" +#include "luaconf.h" + +#include "test.h" +#include "utils.h" + +#include + +/* XXX: Still need normal assert for sanity checks. */ +#undef NDEBUG +#include + +static lua_Alloc old_allocf = NULL; +static void *old_alloc_state = NULL; + +/* Always OOM on reallocation (not on allocation). */ +static void *allocf_null_realloc(void *ud, void *ptr, size_t osize, + size_t nsize) +{ + assert(old_allocf != NULL); + if (ptr != NULL && osize < nsize) + return NULL; + else + return old_allocf(ud, ptr, osize, nsize); +} + +static void enable_allocinject(lua_State *L) +{ + assert(old_allocf == NULL); + old_allocf = lua_getallocf(L, &old_alloc_state); + lua_setallocf(L, allocf_null_realloc, old_alloc_state); +} + +/* Restore the default allocator function. */ +static void disable_allocinject(lua_State *L) +{ + assert(old_allocf != NULL); + lua_setallocf(L, old_allocf, old_alloc_state); + old_allocf = NULL; + old_alloc_state = NULL; +} + +static int checkstack_res = -1; + +static int test_checkstack(lua_State *L) +{ + /* + * There is no stack overflow error, but the OOM error + * due to stack reallocation, before the patch. + */ + checkstack_res = lua_checkstack(L, LUAI_MAXCSTACK / 2); + return 0; +} + +static int oom_on_lua_checkstack(void *test_state) +{ + lua_State *L = test_state; + /* Use fresh-new coroutine for stack manipulations. */ + lua_State *L1 = lua_newthread(L); + + enable_allocinject(L); + /* + * `L1` should have enough space to `cpcall()` without + * stack reallocation. + */ + int status = lua_cpcall(L1, test_checkstack, NULL); + disable_allocinject(L); + + assert_true(status == LUA_OK); + assert_true(checkstack_res == 0); + + return TEST_EXIT_SUCCESS; +} + +int main(void) +{ + lua_State *L = utils_lua_init(); + const struct test_unit tgroup[] = { + test_unit_def(oom_on_lua_checkstack), + }; + const int test_result = test_run_group(tgroup, L); + utils_lua_close(L); + return test_result; +} diff --git a/test/tarantool-tests/lj-1066-err-coroutine-resume.test.lua b/test/tarantool-tests/lj-1066-err-coroutine-resume.test.lua new file mode 100644 index 00000000..3324c351 --- /dev/null +++ b/test/tarantool-tests/lj-1066-err-coroutine-resume.test.lua @@ -0,0 +1,27 @@ +local tap = require('tap') + +-- Test file to demonstrate LuaJIT misbehaviour in case of error +-- on growing stack for the coroutine during `coroutine.resume()`. +-- See also: https://github.com/LuaJIT/LuaJIT/issues/1066. + +local test = tap.test('lj-1066-err-coroutine-resume') + +test:plan(2) + +local function resume_wrap() + local function recurser() + recurser(coroutine.yield()) + end + local co = coroutine.create(recurser) + -- Cause the stack overflow and throws an error with incorrect + -- error message before the patch. Use some arguments to obtain + -- the stack overflow faster. + while coroutine.resume(co, 1, 2, 3, 4, 5, 6, 7, 8, 9) do end +end + +local status, errmsg = pcall(resume_wrap) + +test:is(status, false, 'status is correct') +test:like(errmsg, 'stack overflow', 'error message is correct') + +test:done(true) -- 2.53.0