From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 8C79B6EC40; Tue, 6 Jul 2021 20:41:49 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 8C79B6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1625593309; bh=dKplXZe6TMXMBl+CR1hUenuUlyt1yXD+V1OCEwr65M8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=XpBflojNvoH3AhCOOI4R1M41LNZ1/H16NoXdaPa6I+qwsvCwV+3bMhp37VIPZAf8S Kryz/TNycEAjtwmi3Y24K0DApkOTouXodFOxun5KFzm3Oi+dKTtcrJfO0P4Il54kDx 4bfy6ikPlxifvXjp7DviEbOeLsdFTNS2Ugj27LkQ= Received: from smtp53.i.mail.ru (smtp53.i.mail.ru [94.100.177.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 92D496EC40 for ; Tue, 6 Jul 2021 20:41:16 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 92D496EC40 Received: by smtp53.i.mail.ru with esmtpa (envelope-from ) id 1m0p4N-0008Sr-ET; Tue, 06 Jul 2021 20:41:16 +0300 To: Igor Munkin , Sergey Ostanevich Date: Tue, 6 Jul 2021 20:40:05 +0300 Message-Id: <1733a6045e7ae1ff2cac8c4a49bcdca3388f65aa.1625587322.git.skaplun@tarantool.org> X-Mailer: git-send-email 2.31.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD954DFF1DC42D673FB64496B12B6C6FB871CAFB561BC326F0E182A05F5380850403D7BF4CF3772E8288DBADF0CA65F7EE4BDFCE7FE4157BDF3A7C82193181E4D48 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE78EA80DE462DCD770EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637A9E1EE47BE02EAD28638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8A4AE95BBA99FD5C552FFDB5629B7561B117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD1828451B159A507268D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B6A45692FFBBD75A6A089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C62E2A2CC9B321AEE9D9AAC966D05F211754E223A0E943E0D9C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF309DFB797F6729CB699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34DA1FE609583D493C1D47A262BBBBF22FB118BFE259CA874985CB398254C406EB38A9379E0A89A10A1D7E09C32AA3244C28E040B100ABC1891393F76DB88B5C02F165894D92D62706927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojotfFaHYUgbAxS+pkK/1Bjg== X-Mailru-Sender: 3B9A0136629DC91206CBC582EFEF4CB421B0BA2357800C929FB1F8DF31E0898A6CD0D2698B4FD55BF2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A89437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit 1/2] Add support for full-range 64 bit lightuserdata. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mike Pall (cherry picked from commit e9af1abec542e6f9851ff2368e7f196b6382a44c) LuaJIT uses special NaN-tagging technique to store internal type on the Lua stack. In case LJ_GC64 first 13 bits are set in special NaN type (0xfff8...). FPU generates the only one type. The next 4 bits are used for an internal LuaJIT type of object on stack. The next 47 bits are used for storing this object's content. For userdata, it is its address. In case arm64 the pointer can have more than 47 significant bits [1]. In this case the error BADLU error is raised. For the support of full 64-bit range lightuserdata pointers two new fields in GCState are added: `lightudseg` - vector of segments of lightuserdata storing 32-bit values. MS 25 bits equal to MS bits of lightuserdata address, the rest are filled with zeros. The lentgh of the vector is power of 2. `lightudnum` - the length - 1 of aforementioned vector (up to 255). When lightuserdata is pushed on the stack, if its segment is not stored in vector new value is pushed on top. The maximum amount of segments is 256, BADLU error is raised in case when user tried to add userdata with the new 257-th segment. Also, in this patch all internal usage of lightuserdata (for hooks, profilers, built-in package, ir and so on) is changed to special values on Lua Stack. Also, conversion of TValue to ffi C type with store is no longer compiled for lightuserdata. [1]: https://www.kernel.org/doc/html/latest/arm64/memory.html Sergey Kaplun: * added the description and the test for the problem Needed for tarantool/tarantool#6154 Resolves tarantool/tarantool#2712 --- doc/status.html | 11 ---- src/jit/dump.lua | 4 +- src/lib_debug.c | 12 ++-- src/lib_jit.c | 14 ++--- src/lib_package.c | 8 +-- src/lib_string.c | 2 +- src/lj_api.c | 40 +++++++++++-- src/lj_ccall.c | 2 +- src/lj_cconv.c | 2 +- src/lj_crecord.c | 6 +- src/lj_dispatch.c | 2 +- src/lj_ir.c | 6 +- src/lj_obj.c | 5 +- src/lj_obj.h | 57 ++++++++++++------- src/lj_snap.c | 7 ++- src/lj_state.c | 6 ++ src/lj_strfmt.c | 2 +- test/tarantool-tests/CMakeLists.txt | 1 + .../lj-49-bad-lightuserdata.test.lua | 10 ++++ .../lj-49-bad-lightuserdata/CMakeLists.txt | 1 + .../testlightuserdata.c | 52 +++++++++++++++++ 21 files changed, 183 insertions(+), 67 deletions(-) create mode 100644 test/tarantool-tests/lj-49-bad-lightuserdata.test.lua create mode 100644 test/tarantool-tests/lj-49-bad-lightuserdata/CMakeLists.txt create mode 100644 test/tarantool-tests/lj-49-bad-lightuserdata/testlightuserdata.c diff --git a/doc/status.html b/doc/status.html index cad6ca65..c89255b6 100644 --- a/doc/status.html +++ b/doc/status.html @@ -97,17 +97,6 @@ handled correctly. The error may fall through an on-trace lua_atpanic on x64. This issue will be fixed with the new garbage collector. -
  • -LuaJIT on 64 bit systems provides a limited range of 47 bits for the -legacy lightuserdata data type. -This is only relevant on x64 systems which use the negative part of the -virtual address space in user mode, e.g. Solaris/x64, and on ARM64 systems -configured with a 48 bit or 52 bit VA. -Avoid using lightuserdata to hold pointers that may point outside -of that range, e.g. variables on the stack. In general, avoid this data -type for new code and replace it with (much more performant) FFI bindings. -FFI cdata pointers can address the full 64 bit range. -

    • diff --git a/src/jit/dump.lua b/src/jit/dump.lua index 2bea652b..27b5c2ae 100644 --- a/src/jit/dump.lua +++ b/src/jit/dump.lua @@ -315,7 +315,9 @@ local function formatk(tr, idx, sn) local tn = type(k) local s if tn == "number" then - if band(sn or 0, 0x30000) ~= 0 then + if t < 12 then + s = k == 0 and "NULL" or format("[0x%08x]", k) + elseif band(sn or 0, 0x30000) ~= 0 then s = band(sn, 0x20000) ~= 0 and "contpc" or "ftsz" elseif k == 2^52+2^51 then s = "bias" diff --git a/src/lib_debug.c b/src/lib_debug.c index f112b5bc..8fdfda03 100644 --- a/src/lib_debug.c +++ b/src/lib_debug.c @@ -231,8 +231,8 @@ LJLIB_CF(debug_upvalueid) int32_t n = lj_lib_checkint(L, 2) - 1; if ((uint32_t)n >= fn->l.nupvalues) lj_err_arg(L, 2, LJ_ERR_IDXRNG); - setlightudV(L->top-1, isluafunc(fn) ? (void *)gcref(fn->l.uvptr[n]) : - (void *)&fn->c.upvalue[n]); + lua_pushlightuserdata(L, isluafunc(fn) ? (void *)gcref(fn->l.uvptr[n]) : + (void *)&fn->c.upvalue[n]); return 1; } @@ -283,13 +283,13 @@ LJLIB_CF(debug_setuservalue) /* ------------------------------------------------------------------------ */ -#define KEY_HOOK ((void *)0x3004) +#define KEY_HOOK (U64x(80000000,00000000)|'h') static void hookf(lua_State *L, lua_Debug *ar) { static const char *const hooknames[] = {"call", "return", "line", "count", "tail return"}; - lua_pushlightuserdata(L, KEY_HOOK); + (L->top++)->u64 = KEY_HOOK; lua_rawget(L, LUA_REGISTRYINDEX); if (lua_isfunction(L, -1)) { lua_pushstring(L, hooknames[(int)ar->event]); @@ -334,7 +334,7 @@ LJLIB_CF(debug_sethook) count = luaL_optint(L, arg+3, 0); func = hookf; mask = makemask(smask, count); } - lua_pushlightuserdata(L, KEY_HOOK); + (L->top++)->u64 = KEY_HOOK; lua_pushvalue(L, arg+1); lua_rawset(L, LUA_REGISTRYINDEX); lua_sethook(L, func, mask, count); @@ -349,7 +349,7 @@ LJLIB_CF(debug_gethook) if (hook != NULL && hook != hookf) { /* external hook? */ lua_pushliteral(L, "external hook"); } else { - lua_pushlightuserdata(L, KEY_HOOK); + (L->top++)->u64 = KEY_HOOK; lua_rawget(L, LUA_REGISTRYINDEX); /* get hook */ } lua_pushstring(L, unmakemask(mask, buff)); diff --git a/src/lib_jit.c b/src/lib_jit.c index 22ca0a1a..40aa2b51 100644 --- a/src/lib_jit.c +++ b/src/lib_jit.c @@ -540,15 +540,15 @@ LJLIB_CF(jit_opt_start) /* Not loaded by default, use: local profile = require("jit.profile") */ -static const char KEY_PROFILE_THREAD = 't'; -static const char KEY_PROFILE_FUNC = 'f'; +#define KEY_PROFILE_THREAD (U64x(80000000,00000000)|'t') +#define KEY_PROFILE_FUNC (U64x(80000000,00000000)|'f') static void jit_profile_callback(lua_State *L2, lua_State *L, int samples, int vmstate) { TValue key; cTValue *tv; - setlightudV(&key, (void *)&KEY_PROFILE_FUNC); + key.u64 = KEY_PROFILE_FUNC; tv = lj_tab_get(L, tabV(registry(L)), &key); if (tvisfunc(tv)) { char vmst = (char)vmstate; @@ -575,9 +575,9 @@ LJLIB_CF(jit_profile_start) lua_State *L2 = lua_newthread(L); /* Thread that runs profiler callback. */ TValue key; /* Anchor thread and function in registry. */ - setlightudV(&key, (void *)&KEY_PROFILE_THREAD); + key.u64 = KEY_PROFILE_THREAD; setthreadV(L, lj_tab_set(L, registry, &key), L2); - setlightudV(&key, (void *)&KEY_PROFILE_FUNC); + key.u64 = KEY_PROFILE_FUNC; setfuncV(L, lj_tab_set(L, registry, &key), func); lj_gc_anybarriert(L, registry); luaJIT_profile_start(L, mode ? strdata(mode) : "", @@ -592,9 +592,9 @@ LJLIB_CF(jit_profile_stop) TValue key; luaJIT_profile_stop(L); registry = tabV(registry(L)); - setlightudV(&key, (void *)&KEY_PROFILE_THREAD); + key.u64 = KEY_PROFILE_THREAD; setnilV(lj_tab_set(L, registry, &key)); - setlightudV(&key, (void *)&KEY_PROFILE_FUNC); + key.u64 = KEY_PROFILE_FUNC; setnilV(lj_tab_set(L, registry, &key)); lj_gc_anybarriert(L, registry); return 0; diff --git a/src/lib_package.c b/src/lib_package.c index a0bca572..8573b9f9 100644 --- a/src/lib_package.c +++ b/src/lib_package.c @@ -412,7 +412,7 @@ static int lj_cf_package_loader_preload(lua_State *L) /* ------------------------------------------------------------------------ */ -#define sentinel ((void *)0x4004) +#define KEY_SENTINEL (U64x(80000000,00000000)|'s') static int lj_cf_package_require(lua_State *L) { @@ -422,7 +422,7 @@ static int lj_cf_package_require(lua_State *L) lua_getfield(L, LUA_REGISTRYINDEX, "_LOADED"); lua_getfield(L, 2, name); if (lua_toboolean(L, -1)) { /* is it there? */ - if (lua_touserdata(L, -1) == sentinel) /* check loops */ + if ((L->top-1)->u64 == KEY_SENTINEL) /* check loops */ luaL_error(L, "loop or previous error loading module " LUA_QS, name); return 1; /* package is already loaded */ } @@ -445,14 +445,14 @@ static int lj_cf_package_require(lua_State *L) else lua_pop(L, 1); } - lua_pushlightuserdata(L, sentinel); + (L->top++)->u64 = KEY_SENTINEL; lua_setfield(L, 2, name); /* _LOADED[name] = sentinel */ lua_pushstring(L, name); /* pass name as argument to module */ lua_call(L, 1, 1); /* run loaded module */ if (!lua_isnil(L, -1)) /* non-nil return? */ lua_setfield(L, 2, name); /* _LOADED[name] = returned value */ lua_getfield(L, 2, name); - if (lua_touserdata(L, -1) == sentinel) { /* module did not set a value? */ + if ((L->top-1)->u64 == KEY_SENTINEL) { /* module did not set a value? */ lua_pushboolean(L, 1); /* use true as result */ lua_pushvalue(L, -1); /* extra copy to be returned */ lua_setfield(L, 2, name); /* _LOADED[name] = true */ diff --git a/src/lib_string.c b/src/lib_string.c index 76b0730a..156dae66 100644 --- a/src/lib_string.c +++ b/src/lib_string.c @@ -714,7 +714,7 @@ again: lj_strfmt_putfchar(sb, sf, lj_lib_checkint(L, arg)); break; case STRFMT_PTR: /* No formatting. */ - lj_strfmt_putptr(sb, lj_obj_ptr(L->base+arg-1)); + lj_strfmt_putptr(sb, lj_obj_ptr(G(L), L->base+arg-1)); break; default: lua_assert(0); diff --git a/src/lj_api.c b/src/lj_api.c index c9b5d220..c7a0b327 100644 --- a/src/lj_api.c +++ b/src/lj_api.c @@ -617,7 +617,7 @@ LUA_API void *lua_touserdata(lua_State *L, int idx) if (tvisudata(o)) return uddata(udataV(o)); else if (tvislightud(o)) - return lightudV(o); + return lightudV(G(L), o); else return NULL; } @@ -630,7 +630,7 @@ LUA_API lua_State *lua_tothread(lua_State *L, int idx) LUA_API const void *lua_topointer(lua_State *L, int idx) { - return lj_obj_ptr(index2adr(L, idx)); + return lj_obj_ptr(G(L), index2adr(L, idx)); } /* -- Stack setters (object creation) ------------------------------------- */ @@ -716,9 +716,38 @@ LUA_API void lua_pushboolean(lua_State *L, int b) incr_top(L); } +#if LJ_64 +static void *lightud_intern(lua_State *L, void *p) +{ + global_State *g = G(L); + uint64_t u = (uint64_t)p; + uint32_t up = lightudup(u); + uint32_t *segmap = mref(g->gc.lightudseg, uint32_t); + MSize segnum = g->gc.lightudnum; + if (segmap) { + MSize seg; + for (seg = 0; seg <= segnum; seg++) + if (segmap[seg] == up) /* Fast path. */ + return (void *)(((uint64_t)seg << LJ_LIGHTUD_BITS_LO) | lightudlo(u)); + segnum++; + } + if (!((segnum-1) & segnum) && segnum != 1) { + if (segnum >= (1 << LJ_LIGHTUD_BITS_SEG)) lj_err_msg(L, LJ_ERR_BADLU); + lj_mem_reallocvec(L, segmap, segnum, segnum ? 2*segnum : 2u, uint32_t); + setmref(g->gc.lightudseg, segmap); + } + g->gc.lightudnum = segnum; + segmap[segnum] = up; + return (void *)(((uint64_t)segnum << LJ_LIGHTUD_BITS_LO) | lightudlo(u)); +} +#endif + LUA_API void lua_pushlightuserdata(lua_State *L, void *p) { - setlightudV(L->top, checklightudptr(L, p)); +#if LJ_64 + p = lightud_intern(L, p); +#endif + setrawlightudV(L->top, p); incr_top(L); } @@ -1167,7 +1196,10 @@ static TValue *cpcall(lua_State *L, lua_CFunction func, void *ud) fn->c.f = func; setfuncV(L, top++, fn); if (LJ_FR2) setnilV(top++); - setlightudV(top++, checklightudptr(L, ud)); +#if LJ_64 + ud = lightud_intern(L, ud); +#endif + setrawlightudV(top++, ud); cframe_nres(L->cframe) = 1+0; /* Zero results. */ L->top = top; return top-1; /* Now call the newly allocated C function. */ diff --git a/src/lj_ccall.c b/src/lj_ccall.c index 06d1b3bd..f26f1fea 100644 --- a/src/lj_ccall.c +++ b/src/lj_ccall.c @@ -1150,7 +1150,7 @@ int lj_ccall_func(lua_State *L, GCcdata *cd) lj_vm_ffi_call(&cc); if (cts->cb.slot != ~0u) { /* Blacklist function that called a callback. */ TValue tv; - setlightudV(&tv, (void *)cc.func); + tv.u64 = ((uintptr_t)(void *)cc.func >> 2) | U64x(800000000, 00000000); setboolV(lj_tab_set(L, cts->miscmap, &tv), 1); } ct = (CType *)((intptr_t)ct+(intptr_t)cts->tab); /* May be reallocated. */ diff --git a/src/lj_cconv.c b/src/lj_cconv.c index 13b8230d..ca2a5d30 100644 --- a/src/lj_cconv.c +++ b/src/lj_cconv.c @@ -611,7 +611,7 @@ void lj_cconv_ct_tv(CTState *cts, CType *d, if (ud->udtype == UDTYPE_IO_FILE) tmpptr = *(void **)tmpptr; } else if (tvislightud(o)) { - tmpptr = lightudV(o); + tmpptr = lightudV(cts->g, o); } else if (tvisfunc(o)) { void *p = lj_ccallback_new(cts, d, funcV(o)); if (p) { diff --git a/src/lj_crecord.c b/src/lj_crecord.c index e32ae23e..59186dab 100644 --- a/src/lj_crecord.c +++ b/src/lj_crecord.c @@ -643,8 +643,7 @@ static TRef crec_ct_tv(jit_State *J, CType *d, TRef dp, TRef sp, cTValue *sval) } } else if (tref_islightud(sp)) { #if LJ_64 - sp = emitir(IRT(IR_BAND, IRT_P64), sp, - lj_ir_kint64(J, U64x(00007fff,ffffffff))); + lj_trace_err(J, LJ_TRERR_NYICONV); #endif } else { /* NYI: tref_istab(sp). */ IRType t; @@ -1209,8 +1208,7 @@ static int crec_call(jit_State *J, RecordFFData *rd, GCcdata *cd) TRef tr; TValue tv; /* Check for blacklisted C functions that might call a callback. */ - setlightudV(&tv, - cdata_getptr(cdataptr(cd), (LJ_64 && tp == IRT_P64) ? 8 : 4)); + tv.u64 = ((uintptr_t)cdata_getptr(cdataptr(cd), (LJ_64 && tp == IRT_P64) ? 8 : 4) >> 2) | U64x(800000000, 00000000); if (tvistrue(lj_tab_get(J->L, cts->miscmap, &tv))) lj_trace_err(J, LJ_TRERR_BLACKL); if (ctype_isvoid(ctr->info)) { diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c index b694d8fd..ee735450 100644 --- a/src/lj_dispatch.c +++ b/src/lj_dispatch.c @@ -308,7 +308,7 @@ int luaJIT_setmode(lua_State *L, int idx, int mode) if (idx != 0) { cTValue *tv = idx > 0 ? L->base + (idx-1) : L->top + idx; if (tvislightud(tv)) - g->wrapf = (lua_CFunction)lightudV(tv); + g->wrapf = (lua_CFunction)lightudV(g, tv); else return 0; /* Failed. */ } else { diff --git a/src/lj_ir.c b/src/lj_ir.c index 5baece67..2f7ddb24 100644 --- a/src/lj_ir.c +++ b/src/lj_ir.c @@ -386,8 +386,10 @@ void lj_ir_kvalue(lua_State *L, TValue *tv, const IRIns *ir) case IR_KPRI: setpriV(tv, irt_toitype(ir->t)); break; case IR_KINT: setintV(tv, ir->i); break; case IR_KGC: setgcV(L, tv, ir_kgc(ir), irt_toitype(ir->t)); break; - case IR_KPTR: case IR_KKPTR: setlightudV(tv, ir_kptr(ir)); break; - case IR_KNULL: setlightudV(tv, NULL); break; + case IR_KPTR: case IR_KKPTR: + setnumV(tv, (lua_Number)(uintptr_t)ir_kptr(ir)); + break; + case IR_KNULL: setintV(tv, 0); break; case IR_KNUM: setnumV(tv, ir_knum(ir)->n); break; #if LJ_HASFFI case IR_KINT64: { diff --git a/src/lj_obj.c b/src/lj_obj.c index ee33aeb3..746da6ef 100644 --- a/src/lj_obj.c +++ b/src/lj_obj.c @@ -34,12 +34,13 @@ int LJ_FASTCALL lj_obj_equal(cTValue *o1, cTValue *o2) } /* Return pointer to object or its object data. */ -const void * LJ_FASTCALL lj_obj_ptr(cTValue *o) +const void * LJ_FASTCALL lj_obj_ptr(global_State *g, cTValue *o) { + UNUSED(g); if (tvisudata(o)) return uddata(udataV(o)); else if (tvislightud(o)) - return lightudV(o); + return lightudV(g, o); else if (LJ_HASFFI && tviscdata(o)) return cdataptr(cdataV(o)); else if (tvisgcv(o)) diff --git a/src/lj_obj.h b/src/lj_obj.h index 4a4d77f5..d26e60be 100644 --- a/src/lj_obj.h +++ b/src/lj_obj.h @@ -232,7 +232,7 @@ typedef const TValue cTValue; ** ---MSW---.---LSW--- ** primitive types | itype | | ** lightuserdata | itype | void * | (32 bit platforms) -** lightuserdata |ffff| void * | (64 bit platforms, 47 bit pointers) +** lightuserdata |ffff|seg| ofs | (64 bit platforms) ** GC objects | itype | GCRef | ** int (LJ_DUALNUM)| itype | int | ** number -------double------ @@ -245,7 +245,8 @@ typedef const TValue cTValue; ** ** ------MSW------.------LSW------ ** primitive types |1..1|itype|1..................1| -** GC objects/lightud |1..1|itype|-------GCRef--------| +** GC objects |1..1|itype|-------GCRef--------| +** lightuserdata |1..1|itype|seg|------ofs-------| ** int (LJ_DUALNUM) |1..1|itype|0..0|-----int-------| ** number ------------double------------- ** @@ -285,6 +286,12 @@ typedef const TValue cTValue; #define LJ_GCVMASK (((uint64_t)1 << 47) - 1) #endif +#if LJ_64 +/* To stay within 47 bits, lightuserdata is segmented. */ +#define LJ_LIGHTUD_BITS_SEG 8 +#define LJ_LIGHTUD_BITS_LO (47 - LJ_LIGHTUD_BITS_SEG) +#endif + /* -- String object ------------------------------------------------------- */ /* String object header. String payload follows. */ @@ -597,7 +604,11 @@ typedef struct GCState { uint8_t currentwhite; /* Current white color. */ uint8_t state; /* GC state. */ uint8_t nocdatafin; /* No cdata finalizer called. */ - uint8_t unused2; +#if LJ_64 + uint8_t lightudnum; /* Number of lightuserdata segments - 1. */ +#else + uint8_t unused1; +#endif MSize sweepstr; /* Sweep position in string table. */ GCRef root; /* List of all collectable objects. */ MRef sweep; /* Sweep position in root list. */ @@ -609,6 +620,9 @@ typedef struct GCState { GCSize estimate; /* Estimate of memory actually in use. */ MSize stepmul; /* Incremental GC step granularity. */ MSize pause; /* Pause between successive GC cycles. */ +#if LJ_64 + MRef lightudseg; /* Upper bits of lightuserdata segments. */ +#endif size_t freed; /* Total amount of freed memory. */ size_t allocated; /* Total amount of allocated memory. */ @@ -834,10 +848,23 @@ typedef union GCobj { #endif #define boolV(o) check_exp(tvisbool(o), (LJ_TFALSE - itype(o))) #if LJ_64 -#define lightudV(o) \ - check_exp(tvislightud(o), (void *)((o)->u64 & U64x(00007fff,ffffffff))) +#define lightudseg(u) \ + (((u) >> LJ_LIGHTUD_BITS_LO) & ((1 << LJ_LIGHTUD_BITS_SEG)-1)) +#define lightudlo(u) \ + ((u) & (((uint64_t)1 << LJ_LIGHTUD_BITS_LO) - 1)) +#define lightudup(p) \ + ((uint32_t)(((p) >> LJ_LIGHTUD_BITS_LO) << (LJ_LIGHTUD_BITS_LO-32))) +static LJ_AINLINE void *lightudV(global_State *g, cTValue *o) +{ + uint64_t u = o->u64; + uint64_t seg = lightudseg(u); + uint32_t *segmap = mref(g->gc.lightudseg, uint32_t); + lua_assert(tvislightud(o)); + lua_assert(seg <= g->gc.lightudnum); + return (void *)(((uint64_t)segmap[seg] << 32) | lightudlo(u)); +} #else -#define lightudV(o) check_exp(tvislightud(o), gcrefp((o)->gcr, void)) +#define lightudV(g, o) check_exp(tvislightud(o), gcrefp((o)->gcr, void)) #endif #define gcV(o) check_exp(tvisgcv(o), gcval(o)) #define strV(o) check_exp(tvisstr(o), &gcval(o)->str) @@ -863,7 +890,7 @@ typedef union GCobj { #define setpriV(o, i) (setitype((o), (i))) #endif -static LJ_AINLINE void setlightudV(TValue *o, void *p) +static LJ_AINLINE void setrawlightudV(TValue *o, void *p) { #if LJ_GC64 o->u64 = (uint64_t)p | (((uint64_t)LJ_TLIGHTUD) << 47); @@ -874,24 +901,14 @@ static LJ_AINLINE void setlightudV(TValue *o, void *p) #endif } -#if LJ_64 -#define checklightudptr(L, p) \ - (((uint64_t)(p) >> 47) ? (lj_err_msg(L, LJ_ERR_BADLU), NULL) : (p)) -#else -#define checklightudptr(L, p) (p) -#endif - -#if LJ_FR2 +#if LJ_FR2 || LJ_32 #define contptr(f) ((void *)(f)) #define setcont(o, f) ((o)->u64 = (uint64_t)(uintptr_t)contptr(f)) -#elif LJ_64 +#else #define contptr(f) \ ((void *)(uintptr_t)(uint32_t)((intptr_t)(f) - (intptr_t)lj_vm_asm_begin)) #define setcont(o, f) \ ((o)->u64 = (uint64_t)(void *)(f) - (uint64_t)lj_vm_asm_begin) -#else -#define contptr(f) ((void *)(f)) -#define setcont(o, f) setlightudV((o), contptr(f)) #endif #define tvchecklive(L, o) \ @@ -1014,6 +1031,6 @@ LJ_DATA const char *const lj_obj_itypename[~LJ_TNUMX+1]; /* Compare two objects without calling metamethods. */ LJ_FUNC int LJ_FASTCALL lj_obj_equal(cTValue *o1, cTValue *o2); -LJ_FUNC const void * LJ_FASTCALL lj_obj_ptr(cTValue *o); +LJ_FUNC const void * LJ_FASTCALL lj_obj_ptr(global_State *g, cTValue *o); #endif diff --git a/src/lj_snap.c b/src/lj_snap.c index 4cf27e76..2f7cf80a 100644 --- a/src/lj_snap.c +++ b/src/lj_snap.c @@ -626,7 +626,12 @@ static void snap_restoreval(jit_State *J, GCtrace *T, ExitState *ex, IRType1 t = ir->t; RegSP rs = ir->prev; if (irref_isk(ref)) { /* Restore constant slot. */ - lj_ir_kvalue(J->L, o, ir); + if (ir->o == IR_KPTR) { + o->u64 = (uint64_t)(uintptr_t)ir_kptr(ir); + } else { + lua_assert(!(ir->o == IR_KKPTR || ir->o == IR_KNULL)); + lj_ir_kvalue(J->L, o, ir); + } return; } if (LJ_UNLIKELY(bloomtest(rfilt, ref))) diff --git a/src/lj_state.c b/src/lj_state.c index 448f5134..f82b1b5b 100644 --- a/src/lj_state.c +++ b/src/lj_state.c @@ -186,6 +186,12 @@ static void close_state(lua_State *L) lj_mem_freevec(g, g->strhash, g->strmask+1, GCRef); lj_buf_free(g, &g->tmpbuf); lj_mem_freevec(g, tvref(L->stack), L->stacksize, TValue); +#if LJ_64 + if (mref(g->gc.lightudseg, uint32_t)) { + MSize segnum = g->gc.lightudnum ? (2 << lj_fls(g->gc.lightudnum)) : 2; + lj_mem_freevec(g, mref(g->gc.lightudseg, uint32_t), segnum, uint32_t); + } +#endif lua_assert(g->gc.total == sizeof(GG_State)); #ifndef LUAJIT_USE_SYSMALLOC if (g->allocf == lj_alloc_f) diff --git a/src/lj_strfmt.c b/src/lj_strfmt.c index d7893ce9..237cc575 100644 --- a/src/lj_strfmt.c +++ b/src/lj_strfmt.c @@ -393,7 +393,7 @@ GCstr * LJ_FASTCALL lj_strfmt_obj(lua_State *L, cTValue *o) p = lj_buf_wmem(p, "builtin#", 8); p = lj_strfmt_wint(p, funcV(o)->c.ffid); } else { - p = lj_strfmt_wptr(p, lj_obj_ptr(o)); + p = lj_strfmt_wptr(p, lj_obj_ptr(G(L), o)); } return lj_str_new(L, buf, (size_t)(p - buf)); } diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt index 2fdb4d1f..3e881411 100644 --- a/test/tarantool-tests/CMakeLists.txt +++ b/test/tarantool-tests/CMakeLists.txt @@ -57,6 +57,7 @@ macro(BuildTestCLib lib sources) endmacro() add_subdirectory(gh-4427-ffi-sandwich) +add_subdirectory(lj-49-bad-lightuserdata) add_subdirectory(lj-flush-on-trace) add_subdirectory(misclib-getmetrics-capi) diff --git a/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua b/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua new file mode 100644 index 00000000..111d6a70 --- /dev/null +++ b/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua @@ -0,0 +1,10 @@ +local tap = require('tap') + +local test = tap.test('lj-49-bad-lightuserdata') +test:plan(1) + +local testlightuserdata = require('testlightuserdata') + +test:ok(testlightuserdata.longptr()) + +os.exit(test:check() and 0 or 1) diff --git a/test/tarantool-tests/lj-49-bad-lightuserdata/CMakeLists.txt b/test/tarantool-tests/lj-49-bad-lightuserdata/CMakeLists.txt new file mode 100644 index 00000000..ec6bb62c --- /dev/null +++ b/test/tarantool-tests/lj-49-bad-lightuserdata/CMakeLists.txt @@ -0,0 +1 @@ +BuildTestCLib(testlightuserdata testlightuserdata.c) diff --git a/test/tarantool-tests/lj-49-bad-lightuserdata/testlightuserdata.c b/test/tarantool-tests/lj-49-bad-lightuserdata/testlightuserdata.c new file mode 100644 index 00000000..801c7fe1 --- /dev/null +++ b/test/tarantool-tests/lj-49-bad-lightuserdata/testlightuserdata.c @@ -0,0 +1,52 @@ +#include +#include + +#include +#include + +#undef NDEBUG +#include + +#define START ((void *)-1) + +static int longptr(lua_State *L) +{ + /* + * We know that for arm64 at least 48 bits are available. + * So emulate manually push of lightuseradata within + * this range. + */ + void *longptr = (void *)(1llu << 48); + lua_pushlightuserdata(L, longptr); + assert(longptr == lua_topointer(L, -1)); + /* + * If start mapping address is not NULL, then the kernel + * takes it as a hint about where to place the mapping, so + * we try to get the highest memory address by hint + * equals -1. + */ + const size_t pagesize = getpagesize(); + void *mmaped = mmap(START, pagesize, PROT_NONE, + MAP_PRIVATE | MAP_ANON, -1, 0); + if (mmaped != MAP_FAILED) { + lua_pushlightuserdata(L, mmaped); + assert(mmaped == lua_topointer(L, -1)); + assert(munmap(mmaped, pagesize) == 0); + } + /* Clear our stack. */ + lua_pop(L, 0); + lua_pushboolean(L, 1); + return 1; +} + +static const struct luaL_Reg testlightuserdata[] = { + {"longptr", longptr}, + {NULL, NULL} +}; + +LUA_API int luaopen_testlightuserdata(lua_State *L) +{ + luaL_register(L, "testlightuserdata", testlightuserdata); + return 1; +} + -- 2.31.0