From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 1E55E6EC58; Sun, 1 Aug 2021 19:25:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1E55E6EC58 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627835146; bh=f685NPJWxJyKvoSb+PPTNfVETAG1lJtNL/iMDxutC+M=; h=Date:In-Reply-To:To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ocKpylm3GsCP4HzwDDsQTV8m4gTt2VUdWdQRrifMQvNDYf/l7R9Hz5wi7xEpG911w 4xhGedIK6nepkt28yaP3xZ/aY/Eoay/PNHhPSNMVuu/qMHGGqNH5EAjH9puAibaQmT bYsdTaTk4FSgnqGu4qsBlTXya49nt0tb2P0BwBwE= Received: from smtp31.i.mail.ru (smtp31.i.mail.ru [94.100.177.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 439D26EC58 for ; Sun, 1 Aug 2021 19:25:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 439D26EC58 Received: by smtp31.i.mail.ru with esmtpa (envelope-from ) id 1mAEHX-0008Td-9c; Sun, 01 Aug 2021 19:25:43 +0300 Message-Id: <9C3661B1-0D21-42B7-94F6-C9C14FCEBCCD@tarantool.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_2C35291A-C360-4B43-A68A-DBB2B5CC16EF" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Date: Sun, 1 Aug 2021 19:25:41 +0300 In-Reply-To: To: Sergey Kaplun References: <1733a6045e7ae1ff2cac8c4a49bcdca3388f65aa.1625587322.git.skaplun@tarantool.org> <20210727135941.GR27855@tarantool.org> X-Mailer: Apple Mail (2.3654.100.0.2.22) X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD941C43E597735A9C351B198F4576AC7B20CA14D9DFB46B94A182A05F53808504054D0E230D7331B093651002C8BC3382ED8171387829E6BED8F355032CEF99A6F X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F2393C4755A27B53EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637BA9119251D79ECE08638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D867E962612690257E8000A91F3E65A606117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18BDFBBEFFF4125B51D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B67393CE827C55B5F775ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B50520FF825FF000BA6F2DD3094A10795402 X-C1DE0DAB: 0D63561A33F958A533A192A7EE81D9428F237FD3AAF5708B8FD2511C3ABEE29CD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA751B940EDA0DFB0535410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A2EC120135420F9074FD2930251ADF49FD54524E60BCD518A62A728532A43594E04650AA13C8103D1D7E09C32AA3244C5242D8E2D2ADEB968CA5ED36FD5C88A0A8CE788DE6831205FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojMCfuYI4Predx6gMW/sMoMg== X-Mailru-Sender: 3B9A0136629DC912F4AABCEFC589C81EA88649973730A27CF96AA72C0CAB1C952241BB377472528AAD07DD1419AC565FA614486B47F28B67C5E079CCF3B0523AED31B7EB2E253A9E112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] Add support for full-range 64 bit lightuserdata. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Ostanevich via Tarantool-patches Reply-To: Sergey Ostanevich Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" --Apple-Mail=_2C35291A-C360-4B43-A68A-DBB2B5CC16EF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi! Thanks for the patch! Some minor message fixes, one great gag from Mike=E2=80=99s code and a test request. Regards, Sergos >=20 > The new commit message is the following: >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Add support for full-range 64 bit lightuserdata. >=20 > (cherry picked from commit e9af1abec542e6f9851ff2368e7f196b6382a44c) >=20 > LuaJIT uses special NaN-tagging technique to store internal type on > the Lua stack. In case LJ_GC64 first 13 bits are set in special NaN ^^^^^^^ ^ In case of the > type (0xfff8...). FPU generates the only one type. The next 4 bits are ^^^^^^^^^^^ Which one and how is it relevant?=09 > used for an internal LuaJIT type of object on stack. The next 47 bits > are used for storing this object's content. For userdata, it is its > address. In case arm64 the pointer can have more than 47 significant ^^^^^ For > bits [1]. In this case the error BADLU error is raised. >=20 > For the support of full 64-bit range lightuserdata pointers two new > fields in GCState are added: >=20 > `lightudseg` - vector of segments of lightuserdata. Each element keeps > 32-bit value. 25 MSB equal to MSB of lightuserdata address, the rest = are ^ 64bit > filled with zeros. The length of the vector is power of 2. >=20 > `lightudnum` - the length - 1 of aforementioned vector (up to 255). >=20 > When lightuserdata is pushed on the stack, if its segment is not = stored > in vector new value is appended on top of this vector. The maximum ^^^^^^^^^ to At first I want you to put it as =E2=80=99not found=E2=80=99 instead of = =E2=80=99not stored=E2=80=99.=20 Then I start thinking over =E2=80=98on top=E2=80=99 for a vector and I = got a strange feeling...=20 Now tell me, every time you put a LUD pointer to stack you have to roll over all present segments in this '>>>' plain loop below? --- a/src/lj_api.c +++ b/src/lj_api.c +#if LJ_64 +static void *lightud_intern(lua_State *L, void *p) +{ + global_State *g =3D G(L); + uint64_t u =3D (uint64_t)p; + uint32_t up =3D lightudup(u); + uint32_t *segmap =3D mref(g->gc.lightudseg, uint32_t); + MSize segnum =3D g->gc.lightudnum; + if (segmap) { + MSize seg; >>> + for (seg =3D 0; seg <=3D segnum; seg++) >>> + if (segmap[seg] =3D=3D up) /* Fast path. */ >>> + return (void *)(((uint64_t)seg << LJ_LIGHTUD_BITS_LO) | = lightudlo(u)); + segnum++; + } + if (!((segnum-1) & segnum) && segnum !=3D 1) { + if (segnum >=3D (1 << LJ_LIGHTUD_BITS_SEG)) lj_err_msg(L, = LJ_ERR_BADLU); + lj_mem_reallocvec(L, segmap, segnum, segnum ? 2*segnum : 2u, = uint32_t); + setmref(g->gc.lightudseg, segmap); + } + g->gc.lightudnum =3D segnum; + segmap[segnum] =3D up; + return (void *)(((uint64_t)segnum << LJ_LIGHTUD_BITS_LO) | = lightudlo(u)); +} +#endif + Can=E2=80=99t help to laugh at Mike=E2=80=99s /* Fast path */, brilliant = isn=E2=80=99t it? Perhaps addition of a new segment is not so often - and is counted to = 256 - so we can easily sort the array each time to make it log(n) rather (n) = for each lua_pushlightuserdata()? > >=20 > See the iterative patch below. >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > diff --git a/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua = b/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua This one tests the LUD push/pop to/fro stack. How about those=20 > all internal usage of lightuserdata (for hooks, > profilers, built-in package, IR and so on) is changed to special = values > on Lua Stack. Can you add at least _some_ test to verify memprof is fine? --Apple-Mail=_2C35291A-C360-4B43-A68A-DBB2B5CC16EF Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi! = Thanks for the patch!

Some = minor message fixes, one great gag from Mike=E2=80=99s code and = a
test request.

Regards,
Sergos


The new = commit message is the following:

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Add support = for full-range 64 bit lightuserdata.

(cherry picked from commit = e9af1abec542e6f9851ff2368e7f196b6382a44c)

LuaJIT uses special NaN-tagging technique to store internal = type on
the Lua = stack. In case LJ_GC64 first 13 bits are set in special NaN
^^^^^^^ = ^
In case of     = the
type = (0xfff8...). FPU generates the only one type. The next 4 bits = are
  = ^^^^^^^^^^^
Which one and how = is it relevant? =

used for an = internal LuaJIT type of object on stack. The next 47 bits
are used for = storing this object's content. For userdata, it is its
address. In = case arm64 the pointer can have more than 47 significant
   ^^^^^
=    For
bits [1]. In this case the error BADLU error is = raised.

For the = support of full 64-bit range lightuserdata pointers two new
fields in = GCState are added:

`lightudseg` - vector of segments of lightuserdata. Each = element keeps
32-bit value. = 25 MSB equal to MSB of lightuserdata address, the rest are
          =                     =                     =   ^
= 64bit
filled with = zeros. The length of the vector is power of 2.

`lightudnum` = - the length - 1 of aforementioned vector (up to 255).

When = lightuserdata is pushed on the stack, if its segment is not = stored
in vector new value is appended on top of this vector. The = maximum
=  ^^^^^^^^^ to

At = first I want you to put it as =E2=80=99not found=E2=80=99 instead of = =E2=80=99not stored=E2=80=99. 
Then I start thinking over = =E2=80=98on top=E2=80=99 for a vector and I got a = strange
feeling... 

Now tell = me, every time you put a LUD pointer to stack you have to = roll
over all present segments in this '>>>' plain = loop below?

--- = a/src/lj_api.c
+++ = b/src/lj_api.c
+#if = LJ_64
+static void = *lightud_intern(lua_State *L, void *p)
+{
+ =  global_State *g =3D G(L);
+  uint64_t u =3D (uint64_t)p;
+  uint32_t up =3D = lightudup(u);
+ =  uint32_t *segmap =3D mref(g->gc.lightudseg, = uint32_t);
+ =  MSize segnum =3D g->gc.lightudnum;
+  if (segmap) = {
+ =    MSize seg;
>>> +    for (seg =3D 0; seg <=3D = segnum; seg++)
>>> +      if (segmap[seg] = =3D=3D up)  /* Fast path. */
>>> + = return = (void *)(((uint64_t)seg << LJ_LIGHTUD_BITS_LO) | = lightudlo(u));
+ =    segnum++;
+  }
+ =  if (!((segnum-1) & segnum) && segnum !=3D 1) = {
+    if = (segnum >=3D (1 << LJ_LIGHTUD_BITS_SEG)) lj_err_msg(L, = LJ_ERR_BADLU);
+ =    lj_mem_reallocvec(L, segmap, segnum, segnum ? 2*segnum = : 2u, uint32_t);
+ =    setmref(g->gc.lightudseg, segmap);
+  }
+ =  g->gc.lightudnum =3D segnum;
+  segmap[segnum] =3D up;
+  return (void *)(((uint64_t)segnum << = LJ_LIGHTUD_BITS_LO) | lightudlo(u));
+}
+#endif
+

Can=E2=80=99t help to laugh at Mike=E2=80=99s /* Fast path = */, brilliant isn=E2=80=99t it?
Perhaps addition of a new segment = is not so often - and is counted to 256 -
so we can easily sort the array each = time to make it log(n) rather (n) for
each lua_pushlightuserdata()?

<snipped>

See the iterative patch below.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
diff --git = a/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua = b/test/tarantool-tests/lj-49-bad-lightuserdata.test.lua

This = one tests the LUD push/pop to/fro stack. How about = those 

all internal usage of = lightuserdata (for hooks,
profilers, built-in package, IR = and so on) is changed to special values
on Lua = Stack.

Can you = add at least _some_ test to verify memprof is fine?

= --Apple-Mail=_2C35291A-C360-4B43-A68A-DBB2B5CC16EF--