From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 182E96EC40; Tue, 10 Aug 2021 19:46:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 182E96EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628613988; bh=KW+WvluBzTi19wOQcTUnLL8Lam1Y2YdN10o/XF9IbOg=; h=Date:In-Reply-To:To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=UWQnvDI2dK7UFQ6JYiMrS1t4GdyjDeRyS/KPjknmD+o+ltfK6ezbVBWuwpF1tv5Dq rP1t4XRyynNr8j171yk6hpcxbfw+8kj/G1rxBWebHdc7vYznzNQRN5VLsGPBXIt56l xpcQvzgLBusIFgQil0gmgFsJ1JfFLUVqbXoc84pk= Received: from smtp35.i.mail.ru (smtp35.i.mail.ru [94.100.177.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 939836EC40 for ; Tue, 10 Aug 2021 19:46:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 939836EC40 Received: by smtp35.i.mail.ru with esmtpa (envelope-from ) id 1mDUtV-0005lk-LF; Tue, 10 Aug 2021 19:46:26 +0300 Message-Id: <2D0425B2-B689-43F1-8D1F-1A676E86718B@tarantool.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_F697CB1A-A6E9-40FB-AECB-B343283FAE27" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Date: Tue, 10 Aug 2021 19:46:24 +0300 In-Reply-To: To: Sergey Kaplun References: <1733a6045e7ae1ff2cac8c4a49bcdca3388f65aa.1625587322.git.skaplun@tarantool.org> <20210727135941.GR27855@tarantool.org> <9C3661B1-0D21-42B7-94F6-C9C14FCEBCCD@tarantool.org> X-Mailer: Apple Mail (2.3654.100.0.2.22) X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9D5AC6413C25DCF08CC98B8FCC5CD86F3182A05F53808504038CFBE39D2893AA7640F237D31A8E714553310CB81048168DDAC7711D2E48BC9 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE72F22E6DC541F75D9EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637BE899A9B5C1209058638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D88B1EF3B5776D1D9E441DFD37397C4C45117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735201E561CDFBCA1751F2CC0D3CB04F14752D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B67393CE827C55B5F775ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: 0D63561A33F958A558281E4D66A147477285A42C9F84B4979CC82634498EBF56D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA753177526CD55AFC11410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A1AF43396E40A36F15EA8F12AE18CA75BA02CDEB6CF9C1F06543425E16DFC26B607D6D22A6DC12C61D7E09C32AA3244C4272D91290AC589CD26A9F1537CA892935DA7DC5AF9B58C0FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj6qlzQV0oSZOlJOItXyoOGg== X-Mailru-Sender: 3B9A0136629DC912F4AABCEFC589C81E293454CC7BB1E1719A2484940EF99B3A27C75EA4B357C6AAAD07DD1419AC565FA614486B47F28B67C5E079CCF3B0523AED31B7EB2E253A9E112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] Add support for full-range 64 bit lightuserdata. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Ostanevich via Tarantool-patches Reply-To: Sergey Ostanevich Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" --Apple-Mail=_F697CB1A-A6E9-40FB-AECB-B343283FAE27 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii >>> all internal usage of lightuserdata (for hooks, >>> profilers, built-in package, IR and so on) is changed to special = values >>> on Lua Stack. >>=20 >> Can you add at least _some_ test to verify memprof is fine? >=20 > Memprof avoids such extroversions. Do you mean the test for `jit.p`? >=20 I can see memeprof is out of the business, sure I meant any our new functionality - such as stats on gc - that can be hit by this in some way. =20 > The new commit message is the following: >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Add support for full-range 64 bit lightuserdata. >=20 > (cherry picked from commit e9af1abec542e6f9851ff2368e7f196b6382a44c) >=20 > LuaJIT uses special NaN-tagging technique to store internal type on > the Lua stack. In case of LJ_GC64 the first 13 bits are set in special > NaN type (0xfff8...). The next 4 bits are used for an internal LuaJIT > type of object on stack. The next 47 bits are used for storing this > object's content. For userdata, it is its address. For arm64 a pointer > can have more than 47 significant bits [1]. In this case the error = BADLU > error is raised. >=20 > For the support of full 64-bit range lightuserdata pointers two new > fields in GCState are added: >=20 > `lightudseg` - vector of segments of lightuserdata. Each element keeps > 32-bit value. 25 MSB equal to MSB of lightuserdata 64-bit address, the > rest are filled with zeros. The lentgh of the vector is power of 2. >=20 > `lightudnum` - the length - 1 of aforementioned vector (up to 255). >=20 > When lightuserdata is pushed on the stack, if its segment is not = stored > in vector new value is appended to of this vector. The maximum amount = of > segments is 256. BADLU error is raised in case when user tried to add > userdata with the new 257-th segment, so the whole VA-space isn't > covered by this patch. >=20 > Also, in this patch all internal usage of lightuserdata (for hooks, > profilers, built-in package, IR and so on) is changed to special = values > on Lua Stack. >=20 > Also, conversion of TValue to FFI C type with store is no longer > compiled for lightuserdata. >=20 > [1]: https://www.kernel.org/doc/html/latest/arm64/memory.html = >=20 > Sergey Kaplun: > * added the description and the test for the problem >=20 > Resolves tarantool/tarantool#2712 > Needed for tarantool/tarantool#6154 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Branch is force-pushed. LGTM. Sergos --Apple-Mail=_F697CB1A-A6E9-40FB-AECB-B343283FAE27 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
all internal usage of = lightuserdata (for hooks,
profilers, built-in package, IR = and so on) is changed to special values
on Lua Stack.

Can you add at least _some_ test = to verify memprof is fine?

Memprof = avoids such extroversions. Do you mean the test for `jit.p`?


I can see = memeprof is out of the business, sure I meant any our = new
functionality - such as stats on gc - that can be hit by = this in some
way.
 
The new commit message is the following:

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Add support = for full-range 64 bit lightuserdata.

(cherry picked from commit = e9af1abec542e6f9851ff2368e7f196b6382a44c)

LuaJIT uses special NaN-tagging technique to store internal = type on
the Lua = stack. In case of LJ_GC64 the first 13 bits are set in special
NaN type = (0xfff8...). The next 4 bits are used for an internal LuaJIT
type of = object on stack. The next 47 bits are used for storing this
object's = content. For userdata, it is its address. For arm64 a pointer
can have more = than 47 significant bits [1]. In this case the error BADLU
error is = raised.

For the = support of full 64-bit range lightuserdata pointers two new
fields in = GCState are added:

`lightudseg` - vector of segments of lightuserdata. Each = element keeps
32-bit value. = 25 MSB equal to MSB of lightuserdata 64-bit address, the
rest are = filled with zeros. The lentgh of the vector is power of 2.

`lightudnum` = - the length - 1 of aforementioned vector (up to 255).

When = lightuserdata is pushed on the stack, if its segment is not = stored
in vector new = value is appended to of this vector. The maximum amount of
segments is = 256. BADLU error is raised in case when user tried to add
userdata with = the new 257-th segment, so the whole VA-space isn't
covered by = this patch.

Also, in this = patch all internal usage of lightuserdata (for hooks,
profilers, = built-in package, IR and so on) is changed to special values
on Lua = Stack.

Also, = conversion of TValue to FFI C type with store is no longer
compiled for = lightuserdata.

[1]: https://www.kernel.org/doc/html/latest/arm64/memory.html
Sergey = Kaplun:
* added the = description and the test for the problem

Resolves tarantool/tarantool#2712
Needed for = tarantool/tarantool#6154
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Branch is = force-pushed.

LGTM.

Sergos

= --Apple-Mail=_F697CB1A-A6E9-40FB-AECB-B343283FAE27--