Re: [Tarantool-patches] [PATCH luajit 2/2] Linux/ARM64: Make mremap() non-moving due to VA space woes.

[Igor Munkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Sergey,

Thanks for the patch! LGTM, except a few nits below.

On 06.07.21, Sergey Kaplun wrote:
> From: Mike Pall <mike>
> 
> This reduces overall performance on ARM64, but we have no choice.
> Linux kernel default userspace VA is 48 bit, but we'd need 47 bit.
> mremap() ignores address hints due to a kernel API issue. The mapping
> may move to an undesired address which will cause an assert or crash.
> 
> Reported by Raymond W. Ko.
> 
> (cherry picked from commit 67dbec82f4f05a416a78a560a726553beaa7a223)
> 
> 47-bit VA space is required by LuaJIT for keeping a GC object pointer in
> TValue. When need to reallocate to huge sized block `mrepmap()` on arm64
> may move out VA space from the 47-bit range. `mremap()` accepts the

I'd rather reword the previous sentence the following way:
| In case of huge blobs that are mapped directly, `mremap()` may move
| the chunk out of 47-bit range of VA space on ARM64.

> fifth argument (new address hint) only with MREMAP_FIXED flag. In that
> case it unmaps any other mapping to specified address.
> 
> To avoid this behaviour this patch restricts `mremap()` to relocate
> the mapping to a new virtual address by reset MREMAP_MAYMOVE flag

I'm confused a bit with "reset" word: MREMAP_MAYMOVE is simply changed
to 0 (i.e. CALL_MREMAP_NOMOVE). Moreover, it's better to stay in LuaJIT
terms instead of Linux ones.

> for arm64 architecture.
> 
> Sergey Kaplun:
> * added the description and the test for the problem
> 
> Needed for tarantool/tarantool#6154
> ---
>  src/lj_alloc.c                                |  2 +-
>  .../lj-671-arm64-assert-after-mremap.test.lua | 24 +++++++++++++++++++
>  2 files changed, 25 insertions(+), 1 deletion(-)
>  create mode 100644 test/tarantool-tests/lj-671-arm64-assert-after-mremap.test.lua
> 

<snipped>

> diff --git a/test/tarantool-tests/lj-671-arm64-assert-after-mremap.test.lua b/test/tarantool-tests/lj-671-arm64-assert-after-mremap.test.lua
> new file mode 100644
> index 00000000..0be60a2d
> --- /dev/null
> +++ b/test/tarantool-tests/lj-671-arm64-assert-after-mremap.test.lua
> @@ -0,0 +1,24 @@
> +local tap = require('tap')
> +
> +-- Test file to demonstrate assertion after `mremap()` on arm64.
> +-- See also, https://github.com/LuaJIT/LuaJIT/issues/671.
> +
> +local test = tap.test('lj-671-arm64-assert-after-mremap')
> +test:plan(1)
> +
> +-- `mremap()` is used on Linux for remap directly mapped big

Typo: s/for/to/.

> +-- (>=DEFAULT_MMAP_THRESHOLD) memory chunks.
> +-- The simplest way to test memory move is to allocate the huge
> +-- memory chunk for string buffer directly and reallocate it
> +-- after.
> +-- To allocate buffer exactly to threshold limit for direct chunk
> +-- mapping use `string.rep()` with length equals threshold.
> +-- Then concatenate result string (with length of
> +-- DEFAULT_MMAP_THRESHOLD) with the other one to reallocate
> +-- and remap string buffer.

Just polished two sections above:
| -- To allocate a memory buffer with the size up to the threshold
| -- for direct mapping `string.rep()` is used with the length that
| -- equals to DEFAULT_MMAP_THRESHOLD.
| -- Then concatenate the directly mapped result string with the
| -- other one to trigger buffer reallocation and its remapping.

> +
> +local DEFAULT_MMAP_THRESHOLD = 128 * 1024
> +local s = string.rep('x', DEFAULT_MMAP_THRESHOLD)..'x'
> +test:ok(s)
> +
> +os.exit(test:check() and 0 or 1)
> -- 
> 2.31.0
> 

-- 
Best regards,
IM