[Tarantool-patches] [PATCH luajit v2 0/5] Add ASan support in LuaJIT
Igor Munkin
imun at tarantool.org
Fri Jul 21 11:12:03 MSK 2023
This series implements the second attempt to enable ASan support back
for LuaJIT repository.
The only thing that was missing the whole time was the option per se
(and the corresponding compile flags, obviously). Anyway, when the
option has been finally added, the dedicated CI workflow has been added
too in scope of the second patch in the series. All other info can be
found in the corresponding patches.
Besided, internal LuaJIT memory allocator is not instrumented yet
unfortunately, so to find any memory faults it's better to build LuaJIT
with system provided memory allocator (i.e. run CMake configuration
phase with -DLUAJIT_USE_SYSMALLOC=ON). However, LUAJIT_USE_SYSMALLOC
cannot be enabled on x64 without GC64, since realloc usually doesn't
return addresses in the right address range. For more info, see root
CMakeLists.txt.
Surprisingly, some hidden bugs were found while testing the
aforementioned configuration.
1. The assertions in memprof initialization, checking the state of the
allocator against NULL, can fail if this allocator requires no internal
state (e.g. glibc functions for allocating dynamic memory). In fact,
when building LuaJIT with LUAJIT_USE_SYSMALLOC option enabled, NULL is
given as the second parameter to <lua_newstate> and these assertions
fail as a result. Hence, they are simply removed.
2. Before the patch all tests in tarantool-tests suite (except
<lj-603-err-snap-restore.test.lua>) terminate their execution via
<os.exit> with the status depending on the test results. However, the
second argument of <os.exit> was omitted and Lua universe was not
properly finalized as a result. This behaviour becomes a problem, when
LuaJIT is build with LUAJIT_USE_SYSMALLOC option and AddressSanitizer
support enabled, since the sanitizer starts reporting false positive
errors about the memory allocations without the corresponding memory
releases. To resolve these errors, the second parameter to each
<os.exit> call terminating the test has to be added. To avoid loss of
the aforementioned parameter in future, <test:done> helper has been
added to the TAP module. Depending on the single parameter, the new
helper either properly finalize the test being run, or simply checks all
the test assertions and raises an error if any of them fail. The latter
case is added especially to handle <lj-603-err-snap-restore.test.lua>
specifics and still check that everything works fine.
Last but not least: for all ARM64 jobs in exotic builds testing pipeline
non-GC64 configurations were disabled, since LUAJIT_ENABLE_GC64 takes no
effect for this arch (GC64 is the only option).
Issue: https://github.com/tarantool/tarantool/issues/5878
Branch: https://github.com/tarantool/luajit/tree/hackaton/gh-5878-enable-ASAN
Tarantool related changes and CI can be found in #8846[1].
v1: https://lists.tarantool.org/tarantool-patches/cover.1689195028.git.imun@tarantool.org/T/#t
Changes in v2:
* Fixed comments as per review by Sergey B. and Sergey K
* Enabled LUAJIT_USE_SYSMALLOC option in sanitizers-testing.yml
* Removed two invalid assertions in memprof sources (found via
enabling LUAJIT_USE_SYSMALLOC option)
* Introduced test:done helper for proper test finalization (found via
enabling both LUAJIT_USE_ASAN and LUAJIT_USE_SYSMALLOC)
* Little maintenance of exotic builds workflow
Igor Munkin (5):
ci: clean up workflow for exotic builds
memprof: remove invalid assertions
test: introduce test:done TAP helper
build: introduce LUAJIT_USE_ASAN option
ci: introduce testing workflow with sanitizers
.github/actions/setup-sanitizers/README.md | 12 +++
.github/actions/setup-sanitizers/action.yml | 24 +++++
.github/workflows/exotic-builds-testing.yml | 4 +-
.github/workflows/sanitizers-testing.yml | 89 +++++++++++++++++++
CMakeLists.txt | 33 +++++--
src/lj_memprof.c | 2 -
.../arm64-ccall-fp-convention.test.lua | 2 +-
.../bc-jit-unpatching.test.lua | 2 +-
.../c-library-path-length.test.lua | 2 +-
.../fix-bit-shift-generation.test.lua | 2 +-
.../tarantool-tests/fix-cdata-concat.test.lua | 2 +-
test/tarantool-tests/fix-emit-rma.test.lua | 2 +-
.../fix-fold-simplify-conv-sext.test.lua | 2 +-
.../fix-gc-setupvalue.test.lua | 2 +-
.../fix-slot-check-for-mm-record.test.lua | 2 +-
.../gh-3196-incorrect-string-length.test.lua | 2 +-
.../gh-4199-gc64-fuse.test.lua | 2 +-
.../gh-4427-ffi-sandwich.test.lua | 2 +-
...gh-4476-fix-string-find-recording.test.lua | 2 +-
...gh-4773-tonumber-fail-on-NUL-char.test.lua | 2 +-
.../gh-5813-resolving-of-c-symbols.test.lua | 2 +-
.../gh-6065-jit-library-smoke-tests.test.lua | 2 +-
...-missed-carg1-in-bctsetr-fallback.test.lua | 2 +-
...-6096-external-unwinding-on-arm64.test.lua | 2 +-
...8-fix-side-exit-patching-on-arm64.test.lua | 2 +-
test/tarantool-tests/gh-6163-min-max.test.lua | 2 +-
test/tarantool-tests/gh-6189-cur_L.test.lua | 2 +-
...ytecode-allocator-for-comparisons.test.lua | 2 +-
.../gh-6371-string-char-no-arg.test.lua | 2 +-
...6782-stitching-in-vmevent-handler.test.lua | 2 +-
.../gh-6976-narrowing-of-unary-minus.test.lua | 2 +-
...4-add-proto-trace-sysprof-default.test.lua | 2 +-
.../gh-7745-oom-on-trace.test.lua | 2 +-
.../lj-350-sload-typecheck.test.lua | 2 +-
.../lj-351-print-tostring-number.test.lua | 2 +-
.../lj-356-ir-khash-non-string-obj.test.lua | 2 +-
.../lj-357-arm64-hrefk.test.lua | 2 +-
.../lj-366-strtab-correct-size.test.lua | 2 +-
.../lj-375-ir-bufput-signed-char.test.lua | 2 +-
.../lj-378-string-format-c-null-char.test.lua | 2 +-
.../lj-408-tonumber-cdata-record.test.lua | 2 +-
.../lj-416-xor-before-jcc.test.lua | 2 +-
.../lj-418-assert-any-type.test.lua | 2 +-
...6-arm64-incorrect-check-closed-uv.test.lua | 2 +-
.../lj-430-maxirconst.test.lua | 2 +-
...-arm64-constant-rematerialization.test.lua | 2 +-
...-445-fix-memory-probing-allocator.test.lua | 2 +-
.../lj-463-os-date-oom.test.lua | 2 +-
.../lj-494-table-chain-infinite-loop.test.lua | 2 +-
...lj-505-fold-no-strref-for-ptrdiff.test.lua | 2 +-
.../lj-512-profiler-hook-finalizers.test.lua | 2 +-
.../lj-524-fold-conv-respect-src-irt.test.lua | 2 +-
.../lj-528-tonumber-0.test.lua | 2 +-
.../lj-556-fix-loop-realignment.test.lua | 2 +-
.../lj-574-overflow-unpack.test.lua | 2 +-
...j-584-bad-renames-for-sunk-values.test.lua | 2 +-
.../lj-586-debug-non-string-error.test.lua | 2 +-
.../lj-601-fix-gc-finderrfunc.test.lua | 2 +-
.../lj-603-err-snap-restore.test.lua | 15 ++--
.../lj-671-arm64-assert-after-mremap.test.lua | 2 +-
...lj-672-cdata-allocation-recording.test.lua | 2 +-
.../lj-688-snap-ir-rename.test.lua | 2 +-
.../lj-695-ffi-vararg-call.test.lua | 2 +-
.../lj-698-arm-pcall-panic.test.lua | 2 +-
.../lj-704-bc-varg-use-def.test.lua | 2 +-
.../lj-726-profile-flush-close.test.lua | 2 +-
.../lj-727-lightuserdata-itern.test.lua | 2 +-
.../lj-735-io-close-on-closed-file.test.lua | 2 +-
.../lj-762-pcall-no-arg.test.lua | 2 +-
.../lj-819-fix-missing-uclo.test.lua | 2 +-
.../lj-864-varg-rec-base-offset.test.lua | 2 +-
.../lj-906-fix-err-mem.test.lua | 2 +-
.../tarantool-tests/lj-981-folding-0.test.lua | 2 +-
...instable-types-during-loop-unroll.test.lua | 2 +-
.../lj-flush-on-trace.test.lua | 2 +-
test/tarantool-tests/math-modf.test.lua | 2 +
.../misclib-getmetrics-lapi.test.lua | 2 +-
.../misclib-memprof-lapi.test.lua | 2 +-
.../misclib-sysprof-lapi.test.lua | 2 +-
.../or-144-gc64-asmref-l.test.lua | 2 +-
.../or-232-unsink-64-kptr.test.lua | 2 +-
.../or-94-arm64-ir-ahuvload-bool.test.lua | 2 +-
test/tarantool-tests/tap.lua | 14 ++-
...onumber-negative-non-decimal-base.test.lua | 2 +-
test/tarantool-tests/tools-utils-avl.test.lua | 2 +-
test/tarantool-tests/unit-jit-parse.test.lua | 2 +-
86 files changed, 252 insertions(+), 97 deletions(-)
create mode 100644 .github/actions/setup-sanitizers/README.md
create mode 100644 .github/actions/setup-sanitizers/action.yml
create mode 100644 .github/workflows/sanitizers-testing.yml
[1]: https://github.com/tarantool/tarantool/pull/8846
--
2.30.2
More information about the Tarantool-patches
mailing list