From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 0959D53A8F0; Fri, 21 Jul 2023 11:23:29 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0959D53A8F0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1689927809; bh=YANmOrTRFol9jnUFXdqaXQpEjUf/EQVTZIn3RzMm9t8=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=kDUADlPXzBxMQclrP9r73qHwYibyEHnaZ6LMVZzQaZD9DKvhSWqseqnhI6Q+ZFu4P oTo/V2d/quZv3IEpjBx6W00u2+6HESOdjnu4r4aKakaOrhYsC6vtSPWY5IcSP+jFOA 7+BdmPVULI3GDb9wp8NR515Bx7NDJVj7f99w1eLk= Received: from smtp33.i.mail.ru (smtp33.i.mail.ru [95.163.41.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 3EB9952B8ED for ; Fri, 21 Jul 2023 11:23:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3EB9952B8ED Received: by smtp33.i.mail.ru with esmtpa (envelope-from ) id 1qMlQ6-007UVO-0w; Fri, 21 Jul 2023 11:23:26 +0300 To: Sergey Kaplun , Sergey Bronnikov Date: Fri, 21 Jul 2023 08:12:03 +0000 Message-Id: X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9227D43F3CB451B41616FC75AC98AB10873760FAF3730B0E5182A05F5380850408654C171DDB43A25A2A6DE94675075F277CE3CA328FE4FB88807104B2B7A3519 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE731D82F3F177D3BCDEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637DC205F3977E1285D8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D87721E26D5274BB22BD7C829A9C25CDBF117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC55B19328CBC4F849A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520902A1BE408319B29BDFBBEFFF4125B51D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE26055571C92BF10F4AAC223A686B1DECD8FC6C240DEA76429C9F4D5AE37F343AA9539A8B242431040A6AB1C7CE11FEE36089696B24BB1D192D242C3BD2E3F4C6C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F7900637F765F39FA4E70FFE43847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A5C7C4F8ABB7811763F1FBBE4910BDE6B53FF1683D2892DDEEF87CCE6106E1FC07E67D4AC08A07B9B0AD0E433DBF1FBFA39C5DF10A05D560A950611B66E3DA6D700B0A020F03D25A0997E3FB2386030E77 X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF77DD89D51EBB7742D3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF0DEDC962809A635B1DFEEA3E87BED37FCE056E2B8EA2BBD2863806029F62EC9D7622FCE73A124D837A512D943FCE53BF847A04BDCDB6F1C900311C825BF0C66BA74DFFEFA5DC0E7F02C26D483E81D6BEECAEF3E2CCC1ED8C383653B6C8D9AE0FD16FCAA6493B703A X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbL9S8ysBdXg8CFNaDPEIdZmfP+wOZm7R X-Mailru-Sender: 2FEBA92C8E508479FE7B9A1DF348D5312354B6DF3EEE8F14457232302BAC2B3C780557B2E842FBA42326FE6F2A341ACE0FB9F97486540B4CD9E8847AB8CFED4D9ABF8A61C016C2CFB0DAF586E7D11B3E67EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH luajit v2 0/5] Add ASan support in LuaJIT X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This series implements the second attempt to enable ASan support back for LuaJIT repository. The only thing that was missing the whole time was the option per se (and the corresponding compile flags, obviously). Anyway, when the option has been finally added, the dedicated CI workflow has been added too in scope of the second patch in the series. All other info can be found in the corresponding patches. Besided, internal LuaJIT memory allocator is not instrumented yet unfortunately, so to find any memory faults it's better to build LuaJIT with system provided memory allocator (i.e. run CMake configuration phase with -DLUAJIT_USE_SYSMALLOC=ON). However, LUAJIT_USE_SYSMALLOC cannot be enabled on x64 without GC64, since realloc usually doesn't return addresses in the right address range. For more info, see root CMakeLists.txt. Surprisingly, some hidden bugs were found while testing the aforementioned configuration. 1. The assertions in memprof initialization, checking the state of the allocator against NULL, can fail if this allocator requires no internal state (e.g. glibc functions for allocating dynamic memory). In fact, when building LuaJIT with LUAJIT_USE_SYSMALLOC option enabled, NULL is given as the second parameter to and these assertions fail as a result. Hence, they are simply removed. 2. Before the patch all tests in tarantool-tests suite (except ) terminate their execution via with the status depending on the test results. However, the second argument of was omitted and Lua universe was not properly finalized as a result. This behaviour becomes a problem, when LuaJIT is build with LUAJIT_USE_SYSMALLOC option and AddressSanitizer support enabled, since the sanitizer starts reporting false positive errors about the memory allocations without the corresponding memory releases. To resolve these errors, the second parameter to each call terminating the test has to be added. To avoid loss of the aforementioned parameter in future, helper has been added to the TAP module. Depending on the single parameter, the new helper either properly finalize the test being run, or simply checks all the test assertions and raises an error if any of them fail. The latter case is added especially to handle specifics and still check that everything works fine. Last but not least: for all ARM64 jobs in exotic builds testing pipeline non-GC64 configurations were disabled, since LUAJIT_ENABLE_GC64 takes no effect for this arch (GC64 is the only option). Issue: https://github.com/tarantool/tarantool/issues/5878 Branch: https://github.com/tarantool/luajit/tree/hackaton/gh-5878-enable-ASAN Tarantool related changes and CI can be found in #8846[1]. v1: https://lists.tarantool.org/tarantool-patches/cover.1689195028.git.imun@tarantool.org/T/#t Changes in v2: * Fixed comments as per review by Sergey B. and Sergey K * Enabled LUAJIT_USE_SYSMALLOC option in sanitizers-testing.yml * Removed two invalid assertions in memprof sources (found via enabling LUAJIT_USE_SYSMALLOC option) * Introduced test:done helper for proper test finalization (found via enabling both LUAJIT_USE_ASAN and LUAJIT_USE_SYSMALLOC) * Little maintenance of exotic builds workflow Igor Munkin (5): ci: clean up workflow for exotic builds memprof: remove invalid assertions test: introduce test:done TAP helper build: introduce LUAJIT_USE_ASAN option ci: introduce testing workflow with sanitizers .github/actions/setup-sanitizers/README.md | 12 +++ .github/actions/setup-sanitizers/action.yml | 24 +++++ .github/workflows/exotic-builds-testing.yml | 4 +- .github/workflows/sanitizers-testing.yml | 89 +++++++++++++++++++ CMakeLists.txt | 33 +++++-- src/lj_memprof.c | 2 - .../arm64-ccall-fp-convention.test.lua | 2 +- .../bc-jit-unpatching.test.lua | 2 +- .../c-library-path-length.test.lua | 2 +- .../fix-bit-shift-generation.test.lua | 2 +- .../tarantool-tests/fix-cdata-concat.test.lua | 2 +- test/tarantool-tests/fix-emit-rma.test.lua | 2 +- .../fix-fold-simplify-conv-sext.test.lua | 2 +- .../fix-gc-setupvalue.test.lua | 2 +- .../fix-slot-check-for-mm-record.test.lua | 2 +- .../gh-3196-incorrect-string-length.test.lua | 2 +- .../gh-4199-gc64-fuse.test.lua | 2 +- .../gh-4427-ffi-sandwich.test.lua | 2 +- ...gh-4476-fix-string-find-recording.test.lua | 2 +- ...gh-4773-tonumber-fail-on-NUL-char.test.lua | 2 +- .../gh-5813-resolving-of-c-symbols.test.lua | 2 +- .../gh-6065-jit-library-smoke-tests.test.lua | 2 +- ...-missed-carg1-in-bctsetr-fallback.test.lua | 2 +- ...-6096-external-unwinding-on-arm64.test.lua | 2 +- ...8-fix-side-exit-patching-on-arm64.test.lua | 2 +- test/tarantool-tests/gh-6163-min-max.test.lua | 2 +- test/tarantool-tests/gh-6189-cur_L.test.lua | 2 +- ...ytecode-allocator-for-comparisons.test.lua | 2 +- .../gh-6371-string-char-no-arg.test.lua | 2 +- ...6782-stitching-in-vmevent-handler.test.lua | 2 +- .../gh-6976-narrowing-of-unary-minus.test.lua | 2 +- ...4-add-proto-trace-sysprof-default.test.lua | 2 +- .../gh-7745-oom-on-trace.test.lua | 2 +- .../lj-350-sload-typecheck.test.lua | 2 +- .../lj-351-print-tostring-number.test.lua | 2 +- .../lj-356-ir-khash-non-string-obj.test.lua | 2 +- .../lj-357-arm64-hrefk.test.lua | 2 +- .../lj-366-strtab-correct-size.test.lua | 2 +- .../lj-375-ir-bufput-signed-char.test.lua | 2 +- .../lj-378-string-format-c-null-char.test.lua | 2 +- .../lj-408-tonumber-cdata-record.test.lua | 2 +- .../lj-416-xor-before-jcc.test.lua | 2 +- .../lj-418-assert-any-type.test.lua | 2 +- ...6-arm64-incorrect-check-closed-uv.test.lua | 2 +- .../lj-430-maxirconst.test.lua | 2 +- ...-arm64-constant-rematerialization.test.lua | 2 +- ...-445-fix-memory-probing-allocator.test.lua | 2 +- .../lj-463-os-date-oom.test.lua | 2 +- .../lj-494-table-chain-infinite-loop.test.lua | 2 +- ...lj-505-fold-no-strref-for-ptrdiff.test.lua | 2 +- .../lj-512-profiler-hook-finalizers.test.lua | 2 +- .../lj-524-fold-conv-respect-src-irt.test.lua | 2 +- .../lj-528-tonumber-0.test.lua | 2 +- .../lj-556-fix-loop-realignment.test.lua | 2 +- .../lj-574-overflow-unpack.test.lua | 2 +- ...j-584-bad-renames-for-sunk-values.test.lua | 2 +- .../lj-586-debug-non-string-error.test.lua | 2 +- .../lj-601-fix-gc-finderrfunc.test.lua | 2 +- .../lj-603-err-snap-restore.test.lua | 15 ++-- .../lj-671-arm64-assert-after-mremap.test.lua | 2 +- ...lj-672-cdata-allocation-recording.test.lua | 2 +- .../lj-688-snap-ir-rename.test.lua | 2 +- .../lj-695-ffi-vararg-call.test.lua | 2 +- .../lj-698-arm-pcall-panic.test.lua | 2 +- .../lj-704-bc-varg-use-def.test.lua | 2 +- .../lj-726-profile-flush-close.test.lua | 2 +- .../lj-727-lightuserdata-itern.test.lua | 2 +- .../lj-735-io-close-on-closed-file.test.lua | 2 +- .../lj-762-pcall-no-arg.test.lua | 2 +- .../lj-819-fix-missing-uclo.test.lua | 2 +- .../lj-864-varg-rec-base-offset.test.lua | 2 +- .../lj-906-fix-err-mem.test.lua | 2 +- .../tarantool-tests/lj-981-folding-0.test.lua | 2 +- ...instable-types-during-loop-unroll.test.lua | 2 +- .../lj-flush-on-trace.test.lua | 2 +- test/tarantool-tests/math-modf.test.lua | 2 + .../misclib-getmetrics-lapi.test.lua | 2 +- .../misclib-memprof-lapi.test.lua | 2 +- .../misclib-sysprof-lapi.test.lua | 2 +- .../or-144-gc64-asmref-l.test.lua | 2 +- .../or-232-unsink-64-kptr.test.lua | 2 +- .../or-94-arm64-ir-ahuvload-bool.test.lua | 2 +- test/tarantool-tests/tap.lua | 14 ++- ...onumber-negative-non-decimal-base.test.lua | 2 +- test/tarantool-tests/tools-utils-avl.test.lua | 2 +- test/tarantool-tests/unit-jit-parse.test.lua | 2 +- 86 files changed, 252 insertions(+), 97 deletions(-) create mode 100644 .github/actions/setup-sanitizers/README.md create mode 100644 .github/actions/setup-sanitizers/action.yml create mode 100644 .github/workflows/sanitizers-testing.yml [1]: https://github.com/tarantool/tarantool/pull/8846 -- 2.30.2