[Tarantool-patches] [PATCH luajit 5/6] test: fix lj-802-panic-at-mcode-protfail GCC+ASan

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

From: Sergey Kaplun <skaplun@tarantool.org>

The aforementioned test uses `LD_PRELOAD` to mock the system call to
`mprotect()`. The GCC implementation of libasan requires that the ASan
library go first in the `LD_PRELOAD` list. This patch tweaks the
behaviour to avoid failure of the test. OTOH, this patch intorduces
warnings related to the libc leaks for a couple of tests, but they are
not treated as test failures. This should be fixed by target setting
LD_PRELOAD only for necessary tests when we start using CTest instead of
`prove`.

Part of tarantool/tarantool#9656

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>
---
 test/tarantool-tests/CMakeLists.txt | 15 +++++++++++++++
 test/tarantool-tests/utils/exec.lua | 14 ++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
index e6d12984..35bcc5ef 100644
--- a/test/tarantool-tests/CMakeLists.txt
+++ b/test/tarantool-tests/CMakeLists.txt
@@ -110,6 +110,21 @@ else()
   list(APPEND LUA_TEST_ENV_MORE LD_LIBRARY_PATH=${LD_LIBRARY_PATH})
 endif()
 
+# Some tests use `LD_PRELOAD` to mock system calls (like
+# <lj-802-panic-at-mcode-protfail.test.lua> overwrites
+# `mprotect()`. When compiling with ASan support under GCC, it is
+# required that the ASan library go first in the `LD_PRELOAD`
+# list. Set it manually, test will append it to the executed
+# process.
+# See also: https://github.com/tarantool/tarantool/issues/9656.
+if(LUAJIT_USE_ASAN AND CMAKE_C_COMPILER_ID STREQUAL "GNU")
+  # FIXME: After using CTest instead of `prove` we should set this
+  # environment variable only for the corresponding tests to avoid
+  # warnings from the libc and etc.
+  LibRealPath(LIB_ASAN libasan.so)
+  list(APPEND LUA_TEST_ENV_MORE LD_PRELOAD=${LIB_ASAN})
+endif()
+
 # LUA_CPATH and LD_LIBRARY_PATH variables and also dependencies
 # list with libraries are set in scope of BuildTestLib macro.
 add_custom_command(TARGET tarantool-tests
diff --git a/test/tarantool-tests/utils/exec.lua b/test/tarantool-tests/utils/exec.lua
index 48a08ba5..49f8ef06 100644
--- a/test/tarantool-tests/utils/exec.lua
+++ b/test/tarantool-tests/utils/exec.lua
@@ -24,6 +24,20 @@ local function makeenv(tabenv)
   if tabenv == nil then return '' end
   local flatenv = {}
   for var, value in pairs(tabenv) do
+    if var == 'LD_PRELOAD' then
+      -- XXX: For ASan used by GCC, the ASan library should go
+      -- first in the `LD_PRELOAD` list, or we get the error:
+      -- "ASan runtime does not come first in initial library
+      -- list;".
+      -- See https://github.com/tarantool/tarantool/issues/9656
+      -- for details.
+      -- So, prepend the given library from `LD_PRELOAD` to the
+      -- start of the list. The separator may be ':' or ' ', see
+      -- man ld.so(8).so. Use ':' as the most robust.
+      local ld_preload = os.getenv('LD_PRELOAD')
+      local ld_preload_prefix = ld_preload and ld_preload .. ':' or ''
+      value = ld_preload_prefix .. value
+    end
     table.insert(flatenv, ('%s=%s'):format(var, value))
   end
   return table.concat(flatenv, ' ')
-- 
2.34.1