[Tarantool-patches] [PATCH luajit 0/3][v4] cmake: replace prove with CTest

[Tarantool-patches] [PATCH luajit 0/3][v4] cmake: replace prove with CTest

[Sergey Bronnikov via Tarantool-patches]

From: Sergey Bronnikov <sergeyb@tarantool.org>

Changes v4:
- rebased to master
- applied fixes suggested by Sergey Kaplun
- added a patch with GH Actions job "GCC + ASAN"
- added a patch with updates for CMake macro LibRealPath

PR: https://github.com/tarantool/tarantool/pull/9286
Branch: https://github.com/tarantool/luajit/tree/ligurio/enable_test_target

Sergey Bronnikov (3):
  ci: execute LuaJIT tests with GCC 10 and ASAN
  test: update CMake macro LibRealPath
  cmake: replace prove with CTest

 .github/actions/setup-sanitizers/action.yml | 15 +++-
 .github/workflows/sanitizers-testing.yml    | 11 +++
 .gitignore                                  |  2 +
 CMakeLists.txt                              | 14 ++++
 test/CMakeLists.txt                         | 89 +++++++++++++++------
 test/LuaJIT-tests/CMakeLists.txt            | 61 +++++++++-----
 test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt   | 25 ++++--
 test/lua-Harness-tests/CMakeLists.txt       | 50 +++++++-----
 test/tarantool-c-tests/CMakeLists.txt       | 50 ++++++------
 test/tarantool-tests/CMakeLists.txt         | 69 ++++++++--------
 10 files changed, 252 insertions(+), 134 deletions(-)

-- 
2.34.1

[Tarantool-patches] [PATCH luajit 1/3][v4] ci: execute LuaJIT tests with GCC 10 and ASAN

[Sergey Bronnikov via Tarantool-patches]

From: Sergey Bronnikov <sergeyb@tarantool.org>

The commit fa8e9e9a721e ("test: enable <catch_wrap.lua> LuaJIT test")
bring a workaround to CMake for running tests with LuaJIT built
by GCC with enabled AddressSanitizer. However, we have
no such configuration in CI, it means that sooner or
later it will be broken.

The patch adds a job to GH Actions that builds LuaJIT with GCC 10
with enabled AddressSanitizer and runs LuaJIT tests.
Linux builds built by runners on Ubuntu Focal, where GCC 10
is the latest version [1] available in packages.

Library libstdc++ is needed by the test `catch_wrap.lua`
for a workaround with AddressSanitizer.

1. https://packages.ubuntu.com/focal/gcc-10
---
 .github/actions/setup-sanitizers/action.yml | 15 ++++++++++++---
 .github/workflows/sanitizers-testing.yml    | 11 +++++++++++
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/.github/actions/setup-sanitizers/action.yml b/.github/actions/setup-sanitizers/action.yml
index 4b74f39d..bab40d2b 100644
--- a/.github/actions/setup-sanitizers/action.yml
+++ b/.github/actions/setup-sanitizers/action.yml
@@ -1,5 +1,10 @@
 name: Setup CI environment for testing with sanitizers on Linux
 description: Common part to tweak Linux CI runner environment for sanitizers
+inputs:
+  cc_name:
+    description: C compiler name (for example, gcc-12)
+    required: false
+    default: clang-11
 runs:
   using: composite
   steps:
@@ -15,9 +20,13 @@ runs:
     - name: Install build and test dependencies
       run: |
         apt -y update
-        apt -y install clang-11 cmake ninja-build make perl
+        apt -y install ${CC_NAME} libstdc++-10-dev cmake ninja-build make perl
       shell: bash
-    - name: Set Clang as a default toolchain
+      env:
+        CC_NAME: ${{ inputs.cc_name }}
+    - name: Set C compiler as a default toolchain
       run: |
-        echo CC=clang-11 | tee -a $GITHUB_ENV
+        echo CC=${CC_NAME} | tee -a $GITHUB_ENV
       shell: bash
+      env:
+        CC_NAME: ${{ inputs.cc_name }}
diff --git a/.github/workflows/sanitizers-testing.yml b/.github/workflows/sanitizers-testing.yml
index 4bccfcef..5a24d4ca 100644
--- a/.github/workflows/sanitizers-testing.yml
+++ b/.github/workflows/sanitizers-testing.yml
@@ -33,15 +33,24 @@ jobs:
       matrix:
         # XXX: Let's start with only Linux/x86_64
         BUILDTYPE: [Debug, Release]
+        CC: [gcc-10, clang-11]
         include:
           - BUILDTYPE: Debug
             CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug -DLUA_USE_ASSERT=ON -DLUA_USE_APICHECK=ON
+            CC: gcc-10
+          - BUILDTYPE: Debug
+            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug -DLUA_USE_ASSERT=ON -DLUA_USE_APICHECK=ON
+            CC: clang-11
           - BUILDTYPE: Release
             CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
+        exclude:
+          - BUILDTYPE: Release
+            CC: gcc-10
     runs-on: [self-hosted, regular, Linux, x86_64]
     name: >
       LuaJIT with ASan (Linux/x86_64)
       ${{ matrix.BUILDTYPE }}
+      CC:${{ matrix.CC }}
       GC64:ON SYSMALLOC:ON
     steps:
       - uses: actions/checkout@v3
@@ -50,6 +59,8 @@ jobs:
           submodules: recursive
       - name: setup Linux for sanitizers
         uses: ./.github/actions/setup-sanitizers
+        with:
+          cc_name: ${{ matrix.CC }}
       - name: configure
         # XXX: LuaJIT configuration requires a couple of tweaks:
         # LUAJIT_USE_SYSMALLOC=ON: Unfortunately, internal LuaJIT
-- 
2.34.1

[Tarantool-patches] [PATCH luajit 2/3][v4] test: update CMake macro LibRealPath

[Sergey Bronnikov via Tarantool-patches]

From: Sergey Bronnikov <sergeyb@tarantool.org>

The patch updates CMake macro LibRealPath:

- Replace CMAKE_CXX_COMPILER with CMAKE_C_COMPILER,
  because LuaJIT doesn't require CXX compiler and thus
  it can be unavailable on a build system.
- Add a check for a code returned by a command with compiler.
- Add a check for a value returned by compiler, the value equal to
  passed one means that library was not found in a system.
---
 test/LuaJIT-tests/CMakeLists.txt | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt
index a0fb5440..f744abfe 100644
--- a/test/LuaJIT-tests/CMakeLists.txt
+++ b/test/LuaJIT-tests/CMakeLists.txt
@@ -33,12 +33,27 @@ if(LUAJIT_USE_ASAN)
   # https://github.com/google/sanitizers/issues/934.
   macro(LibRealPath output lib)
     execute_process(
-      COMMAND ${CMAKE_CXX_COMPILER} -print-file-name=${lib}
+      # CMAKE_C_COMPILER is used because it has the same behaviour
+      # as CMAKE_CXX_COMPILER, but CMAKE_CXX_COMPILER is not required
+      # for building LuaJIT and can be missed in GH Actions.
+      COMMAND ${CMAKE_C_COMPILER} -print-file-name=${lib}
       OUTPUT_VARIABLE LIB_LINK
       OUTPUT_STRIP_TRAILING_WHITESPACE
+      RESULT_VARIABLE RES
     )
+    # GCC and Clang returns a passed filename,
+    # when library was not found.
+    if (${LIB_LINK} STREQUAL ${lib})
+      message(FATAL_ERROR "Library '${lib}' is not found")
+    endif()
+    if (NOT RES EQUAL 0)
+      message(FATAL_ERROR "Executing '${CMAKE_C_COMPILER} \
+                           -print-file-name=${lib}' has failed")
+    endif()
+
     # Fortunately, we are not interested in macOS here, so we can
-    # use realpath.
+    # use realpath. Beware, builtin commands always returns
+    # an exit code equal to 0, we cannot check is it failed or not.
     execute_process(
       COMMAND realpath ${LIB_LINK}
       OUTPUT_VARIABLE ${output}
-- 
2.34.1

[Tarantool-patches] [PATCH luajit 3/3][v4] cmake: replace prove with CTest

[Sergey Bronnikov via Tarantool-patches]

From: Sergey Bronnikov <sergeyb@tarantool.org>

The patch replaces the main test runner `prove(1)` with CTest,
see [1] and [2].

Build and test steps looks the same as before:

$ cmake -S . -B build
$ cmake --build build --parallel
$ cmake --build build --target [LuaJIT-test, test]

CMake targets lua-Harness-tests, LuaJIT-tests,
PUC-Rio-Lua-5.1-tests, tarantool-tests and tarantool-c-tests
are still operating.

CMake custom target `test` was replaced by a target with the same
name that is automatically generated by CMake. It is not possible
to attach targets to this automatically generated `test` target.
It means that target `test` is now executing `LuaJIT-test`,
but not `LuaJIT-lint`. To mitigate this a new target
`LuaJIT-check-all` is introduced, it has the same behaviour like
an old `test` target: it runs all functional tests and linters.

One could use `ctest` tool:

$ ctest --print-labels
Test project <snipped>
All Labels:
  LuaJIT-tests
  PUC-Rio-Lua-5.1-tests
  lua-Harness-tests
  tarantool-c-tests
  tarantool-tests
$ ctest                          # Run all tests.
$ ctest -L tarantool-tests       # Run tests by label.
$ ctest -L tarantool-c-tests     # Ditto.
$ ctest -L lua-Harness-tests     # Ditto.
$ ctest -L LuaJIT-tests          # Ditto.
$ ctest -L PUC-Rio-Lua-5.1-tests # Ditto.
$ ctest --rerun-fail             # Run only the tests
                                 # that failed previously.
$ ctest --verbose                # Print details to stdout.
$ ctest --output-on-failure      # Print details to stdout
                                 # on test failure only.

The benefits are:

- less external dependencies, `prove(1)` is gone, `ctest`
  is a part of CMake
- running tests by test title
- extended test running options in comparison to prove(1)
- unified test output (finally!)

Note that the testsuites in `test/LuaJIT-tests` and
`test/PUC-Rio-Lua-5.1` directories have their own test runners
written in Lua, and currently CTest runs these testsuites
as single tests. In the future, we could change these test runners
to specify tests from outside, and after this, we could run tests
separately by CTest in these test suites.

Note that it is not possible to add dependencies to `add_test()`
in CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and
FIXTURES_SETUP [5], but these test properties cannot be used -
used CMake version is lower. This is workarounded by an introduced
macro `add_test_suite_target` that adds a CMake-test for each
testsuite that executes a target that builds test dependencies.

The commit introduce a CMake option LUAJIT_TEST_DEPS that set
dependencies to be used for running and building LuaJIT tests.

1. https://cmake.org/cmake/help/latest/manual/ctest.1.html
2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html
3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774
4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html
5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html
---
 .gitignore                                |  2 +
 CMakeLists.txt                            | 14 ++++
 test/CMakeLists.txt                       | 89 +++++++++++++++++------
 test/LuaJIT-tests/CMakeLists.txt          | 42 +++++++----
 test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt | 25 +++++--
 test/lua-Harness-tests/CMakeLists.txt     | 50 +++++++------
 test/tarantool-c-tests/CMakeLists.txt     | 50 ++++++-------
 test/tarantool-tests/CMakeLists.txt       | 69 +++++++++---------
 8 files changed, 212 insertions(+), 129 deletions(-)

diff --git a/.gitignore b/.gitignore
index dc5ea5fc..dd1f8888 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,7 +15,9 @@
 .ninja_log
 CMakeCache.txt
 CMakeFiles
+CTestTestfile.cmake
 Makefile
+Testing
 build.ninja
 cmake_install.cmake
 cmake_uninstall.cmake
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7f5e2afb..c336fe6b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -345,6 +345,13 @@ set(LUAJIT_TEST_BINARY ${LUAJIT_BINARY} CACHE STRING
   "Lua implementation to be used for tests. Default is 'luajit'."
 )
 
+# If LuaJIT is used in a parent project, provide an option
+# to choose what dependencies to be used for running and building
+# LuaJIT tests.
+set(LUAJIT_TEST_DEPS "luajit-main" CACHE STRING
+  "A list of target dependencies to be used for tests."
+)
+
 # FIXME: If LuaJIT is used in parent project, provide an option
 # to pass Lua code to be run before tests are started.
 # XXX: Attentive reader might point to LUA_INIT environment
@@ -362,6 +369,13 @@ set(LUAJIT_TEST_INIT "${PROJECT_SOURCE_DIR}/test/luajit-test-init.lua" CACHE STR
   "Lua code need to be run before tests are started."
 )
 
+# XXX: Enable testing via CTest. This automatically generates
+# `test` target. Since CTest expects to find a test file in
+# the build directory root, this command should be in the source
+# directory root (hence, in this CMakeLists.txt).
+if (LUAJIT_USE_TEST)
+  enable_testing()
+endif()
 add_subdirectory(test)
 
 # --- Misc rules ---------------------------------------------------------------
diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
index 3ad5d15f..603a56f2 100644
--- a/test/CMakeLists.txt
+++ b/test/CMakeLists.txt
@@ -77,35 +77,76 @@ separate_arguments(LUAJIT_TEST_COMMAND)
 set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
 include(AddTestLib)
 
+# CTEST_FLAGS is used by CMake targets in test suites.
+# XXX: CMake 3.17 introduces CMAKE_CTEST_ARGUMENTS that contains
+# CTest options and is used by `test` target.
+set(CTEST_FLAGS
+  --output-on-failure
+  --schedule-random
+  # Timeout in seconds. Most of LuaJIT tests are fast,
+  # however, some tests can hang or execute infinite loop.
+  # See <tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c>,
+  # commit caa99865c206 ("test: rewrite sysprof test using managed execution").
+  # Moreover, tests for instrumented LuaJIT requires more time.
+  # 30 minutes is a sane timeout.
+  --timeout 1800
+  --parallel ${CMAKE_BUILD_PARALLEL_LEVEL}
+)
+if(CMAKE_VERBOSE_MAKEFILE)
+  list(APPEND CTEST_FLAGS --verbose)
+endif()
+
+function(add_test_suite_target target)
+  set(prefix ARG)
+  set(noValues)
+  set(singleValues LABELS)
+  set(multiValues DEPENDS)
+
+  # FIXME: if we update to CMake >= 3.5, can remove this line.
+  include(CMakeParseArguments)
+  cmake_parse_arguments(${prefix}
+                        "${noValues}"
+                        "${singleValues}"
+                        "${multiValues}"
+                        ${ARGN})
+
+  set(_DEPS_TARGET ${target}-deps)
+
+  add_custom_target(${_DEPS_TARGET} DEPENDS ${ARG_DEPENDS})
+
+  add_test(NAME ${_DEPS_TARGET}
+    COMMAND ${CMAKE_COMMAND}
+            --build ${CMAKE_BINARY_DIR}
+            --target ${_DEPS_TARGET}
+  )
+  set_tests_properties(${_DEPS_TARGET} PROPERTIES
+    LABELS ${ARG_LABELS}
+  )
+
+  message(STATUS "Add test suite ${ARG_LABELS}")
+
+  add_custom_target(${target}
+    COMMAND ${CMAKE_CTEST_COMMAND} -L ${ARG_LABELS} ${CTEST_FLAGS}
+    DEPENDS ${_DEPS_TARGET}
+  )
+endfunction()
+
 add_subdirectory(LuaJIT-tests)
 add_subdirectory(PUC-Rio-Lua-5.1-tests)
 add_subdirectory(lua-Harness-tests)
 add_subdirectory(tarantool-c-tests)
 add_subdirectory(tarantool-tests)
 
-add_custom_target(${PROJECT_NAME}-test DEPENDS
-  LuaJIT-tests
-  PUC-Rio-Lua-5.1-tests
-  lua-Harness-tests
-  tarantool-c-tests
-  tarantool-tests
+add_custom_target(${PROJECT_NAME}-test
+  COMMAND ${CMAKE_CTEST_COMMAND} ${CTEST_FLAGS}
+  DEPENDS tarantool-c-tests-deps
+          tarantool-tests-deps
+          lua-Harness-tests-deps
+          PUC-Rio-Lua-5.1-tests-deps
+          LuaJIT-tests-deps
 )
 
-if(LUAJIT_USE_TEST)
-  if(POLICY CMP0037)
-    if(CMAKE_VERSION VERSION_LESS 3.11)
-      # CMake below 3.11 reserves the name 'test'. Use old policy.
-      # https://cmake.org/cmake/help/v3.11/release/3.11.html#other-changes
-      cmake_policy(SET CMP0037 OLD)
-    else()
-      # Starting from CMake 3.11 the name 'test' is reserved in
-      # special cases and can be used as a target name.
-      cmake_policy(SET CMP0037 NEW)
-    endif()
-  endif(POLICY CMP0037)
-
-  add_custom_target(test DEPENDS
-    ${PROJECT_NAME}-test
-    ${PROJECT_NAME}-lint
-  )
-endif()
+add_custom_target(${PROJECT_NAME}-check-all
+  DEPENDS ${PROJECT_NAME}-test
+          ${PROJECT_NAME}-lint
+)
diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt
index f744abfe..6a9043bc 100644
--- a/test/LuaJIT-tests/CMakeLists.txt
+++ b/test/LuaJIT-tests/CMakeLists.txt
@@ -3,17 +3,13 @@ cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
 
 add_subdirectory(src)
 
-add_custom_target(LuaJIT-tests
-  DEPENDS ${LUAJIT_TEST_BINARY} LuaJIT-tests-libs
-)
-
 make_lua_path(LUA_CPATH
   PATHS
   ${CMAKE_CURRENT_BINARY_DIR}/src/?${CMAKE_SHARED_LIBRARY_SUFFIX}
 )
 
 set(LUAJIT_TESTS_ENV
-  "LUA_CPATH=\"${LUA_CPATH}\""
+  "LUA_CPATH=${LUA_CPATH}"
 )
 
 set(LD_LIBRARY_PATH "${CMAKE_CURRENT_BINARY_DIR}/src/:")
@@ -64,11 +60,12 @@ if(LUAJIT_USE_ASAN)
   # XXX: GCC requires both. Clang requires only libstdc++.
   if(CMAKE_C_COMPILER_ID STREQUAL "GNU")
     LibRealPath(LIB_ASAN libasan.so)
-    # XXX: Don't use " " (separator in LD_PRELOAD) in `list()`.
-    # ";" will expand to " " as we want.
-    list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD="${LIB_ASAN};${LIB_STDCPP}")
+    # XXX: The items of the list in LD_PRELOAD can be separated
+    # by spaces or colons, and there is no support for escaping
+    # either separator, see ld.so(8).
+    list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD=${LIB_ASAN}:${LIB_STDCPP})
   else()
-    list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD="${LIB_STDCPP}")
+    list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD=${LIB_STDCPP})
   endif()
 endif()
 
@@ -79,12 +76,25 @@ if(LUAJIT_NO_UNWIND)
   set(LUAJIT_TEST_TAGS_EXTRA +internal_unwinder)
 endif()
 
-add_custom_command(TARGET LuaJIT-tests
-  COMMENT "Running LuaJIT-tests"
-  COMMAND
-    env
-      ${LUAJIT_TESTS_ENV}
-      ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/test.lua
-      +slow +ffi +bit +jit ${LUAJIT_TEST_TAGS_EXTRA}
+set(TEST_SUITE_NAME "LuaJIT-tests")
+
+# XXX: The call produces both test and target <LuaJIT-tests-deps>
+# as a side effect.
+add_test_suite_target(LuaJIT-tests
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS ${LUAJIT_TEST_DEPS} LuaJIT-tests-libs
+)
+
+# The test suite has its own test runner
+# (test/LuaJIT-tests/test.lua), it is not possible to run these
+# tests one-by-one by CTest.
+add_test(NAME "test/LuaJIT-tests"
+  COMMAND ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/test.lua
+                                 +slow +ffi +bit +jit ${LUAJIT_TEST_TAGS_EXTRA}
   WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
 )
+set_tests_properties("test/LuaJIT-tests" PROPERTIES
+  LABELS ${TEST_SUITE_NAME}
+  ENVIRONMENT "${LUAJIT_TESTS_ENV}"
+  DEPENDS LuaJIT-tests-deps
+)
diff --git a/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt b/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt
index 98277f9a..c1bf1d95 100644
--- a/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt
+++ b/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt
@@ -34,17 +34,26 @@ add_subdirectory(libs)
 # But, unfortunately, <ltests.c> depends on specific PUC-Rio
 # Lua 5.1 internal headers and should be adapted for LuaJIT.
 
-add_custom_target(PUC-Rio-Lua-5.1-tests
-  DEPENDS ${LUAJIT_TEST_BINARY} PUC-Rio-Lua-5.1-tests-prepare
+set(TEST_SUITE_NAME "PUC-Rio-Lua-5.1-tests")
+
+# XXX: The call produces both test and target
+# <PUC-Rio-Lua-5.1-tests-deps> as a side effect.
+add_test_suite_target(PUC-Rio-Lua-5.1-tests
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS ${LUAJIT_TEST_DEPS} PUC-Rio-Lua-5.1-tests-prepare
 )
 
-add_custom_command(TARGET PUC-Rio-Lua-5.1-tests
-  COMMENT "Running PUC-Rio Lua 5.1 tests"
-  COMMAND
-  env
-    LUA_PATH="${LUA_PATH}"
-    ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/all.lua
+# The test suite has its own test runner
+# (test/PUC-Rio-Lua-5.1-tests/all.lua), it is not possible
+# to run these tests one-by-one by CTest.
+add_test(NAME "test/PUC-Rio-Lua-5.1"
+  COMMAND ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/all.lua
   WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
 )
+set_tests_properties("test/PUC-Rio-Lua-5.1" PROPERTIES
+  ENVIRONMENT "LUA_PATH=${LUA_PATH}"
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS PUC-Rio-Lua-5.1-tests-deps
+)
 
 # vim: expandtab tabstop=2 shiftwidth=2
diff --git a/test/lua-Harness-tests/CMakeLists.txt b/test/lua-Harness-tests/CMakeLists.txt
index f748a8fd..6f7b5697 100644
--- a/test/lua-Harness-tests/CMakeLists.txt
+++ b/test/lua-Harness-tests/CMakeLists.txt
@@ -4,12 +4,6 @@
 # See the rationale in the root CMakeLists.txt
 cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
 
-find_program(PROVE prove)
-if(NOT PROVE)
-  message(WARNING "`prove' is not found, so lua-Harness-tests target is not generated")
-  return()
-endif()
-
 # Tests create temporary files (see 303-package.t and 411-luajit.t
 # for example) to be required. Also, they require some files from
 # the original test source directory.
@@ -20,23 +14,37 @@ make_lua_path(LUA_PATH
     ${LUAJIT_SOURCE_DIR}/?.lua
     ${LUAJIT_BINARY_DIR}/?.lua
 )
-set(LUA_TEST_FLAGS --failures --shuffle)
 
-if(CMAKE_VERBOSE_MAKEFILE)
-  list(APPEND LUA_TEST_FLAGS --verbose)
-endif()
+set(TEST_SUITE_NAME "lua-Harness-tests")
 
-add_custom_target(lua-Harness-tests DEPENDS ${LUAJIT_TEST_BINARY})
-add_custom_command(TARGET lua-Harness-tests
-  COMMENT "Running lua-Harness tests"
-  COMMAND
-  env
-    LUA_PATH="${LUA_PATH}"
-    ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR}
-      --exec '${LUAJIT_TEST_COMMAND} -l profile_luajit21'
-      --jobs ${CMAKE_BUILD_PARALLEL_LEVEL}
-      ${LUA_TEST_FLAGS}
-  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+# XXX: The call produces both test and target
+# <lua-Harness-tests-deps> as a side effect.
+add_test_suite_target(lua-Harness-tests
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS ${LUAJIT_TEST_DEPS}
 )
 
+set(LUA_TEST_SUFFIX .t)
+
+file(GLOB tests ${CMAKE_CURRENT_SOURCE_DIR} "*${LUA_TEST_SUFFIX}")
+foreach(test_path ${tests})
+  get_filename_component(test_name ${test_path} NAME)
+  # FIXME: By default, GLOB lists directories.
+  # Directories are omitted in the result if LIST_DIRECTORIES
+  # is set to false. New in version CMake 3.3.
+  if (${test_name} STREQUAL ${TEST_SUITE_NAME})
+    continue()
+  endif()
+  set(test_title "test/${TEST_SUITE_NAME}/${test_name}")
+  add_test(NAME ${test_title}
+    COMMAND ${LUAJIT_TEST_COMMAND} -l profile_luajit21 ${test_path}
+    WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+  )
+  set_tests_properties(${test_title} PROPERTIES
+    ENVIRONMENT "LUA_PATH=${LUA_PATH}"
+    LABELS ${TEST_SUITE_NAME}
+    DEPENDS lua-Harness-tests-deps
+  )
+endforeach()
+
 # vim: expandtab tabstop=2 shiftwidth=2
diff --git a/test/tarantool-c-tests/CMakeLists.txt b/test/tarantool-c-tests/CMakeLists.txt
index 4b1d8832..a4bc3a76 100644
--- a/test/tarantool-c-tests/CMakeLists.txt
+++ b/test/tarantool-c-tests/CMakeLists.txt
@@ -1,15 +1,4 @@
-find_program(PROVE prove)
-if(NOT PROVE)
-  message(WARNING "`prove' is not found, so tarantool-c-tests target is not generated")
-  return()
-endif()
-
 set(C_TEST_SUFFIX .c_test)
-set(C_TEST_FLAGS --failures --shuffle)
-
-if(CMAKE_VERBOSE_MAKEFILE)
-  list(APPEND C_TEST_FLAGS --verbose)
-endif()
 
 # Build libtest.
 
@@ -49,20 +38,29 @@ foreach(test_source ${tests})
   LIST(APPEND TESTS_COMPILED ${exe})
 endforeach()
 
-add_custom_target(tarantool-c-tests
-  DEPENDS libluajit libtest ${TESTS_COMPILED}
-)
+set(TEST_SUITE_NAME "tarantool-c-tests")
 
-add_custom_command(TARGET tarantool-c-tests
-  COMMENT "Running Tarantool C tests"
-  COMMAND
-  ${PROVE}
-    ${CMAKE_CURRENT_BINARY_DIR}
-    --ext ${C_TEST_SUFFIX}
-    --jobs ${CMAKE_BUILD_PARALLEL_LEVEL}
-    # Report any TAP parse errors, if any, since test module is
-    # maintained by us.
-    --parse
-    ${C_TEST_FLAGS}
-  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+# XXX: The call produces both test and target
+# <tarantool-c-tests-deps> as a side effect.
+add_test_suite_target(tarantool-c-tests
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS ${LUAJIT_TEST_DEPS} libluajit libtest ${TESTS_COMPILED}
 )
+
+# XXX: Note that we cannot glob generated files in CMake, see
+# https://stackoverflow.com/questions/44076307/cmake-globbing-generated-files.
+# To workaround this, we glob source files and generated tests
+# with paths to executable binaries.
+file(GLOB_RECURSE tests ${CMAKE_CURRENT_SOURCE_DIR} "*${CTEST_SRC_SUFFIX}")
+foreach(test_path ${tests})
+  get_filename_component(test_name ${test_path} NAME_WE)
+  set(test_title "test/${TEST_SUITE_NAME}/${test_name}${C_TEST_SUFFIX}")
+  add_test(NAME ${test_title}
+    COMMAND ${CMAKE_CURRENT_BINARY_DIR}/${test_name}${C_TEST_SUFFIX}
+    WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+  )
+  set_tests_properties(${test_title} PROPERTIES
+    LABELS ${TEST_SUITE_NAME}
+    DEPENDS tarantool-c-tests-deps
+  )
+endforeach()
diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
index e6d12984..3d9432f9 100644
--- a/test/tarantool-tests/CMakeLists.txt
+++ b/test/tarantool-tests/CMakeLists.txt
@@ -4,19 +4,13 @@
 # See the rationale in the root CMakeLists.txt.
 cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
 
-find_program(PROVE prove)
-if(NOT PROVE)
-  message(WARNING "`prove' is not found, so tarantool-tests target is not generated")
-  return()
-endif()
-
-add_custom_target(tarantool-tests
+add_custom_target(tarantool-tests-libs
   DEPENDS ${LUAJIT_TEST_BINARY}
 )
 
 macro(BuildTestCLib lib sources)
   AddTestLib(${lib} ${sources})
-  add_dependencies(tarantool-tests ${lib})
+  add_dependencies(tarantool-tests-libs ${lib})
   # Add the directory where the lib is built to the list with
   # entries for LUA_CPATH environment variable, so LuaJIT can find
   # and load it. See the comment about extending the list in the
@@ -61,20 +55,12 @@ make_lua_path(LUA_PATH
     ${LUAJIT_SOURCE_DIR}/?.lua
     ${LUAJIT_BINARY_DIR}/?.lua
 )
+
 # Update LUA_CPATH with the library paths collected within
 # <BuildTestLib> macro.
 make_lua_path(LUA_CPATH PATHS ${LUA_CPATHS})
 
 set(LUA_TEST_SUFFIX .test.lua)
-set(LUA_TEST_FLAGS --failures --shuffle)
-set(LUA_TEST_ENV
-  "LUA_PATH=\"${LUA_PATH}\""
-  "LUA_CPATH=\"${LUA_CPATH}\""
-)
-
-if(CMAKE_VERBOSE_MAKEFILE)
-  list(APPEND LUA_TEST_FLAGS --verbose)
-endif()
 
 # XXX: Since the auxiliary libraries are built as a dynamically
 # loaded modules on MacOS instead of shared libraries as it is
@@ -110,22 +96,37 @@ else()
   list(APPEND LUA_TEST_ENV_MORE LD_LIBRARY_PATH=${LD_LIBRARY_PATH})
 endif()
 
-# LUA_CPATH and LD_LIBRARY_PATH variables and also dependencies
-# list with libraries are set in scope of BuildTestLib macro.
-add_custom_command(TARGET tarantool-tests
-  COMMENT "Running Tarantool tests"
-  COMMAND
-  # XXX: We can't move everything to the "inner" env, since there
-  # are some issues with escaping ';' for different shells. As
-  # a result LUA_PATH/LUA_CPATH variables are set via the "outer"
-  # env, since they are not stripped by SIP like LD_*/DYLD_* are.
-  env
-    ${LUA_TEST_ENV}
-    ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR}
-      --exec 'env ${LUA_TEST_ENV_MORE} ${LUAJIT_TEST_COMMAND}'
-      --ext ${LUA_TEST_SUFFIX}
-      --jobs ${CMAKE_BUILD_PARALLEL_LEVEL}
-      ${LUA_TEST_FLAGS}
-  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+set(TEST_SUITE_NAME "tarantool-tests")
+
+# XXX: The call produces both test and target
+# <tarantool-tests-deps> as a side effect.
+add_test_suite_target(tarantool-tests
+  LABELS ${TEST_SUITE_NAME}
+  DEPENDS ${LUAJIT_TEST_DEPS} tarantool-tests-libs
 )
 
+# LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list
+# with dependencies are set in scope of BuildTestLib macro.
+set(TEST_ENV "LUA_PATH=${LUA_PATH};LUA_CPATH=${LUA_CPATH};${LUA_TEST_ENV_MORE}")
+file(GLOB_RECURSE tests ${CMAKE_CURRENT_SOURCE_DIR} "*${LUA_TEST_SUFFIX}")
+foreach(test_path ${tests})
+  get_filename_component(test_name ${test_path} NAME)
+  set(test_title "test/tarantool-tests/${test_name}")
+  add_test(NAME ${test_title}
+    COMMAND ${LUAJIT_TEST_COMMAND} ${test_path}
+    WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+  )
+  set_tests_properties(${test_title} PROPERTIES
+    ENVIRONMENT "${TEST_ENV}"
+    LABELS ${TEST_SUITE_NAME}
+    DEPENDS tarantool-tests-deps
+  )
+endforeach()
+
+# Test is broken when GCC with enabled ASAN is used.
+if (LUAJIT_USE_ASAN AND (CMAKE_C_COMPILER_ID STREQUAL "GNU"))
+  set(test_title test/tarantool-tests/lj-802-panic-at-mcode-protfail.test.lua)
+  set_tests_properties(${test_title} PROPERTIES
+    DISABLED True
+  )
+endif()
-- 
2.34.1

Re: [Tarantool-patches] [PATCH luajit 2/3][v4] test: update CMake macro LibRealPath

[Sergey Kaplun via Tarantool-patches]

Hi, Sergey!
Thanks for the patch!
Please consider the several minor suggestions below.

On 21.03.24, Sergey Bronnikov wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
> 
> The patch updates CMake macro LibRealPath:

Typo: s/CMake/the CMake/

> 
> - Replace CMAKE_CXX_COMPILER with CMAKE_C_COMPILER,

Nit: This line isn't filled enough.

>   because LuaJIT doesn't require CXX compiler and thus

Typo: s/compiler/compiler,/

>   it can be unavailable on a build system.
> - Add a check for a code returned by a command with compiler.

Typo: s/compiler/the compiler/

> - Add a check for a value returned by compiler, the value equal to

Typo: s/compiler, the/the compiler. A/

>   passed one means that library was not found in a system.

Typo: s/passed one/a passed one/
Typo: s/library/the library/
Typo: s/a system/the system/

> ---
>  test/LuaJIT-tests/CMakeLists.txt | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt
> index a0fb5440..f744abfe 100644
> --- a/test/LuaJIT-tests/CMakeLists.txt
> +++ b/test/LuaJIT-tests/CMakeLists.txt
> @@ -33,12 +33,27 @@ if(LUAJIT_USE_ASAN)
>    # https://github.com/google/sanitizers/issues/934.
>    macro(LibRealPath output lib)
>      execute_process(
> -      COMMAND ${CMAKE_CXX_COMPILER} -print-file-name=${lib}
> +      # CMAKE_C_COMPILER is used because it has the same behaviour
> +      # as CMAKE_CXX_COMPILER, but CMAKE_CXX_COMPILER is not required

Nit: Comment length is more than 66 symbols.

> +      # for building LuaJIT and can be missed in GH Actions.
> +      COMMAND ${CMAKE_C_COMPILER} -print-file-name=${lib}
>        OUTPUT_VARIABLE LIB_LINK
>        OUTPUT_STRIP_TRAILING_WHITESPACE
> +      RESULT_VARIABLE RES
>      )
> +    # GCC and Clang returns a passed filename,
> +    # when library was not found.

Nit: Something strange with comment width. You can use more space from
the first line.

> +    if (${LIB_LINK} STREQUAL ${lib})

I suppose, it should be:

| if (LIB_LINK STREQUAL ${lib})

Or we may get the following error if the `LIB_LINK` isn't defined:

| cc: error: unrecognized command-line option '-not-an-option=libstdc++.so'
| cc: fatal error: no input files
| compilation terminated.
| CMake Error at test/LuaJIT-tests/CMakeLists.txt:42 (if):
|   if given arguments:

> +      message(FATAL_ERROR "Library '${lib}' is not found")
> +    endif()
> +    if (NOT RES EQUAL 0)
> +      message(FATAL_ERROR "Executing '${CMAKE_C_COMPILER} \
> +                           -print-file-name=${lib}' has failed")

Please reformat this code like the following to be consistent with the
rest of the code base:

| message(FATAL_ERROR
|   "Executing '${CMAKE_C_COMPILER} -print-file-name=${lib}' has failed"
| )


> +    endif()
> +
>      # Fortunately, we are not interested in macOS here, so we can
> -    # use realpath.
> +    # use realpath. Beware, builtin commands always returns

Typo: s/builtin/built-in/
Typo: s/returns/return/

Also, why do you call it "built-in comands"? Maybe just name it
`realpath`.

> +    # an exit code equal to 0, we cannot check is it failed or not.

Typo: s/, we/, so we/
Typo: s/is it failed/if it is failed/

>      execute_process(
>        COMMAND realpath ${LIB_LINK}
>        OUTPUT_VARIABLE ${output}
> -- 
> 2.34.1
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 2/3][v4] test: update CMake macro LibRealPath

[Sergey Bronnikov via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 4166 bytes --]

Sergey,

see my comments below


On 3/21/24 18:00, Sergey Kaplun wrote:
> Hi, Sergey!
> Thanks for the patch!
> Please consider the several minor suggestions below.
>
> On 21.03.24, Sergey Bronnikov wrote:
>> From: Sergey Bronnikov<sergeyb@tarantool.org>
>>
>> The patch updates CMake macro LibRealPath:
> Typo: s/CMake/the CMake/

Fixed.


>
>> - Replace CMAKE_CXX_COMPILER with CMAKE_C_COMPILER,
> Nit: This line isn't filled enough.


Fixed.


>
>>    because LuaJIT doesn't require CXX compiler and thus
> Typo: s/compiler/compiler,/
Fixed.
>
>>    it can be unavailable on a build system.
>> - Add a check for a code returned by a command with compiler.
> Typo: s/compiler/the compiler/

Fixed.


>
>> - Add a check for a value returned by compiler, the value equal to
> Typo: s/compiler, the/the compiler. A/

Fixed.


>
>>    passed one means that library was not found in a system.
> Typo: s/passed one/a passed one/
> Typo: s/library/the library/
> Typo: s/a system/the system/

Fixed.


>
>> ---
>>   test/LuaJIT-tests/CMakeLists.txt | 19 +++++++++++++++++--
>>   1 file changed, 17 insertions(+), 2 deletions(-)
>>
>> diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt
>> index a0fb5440..f744abfe 100644
>> --- a/test/LuaJIT-tests/CMakeLists.txt
>> +++ b/test/LuaJIT-tests/CMakeLists.txt
>> @@ -33,12 +33,27 @@ if(LUAJIT_USE_ASAN)
>>     #https://github.com/google/sanitizers/issues/934.
>>     macro(LibRealPath output lib)
>>       execute_process(
>> -      COMMAND ${CMAKE_CXX_COMPILER} -print-file-name=${lib}
>> +      # CMAKE_C_COMPILER is used because it has the same behaviour
>> +      # as CMAKE_CXX_COMPILER, but CMAKE_CXX_COMPILER is not required
> Nit: Comment length is more than 66 symbols.

Fixed.


>
>> +      # for building LuaJIT and can be missed in GH Actions.
>> +      COMMAND ${CMAKE_C_COMPILER} -print-file-name=${lib}
>>         OUTPUT_VARIABLE LIB_LINK
>>         OUTPUT_STRIP_TRAILING_WHITESPACE
>> +      RESULT_VARIABLE RES
>>       )
>> +    # GCC and Clang returns a passed filename,
>> +    # when library was not found.
> Nit: Something strange with comment width. You can use more space from
> the first line.

Fixed.


>
>> +    if (${LIB_LINK} STREQUAL ${lib})
> I suppose, it should be:
>
> | if (LIB_LINK STREQUAL ${lib})
>
> Or we may get the following error if the `LIB_LINK` isn't defined:
>
> | cc: error: unrecognized command-line option '-not-an-option=libstdc++.so'
> | cc: fatal error: no input files
> | compilation terminated.
> | CMake Error at test/LuaJIT-tests/CMakeLists.txt:42 (if):
> |   if given arguments:


Fixed.

-- a/test/LuaJIT-tests/CMakeLists.txt
+++ b/test/LuaJIT-tests/CMakeLists.txt
@@ -40,7 +40,7 @@ if(LUAJIT_USE_ASAN)
      )
      # GCC and Clang returns a passed filename, when library
      # was not found.
-    if (${LIB_LINK} STREQUAL ${lib})
+    if (LIB_LINK STREQUAL ${lib})
        message(FATAL_ERROR "Library '${lib}' is not found")
      endif()
      if (NOT RES EQUAL 0)


>> +      message(FATAL_ERROR "Library '${lib}' is not found")
>> +    endif()
>> +    if (NOT RES EQUAL 0)
>> +      message(FATAL_ERROR "Executing '${CMAKE_C_COMPILER} \
>> +                           -print-file-name=${lib}' has failed")
> Please reformat this code like the following to be consistent with the
> rest of the code base:
>
> | message(FATAL_ERROR
> |   "Executing '${CMAKE_C_COMPILER} -print-file-name=${lib}' has failed"
> | )
>
>
Fixed.


>> +    endif()
>> +
>>       # Fortunately, we are not interested in macOS here, so we can
>> -    # use realpath.
>> +    # use realpath. Beware, builtin commands always returns
> Typo: s/builtin/built-in/
> Typo: s/returns/return/

Fixed.


>
> Also, why do you call it "built-in comands"? Maybe just name it
> `realpath`.

because realpath is a special case of shell built-in

updated to realpath

>
>> +    # an exit code equal to 0, we cannot check is it failed or not.
> Typo: s/, we/, so we/
> Typo: s/is it failed/if it is failed/

Fixed.


>
>>       execute_process(
>>         COMMAND realpath ${LIB_LINK}
>>         OUTPUT_VARIABLE ${output}
>> -- 
>> 2.34.1
>>

[-- Attachment #2: Type: text/html, Size: 8898 bytes --]