From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 8B105A653DC; Tue, 26 Mar 2024 11:37:25 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 8B105A653DC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1711442245; bh=Pvx5shSU4Ppi+c6WM1Uzg84+/YFtaJYbU2DCbHaBy4c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Qr+pQc830fTzYKy+cLYJ0CoTSUz/jc0DmZidXULFWOYufdKalLGsfvR8jTM3mI9AL Z4QC+27za29HDoho6p1mNyfc39A/M1I230tbdFI93Pwo570VuR5gHh2otWgbFY0vuw wiHfWBm5AozCH0EuGh/1So/tRC00aNdZPAV5fOQA= Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C4BABA653D5 for ; Tue, 26 Mar 2024 11:34:33 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C4BABA653D5 Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-513cfc93f4eso5641321e87.3 for ; Tue, 26 Mar 2024 01:34:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711442073; x=1712046873; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yex+BHaRf+tkqWdw6aLHfkIFEKXPcqLy33J/xexem6M=; b=GILS5c/8yh8uMlV1oCj4BPI7q8kVfOLj/OClVNXmNVacLHq8ewwysJM9O5rv3maAhG syoIpaXnh7jxEX2dXfvDWnH6gDCXWTF89ZEwpSsfCmw7ohyMjYBjVYFtL56ABnNnYvoC Eb5+MyDeFhpblA77iRYgFcl/OtxVgjLOUczoAotEzh+HYp7H7gvV2zO7RaobN63VIiKS bQkZgH7bh56pM+82amA4+ujPcDm1BGOx8BYc41RkI4d+DTVB6nWdqVdCFUZvaTWNTnKm CfHkpiZa+e0kAPzYH7lEQ2QKjF6Pqn48o5Pj9L0wewYmKCWsVPZw2Ct+RZt8fuUyFRIv kyFA== X-Gm-Message-State: AOJu0YzpCA8oVC0UscoLCzPwCrMLX9qa/2AMSX1KMEyzdt8wNg/iN0gA si42mfyY3Vmgpmk0explkQd5VkZW3CLSDXpx4sxkz4xFMWAaTafbC2A9/7SURuU= X-Google-Smtp-Source: AGHT+IEEMQmrETYSHnrU/amTf1Yi9DfkA6GKpubd90l7J996rFtqioTubTNJuDx9742Urqm7yhvihg== X-Received: by 2002:a05:6512:344c:b0:515:a35f:2524 with SMTP id j12-20020a056512344c00b00515a35f2524mr1431841lfr.22.1711442072434; Tue, 26 Mar 2024 01:34:32 -0700 (PDT) Received: from pony.. ([5.181.62.126]) by smtp.gmail.com with ESMTPSA id y11-20020a056512044b00b005159c7939basm1434099lfk.185.2024.03.26.01.34.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Mar 2024 01:34:32 -0700 (PDT) To: tarantool-patches@dev.tarantool.org, Sergey Kaplun , Maxim Kokryashkin Date: Tue, 26 Mar 2024 11:33:32 +0300 Message-Id: <1d967a53d2d8783ef9f66c0b9d56f64a87a318cb.1711441864.git.sergeyb@tarantool.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit 6/6] cmake: replace prove with CTest X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Sergey Bronnikov The patch replaces the main test runner `prove(1)` with CTest, see [1] and [2]. Build and test steps looks the same as before: $ cmake -S . -B build $ cmake --build build --parallel $ cmake --build build --target [LuaJIT-test, test] CMake targets lua-Harness-tests, LuaJIT-tests, PUC-Rio-Lua-5.1-tests, tarantool-tests and tarantool-c-tests are still operating. CMake custom target `test` was replaced by a target with the same name that is automatically generated by CMake. It is not possible to attach targets to this automatically generated `test` target. It means that target `test` is now executing `LuaJIT-test`, but not `LuaJIT-lint`. To mitigate this a new target `LuaJIT-check-all` is introduced, it has the same behaviour like an old `test` target: it runs all functional tests and linters. One could use `ctest` tool: $ ctest --print-labels Test project All Labels: LuaJIT-tests PUC-Rio-Lua-5.1-tests lua-Harness-tests tarantool-c-tests tarantool-tests $ ctest # Run all tests. $ ctest -L tarantool-tests # Run tests by label. $ ctest -L tarantool-c-tests # Ditto. $ ctest -L lua-Harness-tests # Ditto. $ ctest -L LuaJIT-tests # Ditto. $ ctest -L PUC-Rio-Lua-5.1-tests # Ditto. $ ctest --rerun-fail # Run only the tests # that failed previously. $ ctest --verbose # Print details to stdout. $ ctest --output-on-failure # Print details to stdout # on test failure only. The benefits are: - less external dependencies, `prove(1)` is gone, `ctest` is a part of CMake - running tests by test title - extended test running options in comparison to prove(1) - unified test output (finally!) Note that the testsuites in `test/LuaJIT-tests` and `test/PUC-Rio-Lua-5.1` directories have their own test runners written in Lua, and currently CTest runs these testsuites as single tests. In the future, we could change these test runners to specify tests from outside, and after this, we could run tests separately by CTest in these test suites. Note that it is not possible to add dependencies to `add_test()` in CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and FIXTURES_SETUP [5], but these test properties cannot be used - used CMake version is lower. This is workarounded by an introduced macro `add_test_suite_target` that adds a CMake-test for each testsuite that executes a target that builds test dependencies. The commit introduce a CMake option LUAJIT_TEST_DEPS that set dependencies to be used for running and building LuaJIT tests. 1. https://cmake.org/cmake/help/latest/manual/ctest.1.html 2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html 3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774 4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html 5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html --- .gitignore | 2 + CMakeLists.txt | 14 +++ test/CMakeLists.txt | 107 +++++++++++++++++----- test/LuaJIT-tests/CMakeLists.txt | 33 ++++--- test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt | 25 +++-- test/lua-Harness-tests/CMakeLists.txt | 50 +++++----- test/tarantool-c-tests/CMakeLists.txt | 58 ++++++------ test/tarantool-tests/CMakeLists.txt | 68 +++++++------- 8 files changed, 226 insertions(+), 131 deletions(-) diff --git a/.gitignore b/.gitignore index dc5ea5fc..dd1f8888 100644 --- a/.gitignore +++ b/.gitignore @@ -15,7 +15,9 @@ .ninja_log CMakeCache.txt CMakeFiles +CTestTestfile.cmake Makefile +Testing build.ninja cmake_install.cmake cmake_uninstall.cmake diff --git a/CMakeLists.txt b/CMakeLists.txt index 7f5e2afb..96f85285 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -345,6 +345,13 @@ set(LUAJIT_TEST_BINARY ${LUAJIT_BINARY} CACHE STRING "Lua implementation to be used for tests. Default is 'luajit'." ) +# If LuaJIT is used in a parent project, provide an option +# to choose what dependencies to be used for running and building +# LuaJIT tests. +set(LUAJIT_TEST_DEPS "luajit-main" CACHE STRING + "A list of target dependencies to be used for tests." +) + # FIXME: If LuaJIT is used in parent project, provide an option # to pass Lua code to be run before tests are started. # XXX: Attentive reader might point to LUA_INIT environment @@ -362,6 +369,13 @@ set(LUAJIT_TEST_INIT "${PROJECT_SOURCE_DIR}/test/luajit-test-init.lua" CACHE STR "Lua code need to be run before tests are started." ) +# XXX: Enable testing via CTest. This automatically generates +# the `test` target. Since CTest expects to find a test file in +# the build directory root, this command should be in the source +# directory root (hence, in this CMakeLists.txt). +if (LUAJIT_USE_TEST) + enable_testing() +endif() add_subdirectory(test) # --- Misc rules --------------------------------------------------------------- diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 62478441..06a1d365 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -78,35 +78,94 @@ set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") include(AddTestLib) include(LibRealPath) +# CTEST_FLAGS is used by CMake targets in test suites. +# XXX: CMake 3.17 introduces CMAKE_CTEST_ARGUMENTS that contains +# CTest options and is used by `test` target. +set(CTEST_FLAGS + --output-on-failure + --schedule-random + # Timeout in seconds. Most of LuaJIT tests are fast, + # however, some tests can hang or execute infinite loop. + # See , + # commit caa99865c206 ("test: rewrite sysprof test using managed execution"). + # Moreover, tests for instrumented LuaJIT requires more time. + # 30 minutes is a sane timeout. + --timeout 1800 + --parallel ${CMAKE_BUILD_PARALLEL_LEVEL} +) +if(CMAKE_VERBOSE_MAKEFILE) + list(APPEND CTEST_FLAGS --verbose) +endif() + +# It is not possible to add dependencies to `add_test()` +# in CMake, see [1]. CMake 3.7 introduces FIXTURES_REQUIRED [2] +# and FIXTURES_SETUP [3], but these test properties cannot be +# used - it is unsupported in a current CMake version. +# To workaround this a function `add_test_suite_target` is +# introduced, it adds a CMake target that builds testsuite's +# prerequisites and CMake test that executes that target. +# +# 1. https://gitlab.kitware.com/cmake/cmake/-/issues/8774 +# 2. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html +# 3. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html +function(add_test_suite_target target) + set(prefix ARG) + set(noValues) + set(singleValues LABELS) + set(multiValues DEPENDS) + + # FIXME: if we update to CMake >= 3.5, can remove this line. + include(CMakeParseArguments) + cmake_parse_arguments(${prefix} + "${noValues}" + "${singleValues}" + "${multiValues}" + ${ARGN}) + + set(_DEPS_TARGET ${target}-deps) + + add_custom_target(${_DEPS_TARGET} DEPENDS ${ARG_DEPENDS}) + + add_test(NAME ${_DEPS_TARGET} + COMMAND ${CMAKE_COMMAND} + --build ${CMAKE_BINARY_DIR} + --target ${_DEPS_TARGET} + ) + set_tests_properties(${_DEPS_TARGET} PROPERTIES + LABELS ${ARG_LABELS} + ) + + message(STATUS "Add test suite ${ARG_LABELS}") + + add_custom_target(${target} + COMMAND ${CMAKE_CTEST_COMMAND} -L ${ARG_LABELS} ${CTEST_FLAGS} + DEPENDS ${_DEPS_TARGET} + ) +endfunction() + add_subdirectory(LuaJIT-tests) add_subdirectory(PUC-Rio-Lua-5.1-tests) add_subdirectory(lua-Harness-tests) add_subdirectory(tarantool-c-tests) add_subdirectory(tarantool-tests) -add_custom_target(${PROJECT_NAME}-test DEPENDS - LuaJIT-tests - PUC-Rio-Lua-5.1-tests - lua-Harness-tests - tarantool-c-tests - tarantool-tests +# Each testsuite has it's own CMake target, but combining these +# target to a single one is not desired, because each target +# runs it's own ctest command, that each time enumerates tests +# from zero and prints test summary at the end of test run. +# For common target this output looks inconvenient. +# Therefore target below executes a single instance of ctest +# command that runs all generated CMake tests. +add_custom_target(${PROJECT_NAME}-test + COMMAND ${CMAKE_CTEST_COMMAND} ${CTEST_FLAGS} + DEPENDS tarantool-c-tests-deps + tarantool-tests-deps + lua-Harness-tests-deps + PUC-Rio-Lua-5.1-tests-deps + LuaJIT-tests-deps ) -if(LUAJIT_USE_TEST) - if(POLICY CMP0037) - if(CMAKE_VERSION VERSION_LESS 3.11) - # CMake below 3.11 reserves the name 'test'. Use old policy. - # https://cmake.org/cmake/help/v3.11/release/3.11.html#other-changes - cmake_policy(SET CMP0037 OLD) - else() - # Starting from CMake 3.11 the name 'test' is reserved in - # special cases and can be used as a target name. - cmake_policy(SET CMP0037 NEW) - endif() - endif(POLICY CMP0037) - - add_custom_target(test DEPENDS - ${PROJECT_NAME}-test - ${PROJECT_NAME}-lint - ) -endif() +add_custom_target(${PROJECT_NAME}-check-all + DEPENDS ${PROJECT_NAME}-test + ${PROJECT_NAME}-lint +) diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt index 003f8de6..6ad05aa6 100644 --- a/test/LuaJIT-tests/CMakeLists.txt +++ b/test/LuaJIT-tests/CMakeLists.txt @@ -3,17 +3,13 @@ cmake_minimum_required(VERSION 3.1 FATAL_ERROR) add_subdirectory(src) -add_custom_target(LuaJIT-tests - DEPENDS ${LUAJIT_TEST_BINARY} LuaJIT-tests-libs -) - make_lua_path(LUA_CPATH PATHS ${CMAKE_CURRENT_BINARY_DIR}/src/?${CMAKE_SHARED_LIBRARY_SUFFIX} ) set(LUAJIT_TESTS_ENV - "LUA_CPATH=\"${LUA_CPATH}\"" + "LUA_CPATH=${LUA_CPATH}" ) set(LD_LIBRARY_PATH "${CMAKE_CURRENT_BINARY_DIR}/src/:") @@ -56,12 +52,25 @@ if(LUAJIT_NO_UNWIND) set(LUAJIT_TEST_TAGS_EXTRA +internal_unwinder) endif() -add_custom_command(TARGET LuaJIT-tests - COMMENT "Running LuaJIT-tests" - COMMAND - env - ${LUAJIT_TESTS_ENV} - ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/test.lua - +slow +ffi +bit +jit ${LUAJIT_TEST_TAGS_EXTRA} +set(TEST_SUITE_NAME "LuaJIT-tests") + +# XXX: The call produces both test and target +# as a side effect. +add_test_suite_target(LuaJIT-tests + LABELS ${TEST_SUITE_NAME} + DEPENDS ${LUAJIT_TEST_DEPS} LuaJIT-tests-libs +) + +# The test suite has its own test runner +# (test/LuaJIT-tests/test.lua), it is not possible to run these +# tests one-by-one by CTest. +add_test(NAME "test/LuaJIT-tests" + COMMAND ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/test.lua + +slow +ffi +bit +jit ${LUAJIT_TEST_TAGS_EXTRA} WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} ) +set_tests_properties("test/LuaJIT-tests" PROPERTIES + LABELS ${TEST_SUITE_NAME} + ENVIRONMENT "${LUAJIT_TESTS_ENV}" + DEPENDS LuaJIT-tests-deps +) diff --git a/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt b/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt index 98277f9a..a26e38d4 100644 --- a/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt +++ b/test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt @@ -34,17 +34,26 @@ add_subdirectory(libs) # But, unfortunately, depends on specific PUC-Rio # Lua 5.1 internal headers and should be adapted for LuaJIT. -add_custom_target(PUC-Rio-Lua-5.1-tests - DEPENDS ${LUAJIT_TEST_BINARY} PUC-Rio-Lua-5.1-tests-prepare +set(TEST_SUITE_NAME "PUC-Rio-Lua-5.1-tests") + +# XXX: The call produces both test and target +# as a side effect. +add_test_suite_target(PUC-Rio-Lua-5.1-tests + LABELS ${TEST_SUITE_NAME} + DEPENDS ${LUAJIT_TEST_DEPS} PUC-Rio-Lua-5.1-tests-prepare ) -add_custom_command(TARGET PUC-Rio-Lua-5.1-tests - COMMENT "Running PUC-Rio Lua 5.1 tests" - COMMAND - env - LUA_PATH="${LUA_PATH}" - ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/all.lua +# The test suite has its own test runner +# (test/PUC-Rio-Lua-5.1-tests/all.lua), it is not possible +# to run these tests one-by-one by CTest. +add_test(NAME "test/PUC-Rio-Lua-5.1-tests" + COMMAND ${LUAJIT_TEST_COMMAND} ${CMAKE_CURRENT_SOURCE_DIR}/all.lua WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} ) +set_tests_properties("test/PUC-Rio-Lua-5.1-tests" PROPERTIES + ENVIRONMENT "LUA_PATH=${LUA_PATH}" + LABELS ${TEST_SUITE_NAME} + DEPENDS PUC-Rio-Lua-5.1-tests-deps +) # vim: expandtab tabstop=2 shiftwidth=2 diff --git a/test/lua-Harness-tests/CMakeLists.txt b/test/lua-Harness-tests/CMakeLists.txt index f748a8fd..6f7b5697 100644 --- a/test/lua-Harness-tests/CMakeLists.txt +++ b/test/lua-Harness-tests/CMakeLists.txt @@ -4,12 +4,6 @@ # See the rationale in the root CMakeLists.txt cmake_minimum_required(VERSION 3.1 FATAL_ERROR) -find_program(PROVE prove) -if(NOT PROVE) - message(WARNING "`prove' is not found, so lua-Harness-tests target is not generated") - return() -endif() - # Tests create temporary files (see 303-package.t and 411-luajit.t # for example) to be required. Also, they require some files from # the original test source directory. @@ -20,23 +14,37 @@ make_lua_path(LUA_PATH ${LUAJIT_SOURCE_DIR}/?.lua ${LUAJIT_BINARY_DIR}/?.lua ) -set(LUA_TEST_FLAGS --failures --shuffle) -if(CMAKE_VERBOSE_MAKEFILE) - list(APPEND LUA_TEST_FLAGS --verbose) -endif() +set(TEST_SUITE_NAME "lua-Harness-tests") -add_custom_target(lua-Harness-tests DEPENDS ${LUAJIT_TEST_BINARY}) -add_custom_command(TARGET lua-Harness-tests - COMMENT "Running lua-Harness tests" - COMMAND - env - LUA_PATH="${LUA_PATH}" - ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR} - --exec '${LUAJIT_TEST_COMMAND} -l profile_luajit21' - --jobs ${CMAKE_BUILD_PARALLEL_LEVEL} - ${LUA_TEST_FLAGS} - WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} +# XXX: The call produces both test and target +# as a side effect. +add_test_suite_target(lua-Harness-tests + LABELS ${TEST_SUITE_NAME} + DEPENDS ${LUAJIT_TEST_DEPS} ) +set(LUA_TEST_SUFFIX .t) + +file(GLOB tests ${CMAKE_CURRENT_SOURCE_DIR} "*${LUA_TEST_SUFFIX}") +foreach(test_path ${tests}) + get_filename_component(test_name ${test_path} NAME) + # FIXME: By default, GLOB lists directories. + # Directories are omitted in the result if LIST_DIRECTORIES + # is set to false. New in version CMake 3.3. + if (${test_name} STREQUAL ${TEST_SUITE_NAME}) + continue() + endif() + set(test_title "test/${TEST_SUITE_NAME}/${test_name}") + add_test(NAME ${test_title} + COMMAND ${LUAJIT_TEST_COMMAND} -l profile_luajit21 ${test_path} + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + ) + set_tests_properties(${test_title} PROPERTIES + ENVIRONMENT "LUA_PATH=${LUA_PATH}" + LABELS ${TEST_SUITE_NAME} + DEPENDS lua-Harness-tests-deps + ) +endforeach() + # vim: expandtab tabstop=2 shiftwidth=2 diff --git a/test/tarantool-c-tests/CMakeLists.txt b/test/tarantool-c-tests/CMakeLists.txt index 4b1d8832..5affbb9e 100644 --- a/test/tarantool-c-tests/CMakeLists.txt +++ b/test/tarantool-c-tests/CMakeLists.txt @@ -1,15 +1,4 @@ -find_program(PROVE prove) -if(NOT PROVE) - message(WARNING "`prove' is not found, so tarantool-c-tests target is not generated") - return() -endif() - set(C_TEST_SUFFIX .c_test) -set(C_TEST_FLAGS --failures --shuffle) - -if(CMAKE_VERBOSE_MAKEFILE) - list(APPEND C_TEST_FLAGS --verbose) -endif() # Build libtest. @@ -30,6 +19,23 @@ AppendFlags(TESTS_C_FLAGS ${TARGET_C_FLAGS}) # test binary can be run from any directory. AppendFlags(TESTS_C_FLAGS "-D__LJ_TEST_DIR__='\"${CMAKE_CURRENT_SOURCE_DIR}\"'") +set(TEST_SUITE_NAME "tarantool-c-tests") + +# Proxy CMake target with all targets that builds C tests. +# This is needed because targets for each C test are generated +# at the same time with CMake tests, and all prerequisites must be +# already exist at this moment. +add_custom_target(tarantool-c-tests-build) + +# XXX: The call produces both test and target +# as a side effect. +add_test_suite_target(tarantool-c-tests + LABELS ${TEST_SUITE_NAME} + # XXX: LUAJIT_TEST_DEPS is not needed, we know dependencies + # in advance. + DEPENDS libluajit libtest tarantool-c-tests-build +) + set(CTEST_SRC_SUFFIX ".test.c") file(GLOB tests "${CMAKE_CURRENT_SOURCE_DIR}/*${CTEST_SRC_SUFFIX}") foreach(test_source ${tests}) @@ -47,22 +53,16 @@ foreach(test_source ${tests}) ) target_link_libraries(${exe} libtest ${LUAJIT_LIBRARY}) LIST(APPEND TESTS_COMPILED ${exe}) -endforeach() + add_dependencies(tarantool-c-tests-build ${exe}) -add_custom_target(tarantool-c-tests - DEPENDS libluajit libtest ${TESTS_COMPILED} -) - -add_custom_command(TARGET tarantool-c-tests - COMMENT "Running Tarantool C tests" - COMMAND - ${PROVE} - ${CMAKE_CURRENT_BINARY_DIR} - --ext ${C_TEST_SUFFIX} - --jobs ${CMAKE_BUILD_PARALLEL_LEVEL} - # Report any TAP parse errors, if any, since test module is - # maintained by us. - --parse - ${C_TEST_FLAGS} - WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} -) + # Generate CMake tests. + set(test_title "test/${TEST_SUITE_NAME}/${exe}${C_TEST_SUFFIX}") + add_test(NAME ${test_title} + COMMAND ${CMAKE_CURRENT_BINARY_DIR}/${exe}${C_TEST_SUFFIX} + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + ) + set_tests_properties(${test_title} PROPERTIES + LABELS ${TEST_SUITE_NAME} + DEPENDS tarantool-c-tests-deps + ) +endforeach() diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt index 35bcc5ef..9086a890 100644 --- a/test/tarantool-tests/CMakeLists.txt +++ b/test/tarantool-tests/CMakeLists.txt @@ -4,19 +4,13 @@ # See the rationale in the root CMakeLists.txt. cmake_minimum_required(VERSION 3.1 FATAL_ERROR) -find_program(PROVE prove) -if(NOT PROVE) - message(WARNING "`prove' is not found, so tarantool-tests target is not generated") - return() -endif() - -add_custom_target(tarantool-tests +add_custom_target(tarantool-tests-libs DEPENDS ${LUAJIT_TEST_BINARY} ) macro(BuildTestCLib lib sources) AddTestLib(${lib} ${sources}) - add_dependencies(tarantool-tests ${lib}) + add_dependencies(tarantool-tests-libs ${lib}) # Add the directory where the lib is built to the list with # entries for LUA_CPATH environment variable, so LuaJIT can find # and load it. See the comment about extending the list in the @@ -61,20 +55,12 @@ make_lua_path(LUA_PATH ${LUAJIT_SOURCE_DIR}/?.lua ${LUAJIT_BINARY_DIR}/?.lua ) + # Update LUA_CPATH with the library paths collected within # macro. make_lua_path(LUA_CPATH PATHS ${LUA_CPATHS}) set(LUA_TEST_SUFFIX .test.lua) -set(LUA_TEST_FLAGS --failures --shuffle) -set(LUA_TEST_ENV - "LUA_PATH=\"${LUA_PATH}\"" - "LUA_CPATH=\"${LUA_CPATH}\"" -) - -if(CMAKE_VERBOSE_MAKEFILE) - list(APPEND LUA_TEST_FLAGS --verbose) -endif() # XXX: Since the auxiliary libraries are built as a dynamically # loaded modules on MacOS instead of shared libraries as it is @@ -118,29 +104,37 @@ endif() # process. # See also: https://github.com/tarantool/tarantool/issues/9656. if(LUAJIT_USE_ASAN AND CMAKE_C_COMPILER_ID STREQUAL "GNU") - # FIXME: After using CTest instead of `prove` we should set this - # environment variable only for the corresponding tests to avoid - # warnings from the libc and etc. + # FIXME: test_title test/tarantool-tests/lj-802-panic-at-mcode-protfail.test.lua + # With CTest we should set this environment variable only + # for the corresponding tests to avoid warnings from + # the GNU libc and other libc implementations. LibRealPath(LIB_ASAN libasan.so) list(APPEND LUA_TEST_ENV_MORE LD_PRELOAD=${LIB_ASAN}) endif() -# LUA_CPATH and LD_LIBRARY_PATH variables and also dependencies -# list with libraries are set in scope of BuildTestLib macro. -add_custom_command(TARGET tarantool-tests - COMMENT "Running Tarantool tests" - COMMAND - # XXX: We can't move everything to the "inner" env, since there - # are some issues with escaping ';' for different shells. As - # a result LUA_PATH/LUA_CPATH variables are set via the "outer" - # env, since they are not stripped by SIP like LD_*/DYLD_* are. - env - ${LUA_TEST_ENV} - ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR} - --exec 'env ${LUA_TEST_ENV_MORE} ${LUAJIT_TEST_COMMAND}' - --ext ${LUA_TEST_SUFFIX} - --jobs ${CMAKE_BUILD_PARALLEL_LEVEL} - ${LUA_TEST_FLAGS} - WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} +set(TEST_SUITE_NAME "tarantool-tests") + +# XXX: The call produces both test and target +# as a side effect. +add_test_suite_target(tarantool-tests + LABELS ${TEST_SUITE_NAME} + DEPENDS ${LUAJIT_TEST_DEPS} tarantool-tests-libs ) +# LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list +# with dependencies are set in scope of BuildTestLib macro. +set(TEST_ENV "LUA_PATH=${LUA_PATH};LUA_CPATH=${LUA_CPATH};${LUA_TEST_ENV_MORE}") +file(GLOB_RECURSE tests ${CMAKE_CURRENT_SOURCE_DIR} "*${LUA_TEST_SUFFIX}") +foreach(test_path ${tests}) + get_filename_component(test_name ${test_path} NAME) + set(test_title "test/tarantool-tests/${test_name}") + add_test(NAME ${test_title} + COMMAND ${LUAJIT_TEST_COMMAND} ${test_path} + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + ) + set_tests_properties(${test_title} PROPERTIES + ENVIRONMENT "${TEST_ENV}" + LABELS ${TEST_SUITE_NAME} + DEPENDS tarantool-tests-deps + ) +endforeach() -- 2.34.1