From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id C3990B4D7C4; Sat, 16 Mar 2024 20:27:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C3990B4D7C4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1710610077; bh=TtfNOO6nMmvwnkTNNoDuaskdJ+z0K4+a2HNSWe7gqDY=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=i/WSNkUSlrS67X7tXwrB0v0k56q+2pPiAhzS3Mp0b2iO6Pz6F2oKmVUx0/QMozmj6 8cDZVo3ZlXH8J/wknCsmhI0NIVCgF2H3iKohvjbgg2OQg+y9QLdzAk3MsixzgV3ekL y/gDy+obuv/Pdb+d58mIemZeAWHcF3/kmbYX0uMY= Received: from smtp43.i.mail.ru (smtp43.i.mail.ru [95.163.41.66]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 9084DB4D7C3 for ; Sat, 16 Mar 2024 20:27:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9084DB4D7C3 Received: by smtp43.i.mail.ru with esmtpa (envelope-from ) id 1rlXp5-0000000GTfl-2NED; Sat, 16 Mar 2024 20:27:56 +0300 Date: Sat, 16 Mar 2024 17:12:01 +0000 To: Sergey Bronnikov Message-ID: References: <62428a61c98eee87dfc91f2d8972988d80589d3e.1710226958.git.sergeyb@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <62428a61c98eee87dfc91f2d8972988d80589d3e.1710226958.git.sergeyb@tarantool.org> X-Clacks-Overhead: GNU Terry Pratchett X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD987C0EE6E7F0A597D8D64E6F18B2E2E3258EF6A9BB0633F3C182A05F538085040D0414269DB6E4E95479CDAE959BF6424F867A953F44EECA5F8828664800E43C8810A56970F496ED6 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7CC84CC3AD347B910EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006375101D255E984898F8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8A6505092EA7653571E5448187A557B9E0CAC35F5AA3D720ACC7F00164DA146DAFE8445B8C89999728AA50765F7900637F3E38EE449E3E2AE389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8D2DCF9CF1F528DBCF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947C6FF1B927F95F33162D242C3BD2E3F4C64AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C3480B02F9DC9E23DABA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CFED8438A78DFE0A9E1DD303D21008E298D5E8D9A59859A8B6B372FE9A2E580EFC725E5C173C3A84C3517AAFC1974B5A1735872C767BF85DA2F004C90652538430E4A6367B16DE6309 X-C1DE0DAB: 0D63561A33F958A5AD95D65C2013377B5002B1117B3ED69645A77B0AE216CB063E67C18142C611B7823CB91A9FED034534781492E4B8EEAD003C2D46C52F18F2BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF45981E99E96AA97181C76496483AAF95F692BE62B5219585F68192BE724AD593136DDA45E01CE4BE6CD0533D1BDF67ED380E5FE4A4518D0DCB57650DE097CD8D57FADBE7965C09AF77C8E16086367389913E6812662D5F2A74A26541E23E71317A83BD0C44CE203720ABEDE4BBDD9CDD X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojpAzVD49/gcMDG2iUgmU9iw== X-Mailru-Sender: 2FEBA92C8E508479FE7B9A1DF348D531E5C3566321BC84B6AC4385294E02B5FB6048E2F0D9C663C83E92BACC0D92A683F5EE5212A79745119D086FC03DADA47A24EDF3BD9585A06DEEABBE9EE6587A054198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit][v3] cmake: replace prove with CTest X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: max.kokryashkin@gmail.com, tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Sergey, Thanks for the fixes! The patch L(almost)GTM now, but I still make some comments. You can see my changes here below. On 12.03.24, Sergey Bronnikov wrote: > From: Sergey Bronnikov > > The patch replaces the main test runner `prove(1)` with CTest, > see [1] and [2]. > > Build and test steps looks the same as before: > > $ cmake -S . -B build > $ cmake --build build --parallel > $ cmake --build build --target [LuaJIT-test, test] > > CMake targets lua-Harness-tests, LuaJIT-tests, PUC-Rio-Lua-5.1-tests, > tarantool-tests and tarantool-c-tests are still functional. > > One could use `ctest` tool: > > $ ctest --print-labels > Test project > All Labels: > LuaJIT > PUC-Rio-Lua-5.1 > lua-Harness > tarantool > tarantool-c Typo: Labels are updated since v2. > $ ctest # Run all tests. > $ ctest -L tarantool # Run tests in tarantool-tests dir. > $ ctest -L tarantool-c # Run tests in tarantool-c-tests dir. > $ ctest -L lua-Harness # Run tests in lua-Harness-tests dir. > $ ctest -L luajit # Run tests in LuaJIT-tests dir. > $ ctest -L PUC-Rio-Lua-5.1 # Run tests in PUC-Rio-Lua-5.1-tests dir. > $ ctest --rerun-fail > $ ctest --output-on-failure > > The benefits are: > > - less external dependencies, `prove(1)` is gone, `ctest` is a part of CMake > - running tests by test title > - extended test running options in comparison to prove(1) > - unified test output (finally!) > > Note that it is not possible to attach targets to the automatically > generated `test` target. It means that target `test` is now executing > `LuaJIT-test`, but not `LuaJIT-lint`. Side note: I guees something like can save the behaviour broken by this change. However, do we really need this? I left the final decision to skaplun@ here. > > Note that the testsuites in `test/LuaJIT-tests` and > `test/PUC-Rio-Lua-5.1` directories have their own test runners written > in Lua and currently CTest runs these testsuites as single tests. In the > future we could change these test runners to specify tests from outside, > and after this we could run tests separately by CTest in these test > suites. > > Note that it is not possible to add dependencies to `add_test()` in > CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and > FIXTURES_SETUP [5], but these test properties cannot be used - used > CMake version is lower. This workarounded by introducing a special test > for each testsuite that executes a target that builds test dependencies. Minor: Maybe it's worth to mention macro here? > > 1. https://cmake.org/cmake/help/latest/manual/ctest.1.html > 2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html > 3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774 > 4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html > 5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html > --- > Changes v3: > - rebased to master > - applied fixes suggested by Igor Munkin > > PR: https://github.com/tarantool/tarantool/pull/9286 > Branch: https://github.com/tarantool/luajit/tree/ligurio/enable_test_target > > .gitignore | 2 + > CMakeLists.txt | 11 ++++ > test/CMakeLists.txt | 78 +++++++++++++++-------- > test/LuaJIT-tests/CMakeLists.txt | 40 +++++++----- > test/LuaJIT-tests/src/CMakeLists.txt | 2 +- > test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt | 25 +++++--- > test/lua-Harness-tests/CMakeLists.txt | 48 ++++++++------ > test/tarantool-c-tests/CMakeLists.txt | 56 ++++++++-------- > test/tarantool-tests/CMakeLists.txt | 62 ++++++++---------- > 9 files changed, 194 insertions(+), 130 deletions(-) > > diff --git a/CMakeLists.txt b/CMakeLists.txt > index 7f5e2afb..6b2f855f 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > @@ -345,6 +345,13 @@ set(LUAJIT_TEST_BINARY ${LUAJIT_BINARY} CACHE STRING > "Lua implementation to be used for tests. Default is 'luajit'." > ) > > +# If LuaJIT is used in a parent project, provide an option > +# to choose what dependencies to be used for running and building > +# LuaJIT tests. > +set(LUAJIT_TEST_DEPS "luajit-main" CACHE STRING > + "A list of target dependencies to be used for tests." > +) This is worth to be mentioned in the commit message (at least). > + > # FIXME: If LuaJIT is used in parent project, provide an option > # to pass Lua code to be run before tests are started. > # XXX: Attentive reader might point to LUA_INIT environment > diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt > index 3ad5d15f..90aaead2 100644 > --- a/test/CMakeLists.txt > +++ b/test/CMakeLists.txt > @@ -77,35 +77,63 @@ separate_arguments(LUAJIT_TEST_COMMAND) > set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") > include(AddTestLib) > > +# TEST_FLAGS is used by CMake targets in test suites. > +# XXX: CMake 3.17 introduces CMAKE_CTEST_ARGUMENTS that contains CTest options Typo: We still follows 66 symbol limit for comments in tarantool/luajit, but I also left the final decision to skaplun@ here. > +# and is used by `test` target. > -add_custom_target(${PROJECT_NAME}-test DEPENDS > - LuaJIT-tests > - PUC-Rio-Lua-5.1-tests > - lua-Harness-tests > - tarantool-c-tests > - tarantool-tests > +add_custom_target(${PROJECT_NAME}-test > + COMMAND ${CMAKE_CTEST_COMMAND} ${TEST_FLAGS} > + DEPENDS tarantool-c-tests-deps > + tarantool-tests-deps > + lua-Harness-tests-deps > + PUC-Rio-Lua-5.1-tests-deps > + LuaJIT-tests-deps I suggest to make LuaJIT-test an umbrella rule without the particular implementation. See the changes below: ================================================================================ diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 90aaead2..ad447e72 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -130,10 +130,9 @@ add_subdirectory(tarantool-c-tests) add_subdirectory(tarantool-tests) add_custom_target(${PROJECT_NAME}-test - COMMAND ${CMAKE_CTEST_COMMAND} ${TEST_FLAGS} - DEPENDS tarantool-c-tests-deps - tarantool-tests-deps - lua-Harness-tests-deps - PUC-Rio-Lua-5.1-tests-deps - LuaJIT-tests-deps + DEPENDS tarantool-c-tests + tarantool-tests + lua-Harness-tests + PUC-Rio-Lua-5.1-tests + LuaJIT-tests ) ================================================================================ > ) > diff --git a/test/LuaJIT-tests/CMakeLists.txt b/test/LuaJIT-tests/CMakeLists.txt > index a0fb5440..1080987a 100644 > --- a/test/LuaJIT-tests/CMakeLists.txt > +++ b/test/LuaJIT-tests/CMakeLists.txt > @@ -3,17 +3,13 @@ cmake_minimum_required(VERSION 3.1 FATAL_ERROR) > > add_subdirectory(src) > > -add_custom_target(LuaJIT-tests > - DEPENDS ${LUAJIT_TEST_BINARY} LuaJIT-tests-libs > -) > - > make_lua_path(LUA_CPATH > PATHS > ${CMAKE_CURRENT_BINARY_DIR}/src/?${CMAKE_SHARED_LIBRARY_SUFFIX} > ) > > set(LUAJIT_TESTS_ENV > - "LUA_CPATH=\"${LUA_CPATH}\"" > + "LUA_CPATH=${LUA_CPATH}\;\;" The trailing semicolons are automatically added by . My bad, remove it please. > ) > > set(LD_LIBRARY_PATH "${CMAKE_CURRENT_BINARY_DIR}/src/:") > @@ -50,10 +46,11 @@ if(LUAJIT_USE_ASAN) > if(CMAKE_C_COMPILER_ID STREQUAL "GNU") > LibRealPath(LIB_ASAN libasan.so) > # XXX: Don't use " " (separator in LD_PRELOAD) in `list()`. > - # ";" will expand to " " as we want. > - list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD="${LIB_ASAN};${LIB_STDCPP}") > + # ";" will expand to " " as we want after wrapping > + # LUAJIT_TESTS_ENV by double quotes. > + list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD=${LIB_ASAN};${LIB_STDCPP}) > else() > - list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD="${LIB_STDCPP}") > + list(APPEND LUAJIT_TESTS_ENV LD_PRELOAD=${LIB_STDCPP}) > endif() > endif() I really do not understand, why this quotation is required in the original series and breaks everything now. If skaplun@ is bothered more than me, he can spend more time with CMake to understand this. Sad, but OK, so I'm out here. > > diff --git a/test/lua-Harness-tests/CMakeLists.txt b/test/lua-Harness-tests/CMakeLists.txt > index f748a8fd..d454ba41 100644 > --- a/test/lua-Harness-tests/CMakeLists.txt > +++ b/test/lua-Harness-tests/CMakeLists.txt > +file(GLOB tests ${CMAKE_CURRENT_SOURCE_DIR} "*.t") > +foreach(test_path ${tests}) > + get_filename_component(test_name ${test_path} NAME) > + # FIXME: By default GLOB lists directories. > + # Directories are omitted in the result if LIST_DIRECTORIES > + # is set to false. New in version CMake 3.3. > + if (${test_name} STREQUAL ${TEST_SUITE_NAME}) > + continue() > + endif (${test_name} STREQUAL ${TEST_SUITE_NAME}) Typo: Excess whitespace and condition in . > + set(test_title "test/${TEST_SUITE_NAME}/${test_name}") > + add_test(NAME ${test_title} > + COMMAND ${LUAJIT_TEST_COMMAND} -l profile_luajit21 ${test_path} > + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} Typo: Looks like misindent, doesn't it? > + ) > + set_tests_properties(${test_title} PROPERTIES > + ENVIRONMENT "LUA_PATH=${LUA_PATH}\;" Typo: The semicolon looks excess here. > + LABELS ${TEST_SUITE_NAME} > + DEPENDS lua-Harness-tests-deps > + ) > +endforeach() > + > # vim: expandtab tabstop=2 shiftwidth=2 > diff --git a/test/tarantool-c-tests/CMakeLists.txt b/test/tarantool-c-tests/CMakeLists.txt > index 4b1d8832..2de4a0c8 100644 > --- a/test/tarantool-c-tests/CMakeLists.txt > +++ b/test/tarantool-c-tests/CMakeLists.txt > @@ -49,20 +38,35 @@ foreach(test_source ${tests}) > +# XXX: The call produces both test and target > +# as a side effect. Typo: Miscopypasting :) > +add_test_suite_target(tarantool-c-tests > + LABEL ${TEST_SUITE_NAME} > + DEPENDS ${LUAJIT_TEST_DEPS} libluajit libtest ${TESTS_COMPILED} > ) > + > +# XXX: Note that we cannot glob generated files in CMake, see > +# https://stackoverflow.com/questions/44076307/cmake-globbing-generated-files > +# To workaround this, we glob source files and generated tests > +# with paths to executable binaries. > +file(GLOB tests ${CMAKE_CURRENT_SOURCE_DIR} "*.test.c") > +foreach(test_path ${tests}) > + get_filename_component(test_name ${test_path} NAME_WE) > + # FIXME: By default GLOB lists directories. > + # Directories are omitted in the result if LIST_DIRECTORIES > + # is set to false. New in version CMake 3.3. > + if (${test_name} STREQUAL ${TEST_SUITE_NAME}) > + continue() > + endif (${test_name} STREQUAL ${TEST_SUITE_NAME}) Typo: Excess whitespace and condition in . > + set(test_title "test/${TEST_SUITE_NAME}/${test_name}${C_TEST_SUFFIX}") > diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt > index e6d12984..bb41e53b 100644 > --- a/test/tarantool-tests/CMakeLists.txt > +++ b/test/tarantool-tests/CMakeLists.txt > @@ -60,21 +54,14 @@ make_lua_path(LUA_PATH > ${PROJECT_SOURCE_DIR}/tools/?.lua > ${LUAJIT_SOURCE_DIR}/?.lua > ${LUAJIT_BINARY_DIR}/?.lua > + ${PROJECT_BINARY_DIR}/src/?.lua This is the same directory as a ${LUAJIT_BINARY_DIR}/?.lua. > ) > @@ -110,22 +97,29 @@ else() > +# LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list > +# with dependencies are set in scope of BuildTestLib macro. > +set(TEST_ENV "LUA_PATH=${LUA_PATH};\;\;;LUA_CPATH=${LUA_CPATH}\;\;;${LUA_TEST_ENV_MORE}") The trailing semicolons are automatically added by . Please, remove it. > +file(GLOB_RECURSE tests ${CMAKE_CURRENT_SOURCE_DIR} "*${LUA_TEST_SUFFIX}") Minor: Why is used here, and the hack with manual directory name filtering is omitted? > +foreach(test_path ${tests}) > + get_filename_component(test_name ${test_path} NAME) > + set(test_title "test/tarantool-tests/${test_name}") > + add_test(NAME ${test_title} > + COMMAND ${LUAJIT_TEST_COMMAND} ${test_path} > + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} > + ) > + set_tests_properties(${test_title} PROPERTIES > + ENVIRONMENT "${TEST_ENV}" > + LABELS ${TEST_SUITE_NAME} > + DEPENDS tarantool-tests-deps > + ) > +endforeach() > -- > 2.34.1 > -- Best regards, IM