From: Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: Sergey Kaplun <skaplun@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: Re: [Tarantool-patches] [PATCH v2 luajit 6/7] test: skip flaky tests with enabled table bump Date: Tue, 24 Sep 2024 17:17:40 +0300 [thread overview] Message-ID: <056c98ce-9df4-46c2-84db-eaabfb32f5ef@tarantool.org> (raw) In-Reply-To: <ZvKmTM83r7scDSqg@root> [-- Attachment #1: Type: text/plain, Size: 3752 bytes --] Hi, Sergey, On 24.09.2024 14:45, Sergey Kaplun wrote: > Hi, Sergey! > Thanks for the review! > > On 24.09.24, Sergey Bronnikov wrote: >> Hi, Sergey, >> >> test statuses from TAP report produced by tests is not propagated to CTest. >> >> I propose to disable these tests in CTest: >> >> >> --- a/test/tarantool-tests/CMakeLists.txt >> +++ b/test/tarantool-tests/CMakeLists.txt >> @@ -166,3 +167,14 @@ if(LUAJIT_USE_ASAN AND CMAKE_C_COMPILER_ID STREQUAL >> "GNU") >> LD_PRELOAD ${LIB_ASAN} >> ) >> endif() >> + >> +set_tests_properties( >> + test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua >> + test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua >> + test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua >> + >> test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua >> + >> test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua >> + test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua >> + test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua >> + PROPERTIES DISABLED TRUE >> +) >> >> >> CTest will always report that these tests are disabled and we will never >> forget to enable them back: >> >> >> 100% tests passed, 0 tests failed out of 158 >> >> Label Time Summary: >> tarantool-tests = 14.66 sec*proc (165 tests) >> >> Total Test time (real) = 14.73 sec >> >> The following tests did not run: >> 222 - >> test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua (Disabled) >> 223 - >> test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua >> (Disabled) >> 224 - >> test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua >> (Disabled) >> 225 - >> test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua >> (Disabled) >> 226 - >> test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua >> (Disabled) >> 227 - >> test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua (Disabled) >> 228 - >> test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua (Disabled) >> >> >> What do you think? >> > Side note: this is new in 3.9, so we need to update CMake first. > > I'm not sure about this one. > We also have all skipconditions inside the test itself, which makes it > "self-sufficient" -- so if we look at the test, we know all builds when > it is disabled. Adding another location may be confusing. I agree. The ideal option for us is when TAP statuses are propagated to CTest, but it is not an option now. > > OTOH, we can see skipped tests in -V mode (but I agree, that it isn't > very convenient). > > I am open for discussion :). Feel free to ignore, I'll not insist. LGTM >> On 24.09.2024 13:29, Sergey Kaplun wrote: >>> Profilers tests with LUAJIT_ENABLE_TABLE_BUMP are flaky due to the bug >>> in the TDUP patching. Disable them for now. >>> --- >>> test/tarantool-tests/CMakeLists.txt | 6 ++++++ >>> .../profilers/gh-5688-tool-cli-flag.test.lua | 2 ++ >>> .../gh-5813-resolving-of-c-symbols.test.lua | 2 ++ >>> .../gh-5994-memprof-human-readable.test.lua | 2 ++ >>> ...7264-add-proto-trace-sysprof-default.test.lua | 2 ++ >>> ...-9217-profile-parsers-error-handling.test.lua | 2 ++ >>> .../profilers/misclib-memprof-lapi.test.lua | 16 +++++++++------- >>> .../profilers/misclib-sysprof-lapi.test.lua | 2 ++ >>> 8 files changed, 27 insertions(+), 7 deletions(-) >>> >>> diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt > <snipped> > >>> test:plan(19) [-- Attachment #2: Type: text/html, Size: 4869 bytes --]
next prev parent reply other threads:[~2024-09-24 14:17 UTC|newest] Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-09-24 10:29 [Tarantool-patches] [PATCH v2 luajit 0/7] Shrink test env and fix flaky tests Sergey Kaplun via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 1/7] test: move profilers tests to subdirectory Sergey Kaplun via Tarantool-patches 2024-09-24 11:07 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:28 ` Maxim Kokryashkin via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 2/7] test: rename <arm64-ccall-fp-convention.test.lua> Sergey Kaplun via Tarantool-patches 2024-09-24 11:07 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:29 ` Maxim Kokryashkin via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 3/7] cmake: introduce AppendTestEnvVar macro Sergey Kaplun via Tarantool-patches 2024-09-24 11:07 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:30 ` Maxim Kokryashkin via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 4/7] test: shrink LUA_PATH environment variable Sergey Kaplun via Tarantool-patches 2024-09-24 11:15 ` Sergey Bronnikov via Tarantool-patches 2024-09-24 11:33 ` Sergey Kaplun via Tarantool-patches [not found] ` <ZvO5g6ryIcyhM1vg@root> 2024-09-25 7:22 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:31 ` Maxim Kokryashkin via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 5/7] test: shrink LUA_CPATH and {DY}LD_LIBRARY_PATH Sergey Kaplun via Tarantool-patches 2024-09-24 11:16 ` Sergey Bronnikov via Tarantool-patches 2024-09-24 11:28 ` Sergey Kaplun via Tarantool-patches 2024-09-24 14:18 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:37 ` Maxim Kokryashkin via Tarantool-patches 2024-10-12 7:00 ` Sergey Kaplun via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 6/7] test: skip flaky tests with enabled table bump Sergey Kaplun via Tarantool-patches 2024-09-24 11:26 ` Sergey Bronnikov via Tarantool-patches 2024-09-24 11:45 ` Sergey Kaplun via Tarantool-patches 2024-09-24 14:17 ` Sergey Bronnikov via Tarantool-patches [this message] 2024-10-11 18:38 ` Maxim Kokryashkin via Tarantool-patches 2024-09-24 10:29 ` [Tarantool-patches] [PATCH v2 luajit 7/7] test: set LD_PRELOAD only when necessary Sergey Kaplun via Tarantool-patches 2024-09-24 14:13 ` Sergey Bronnikov via Tarantool-patches 2024-09-24 15:23 ` Sergey Kaplun via Tarantool-patches 2024-09-24 17:41 ` Sergey Bronnikov via Tarantool-patches 2024-10-11 18:46 ` Maxim Kokryashkin via Tarantool-patches 2024-10-18 15:18 ` [Tarantool-patches] [PATCH v2 luajit 0/7] Shrink test env and fix flaky tests Sergey Kaplun via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=056c98ce-9df4-46c2-84db-eaabfb32f5ef@tarantool.org \ --to=tarantool-patches@dev.tarantool.org \ --cc=sergeyb@tarantool.org \ --cc=skaplun@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v2 luajit 6/7] test: skip flaky tests with enabled table bump' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox