From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 07A80EBBD96; Tue, 24 Sep 2024 17:17:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 07A80EBBD96 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1727187466; bh=uF3A3RDs4ZQOSjEFY/kL83XEXRitZjw77OxmnP83/f0=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=NSssoLxcn47AhbV79W+IarHPVSy5BE+IRNlrH15o1E/mqxiqkSGsJGdKy+xeBUBQd volwuaT/KE7Tkf3BxOUrzegwpD9V9NYOQUWUjy4XeT289ckx75btG+Fx0jfGLDnbBs 4L5SOwYzijclSxrpCWXPaIT0kfNaOLDaHkf4+Sp4= Received: from smtp42.i.mail.ru (smtp42.i.mail.ru [95.163.41.65]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 96B17EBBD96 for ; Tue, 24 Sep 2024 17:17:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 96B17EBBD96 Received: by smtp42.i.mail.ru with esmtpa (envelope-from ) id 1st6MJ-0000000FBzO-213W; Tue, 24 Sep 2024 17:17:44 +0300 Content-Type: multipart/alternative; boundary="------------xPIigbNGYS9zPGNJOHmN0Low" Message-ID: <056c98ce-9df4-46c2-84db-eaabfb32f5ef@tarantool.org> Date: Tue, 24 Sep 2024 17:17:40 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun References: <746090ddd45350e52ecd3847758c0fd51cba23df.1727172936.git.skaplun@tarantool.org> <89ff148e-27d8-485d-a343-e1daff22d3ec@tarantool.org> In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD941CBEC3D08EBAA096DD98E365DE151DD85552754D3CB7C05182A05F5380850404C228DA9ACA6FE275EF2C2320CD3BA55411046492FDDF8061B406D485B722023ECBEE71F5DB76FAC2D7423A3730B98E4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE766DBE83FD69AB645EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006370EBB47D88F71BB738638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D889BA1C584F58263557788A3C9AD047CF0018B11013AD17B6CC7F00164DA146DAFE8445B8C89999728AA50765F7900637D0FEED2715E18529389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC821E93C0F2A571C7BF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947CCEBC872EF0A1BB656E0066C2D8992A164AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C30F8E59DFC38C49F1BA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE72958EBC9E2F10130731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: 0D63561A33F958A5B7179DA7DC6749455002B1117B3ED696168D816FF969427D1A1B8FE1FED62FE8823CB91A9FED034534781492E4B8EEAD2B25D9E4C92BC8ACBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF214C368CFFAD33D556CC6385D3D8670870A46F0EE71C4EEEB3B85E40FE8ABE4FA7825560E8A5912CA4322FFB9F042FEF2352DAF26C42E775923EB69796B6FDFB703EDFB6DC785E905F4332CA8FE04980913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojamSMdo1tbG9/hxBf3EXDhw== X-Mailru-Sender: 520A125C2F17F0B1E52FEF5D219D614065E728D275460DB25A686361E53CBA2F1046339A6412E6B60152A3D17938EB451EB5A0BCEC6A560B3DDE9B364B0DF289BE2DA36745F2EEB5CEBA01FB949A1F1EEAB4BC95F72C04283CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 luajit 6/7] test: skip flaky tests with enabled table bump X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------xPIigbNGYS9zPGNJOHmN0Low Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, Sergey, On 24.09.2024 14:45, Sergey Kaplun wrote: > Hi, Sergey! > Thanks for the review! > > On 24.09.24, Sergey Bronnikov wrote: >> Hi, Sergey, >> >> test statuses from TAP report produced by tests is not propagated to CTest. >> >> I propose to disable these tests in CTest: >> >> >> --- a/test/tarantool-tests/CMakeLists.txt >> +++ b/test/tarantool-tests/CMakeLists.txt >> @@ -166,3 +167,14 @@ if(LUAJIT_USE_ASAN AND CMAKE_C_COMPILER_ID STREQUAL >> "GNU") >>      LD_PRELOAD ${LIB_ASAN} >>    ) >>  endif() >> + >> +set_tests_properties( >> +  test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua >> + test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua >> + test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua >> + >> test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua >> + >> test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua >> +  test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua >> +  test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua >> +  PROPERTIES DISABLED TRUE >> +) >> >> >> CTest will always report that these tests are disabled and we will never >> forget to enable them back: >> >> >> 100% tests passed, 0 tests failed out of 158 >> >> Label Time Summary: >> tarantool-tests    =  14.66 sec*proc (165 tests) >> >> Total Test time (real) =  14.73 sec >> >> The following tests did not run: >>         222 - >> test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua (Disabled) >>         223 - >> test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua >> (Disabled) >>         224 - >> test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua >> (Disabled) >>         225 - >> test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua >> (Disabled) >>         226 - >> test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua >> (Disabled) >>         227 - >> test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua (Disabled) >>         228 - >> test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua (Disabled) >> >> >> What do you think? >> > Side note: this is new in 3.9, so we need to update CMake first. > > I'm not sure about this one. > We also have all skipconditions inside the test itself, which makes it > "self-sufficient" -- so if we look at the test, we know all builds when > it is disabled. Adding another location may be confusing. I agree. The ideal option for us is when TAP statuses are propagated to CTest, but it is not an option now. > > OTOH, we can see skipped tests in -V mode (but I agree, that it isn't > very convenient). > > I am open for discussion :). Feel free to ignore, I'll not insist. LGTM >> On 24.09.2024 13:29, Sergey Kaplun wrote: >>> Profilers tests with LUAJIT_ENABLE_TABLE_BUMP are flaky due to the bug >>> in the TDUP patching. Disable them for now. >>> --- >>> test/tarantool-tests/CMakeLists.txt | 6 ++++++ >>> .../profilers/gh-5688-tool-cli-flag.test.lua | 2 ++ >>> .../gh-5813-resolving-of-c-symbols.test.lua | 2 ++ >>> .../gh-5994-memprof-human-readable.test.lua | 2 ++ >>> ...7264-add-proto-trace-sysprof-default.test.lua | 2 ++ >>> ...-9217-profile-parsers-error-handling.test.lua | 2 ++ >>> .../profilers/misclib-memprof-lapi.test.lua | 16 +++++++++------- >>> .../profilers/misclib-sysprof-lapi.test.lua | 2 ++ >>> 8 files changed, 27 insertions(+), 7 deletions(-) >>> >>> diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt > > >>> test:plan(19) --------------xPIigbNGYS9zPGNJOHmN0Low Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi, Sergey,

On 24.09.2024 14:45, Sergey Kaplun wrote:
Hi, Sergey!
Thanks for the review!

On 24.09.24, Sergey Bronnikov wrote:

Hi, Sergey,

test statuses from TAP report produced by tests is not propagated to CTest.

I propose to disable these tests in CTest:


--- a/test/tarantool-tests/CMakeLists.txt
+++ b/test/tarantool-tests/CMakeLists.txt
@@ -166,3 +167,14 @@ if(LUAJIT_USE_ASAN AND CMAKE_C_COMPILER_ID STREQUAL 
"GNU")
      LD_PRELOAD ${LIB_ASAN}
    )
  endif()
+
+set_tests_properties(
+  test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua
+ test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua
+ test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua
+ 
test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua
+ 
test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua
+  test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua
+  test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua
+  PROPERTIES DISABLED TRUE
+)


CTest will always report that these tests are disabled and we will never 
forget to enable them back:


100% tests passed, 0 tests failed out of 158

Label Time Summary:
tarantool-tests    =  14.66 sec*proc (165 tests)

Total Test time (real) =  14.73 sec

The following tests did not run:
         222 - 
test/tarantool-tests/profilers/gh-5688-tool-cli-flag.test.lua (Disabled)
         223 - 
test/tarantool-tests/profilers/gh-5813-resolving-of-c-symbols.test.lua 
(Disabled)
         224 - 
test/tarantool-tests/profilers/gh-5994-memprof-human-readable.test.lua 
(Disabled)
         225 - 
test/tarantool-tests/profilers/gh-7264-add-proto-trace-sysprof-default.test.lua 
(Disabled)
         226 - 
test/tarantool-tests/profilers/gh-9217-profile-parsers-error-handling.test.lua 
(Disabled)
         227 - 
test/tarantool-tests/profilers/misclib-memprof-lapi.test.lua (Disabled)
         228 - 
test/tarantool-tests/profilers/misclib-sysprof-lapi.test.lua (Disabled)


What do you think?


Side note: this is new in 3.9, so we need to update CMake first.

I'm not sure about this one.
We also have all skipconditions inside the test itself, which makes it
"self-sufficient" -- so if we look at the test, we know all builds when
it is disabled. Adding another location may be confusing.

I agree. The ideal option for us is when TAP statuses are propagated to CTest,

but it is not an option now.


OTOH, we can see skipped tests in -V mode (but I agree, that it isn't
very convenient).

I am open for discussion :).

Feel free to ignore, I'll not insist.


LGTM


      

        
On 24.09.2024 13:29, Sergey Kaplun wrote:


        

          
Profilers tests with LUAJIT_ENABLE_TABLE_BUMP are flaky due to the bug
in the TDUP patching. Disable them for now.
---
  test/tarantool-tests/CMakeLists.txt              |  6 ++++++
  .../profilers/gh-5688-tool-cli-flag.test.lua     |  2 ++
  .../gh-5813-resolving-of-c-symbols.test.lua      |  2 ++
  .../gh-5994-memprof-human-readable.test.lua      |  2 ++
  ...7264-add-proto-trace-sysprof-default.test.lua |  2 ++
  ...-9217-profile-parsers-error-handling.test.lua |  2 ++
  .../profilers/misclib-memprof-lapi.test.lua      | 16 +++++++++-------
  .../profilers/misclib-sysprof-lapi.test.lua      |  2 ++
  8 files changed, 27 insertions(+), 7 deletions(-)

diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt


        

      

      
<snipped>



      

        

          
  test:plan(19)


        

      

      

    

  
  


--------------xPIigbNGYS9zPGNJOHmN0Low--