From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id AF9F7566C81; Tue, 1 Aug 2023 18:25:29 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org AF9F7566C81 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1690903529; bh=sV6oNp0mDo8MyfHyr9klvH91E/S87xdliryEZ8USg6Q=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=vOs7JlR3N4U09fd8u9nTHs6o+tqwcPFIg7pnAB3JybmsHzwLhTYUkReN1ixfgImbZ 3Dq7Z8l2KiDxFSSKJu85VZoQKca00PYSXy0rs3ckiartMVsZnuNGtzw7EseTKpNnUf em/6d3z6YRTHX6SCfT/RBcNclBTwrlm39qQ4vC+M= Received: from smtp33.i.mail.ru (smtp33.i.mail.ru [95.163.41.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 043BA3A4690 for ; Tue, 1 Aug 2023 18:25:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 043BA3A4690 Received: by smtp33.i.mail.ru with esmtpa (envelope-from ) id 1qQrFX-005M5J-0X; Tue, 01 Aug 2023 18:25:27 +0300 Message-ID: <58297844-8590-cf70-b3d5-3ed8908aef9a@tarantool.org> Date: Tue, 1 Aug 2023 18:25:26 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 To: Maxim Kokryashkin References: <4a26e6a1f06191178c385a1df62d61763a5743e3.1690293120.git.sergeyb@tarantool.org> <60fc85bf-2ac0-fa2c-9e5c-de9b3b71a726@tarantool.org> <62sqm37aohluqmyfgqgdeupxavzfnhszawdtns3yg7ctk5mppf@d535jjttice5> Content-Language: en-US In-Reply-To: <62sqm37aohluqmyfgqgdeupxavzfnhszawdtns3yg7ctk5mppf@d535jjttice5> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD93761F2630DFFAF41A40BE88C1D2246309964D810A4987D50182A05F538085040EC44712CEF69D264F70833F0C9E0228350ACCA52B25B7B0390F0988343784BA5 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7196003627DEC9496EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006375D8840FA58F505298638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D88DF82CBAC466140CE33515958152C733117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC3A703B70628EAD7BA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520599709FD55CB46A6F04B652EEC242312D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE9647ADFADE5905B197A342136F30543AD8FC6C240DEA76429C9F4D5AE37F343AA9539A8B242431040A6AB1C7CE11FEE34E7D9683544204AF6136E347CC761E07C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947C744B801E316CB65F2E808ACE2090B5E1725E5C173C3A84C362968DCAA3E4B45B089D37D7C0E48F6C8AA50765F79006376A91CFDE938F542CEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C1DE0DAB: 0D63561A33F958A53AC4E8130D5C48CDF0058AC840C4B4B3C82122E601244C1BF87CCE6106E1FC07E67D4AC08A07B9B0B355ED1E20F5346ACB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF8BBF16BEE5A385327CD32E7E893B88A9E8933878255DA36E4524138C91C26394FB7A4CF61EC7A84F337391BE08E8D56EDD9F73D256E4E818C70A7D771E758E30461A413F07889F2102C26D483E81D6BE0DBAE6F56676BC7117BB6831D7356A2DEC5B5AD62611EEC62B5AFB4261A09AF0 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojV0vAgLNnFueryEQRKK38FQ== X-Mailru-Sender: 49D287FBCBBF3A5C10E199891D96F069CCCA4A8F5EBE9BDED4E7D2FCB24145915FC9CED9F47C053A645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH] cmake: add code coverage support X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Cc: max.kokryashkin@gmail.com, tarantool-patches@dev.tarantool.org, Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Max! thanks for your comments. See my comments below. On 7/30/23 16:34, Maxim Kokryashkin wrote: > Here is the commit message on the branch: > | cmake: add code coverage support > | > | The patch adds building code coverage support using gcovr [1] and gcov. > | gcovr is a better version of lcov, see [2]. CMake target LuaJIT-coverage > | executes regression tests, proccess *.gcno and *.gcda files with gcov, > Typo: s/process/processes/ > | builds detailed HTML report and prints summary about code coverage. > Typo: s/detailed/a detailed > Typo: s/summary/a summary/ Fixed. >> >>>> +if (ENABLE_COVERAGE) >>>> + AppendFlags(CMAKE_C_FLAGS >>>> + -fprofile-arcs >>>> + -ftest-coverage >>> Should we use just --coverage? >> Sure, updated. > Now the alignments seems kinda strange. I believe it would be > better to make it a signle line now, like: > | AppendFlags(CMAKE_C_FLAGS --coverage) Fixed. Actually I left it on a separate line intentionally. This will minimize a diff in case adding more options to CFLAGS. >>> | --coverage >>> | This option is used to compile and link code instrumented >>> | for coverage analysis. The option is a synonym for >>> | -fprofile-arcs -ftest-coverage (when compiling) and -lgcov >>> | (when linking). >>> >>> See man gcc(1) for details. >>> >>>> + ) >>>> +endif() >>>> + >>>> # Auxiliary flags for main targets (libraries, binaries). >>>> AppendFlags(TARGET_C_FLAGS >>>> -D_FILE_OFFSET_BITS=64 > I suggest moving most of the logic with finding gcov/running coverage into > a separate cmake module, just to make the test-related one more readable. > What do you think? Good idea. Moved to cmake/CodeCoverage.cmake. >>>> + endif() >>>> + >>>> + add_custom_target(${PROJECT_NAME}-coverage) >>>> + add_custom_command(TARGET ${PROJECT_NAME}-coverage >>>> + COMMENT "Building coverage report" >>>> + COMMAND >>>> + ${GCOVR} >>>> + # See https://gcovr.com/en/stable/guide/configuration.html >>>> + --root ${PROJECT_SOURCE_DIR} >>>> + --filter ${PROJECT_SOURCE_DIR}/src >>> I suppose some files like *.dasc files or buildvm sources should be >>> excluded too, since they are not informative. >> Done. > You did not include the actual diff for the change, so I've checked it > on GitHub. Here are two comments, that were added: > | # Exclude DynASM files, that contains a low-level VM code for CPUs. > Typo: s/that contains/that contain/ > > | # Exclude buildvm source code, it's a project's infrastructure. > Typo: s/a project's/the project's/ Fixed. >> >>>> + --print-summary >>>> + --output ${COVERAGE_HTML_REPORT} >>>> + --coveralls ${COVERAGE_JSON_REPORT} >>> I see also output >>> >>>> + --html >>>> + --html-title "LuaJIT Code Coverage Report" >>>> + --html-details >>>> + --sort-percentage >>> I can't see an option for decreasing percentage of uncovered lines? >>> Is its default value? If it is, then it should be used (as far as we want >>> to see uncovered cases first). >> >> Seems it can sort in that order only. >> >>>> + --branches >>>> + --decisions >>> Do we need this option? AFAICS, this is mostly about Functional Safety >>> applications (road vehicles, etc.). >> I would leave it. >> >> >>>> + --verbose >>> I suppose that we should use --verbose option only when VERBOSE=1 env >>> var is set. >>> >>> Also, I'm suprised by lines like this: >>> | (DEBUG) Parsing coverage data for file /home/burii/reviews/luajit/gcov/src/lj_tab.h >>> | (DEBUG) Starting the decision analysis >>> | (DEBUG) Decision Analysis finished! >>> | (DEBUG) Finding source file corresponding to a gcov data file >>> Is it the part of the --verbose options? >> Right, it is enabled only when --verbose is passed. Removed option --verbose >> at all, >> >> I believe we don't need it in normal use. >> >> >>>> + -j ${CMAKE_BUILD_PARALLEL_LEVEL} >>>> + WORKING_DIRECTORY ${PROJECT_SOURCE_DIR} >>>> + ) >>>> + message(STATUS "Code coverage HTML report: ${COVERAGE_HTML_REPORT}") >>>> + message(STATUS "Code coverage JSON report: ${COVERAGE_JSON_REPORT}") >>>> +endif(ENABLE_COVERAGE) >>>> + >>>> if(LUAJIT_USE_TEST) >>>> if(POLICY CMP0037) >>>> if(CMAKE_VERSION VERSION_LESS 3.11) >>>> @@ -76,4 +114,6 @@ if(LUAJIT_USE_TEST) >>>> ${PROJECT_NAME}-test >>>> ${PROJECT_NAME}-luacheck >>>> ) >>>> + >>>> + add_dependencies(${PROJECT_NAME}-coverage ${PROJECT_NAME}-test) >>> Is it necessary to rerun all tests to generate this analisys every >>> time? >> What could be a reasons to regenerate  coverage report without rerunning >> tests? >> >> I can make a target LuaJIT-coverage that will generate report and add target >> coverage that will >> >> run tests and then call LuaJIT-coverage. What do you think? > Seems adequate to me. Added: --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -78,6 +78,9 @@ if(LUAJIT_USE_TEST)    )    if (LUAJIT_ENABLE_COVERAGE) -    add_dependencies(${PROJECT_NAME}-coverage ${PROJECT_NAME}-test) +    add_custom_target(coverage DEPENDS +      ${PROJECT_NAME}-test +      ${PROJECT_NAME}-coverage +    )    endif (LUAJIT_ENABLE_COVERAGE)  endif() > Maxim Kokryashkin