Re: [Tarantool-patches] [PATCH 2/2 v2] ci: support coveralls

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Hi, Sergey

thanks for a long-awaited review!

On 8/6/23 14:41, Sergey Kaplun via Tarantool-patches wrote:
> Hi, Maxim, Sergey!
> Thanks for the patch and the review!
> LGTM, just a minor comments below.
>
> On 02.08.23, Maxim Kokryashkin wrote:
>> Hi, Sergey!
>> Thanks for the patch!
>> LGTM, except for a few nits regarding the commit message.
>>
>> Best regards,
>> Maxim Kokryashkin
>>
>> On Tue, Aug 01, 2023 at 09:46:10PM +0300, Sergey Bronnikov via Tarantool-patches wrote:
>>> From: Sergey Bronnikov <sergeyb@tarantool.org>
>>>
>>> The patch adds a workflow that runs regression test suites, produces a
>>> summary about current code coverage and send code coverage data to
>> Typo: s/about/of/
>> Typo: s/send code/sends code/
>>> Coveralls. Coveralls is a web-service that lets you inspect every detail
>> Typo: s/web-service/web service/
>>> of your coverage. See Tarantool's LuaJIT page on Coveralls [1].
>>>
>>> 1. https://coveralls.io/github/tarantool/luajit
>>> ---
>>>   .github/actions/setup-linux/action.yml |  1 +
>>>   .github/workflows/coverage.yml         | 60 ++++++++++++++++++++++++++
>>>   2 files changed, 61 insertions(+)
>>>   create mode 100644 .github/workflows/coverage.yml
>>>
>>> diff --git a/.github/actions/setup-linux/action.yml b/.github/actions/setup-linux/action.yml
>>> index f0171b83..71619a60 100644
>>> --- a/.github/actions/setup-linux/action.yml
>>> +++ b/.github/actions/setup-linux/action.yml
>>> @@ -16,4 +16,5 @@ runs:
>>>         run: |
>>>           apt -y update
>>>           apt -y install cmake gcc make ninja-build perl
>>> +        pip3 install gcovr
> Should it be done in the separate action, like it is done for ASan?
> Because we don't need gcovr for all our linux testing.

Well, added a separate GH action for that.


>
>>>         shell: bash
>>> diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml
>>> new file mode 100644
>>> index 00000000..9fff06c7
>>> --- /dev/null
>>> +++ b/.github/workflows/coverage.yml
>>> @@ -0,0 +1,60 @@
>>> +name: Code coverage
>>> +
>>> +on:
>>> +  push:
>>> +    branches-ignore:
>>> +      - '**-notest'
>>> +      - 'upstream-**'
>>> +    tags-ignore:
>>> +      - '**'
>>> +
>>> +concurrency:
>>> +  # An update of a developer branch cancels the previously
>>> +  # scheduled workflow run for this branch. However, the default
>>> +  # branch, and long-term branch (tarantool/release/2.11,
>>> +  # tarantool/release/2.10, etc) workflow runs are never canceled.
>>> +  #
>>> +  # We use a trick here: define the concurrency group as 'workflow
>>> +  # run ID' + # 'workflow run attempt' because it is a unique
>>> +  # combination for any run. So it effectively discards grouping.
>>> +  #
>>> +  # XXX: we cannot use `github.sha` as a unique identifier because
>>> +  # pushing a tag may cancel a run that works on a branch push
>>> +  # event.
>>> +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
>>> +    && format('{0}-{1}', github.run_id, github.run_attempt)
>>> +    || format('{0}-{1}', github.workflow, github.ref) }}
>>> +  cancel-in-progress: true
>>> +
>>> +jobs:
>>> +  coverage:
>>> +    strategy:
>>> +      fail-fast: false
>>> +    runs-on: [self-hosted, regular, x86_64, Linux]
>>> +    steps:
>>> +      - uses: actions/checkout@v3
>>> +        with:
>>> +          fetch-depth: 0
>>> +          submodules: recursive
>>> +      - name: setup Linux
>>> +        uses: ./.github/actions/setup-linux
>>> +      - name: configure
>>> +        run: >
>>> +          cmake -S . -B ${{ env.BUILDDIR }}
>>> +          -G Ninja
>>> +          -DCMAKE_BUILD_TYPE=RelWithDebInfo
>>> +          -DLUAJIT_ENABLE_COVERAGE=ON
>>> +          -DLUAJIT_ENABLE_GC64=ON
> Is there a way to joing GC64/non-GC64 testsing?
> Same for ARM64.

Yes, there is a thing named "parallel build" that allows reporting

code coverage data from a number of CI jobs, see [1].

We definitely need this for jobs with tests that covers non-common parts 
of code like GC64/ARM64 etc,

but in scope of other patch series.


1. https://docs.coveralls.io/parallel-builds#whats-a-parallel-build


>>> +      - name: build
>>> +        run: cmake --build . --parallel
>>> +        working-directory: ${{ env.BUILDDIR }}
>>> +      - name: test and generate code coverage report
>>> +        run: cmake --build ${{ env.BUILDDIR }} --parallel --target coverage
> I see no test target here, so will we get the correct coverage results?
> Am I missing something? If yes, than comment is desirable.

- target "LuaJIT-coverage" runs gcov and gcovr and builds a code 
coverage report

- target "coverage" connected to "test" (running tests) and 
"LuaJIT-coverage" (builds code coverage report)

I suppose comment is not required here.

>
>>> +      - name: send code coverage to coveralls.io
>>> +        run: |
>>> +          curl -LO https://coveralls.io/coveralls-linux.tar.gz
>>> +          tar xvzf coveralls-linux.tar.gz
>>> +          ./coveralls -f ./coverage/luajit.xml
>>> +        working-directory: ${{ env.BUILDDIR }}
>>> +        env:
>>> +          COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
>>> -- 
>>> 2.34.1
>>>