Re: [Tarantool-patches] [PATCH luajit] ci: extend tarantool integration testing

[Maxim Kokryashkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Hi, Sergey!
Thanks for the review!
Consider my answers below.
On Mon, Mar 04, 2024 at 05:05:31PM +0300, Sergey Bronnikov wrote:
> Hi, Max
>
> thanks for updates. LGTM with minor comments below.
>
>
> On 3/1/24 11:25, Maxim Kokryashkin wrote:
> > This patch makes use of reusable workflows from the
> > Tarantool repository, so now LuaJIT CI tests integration
> > in all relevant workflows.
> >
> > Also, this patch disables MacOS testing for Tarantool
> > integration since those runs were made nightly in the
> > Tarantool repo.
> >
> > As noted in the documentation[1], it is impossible for a single
> > workflow call tree to include more than 20 workflows. To
> > overcome this limitation, workflows are started in a bunch of
> > separate files. This makes it impossible to depend on LuaJIT-only
> > jobs for integration workflow since it is impossible to make
> > cross-dependencies between workflow files.
> >
> > The name of a callee workflow cannot be substituted using a
> > matrix[2], so workflow calls are copied and pasted instead.
> >
> > Table below shows which workflows are enabled and why.
> >
> > Workflow name                   |+/-| Reason
> > -----------------------------------------------------------------
> > codeql                          | - | Not relevant to LuaJIT.
> > coverage                        | + | Long tests for profilers.
> > coverity                        | - | Cron workflow.
> > debug                           | + | Tarantool debug build.
> > debug_aarch64                   | + | Tarantool debug build.
> > debug_asan_clang                | + | Tarantool debug build.
> > default_gcc_centos_7            | + | Ancient gcc build.
> better: "gcc version". Fteel free to ignore.
Fixed.
> > freebsd                         | - | Nightly build.
> > fuzzing                         | - | Impossible to bump LuaJIT.
> > integration                     | + | Tarantool ecosystem.
> > jepsen-cluster-txm              | - | Manual workflow.
> > jepsen-cluster                  | - | Manual workflow.
> > jepsen-single-instance-txm      | - | Cron workflow.
> > jepsen-single-instance          | - | Cron workflow.
> > lango-stale-reviews             | - | Cron workflow.
> > lint                            | - | LuaJIT has its own lint.
> > luajit-integration              | + | Exotic LuaJIT options.
> > memtx_allocator_based_on_malloc | + | Not relevant to LuaJIT.
> typo: s/+/-/
Fixed.
> > osx                             | - | Nightly build.
> > out_of_source                   | - | Not relevent to LuaJIT.
> > packaging                       | - | No LuaJIT-relevant variety.
> > perf_cbench                     | - | Not enabled for PRs.
> > perf_linkbench_ssd              | - | Not enabled for PRs.
> > perf_micro                      | - | Not relevant to LuaJIT.
> > perf_nosqlbench_hash            | - | Not enabled for PRs.
> > perf_nosqlbench_tree            | - | Not enabled for PRs.
> > perf_sysbench                   | - | Not enabled for PRs.
> > perf_tpcc                       | - | Not enabled for PRs.
> > perf_tpch                       | - | Not enabled for PRs.
> > perf_ycsb_hash                  | - | Not enabled for PRs.
> > perf_ycsb_tree                  | - | Not enabled for PRs.
> > publish-module-api-doc          | - | No Doxygen in LuaJIT.
> > release                         | + | Tarantool release build.
> > release_asan_clang              | + | Tarantool release build.
> > release_clang                   | + | Tarantool release build.
> > release_lto                     | + | Tarantool release build.
> > release_lto_clang               | + | Tarantool release build.
> > reusable_build                  | - | Utility for integration.
> > source                          | - | Not enabled for PRs.
> > static_build                    | + | Tarantool static build.
> > static_build_cmake_linux        | + | Tarantool static build.
> > static_build_pack_test_deploy   | - | Utility for packaging.
>
> I would left "static_build" [1] and skip "static_build_cmake_linux" [2].
>
> I don't see a big difference for static_build and static_build_cmake_linux,
> build commands are the same,
>
> both workflows are running on ubuntu-20.04-self-hosted. The difference is in
> builddir location:
>
> first is in source build, the second one is out of source build, see [4].
Fixed.
>
>
> 1. https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.test.mk#L165-L171
>
> 2. https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.github/workflows/static_build_cmake_linux.yml#L1
>
> 4. https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.test.mk#L156-L171
>
> > submodule_update                | - | Not enabled for PRs.
> >
> > [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
> > [2]: https://github.com/orgs/community/discussions/45342#discussioncomment-4779360
> > ---
> > Branch: https://github.com/tarantool/luajit/tree/fckxorg/integration-testing
> >
> >   .github/workflows/exotic-tarantool.yml    |  62 ++++++++++++
> >   .github/workflows/tarantool-ecosystem.yml |  35 +++++++
> >   .github/workflows/testing.yml             | 116 ++++++++++++++++------
> >   3 files changed, 183 insertions(+), 30 deletions(-)
> >   create mode 100644 .github/workflows/exotic-tarantool.yml
> >   create mode 100644 .github/workflows/tarantool-ecosystem.yml
> >
> > diff --git a/.github/workflows/exotic-tarantool.yml b/.github/workflows/exotic-tarantool.yml
> > new file mode 100644
> > index 00000000..be55291a
> > --- /dev/null
> > +++ b/.github/workflows/exotic-tarantool.yml
> > @@ -0,0 +1,62 @@
> > +name: "Exotic Tarantool builds testing"
> > +
> > +on:
> > +  push:
> > +    branches-ignore:
> > +      - '**-notest'
> > +      - 'upstream-**'
> > +    tags-ignore:
> > +      - '**'
> > +
> > +concurrency:
> > +  # An update of a developer branch cancels the previously
> > +  # scheduled workflow run for this branch. However, the default
> > +  # branch, and long-term branch (tarantool/release/2.11,
> > +  # tarantool/release/2.10, etc) workflow runs are never canceled.
> > +  #
> > +  # We use a trick here: define the concurrency group as 'workflow
> > +  # run ID' + # 'workflow run attempt' because it is a unique
> > +  # combination for any run. So it effectively discards grouping.
> > +  #
> > +  # XXX: we cannot use `github.sha` as a unique identifier because
> > +  # pushing a tag may cancel a run that works on a branch push
> > +  # event.
> > +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
> > +    && format('{0}-{1}', github.run_id, github.run_attempt)
> > +    || format('{0}-{1}', github.workflow, github.ref) }}
> > +  cancel-in-progress: true
> > +
> > +jobs:
> > +  test-exotic-tarantool:
> > +    strategy:
> > +      fail-fast: false
> > +      matrix:
> > +        ARCH: [ARM64, x86_64]
> > +        BUILDTYPE: [Debug, Release]
> > +        GC64: [ON, OFF]
> > +        OS: [Linux, macOS]
> > +        include:
> > +          - BUILDTYPE: Debug
> > +            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
> > +          - BUILDTYPE: Release
> > +            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> > +        exclude:
> > +          - ARCH: ARM64
> > +            GC64: OFF
> > +          - OS: macOS
> > +            GC64: OFF
> > +    name: >
> > +      Tarantool
> > +      (${{ matrix.OS }}/${{ matrix.ARCH }})
> > +      ${{ matrix.BUILDTYPE }}
> > +      GC64:${{ matrix.GC64 }}
> > +    # XXX: Only LuaJIT-tests are running in this workflow.
> > +    uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
> > +    with:
> > +      CMAKE_EXTRA_PARAMS: >
> > +        -G Ninja
> > +        ${{ matrix.CMAKEFLAGS }}
> > +        -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> > +      arch: ${{ matrix.ARCH }}
> > +      os: ${{ matrix.OS }}
> > +      revision: ${{ github.sha }}
> > diff --git a/.github/workflows/tarantool-ecosystem.yml b/.github/workflows/tarantool-ecosystem.yml
> > new file mode 100644
> > index 00000000..82a47fe1
> > --- /dev/null
> > +++ b/.github/workflows/tarantool-ecosystem.yml
> > @@ -0,0 +1,35 @@
> > +name: Tarantool ecosystem integration testing
> > +
> > +on:
> > +  push:
> > +    branches-ignore:
> > +      - '**-notest'
> > +      - 'upstream-**'
> > +    tags-ignore:
> > +      - '**'
> > +
> > +concurrency:
> > +  # An update of a developer branch cancels the previously
> > +  # scheduled workflow run for this branch. However, the default
> > +  # branch, and long-term branch (tarantool/release/2.11,
> > +  # tarantool/release/2.10, etc) workflow runs are never canceled.
> > +  #
> > +  # We use a trick here: define the concurrency group as 'workflow
> > +  # run ID' + # 'workflow run attempt' because it is a unique
> > +  # combination for any run. So it effectively discards grouping.
> > +  #
> > +  # XXX: we cannot use `github.sha` as a unique identifier because
> > +  # pushing a tag may cancel a run that works on a branch push
> > +  # event.
> > +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
> > +    && format('{0}-{1}', github.run_id, github.run_attempt)
> > +    || format('{0}-{1}', github.workflow, github.ref) }}
> > +  cancel-in-progress: true
> > +
> > +jobs:
> > +  test-tarantool-integration:
> > +    uses: tarantool/tarantool/.github/workflows/integration.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
> > index cb4ba57b..7bd527be 100644
> > --- a/.github/workflows/testing.yml
> > +++ b/.github/workflows/testing.yml
> > @@ -83,37 +83,93 @@ jobs:
> >           run: cmake --build . --parallel --target LuaJIT-test
> >           working-directory: ${{ env.BUILDDIR }}
> >
> > +  test-tarantool-coverage:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/coverage.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> >
> > -  test-tarantool:
> > -    strategy:
> > -      fail-fast: false
> > -      matrix:
> > -        ARCH: [ARM64, x86_64]
> > -        BUILDTYPE: [Debug, Release]
> > -        GC64: [ON, OFF]
> > -        OS: [Linux, macOS]
> > -        include:
> > -          - BUILDTYPE: Debug
> > -            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
> > -          - BUILDTYPE: Release
> > -            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> > -        exclude:
> > -          - ARCH: ARM64
> > -            GC64: OFF
> > -          - OS: macOS
> > -            GC64: OFF
> > -    name: >
> > -      Tarantool
> > -      (${{ matrix.OS }}/${{ matrix.ARCH }})
> > -      ${{ matrix.BUILDTYPE }}
> > -      GC64:${{ matrix.GC64 }}
> > +  test-tarantool-debug:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/debug.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-debug_aarch64:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/debug_aarch64.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-debug_asan_clang:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/debug_asan_clang.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-default_gcc_centos_7:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/default_gcc_centos_7.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-out_of_source:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/out_of_source.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-release:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/release.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-release_asan_clang:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/release_asan_clang.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-release_clang:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/release_clang.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-release_lto:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/release_lto.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-release_lto_clang:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/release_lto_clang.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-static_build:
> > +    needs: test-luajit
> > +    uses: tarantool/tarantool/.github/workflows/static_build.yml@master
> > +    with:
> > +      submodule: luajit
> > +      revision: ${{ github.sha }}
> > +
> > +  test-tarantool-static_build_cmake_linux:
> >       needs: test-luajit
> > -    uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
> > +    uses: tarantool/tarantool/.github/workflows/static_build_cmake_linux.yml@master
> >       with:
> > -      CMAKE_EXTRA_PARAMS: >
> > -        -G Ninja
> > -        ${{ matrix.CMAKEFLAGS }}
> > -        -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> > -      arch: ${{ matrix.ARCH }}
> > -      os: ${{ matrix.OS }}
> > +      submodule: luajit
> >         revision: ${{ github.sha }}
> > --
> > 2.43.0
> >