Re: [Tarantool-patches] [PATCH luajit] ci: extend tarantool integration testing

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Hi, Max

thanks for updates. LGTM with minor comments below.


On 3/1/24 11:25, Maxim Kokryashkin wrote:
> This patch makes use of reusable workflows from the
> Tarantool repository, so now LuaJIT CI tests integration
> in all relevant workflows.
>
> Also, this patch disables MacOS testing for Tarantool
> integration since those runs were made nightly in the
> Tarantool repo.
>
> As noted in the documentation[1], it is impossible for a single
> workflow call tree to include more than 20 workflows. To
> overcome this limitation, workflows are started in a bunch of
> separate files. This makes it impossible to depend on LuaJIT-only
> jobs for integration workflow since it is impossible to make
> cross-dependencies between workflow files.
>
> The name of a callee workflow cannot be substituted using a
> matrix[2], so workflow calls are copied and pasted instead.
>
> Table below shows which workflows are enabled and why.
>
> Workflow name                   |+/-| Reason
> -----------------------------------------------------------------
> codeql                          | - | Not relevant to LuaJIT.
> coverage                        | + | Long tests for profilers.
> coverity                        | - | Cron workflow.
> debug                           | + | Tarantool debug build.
> debug_aarch64                   | + | Tarantool debug build.
> debug_asan_clang                | + | Tarantool debug build.
> default_gcc_centos_7            | + | Ancient gcc build.
better: "gcc version". Feel free to ignore.
> freebsd                         | - | Nightly build.
> fuzzing                         | - | Impossible to bump LuaJIT.
> integration                     | + | Tarantool ecosystem.
> jepsen-cluster-txm              | - | Manual workflow.
> jepsen-cluster                  | - | Manual workflow.
> jepsen-single-instance-txm      | - | Cron workflow.
> jepsen-single-instance          | - | Cron workflow.
> lango-stale-reviews             | - | Cron workflow.
> lint                            | - | LuaJIT has its own lint.
> luajit-integration              | + | Exotic LuaJIT options.
> memtx_allocator_based_on_malloc | + | Not relevant to LuaJIT.
typo: s/+/-/
> osx                             | - | Nightly build.
> out_of_source                   | - | Not relevent to LuaJIT.
> packaging                       | - | No LuaJIT-relevant variety.
> perf_cbench                     | - | Not enabled for PRs.
> perf_linkbench_ssd              | - | Not enabled for PRs.
> perf_micro                      | - | Not relevant to LuaJIT.
> perf_nosqlbench_hash            | - | Not enabled for PRs.
> perf_nosqlbench_tree            | - | Not enabled for PRs.
> perf_sysbench                   | - | Not enabled for PRs.
> perf_tpcc                       | - | Not enabled for PRs.
> perf_tpch                       | - | Not enabled for PRs.
> perf_ycsb_hash                  | - | Not enabled for PRs.
> perf_ycsb_tree                  | - | Not enabled for PRs.
> publish-module-api-doc          | - | No Doxygen in LuaJIT.
> release                         | + | Tarantool release build.
> release_asan_clang              | + | Tarantool release build.
> release_clang                   | + | Tarantool release build.
> release_lto                     | + | Tarantool release build.
> release_lto_clang               | + | Tarantool release build.
> reusable_build                  | - | Utility for integration.
> source                          | - | Not enabled for PRs.
> static_build                    | + | Tarantool static build.
> static_build_cmake_linux        | + | Tarantool static build.
> static_build_pack_test_deploy   | - | Utility for packaging.

I would left "static_build" [1] and skip "static_build_cmake_linux" [2].

I don't see a big difference for static_build and 
static_build_cmake_linux, build commands are the same,

both workflows are running on ubuntu-20.04-self-hosted. The difference 
is in builddir location:

first is in source build, the second one is out of source build, see [4].


1. 
https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.test.mk#L165-L171

2. 
https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.github/workflows/static_build_cmake_linux.yml#L1

4. 
https://github.com/tarantool/tarantool/blob/f267af24058c7cb49bf2a6403b1f3c51b41d22ae/.test.mk#L156-L171

> submodule_update                | - | Not enabled for PRs.
>
> [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
> [2]: https://github.com/orgs/community/discussions/45342#discussioncomment-4779360
> ---
> Branch: https://github.com/tarantool/luajit/tree/fckxorg/integration-testing
>
>   .github/workflows/exotic-tarantool.yml    |  62 ++++++++++++
>   .github/workflows/tarantool-ecosystem.yml |  35 +++++++
>   .github/workflows/testing.yml             | 116 ++++++++++++++++------
>   3 files changed, 183 insertions(+), 30 deletions(-)
>   create mode 100644 .github/workflows/exotic-tarantool.yml
>   create mode 100644 .github/workflows/tarantool-ecosystem.yml
>
> diff --git a/.github/workflows/exotic-tarantool.yml b/.github/workflows/exotic-tarantool.yml
> new file mode 100644
> index 00000000..be55291a
> --- /dev/null
> +++ b/.github/workflows/exotic-tarantool.yml
> @@ -0,0 +1,62 @@
> +name: "Exotic Tarantool builds testing"
> +
> +on:
> +  push:
> +    branches-ignore:
> +      - '**-notest'
> +      - 'upstream-**'
> +    tags-ignore:
> +      - '**'
> +
> +concurrency:
> +  # An update of a developer branch cancels the previously
> +  # scheduled workflow run for this branch. However, the default
> +  # branch, and long-term branch (tarantool/release/2.11,
> +  # tarantool/release/2.10, etc) workflow runs are never canceled.
> +  #
> +  # We use a trick here: define the concurrency group as 'workflow
> +  # run ID' + # 'workflow run attempt' because it is a unique
> +  # combination for any run. So it effectively discards grouping.
> +  #
> +  # XXX: we cannot use `github.sha` as a unique identifier because
> +  # pushing a tag may cancel a run that works on a branch push
> +  # event.
> +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
> +    && format('{0}-{1}', github.run_id, github.run_attempt)
> +    || format('{0}-{1}', github.workflow, github.ref) }}
> +  cancel-in-progress: true
> +
> +jobs:
> +  test-exotic-tarantool:
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        ARCH: [ARM64, x86_64]
> +        BUILDTYPE: [Debug, Release]
> +        GC64: [ON, OFF]
> +        OS: [Linux, macOS]
> +        include:
> +          - BUILDTYPE: Debug
> +            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
> +          - BUILDTYPE: Release
> +            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> +        exclude:
> +          - ARCH: ARM64
> +            GC64: OFF
> +          - OS: macOS
> +            GC64: OFF
> +    name: >
> +      Tarantool
> +      (${{ matrix.OS }}/${{ matrix.ARCH }})
> +      ${{ matrix.BUILDTYPE }}
> +      GC64:${{ matrix.GC64 }}
> +    # XXX: Only LuaJIT-tests are running in this workflow.
> +    uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
> +    with:
> +      CMAKE_EXTRA_PARAMS: >
> +        -G Ninja
> +        ${{ matrix.CMAKEFLAGS }}
> +        -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> +      arch: ${{ matrix.ARCH }}
> +      os: ${{ matrix.OS }}
> +      revision: ${{ github.sha }}
> diff --git a/.github/workflows/tarantool-ecosystem.yml b/.github/workflows/tarantool-ecosystem.yml
> new file mode 100644
> index 00000000..82a47fe1
> --- /dev/null
> +++ b/.github/workflows/tarantool-ecosystem.yml
> @@ -0,0 +1,35 @@
> +name: Tarantool ecosystem integration testing
> +
> +on:
> +  push:
> +    branches-ignore:
> +      - '**-notest'
> +      - 'upstream-**'
> +    tags-ignore:
> +      - '**'
> +
> +concurrency:
> +  # An update of a developer branch cancels the previously
> +  # scheduled workflow run for this branch. However, the default
> +  # branch, and long-term branch (tarantool/release/2.11,
> +  # tarantool/release/2.10, etc) workflow runs are never canceled.
> +  #
> +  # We use a trick here: define the concurrency group as 'workflow
> +  # run ID' + # 'workflow run attempt' because it is a unique
> +  # combination for any run. So it effectively discards grouping.
> +  #
> +  # XXX: we cannot use `github.sha` as a unique identifier because
> +  # pushing a tag may cancel a run that works on a branch push
> +  # event.
> +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
> +    && format('{0}-{1}', github.run_id, github.run_attempt)
> +    || format('{0}-{1}', github.workflow, github.ref) }}
> +  cancel-in-progress: true
> +
> +jobs:
> +  test-tarantool-integration:
> +    uses: tarantool/tarantool/.github/workflows/integration.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
> index cb4ba57b..7bd527be 100644
> --- a/.github/workflows/testing.yml
> +++ b/.github/workflows/testing.yml
> @@ -83,37 +83,93 @@ jobs:
>           run: cmake --build . --parallel --target LuaJIT-test
>           working-directory: ${{ env.BUILDDIR }}
>
> +  test-tarantool-coverage:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/coverage.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
>
> -  test-tarantool:
> -    strategy:
> -      fail-fast: false
> -      matrix:
> -        ARCH: [ARM64, x86_64]
> -        BUILDTYPE: [Debug, Release]
> -        GC64: [ON, OFF]
> -        OS: [Linux, macOS]
> -        include:
> -          - BUILDTYPE: Debug
> -            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
> -          - BUILDTYPE: Release
> -            CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> -        exclude:
> -          - ARCH: ARM64
> -            GC64: OFF
> -          - OS: macOS
> -            GC64: OFF
> -    name: >
> -      Tarantool
> -      (${{ matrix.OS }}/${{ matrix.ARCH }})
> -      ${{ matrix.BUILDTYPE }}
> -      GC64:${{ matrix.GC64 }}
> +  test-tarantool-debug:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/debug.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-debug_aarch64:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/debug_aarch64.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-debug_asan_clang:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/debug_asan_clang.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-default_gcc_centos_7:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/default_gcc_centos_7.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-out_of_source:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/out_of_source.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-release:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/release.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-release_asan_clang:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/release_asan_clang.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-release_clang:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/release_clang.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-release_lto:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/release_lto.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-release_lto_clang:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/release_lto_clang.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-static_build:
> +    needs: test-luajit
> +    uses: tarantool/tarantool/.github/workflows/static_build.yml@master
> +    with:
> +      submodule: luajit
> +      revision: ${{ github.sha }}
> +
> +  test-tarantool-static_build_cmake_linux:
>       needs: test-luajit
> -    uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
> +    uses: tarantool/tarantool/.github/workflows/static_build_cmake_linux.yml@master
>       with:
> -      CMAKE_EXTRA_PARAMS: >
> -        -G Ninja
> -        ${{ matrix.CMAKEFLAGS }}
> -        -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> -      arch: ${{ matrix.ARCH }}
> -      os: ${{ matrix.OS }}
> +      submodule: luajit
>         revision: ${{ github.sha }}
> --
> 2.43.0
>