[Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Maksim Kokryashkin via Tarantool-patches]

From: Maxim Kokryashkin <m.kokryashkin@tarantool.org>

This patch makes use of reusable workflows from the
Tarantool repository, so now LuaJIT CI tests integration
in all relevant workflows.

Also, this patch disables MacOS testing for Tarantool
integration since those runs were made nightly in the
Tarantool repo.

As noted in the documentation [1], it is impossible for a single
workflow call tree to include more than 20 workflows. To
overcome this limitation, workflows are started in a bunch of
separate files. This makes it impossible to depend on LuaJIT-only
jobs for integration workflows such as `tarantool-ecosystem.yml`
or `tarantool-exotic.yml` since it is impossible to make
cross-dependencies between workflow files.

The name of a callee workflow cannot be substituted using a
matrix [2], so workflow calls are copied and pasted instead.

Table below shows which workflows are enabled and why.

Workflow name                   |+/-| Reason
-----------------------------------------------------------------
codeql                          | - | Not relevant to LuaJIT.
coverage                        | + | Long tests for profilers.
coverity                        | - | Cron workflow.
debug                           | + | Tarantool debug build.
debug_aarch64                   | + | Tarantool debug build.
debug_asan_clang                | + | Tarantool debug build.
default_gcc_centos_7            | + | Old gcc version (7).
freebsd                         | - | Nightly build.
fuzzing                         | - | Impossible to bump LuaJIT.
integration                     | + | Tarantool ecosystem.
jepsen-cluster-txm              | - | Manual workflow.
jepsen-cluster                  | - | Manual workflow.
jepsen-single-instance-txm      | - | Cron workflow.
jepsen-single-instance          | - | Cron workflow.
lango-stale-reviews             | - | Cron workflow.
lint                            | - | LuaJIT has its own lint.
luajit-integration              | + | Run tests under Tarantool.
memtx_allocator_based_on_malloc | - | Not relevant to LuaJIT.
osx                             | - | Nightly build.
out_of_source                   | + | Out of source build.
packaging                       | - | No LuaJIT-relevant variety.
perf_cbench                     | - | Not enabled for PRs.
perf_linkbench_ssd              | - | Not enabled for PRs.
perf_micro                      | - | Not relevant to LuaJIT.
perf_nosqlbench_hash            | - | Not enabled for PRs.
perf_nosqlbench_tree            | - | Not enabled for PRs.
perf_sysbench                   | - | Not enabled for PRs.
perf_tpcc                       | - | Not enabled for PRs.
perf_tpch                       | - | Not enabled for PRs.
perf_ycsb_hash                  | - | Not enabled for PRs.
perf_ycsb_tree                  | - | Not enabled for PRs.
publish-module-api-doc          | - | No Doxygen in LuaJIT.
release                         | + | Tarantool release build.
release_asan_clang              | + | Tarantool release build.
release_clang                   | + | Tarantool release build.
release_lto                     | + | Tarantool release build.
release_lto_clang               | + | Tarantool release build.
reusable_build                  | - | Utility for integration.
source                          | - | Not enabled for PRs.
static_build                    | + | Tarantool static build.
static_build_cmake_linux        | - | Just an OOS static build.
static_build_pack_test_deploy   | - | Utility for packaging.
submodule_update                | - | Not enabled for PRs.

[1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
[2]: https://github.com/orgs/community/discussions/45342#discussioncomment-4779360
---

Changes in v5:
- Added secrets section for integration-tarantool-ecosystem.yml to
  resolve the issue with etcd-client integration.

Branch: https://github.com/tarantool/luajit/tree/fckxorg/integration-testing

 .../integration-tarantool-ecosystem.yml       |  41 ++++++
 .github/workflows/integration-tarantool.yml   | 118 ++++++++++++++++++
 .github/workflows/testing.yml                 |   9 ++
 3 files changed, 168 insertions(+)
 create mode 100644 .github/workflows/integration-tarantool-ecosystem.yml
 create mode 100644 .github/workflows/integration-tarantool.yml

diff --git a/.github/workflows/integration-tarantool-ecosystem.yml b/.github/workflows/integration-tarantool-ecosystem.yml
new file mode 100644
index 00000000..ccafd601
--- /dev/null
+++ b/.github/workflows/integration-tarantool-ecosystem.yml
@@ -0,0 +1,41 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# This workflow file was created to avoid this limitation.
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
+name: 'Integration / Tarantool ecosystem'
+
+on:
+  push:
+    branches-ignore:
+      - '**-notest'
+      - '**-nointegration'
+      - 'upstream-**'
+    tags-ignore:
+      - '**'
+
+concurrency:
+  # An update of a developer branch cancels the previously
+  # scheduled workflow run for this branch. However, the default
+  # branch, and long-term branch (tarantool/release/2.11,
+  # tarantool/release/2.10, etc) workflow runs are never canceled.
+  #
+  # We use a trick here: define the concurrency group as 'workflow
+  # run ID' + # 'workflow run attempt' because it is a unique
+  # combination for any run. So it effectively discards grouping.
+  #
+  # XXX: we cannot use `github.sha` as a unique identifier because
+  # pushing a tag may cancel a run that works on a branch push
+  # event.
+  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
+    && format('{0}-{1}', github.run_id, github.run_attempt)
+    || format('{0}-{1}', github.workflow, github.ref) }}
+  cancel-in-progress: true
+
+jobs:
+  test-tarantool-integration:
+    uses: tarantool/tarantool/.github/workflows/integration.yml@master
+    secrets: inherit
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
diff --git a/.github/workflows/integration-tarantool.yml b/.github/workflows/integration-tarantool.yml
new file mode 100644
index 00000000..cf5c7fb5
--- /dev/null
+++ b/.github/workflows/integration-tarantool.yml
@@ -0,0 +1,118 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# This workflow file was created to avoid this limitation.
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
+name: 'Integration / Tarantool'
+
+on:
+  push:
+    branches-ignore:
+      - '**-notest'
+      - '**-nointegration'
+      - 'upstream-**'
+    tags-ignore:
+      - '**'
+
+concurrency:
+  # An update of a developer branch cancels the previously
+  # scheduled workflow run for this branch. However, the default
+  # branch, and long-term branch (tarantool/release/2.11,
+  # tarantool/release/2.10, etc) workflow runs are never canceled.
+  #
+  # We use a trick here: define the concurrency group as 'workflow
+  # run ID' + # 'workflow run attempt' because it is a unique
+  # combination for any run. So it effectively discards grouping.
+  #
+  # XXX: we cannot use `github.sha` as a unique identifier because
+  # pushing a tag may cancel a run that works on a branch push
+  # event.
+  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
+    && format('{0}-{1}', github.run_id, github.run_attempt)
+    || format('{0}-{1}', github.workflow, github.ref) }}
+  cancel-in-progress: true
+
+jobs:
+  test-tarantool-coverage:
+    name: coverage
+    uses: tarantool/tarantool/.github/workflows/coverage.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-debug:
+    name: debug
+    uses: tarantool/tarantool/.github/workflows/debug.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-debug_aarch64:
+    name: debug aarch64
+    uses: tarantool/tarantool/.github/workflows/debug_aarch64.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-debug_asan_clang:
+    name: debug ASAN clang
+    uses: tarantool/tarantool/.github/workflows/debug_asan_clang.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-default_gcc_centos_7:
+    name: gcc centos 7
+    uses: tarantool/tarantool/.github/workflows/default_gcc_centos_7.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-out_of_source:
+    name: out of source
+    uses: tarantool/tarantool/.github/workflows/out_of_source.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-release:
+    name: release
+    uses: tarantool/tarantool/.github/workflows/release.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-release_asan_clang:
+    name: release ASAN clang
+    uses: tarantool/tarantool/.github/workflows/release_asan_clang.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-release_clang:
+    name: release clang
+    uses: tarantool/tarantool/.github/workflows/release_clang.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-release_lto:
+    name: release lto
+    uses: tarantool/tarantool/.github/workflows/release_lto.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-release_lto_clang:
+    name: release lto clang
+    uses: tarantool/tarantool/.github/workflows/release_lto_clang.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
+
+  test-tarantool-static_build:
+    name: static build
+    uses: tarantool/tarantool/.github/workflows/static_build.yml@master
+    with:
+      submodule: luajit
+      revision: ${{ github.sha }}
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index cb4ba57b..1581c8fc 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -1,3 +1,11 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# Some other jobs are started in separate workflow files to
+# overcome the limitation. The jobs are Tarantool ecosystem
+# integration testing (tarantool-ecosystem.yml) and exotic
+# Tarantool builds (tarantool-exotic.yml).
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
 name: Testing

 on:
@@ -108,6 +116,7 @@ jobs:
       ${{ matrix.BUILDTYPE }}
       GC64:${{ matrix.GC64 }}
     needs: test-luajit
+    # XXX: Only LuaJIT-tests are running in this workflow.
     uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
     with:
       CMAKE_EXTRA_PARAMS: >
--
2.39.3 (Apple Git-145)

Re: [Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Sergey Bronnikov via Tarantool-patches]

Max,


On 3/11/24 23:04, Maksim Kokryashkin wrote:
> From: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
>
> This patch makes use of reusable workflows from the
> Tarantool repository, so now LuaJIT CI tests integration
> in all relevant workflows.
>
thanks for the patch! LGTM


<snipped>

Re: [Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Sergey Kaplun via Tarantool-patches]

Hi, Max!
Patch LGTM, except a few nits regarding the commit message and
comments.

Also, may you please provide two separate patches regarding backporting
to the tarantool/release/3.0 and tarantool/release/2.11 branches?

On 11.03.24, Maksim Kokryashkin wrote:
> From: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
> 
> This patch makes use of reusable workflows from the
> Tarantool repository, so now LuaJIT CI tests integration
> in all relevant workflows.
> 
> Also, this patch disables MacOS testing for Tarantool
> integration since those runs were made nightly in the
> Tarantool repo.
> 
> As noted in the documentation [1], it is impossible for a single
> workflow call tree to include more than 20 workflows. To
> overcome this limitation, workflows are started in a bunch of
> separate files. This makes it impossible to depend on LuaJIT-only
> jobs for integration workflows such as `tarantool-ecosystem.yml`
> or `tarantool-exotic.yml` since it is impossible to make

Nit: I suppose it should be "<integration-tarantool-ecosystem.yml> or
<integration-tarantool.yml>".

> cross-dependencies between workflow files.
> 

<snipped>

> ---
> 
> Changes in v5:
> - Added secrets section for integration-tarantool-ecosystem.yml to
>   resolve the issue with etcd-client integration.
> 
> Branch: https://github.com/tarantool/luajit/tree/fckxorg/integration-testing
> 
>  .../integration-tarantool-ecosystem.yml       |  41 ++++++
>  .github/workflows/integration-tarantool.yml   | 118 ++++++++++++++++++
>  .github/workflows/testing.yml                 |   9 ++
>  3 files changed, 168 insertions(+)
>  create mode 100644 .github/workflows/integration-tarantool-ecosystem.yml
>  create mode 100644 .github/workflows/integration-tarantool.yml
> 
> diff --git a/.github/workflows/integration-tarantool-ecosystem.yml b/.github/workflows/integration-tarantool-ecosystem.yml
> new file mode 100644
> index 00000000..ccafd601
> --- /dev/null
> +++ b/.github/workflows/integration-tarantool-ecosystem.yml

<snipped>

> diff --git a/.github/workflows/integration-tarantool.yml b/.github/workflows/integration-tarantool.yml
> new file mode 100644
> index 00000000..cf5c7fb5
> --- /dev/null
> +++ b/.github/workflows/integration-tarantool.yml

<snipped>

> diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
> index cb4ba57b..1581c8fc 100644
> --- a/.github/workflows/testing.yml
> +++ b/.github/workflows/testing.yml
> @@ -1,3 +1,11 @@
> +# XXX: A single call tree for reusable workflows can't exceed the
> +# total of 20 workflows, as stated in the documentation [1].
> +# Some other jobs are started in separate workflow files to
> +# overcome the limitation. The jobs are Tarantool ecosystem
> +# integration testing (tarantool-ecosystem.yml) and exotic
> +# Tarantool builds (tarantool-exotic.yml).

Nit: I suppose it should be "integration testing of Tarantool
<integration-tarantool.yml> and its ecosystem
<integration-tarantool-ecosystem.yml>".

> +#
> +# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
>  name: Testing
> 
>  on:
> @@ -108,6 +116,7 @@ jobs:

<snipped>

> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Maxim Kokryashkin via Tarantool-patches]

Hi, Sergey!
Thanks for the review!
Fixed, branch is force-pushed.
On Wed, Mar 13, 2024 at 10:18:02AM +0300, Sergey Kaplun wrote:
> Hi, Max!
> Patch LGTM, except a few nits regarding the commit message and
> comments.
>
> Also, may you please provide two separate patches regarding backporting
> to the tarantool/release/3.0 and tarantool/release/2.11 branches?
>
> On 11.03.24, Maksim Kokryashkin wrote:
> > From: Maxim Kokryashkin <m.kokryashkin@tarantool.org>
> >
> > This patch makes use of reusable workflows from the
> > Tarantool repository, so now LuaJIT CI tests integration
> > in all relevant workflows.
> >
> > Also, this patch disables MacOS testing for Tarantool
> > integration since those runs were made nightly in the
> > Tarantool repo.
> >
> > As noted in the documentation [1], it is impossible for a single
> > workflow call tree to include more than 20 workflows. To
> > overcome this limitation, workflows are started in a bunch of
> > separate files. This makes it impossible to depend on LuaJIT-only
> > jobs for integration workflows such as `tarantool-ecosystem.yml`
> > or `tarantool-exotic.yml` since it is impossible to make
>
> Nit: I suppose it should be "<integration-tarantool-ecosystem.yml> or
> <integration-tarantool.yml>".
>
> > cross-dependencies between workflow files.
Changed this part to the following:
| As noted in the documentation [1], it is impossible for a single
| workflow call tree to include more than 20 workflows. To overcome
| this limitation, workflows are started in a bunch of separate
| files. This makes it impossible to depend on LuaJIT-only jobs for
| integration workflows such as <integration-tarantool.yml> or
| <integration-tarantool-ecosystem.yml> since it is impossible to
| make cross-dependencies between workflow files.

> >
>
> <snipped>
>
> > ---
> >
> > Changes in v5:
> > - Added secrets section for integration-tarantool-ecosystem.yml to
> >   resolve the issue with etcd-client integration.
> >
> > Branch: https://github.com/tarantool/luajit/tree/fckxorg/integration-testing
> >
> >  .../integration-tarantool-ecosystem.yml       |  41 ++++++
> >  .github/workflows/integration-tarantool.yml   | 118 ++++++++++++++++++
> >  .github/workflows/testing.yml                 |   9 ++
> >  3 files changed, 168 insertions(+)
> >  create mode 100644 .github/workflows/integration-tarantool-ecosystem.yml
> >  create mode 100644 .github/workflows/integration-tarantool.yml
> >
> > diff --git a/.github/workflows/integration-tarantool-ecosystem.yml b/.github/workflows/integration-tarantool-ecosystem.yml
> > new file mode 100644
> > index 00000000..ccafd601
> > --- /dev/null
> > +++ b/.github/workflows/integration-tarantool-ecosystem.yml
>
> <snipped>
>
> > diff --git a/.github/workflows/integration-tarantool.yml b/.github/workflows/integration-tarantool.yml
> > new file mode 100644
> > index 00000000..cf5c7fb5
> > --- /dev/null
> > +++ b/.github/workflows/integration-tarantool.yml
>
> <snipped>
>
> > diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
> > index cb4ba57b..1581c8fc 100644
> > --- a/.github/workflows/testing.yml
> > +++ b/.github/workflows/testing.yml
> > @@ -1,3 +1,11 @@
> > +# XXX: A single call tree for reusable workflows can't exceed the
> > +# total of 20 workflows, as stated in the documentation [1].
> > +# Some other jobs are started in separate workflow files to
> > +# overcome the limitation. The jobs are Tarantool ecosystem
> > +# integration testing (tarantool-ecosystem.yml) and exotic
> > +# Tarantool builds (tarantool-exotic.yml).
>
> Nit: I suppose it should be "integration testing of Tarantool
> <integration-tarantool.yml> and its ecosystem
> <integration-tarantool-ecosystem.yml>".
Fixed.
Here is the diff:
===
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index 1581c8fc..59761963 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -1,9 +1,9 @@
 # XXX: A single call tree for reusable workflows can't exceed the
 # total of 20 workflows, as stated in the documentation [1].
 # Some other jobs are started in separate workflow files to
-# overcome the limitation. The jobs are Tarantool ecosystem
-# integration testing (tarantool-ecosystem.yml) and exotic
-# Tarantool builds (tarantool-exotic.yml).
+# overcome the limitation. The jobs are integration testing of
+# Tarantool <integration-tarantool.yml> and its ecosystem
+# <integration-tarantool-ecosystem.yml>.
 #
 # [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
 name: Testing
===
>
> > +#
> > +# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
> >  name: Testing
> >
> >  on:
> > @@ -108,6 +116,7 @@ jobs:
>
> <snipped>
>
> >
>
> --
> Best regards,
> Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit v5] ci: extend tarantool integration testing

[Sergey Kaplun via Tarantool-patches]

Maxim,

I've checked the patch into tarantool/master branch in
tarantool/luajit and bumped a new version in master [1].

[1]: https://github.com/tarantool/tarantool/pull/9832

-- 
Best regards,
Sergey Kaplun