* [Tarantool-patches] [PATCH luajit v3] ci: extend tarantool integration testing
@ 2024-03-06 21:47 Maxim Kokryashkin via Tarantool-patches
2024-03-07 15:22 ` Sergey Kaplun via Tarantool-patches
0 siblings, 1 reply; 2+ messages in thread
From: Maxim Kokryashkin via Tarantool-patches @ 2024-03-06 21:47 UTC (permalink / raw)
To: tarantool-patches, skaplun, sergeyb
This patch makes use of reusable workflows from the
Tarantool repository, so now LuaJIT CI tests integration
in all relevant workflows.
Also, this patch disables MacOS testing for Tarantool
integration since those runs were made nightly in the
Tarantool repo.
As noted in the documentation [1], it is impossible for a single
workflow call tree to include more than 20 workflows. To
overcome this limitation, workflows are started in a bunch of
separate files. This makes it impossible to depend on LuaJIT-only
jobs for integration workflows such as `tarantool-ecosystem.yml`
or `tarantool-exotic.yml` since it is impossible to make
cross-dependencies between workflow files.
The name of a callee workflow cannot be substituted using a
matrix [2], so workflow calls are copied and pasted instead.
Table below shows which workflows are enabled and why.
Workflow name |+/-| Reason
-----------------------------------------------------------------
codeql | - | Not relevant to LuaJIT.
coverage | + | Long tests for profilers.
coverity | - | Cron workflow.
debug | + | Tarantool debug build.
debug_aarch64 | + | Tarantool debug build.
debug_asan_clang | + | Tarantool debug build.
default_gcc_centos_7 | + | gcc version (7)
freebsd | - | Nightly build.
fuzzing | - | Impossible to bump LuaJIT.
integration | + | Tarantool ecosystem.
jepsen-cluster-txm | - | Manual workflow.
jepsen-cluster | - | Manual workflow.
jepsen-single-instance-txm | - | Cron workflow.
jepsen-single-instance | - | Cron workflow.
lango-stale-reviews | - | Cron workflow.
lint | - | LuaJIT has its own lint.
luajit-integration | + | Exotic LuaJIT options.
memtx_allocator_based_on_malloc | - | Not relevant to LuaJIT.
osx | - | Nightly build.
out_of_source | + | Out of source build.
packaging | - | No LuaJIT-relevant variety.
perf_cbench | - | Not enabled for PRs.
perf_linkbench_ssd | - | Not enabled for PRs.
perf_micro | - | Not relevant to LuaJIT.
perf_nosqlbench_hash | - | Not enabled for PRs.
perf_nosqlbench_tree | - | Not enabled for PRs.
perf_sysbench | - | Not enabled for PRs.
perf_tpcc | - | Not enabled for PRs.
perf_tpch | - | Not enabled for PRs.
perf_ycsb_hash | - | Not enabled for PRs.
perf_ycsb_tree | - | Not enabled for PRs.
publish-module-api-doc | - | No Doxygen in LuaJIT.
release | + | Tarantool release build.
release_asan_clang | + | Tarantool release build.
release_clang | + | Tarantool release build.
release_lto | + | Tarantool release build.
release_lto_clang | + | Tarantool release build.
reusable_build | - | Utility for integration.
source | - | Not enabled for PRs.
static_build | + | Tarantool static build.
static_build_cmake_linux | - | Just an OOS static build.
static_build_pack_test_deploy | - | Utility for packaging.
submodule_update | - | Not enabled for PRs.
[1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
[2]: https://github.com/orgs/community/discussions/45342#discussioncomment-4779360
---
Changes in v3:
- Fixed comments as per review by Sergey Kaplun
- Fixed comments as per review by Sergey Bronnikov
.github/workflows/tarantool-ecosystem.yml | 39 ++++++++
.github/workflows/tarantool-exotic.yml | 67 +++++++++++++
.github/workflows/testing.yml | 117 ++++++++++++++++------
3 files changed, 193 insertions(+), 30 deletions(-)
create mode 100644 .github/workflows/tarantool-ecosystem.yml
create mode 100644 .github/workflows/tarantool-exotic.yml
diff --git a/.github/workflows/tarantool-ecosystem.yml b/.github/workflows/tarantool-ecosystem.yml
new file mode 100644
index 00000000..d8c49638
--- /dev/null
+++ b/.github/workflows/tarantool-ecosystem.yml
@@ -0,0 +1,39 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# This workflow file was created to avoid this limitation.
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
+name: Tarantool ecosystem integration testing
+
+on:
+ push:
+ branches-ignore:
+ - '**-notest'
+ - 'upstream-**'
+ tags-ignore:
+ - '**'
+
+concurrency:
+ # An update of a developer branch cancels the previously
+ # scheduled workflow run for this branch. However, the default
+ # branch, and long-term branch (tarantool/release/2.11,
+ # tarantool/release/2.10, etc) workflow runs are never canceled.
+ #
+ # We use a trick here: define the concurrency group as 'workflow
+ # run ID' + # 'workflow run attempt' because it is a unique
+ # combination for any run. So it effectively discards grouping.
+ #
+ # XXX: we cannot use `github.sha` as a unique identifier because
+ # pushing a tag may cancel a run that works on a branch push
+ # event.
+ group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
+ && format('{0}-{1}', github.run_id, github.run_attempt)
+ || format('{0}-{1}', github.workflow, github.ref) }}
+ cancel-in-progress: true
+
+jobs:
+ test-tarantool-integration:
+ uses: tarantool/tarantool/.github/workflows/integration.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
diff --git a/.github/workflows/tarantool-exotic.yml b/.github/workflows/tarantool-exotic.yml
new file mode 100644
index 00000000..8214c088
--- /dev/null
+++ b/.github/workflows/tarantool-exotic.yml
@@ -0,0 +1,67 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# This workflow file was created to avoid this limitation.
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
+name: "Tarantool exotic builds testing"
+
+on:
+ push:
+ branches-ignore:
+ - '**-notest'
+ - 'upstream-**'
+ tags-ignore:
+ - '**'
+
+concurrency:
+ # An update of a developer branch cancels the previously
+ # scheduled workflow run for this branch. However, the default
+ # branch, and long-term branch (tarantool/release/2.11,
+ # tarantool/release/2.10, etc) workflow runs are never canceled.
+ #
+ # We use a trick here: define the concurrency group as 'workflow
+ # run ID' + # 'workflow run attempt' because it is a unique
+ # combination for any run. So it effectively discards grouping.
+ #
+ # XXX: we cannot use `github.sha` as a unique identifier because
+ # pushing a tag may cancel a run that works on a branch push
+ # event.
+ group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
+ && format('{0}-{1}', github.run_id, github.run_attempt)
+ || format('{0}-{1}', github.workflow, github.ref) }}
+ cancel-in-progress: true
+
+jobs:
+ test-tarantool-exotic:
+ strategy:
+ fail-fast: false
+ matrix:
+ ARCH: [ARM64, x86_64]
+ BUILDTYPE: [Debug, Release]
+ GC64: [ON, OFF]
+ OS: [Linux, macOS]
+ include:
+ - BUILDTYPE: Debug
+ CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
+ - BUILDTYPE: Release
+ CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
+ exclude:
+ - ARCH: ARM64
+ GC64: OFF
+ - OS: macOS
+ GC64: OFF
+ name: >
+ Tarantool
+ (${{ matrix.OS }}/${{ matrix.ARCH }})
+ ${{ matrix.BUILDTYPE }}
+ GC64:${{ matrix.GC64 }}
+ # XXX: Only LuaJIT-tests are running in this workflow.
+ uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
+ with:
+ CMAKE_EXTRA_PARAMS: >
+ -G Ninja
+ ${{ matrix.CMAKEFLAGS }}
+ -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
+ arch: ${{ matrix.ARCH }}
+ os: ${{ matrix.OS }}
+ revision: ${{ github.sha }}
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index cb4ba57b..b4d037cc 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -1,3 +1,11 @@
+# XXX: A single call tree for reusable workflows can't exceed the
+# total of 20 workflows, as stated in the documentation [1].
+# Some other jobs are started in separate workflow files to
+# overcome the limitation. The jobs are Tarantool ecosystem
+# integration testing (tarantool-ecosystem.yml) and exotic
+# Tarantool builds (tarantool-exotic.yml).
+#
+# [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
name: Testing
on:
@@ -83,37 +91,86 @@ jobs:
run: cmake --build . --parallel --target LuaJIT-test
working-directory: ${{ env.BUILDDIR }}
+ test-tarantool-coverage:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/coverage.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
- test-tarantool:
- strategy:
- fail-fast: false
- matrix:
- ARCH: [ARM64, x86_64]
- BUILDTYPE: [Debug, Release]
- GC64: [ON, OFF]
- OS: [Linux, macOS]
- include:
- - BUILDTYPE: Debug
- CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug
- - BUILDTYPE: Release
- CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
- exclude:
- - ARCH: ARM64
- GC64: OFF
- - OS: macOS
- GC64: OFF
- name: >
- Tarantool
- (${{ matrix.OS }}/${{ matrix.ARCH }})
- ${{ matrix.BUILDTYPE }}
- GC64:${{ matrix.GC64 }}
+ test-tarantool-debug:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/debug.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-debug_aarch64:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/debug_aarch64.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-debug_asan_clang:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/debug_asan_clang.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-default_gcc_centos_7:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/default_gcc_centos_7.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-out_of_source:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/out_of_source.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-release:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/release.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-release_asan_clang:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/release_asan_clang.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-release_clang:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/release_clang.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-release_lto:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/release_lto.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-release_lto_clang:
+ needs: test-luajit
+ uses: tarantool/tarantool/.github/workflows/release_lto_clang.yml@master
+ with:
+ submodule: luajit
+ revision: ${{ github.sha }}
+
+ test-tarantool-static_build:
needs: test-luajit
- uses: tarantool/tarantool/.github/workflows/luajit-integration.yml@master
+ uses: tarantool/tarantool/.github/workflows/static_build.yml@master
with:
- CMAKE_EXTRA_PARAMS: >
- -G Ninja
- ${{ matrix.CMAKEFLAGS }}
- -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
- arch: ${{ matrix.ARCH }}
- os: ${{ matrix.OS }}
+ submodule: luajit
revision: ${{ github.sha }}
--
2.43.2
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [Tarantool-patches] [PATCH luajit v3] ci: extend tarantool integration testing
2024-03-06 21:47 [Tarantool-patches] [PATCH luajit v3] ci: extend tarantool integration testing Maxim Kokryashkin via Tarantool-patches
@ 2024-03-07 15:22 ` Sergey Kaplun via Tarantool-patches
0 siblings, 0 replies; 2+ messages in thread
From: Sergey Kaplun via Tarantool-patches @ 2024-03-07 15:22 UTC (permalink / raw)
To: Maxim Kokryashkin; +Cc: tarantool-patches
Hi, Max!
Thanks for the fixes!
Please, consider my general comments below.
1) Now we have 6 types of names (please correct me if I forgot something):
* Build test ... / LuaJIT ...
* Exotic builds testing / LuaJIT ...
* Sanitizers testing / LuaJIT ...
* Static analysis / LuaJIT linters
* Testing / LuaJIT ...
* Testing / Tarantool ...
As part of the patch:
I) "Testing / Tarantool ..." is renamed to "Tarantool exotic builds testing / Tarantool ...".
I do not agree that these are exotic builds -- these are ordinary
builds, where we run the integration of LuaJIT tests under Tarantool.
Plus, there is duplicate information that this is Tarantool-related
testing.
Perhaps we should not change the name of the jobs, meaning that by all
"Testing" we mean LuaJIT tests.
II) We bring all sorts of other things from Tarantool:
* Testing / test-tarantool-static_build / static_build (push)
* Tarantool ecosystem integration testing / test-tarantool-integration / tarantool-python / run_tests (push)
Here, we have a lot of names that are mixed together and don’t look
consistent with the older ones. I would suggest at least starting to fit
into the overall picture by renaming it like this, using the following
naming (if I understand correctly that we cannot change the job names at
the end):
* "Integration / Tarantool static_build" or
"Integration / Tarantool / static_build"
* "Integration / Tarantool ecosystem / ..." or
"Integration / Tarantool / ecosystem / ..."
I like the first mentioned options more than the second.
2) About all "Integration" workflows mentioned above: I suggest file naming
like integration-tarantool-ecosystem.yml etc. for consistency.
3) It would be nice to have an option to skip integration testing.
This can be done later. I just thought that sometimes it would be useful
to specify the "-no-integration" branch suffix and then not run
integration tests with Tarantool (namely "Integration", not our tests
under Tarantool) - useful if you are debugging only our tests.
On 07.03.24, Maxim Kokryashkin wrote:
<snipped>
>
> Workflow name |+/-| Reason
> -----------------------------------------------------------------
> codeql | - | Not relevant to LuaJIT.
> coverage | + | Long tests for profilers.
> coverity | - | Cron workflow.
> debug | + | Tarantool debug build.
> debug_aarch64 | + | Tarantool debug build.
> debug_asan_clang | + | Tarantool debug build.
> default_gcc_centos_7 | + | gcc version (7)
Minor: Old gcc version?
> freebsd | - | Nightly build.
> fuzzing | - | Impossible to bump LuaJIT.
> integration | + | Tarantool ecosystem.
> jepsen-cluster-txm | - | Manual workflow.
> jepsen-cluster | - | Manual workflow.
> jepsen-single-instance-txm | - | Cron workflow.
> jepsen-single-instance | - | Cron workflow.
> lango-stale-reviews | - | Cron workflow.
> lint | - | LuaJIT has its own lint.
> luajit-integration | + | Exotic LuaJIT options.
I suggest the following reasoning: "Run tests under Tarantool" or
something like that.
> memtx_allocator_based_on_malloc | - | Not relevant to LuaJIT.
> osx | - | Nightly build.
> out_of_source | + | Out of source build.
> packaging | - | No LuaJIT-relevant variety.
> perf_cbench | - | Not enabled for PRs.
> perf_linkbench_ssd | - | Not enabled for PRs.
> perf_micro | - | Not relevant to LuaJIT.
> perf_nosqlbench_hash | - | Not enabled for PRs.
> perf_nosqlbench_tree | - | Not enabled for PRs.
> perf_sysbench | - | Not enabled for PRs.
> perf_tpcc | - | Not enabled for PRs.
> perf_tpch | - | Not enabled for PRs.
> perf_ycsb_hash | - | Not enabled for PRs.
> perf_ycsb_tree | - | Not enabled for PRs.
> publish-module-api-doc | - | No Doxygen in LuaJIT.
> release | + | Tarantool release build.
> release_asan_clang | + | Tarantool release build.
> release_clang | + | Tarantool release build.
> release_lto | + | Tarantool release build.
> release_lto_clang | + | Tarantool release build.
> reusable_build | - | Utility for integration.
> source | - | Not enabled for PRs.
> static_build | + | Tarantool static build.
> static_build_cmake_linux | - | Just an OOS static build.
> static_build_pack_test_deploy | - | Utility for packaging.
> submodule_update | - | Not enabled for PRs.
>
> [1]: https://docs.github.com/en/actions/using-workflows/reusing-workflows#limitations
> [2]: https://github.com/orgs/community/discussions/45342#discussioncomment-4779360
> ---
> Changes in v3:
> - Fixed comments as per review by Sergey Kaplun
> - Fixed comments as per review by Sergey Bronnikov
>
> .github/workflows/tarantool-ecosystem.yml | 39 ++++++++
> .github/workflows/tarantool-exotic.yml | 67 +++++++++++++
> .github/workflows/testing.yml | 117 ++++++++++++++++------
<snipped>
--
Best regards,
Sergey Kaplun
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2024-03-07 15:26 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-06 21:47 [Tarantool-patches] [PATCH luajit v3] ci: extend tarantool integration testing Maxim Kokryashkin via Tarantool-patches
2024-03-07 15:22 ` Sergey Kaplun via Tarantool-patches
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox