[Tarantool-patches] [PATCH luajit 8/8] ci: merge Linux and macOS workflows
Sergey Bronnikov
sergeyb at tarantool.org
Mon Aug 15 15:27:36 MSK 2022
LGTM, thanks
I'm glad to see lesser number of GH workflows :)
On 11.08.2022 14:17, Igor Munkin wrote:
> As a result of two previous commits, there is no difference left between
> Linux and macOS workflows except the environment setup (environment
> variables, dependencies installation). Hence there is no reason to have a
> separate workflow file for each OS type.
>
> Signed-off-by: Igor Munkin <imun at tarantool.org>
> ---
> .github/actions/environment/action.yml | 18 ---
> .github/actions/setup-linux/README.md | 12 ++
> .github/actions/setup-linux/action.yml | 19 +++
> .github/actions/setup-macos/README.md | 12 ++
> .github/actions/setup-macos/action.yml | 29 ++++
> .../actions/{environment => setup}/README.md | 5 +-
> .github/actions/setup/action.yml | 10 ++
> .github/workflows/lint.yml | 8 +-
> .github/workflows/linux-x86_64-ninja.yml | 6 +-
> .github/workflows/macos.yml | 142 ------------------
> .github/workflows/{linux.yml => testing.yml} | 113 +++++++++++---
> 11 files changed, 186 insertions(+), 188 deletions(-)
> delete mode 100644 .github/actions/environment/action.yml
> create mode 100644 .github/actions/setup-linux/README.md
> create mode 100644 .github/actions/setup-linux/action.yml
> create mode 100644 .github/actions/setup-macos/README.md
> create mode 100644 .github/actions/setup-macos/action.yml
> rename .github/actions/{environment => setup}/README.md (72%)
> create mode 100644 .github/actions/setup/action.yml
> delete mode 100644 .github/workflows/macos.yml
> rename .github/workflows/{linux.yml => testing.yml} (52%)
>
> diff --git a/.github/actions/environment/action.yml b/.github/actions/environment/action.yml
> deleted file mode 100644
> index 7fb2625f..00000000
> --- a/.github/actions/environment/action.yml
> +++ /dev/null
> @@ -1,18 +0,0 @@
> -name: 'Setup CI environment'
> -description: 'Common part to tweak CI runner environment'
> -runs:
> - using: 'composite'
> - steps:
> - - run: |
> - # Set CMAKE_BUILD_PARALLEL_LEVEL environment variable to
> - # limit the number of parallel jobs for build/test step.
> - # Try the exotic Darwin way at first and fallback to nproc
> - # that is the default for the normal systems.
> - NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null || nproc)
> - echo CMAKE_BUILD_PARALLEL_LEVEL=$(($NPROC + 1)) | tee -a $GITHUB_ENV
> - shell: bash
> - - run: |
> - # Set BUILDDIR environment variable to specify LuaJIT
> - # build directory.
> - echo "BUILDDIR=${{ runner.temp }}/build-${{ github.run_id }}" | tee -a $GITHUB_ENV
> - shell: bash
> diff --git a/.github/actions/setup-linux/README.md b/.github/actions/setup-linux/README.md
> new file mode 100644
> index 00000000..48ae9e01
> --- /dev/null
> +++ b/.github/actions/setup-linux/README.md
> @@ -0,0 +1,12 @@
> +# Setup environment on Linux
> +
> +Action setups the environment on Linux runners (install requirements, setup the
> +workflow environment, etc).
> +
> +## How to use Github Action from Github workflow
> +
> +Add the following code to the running steps before LuaJIT configuration:
> +```
> +- uses: ./.github/actions/setup-linux
> + if: ${{ matrix.OS == 'Linux' }}
> +```
> diff --git a/.github/actions/setup-linux/action.yml b/.github/actions/setup-linux/action.yml
> new file mode 100644
> index 00000000..a13f143c
> --- /dev/null
> +++ b/.github/actions/setup-linux/action.yml
> @@ -0,0 +1,19 @@
> +name: Setup CI environment on Linux
> +description: Common part to tweak Linux CI runner environment
> +runs:
> + using: composite
> + steps:
> + - name: Setup CI environment
> + uses: ./.github/actions/setup
> + - name: Set CMAKE_BUILD_PARALLEL_LEVEL
> + run: |
> + # Set CMAKE_BUILD_PARALLEL_LEVEL environment variable to
> + # limit the number of parallel jobs for build/test step.
> + NPROC=$(nproc)
> + echo CMAKE_BUILD_PARALLEL_LEVEL=$(($NPROC + 1)) | tee -a $GITHUB_ENV
> + shell: bash
> + - name: Install build and test dependencies
> + run: |
> + apt -y update
> + apt -y install cmake gcc make perl
> + shell: bash
> diff --git a/.github/actions/setup-macos/README.md b/.github/actions/setup-macos/README.md
> new file mode 100644
> index 00000000..ae70cb56
> --- /dev/null
> +++ b/.github/actions/setup-macos/README.md
> @@ -0,0 +1,12 @@
> +# Setup environment on macOS
> +
> +Action setups the environment on macOS runners (install requirements, setup the
> +workflow environment, etc).
> +
> +## How to use Github Action from Github workflow
> +
> +Add the following code to the running steps before LuaJIT configuration:
> +```
> +- uses: ./.github/actions/setup-macos
> + if: ${{ matrix.OS == 'macOS' }}
> +```
> diff --git a/.github/actions/setup-macos/action.yml b/.github/actions/setup-macos/action.yml
> new file mode 100644
> index 00000000..7a98065c
> --- /dev/null
> +++ b/.github/actions/setup-macos/action.yml
> @@ -0,0 +1,29 @@
> +name: Setup CI environment on macOS
> +description: Common part to tweak macOS CI runner environment
> +runs:
> + using: composite
> + steps:
> + - name: Setup CI environment
> + uses: ./.github/actions/setup
> + - name: Set CMAKE_BUILD_PARALLEL_LEVEL
> + run: |
> + # Set CMAKE_BUILD_PARALLEL_LEVEL environment variable to
> + # limit the number of parallel jobs for build/test step.
> + NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null)
> + echo CMAKE_BUILD_PARALLEL_LEVEL=$(($NPROC + 1)) | tee -a $GITHUB_ENV
> + shell: bash
> + - name: Install build and test dependencies
> + run: |
> + # Install brew using the command from Homebrew repository
> + # instructions: https://github.com/Homebrew/install.
> + # XXX: 'echo' command below is required since brew
> + # installation script obliges the one to enter a newline
> + # for confirming the installation via Ruby script.
> + brew update ||
> + echo | /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
> + # Try to install the packages either upgrade it to avoid
> + # of fails if the package already exists with the previous
> + # version.
> + brew install --force cmake gcc make perl ||
> + brew upgrade cmake gcc make perl
> + shell: bash
> diff --git a/.github/actions/environment/README.md b/.github/actions/setup/README.md
> similarity index 72%
> rename from .github/actions/environment/README.md
> rename to .github/actions/setup/README.md
> index 1ed0d0ca..87b5bf8d 100644
> --- a/.github/actions/environment/README.md
> +++ b/.github/actions/setup/README.md
> @@ -6,7 +6,8 @@ It is used as a common environment setup for the different testing workflows.
>
> ## How to use Github Action from Github workflow
>
> -Add the following code to the running steps after checkout is finished:
> +Add the following code to the running steps or OS-specific GitHub Actions after
> +checkout step is finished:
> ```
> -- uses: ./.github/actions/environment
> +- uses: ./.github/actions/setup
> ```
> diff --git a/.github/actions/setup/action.yml b/.github/actions/setup/action.yml
> new file mode 100644
> index 00000000..f153b0d9
> --- /dev/null
> +++ b/.github/actions/setup/action.yml
> @@ -0,0 +1,10 @@
> +name: Setup CI environment
> +description: Common part to tweak CI runner environment
> +runs:
> + using: composite
> + steps:
> + - run: |
> + # Set BUILDDIR environment variable to specify LuaJIT
> + # build directory.
> + echo BUILDDIR=${{ runner.temp }}/build-${{ github.run_id }} | tee -a $GITHUB_ENV
> + shell: bash
> diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
> index 64e8f992..04b810f2 100644
> --- a/.github/workflows/lint.yml
> +++ b/.github/workflows/lint.yml
> @@ -1,4 +1,4 @@
> -name: "LuaJIT static analysis"
> +name: Static analysis
>
> on:
> push:
> @@ -38,12 +38,16 @@ jobs:
> fetch-depth: 0
> submodules: recursive
> - name: environment
> - uses: ./.github/actions/environment
> + uses: ./.github/actions/setup
> - name: setup
> run: |
> + # TODO: Move this step to a separate action.
> sudo apt -y update
> sudo apt -y install cmake make lua5.1 luarocks
> sudo luarocks install luacheck
> + # Set CMAKE_BUILD_PARALLEL_LEVEL environment variable to
> + # limit the number of parallel jobs for build/test step.
> + echo CMAKE_BUILD_PARALLEL_LEVEL=$(($(nproc) + 1)) | tee -a $GITHUB_ENV
> - name: configure
> run: cmake -S . -B ${{ env.BUILDDIR }}
> - name: test
> diff --git a/.github/workflows/linux-x86_64-ninja.yml b/.github/workflows/linux-x86_64-ninja.yml
> index 72d56d54..a0422d09 100644
> --- a/.github/workflows/linux-x86_64-ninja.yml
> +++ b/.github/workflows/linux-x86_64-ninja.yml
> @@ -38,11 +38,15 @@ jobs:
> fetch-depth: 0
> submodules: recursive
> - name: environment
> - uses: ./.github/actions/environment
> + uses: ./.github/actions/setup
> - name: setup
> run: |
> + # TODO: Move this step to a separate action.
> sudo apt -y update
> sudo apt -y install cmake gcc ninja-build perl
> + # Set CMAKE_BUILD_PARALLEL_LEVEL environment variable to
> + # limit the number of parallel jobs for build/test step.
> + echo CMAKE_BUILD_PARALLEL_LEVEL=$(($(nproc) + 1)) | tee -a $GITHUB_ENV
> - name: configure
> run: >
> cmake -S . -B ${{ env.BUILDDIR }}
> diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml
> deleted file mode 100644
> index 2fa97f1e..00000000
> --- a/.github/workflows/macos.yml
> +++ /dev/null
> @@ -1,142 +0,0 @@
> -name: "macOS test workflow"
> -
> -on:
> - push:
> - branches-ignore:
> - - '**-notest'
> - - 'upstream-**'
> - tags-ignore:
> - - '**'
> -
> -concurrency:
> - # An update of a developer branch cancels the previously
> - # scheduled workflow run for this branch. However, the default
> - # branch, and long-term branch (tarantool-1.10, tarantool-2.8,
> - # etc.) workflow runs are never canceled.
> - #
> - # We use a trick here: define the concurrency group as 'workflow
> - # run ID' + # 'workflow run attempt' because it is a unique
> - # combination for any run. So it effectively discards grouping.
> - #
> - # XXX: we cannot use `github.sha` as a unique identifier because
> - # pushing a tag may cancel a run that works on a branch push
> - # event.
> - group: ${{ (
> - github.ref == 'refs/heads/tarantool' ||
> - startsWith(github.ref, 'refs/heads/tarantool-')) &&
> - format('{0}-{1}', github.run_id, github.run_attempt) ||
> - format('{0}-{1}', github.workflow, github.ref) }}
> - cancel-in-progress: true
> -
> -jobs:
> - test-luajit:
> - strategy:
> - fail-fast: false
> - matrix:
> - ARCH: [M1, x86_64]
> - BUILDTYPE: [Debug, Release]
> - GC64: [ON, OFF]
> - include:
> - - ARCH: M1
> - RUNNER: macos-11-m1
> - - ARCH: x86_64
> - RUNNER: macos-11
> - - BUILDTYPE: Debug
> - CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug -DLUA_USE_ASSERT=ON -DLUA_USE_APICHECK=ON
> - - BUILDTYPE: Release
> - CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> - exclude:
> - - ARCH: M1
> - GC64: OFF
> - runs-on: ${{ matrix.RUNNER }}
> - name: >
> - LuaJIT
> - ${{ matrix.ARCH }}
> - ${{ matrix.BUILDTYPE }}
> - GC64:${{ matrix.GC64 }}
> - steps:
> - - uses: actions/checkout at v2.3.4
> - with:
> - fetch-depth: 0
> - submodules: recursive
> - - name: environment
> - uses: ./.github/actions/environment
> - - name: setup
> - run: |
> - # Install brew using the command from Homebrew repository
> - # instructions: https://github.com/Homebrew/install.
> - # XXX: 'echo' command below is required since brew installation
> - # script obliges the one to enter a newline for confirming the
> - # installation via Ruby script.
> - brew update ||
> - echo | /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
> - # Try to install the packages either upgrade it to avoid of fails
> - # if the package already exists with the previous version.
> - brew install --force cmake gcc make perl ||
> - brew upgrade cmake gcc make perl
> - - name: configure
> - run: >
> - cmake -S . -B ${{ env.BUILDDIR }}
> - ${{ matrix.CMAKEFLAGS }}
> - -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> - - name: build
> - run: cmake --build . --parallel
> - working-directory: ${{ env.BUILDDIR }}
> - - name: test
> - run: cmake --build . --parallel --target test
> - working-directory: ${{ env.BUILDDIR }}
> -
> - test-tarantool-x86_64-debug-wo-GC64:
> - name: Tarantool x86_64 Debug GC64:OFF
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: OFF
> - buildtype: Debug
> - host: macos-11
> - revision: ${{ github.sha }}
> - test-tarantool-x86_64-debug-w-GC64:
> - name: Tarantool x86_64 Debug GC64:ON
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: ON
> - buildtype: Debug
> - host: macos-11
> - revision: ${{ github.sha }}
> - test-tarantool-x86_64-release-wo-GC64:
> - name: Tarantool x86_64 Release GC64:OFF
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: OFF
> - buildtype: RelWithDebInfo
> - host: macos-11
> - revision: ${{ github.sha }}
> - test-tarantool-x86_64-release-w-GC64:
> - name: Tarantool x86_64 Release GC64:ON
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: ON
> - buildtype: RelWithDebInfo
> - host: macos-11
> - revision: ${{ github.sha }}
> - test-tarantool-m1-debug-w-GC64:
> - name: Tarantool M1 Debug GC64:ON
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: ON
> - buildtype: Debug
> - host: macos-11-m1
> - revision: ${{ github.sha }}
> - test-tarantool-m1-release-w-GC64:
> - name: Tarantool M1 Release GC64:ON
> - needs: test-luajit
> - uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> - with:
> - GC64: ON
> - buildtype: RelWithDebInfo
> - host: macos-11-m1
> - revision: ${{ github.sha }}
> diff --git a/.github/workflows/linux.yml b/.github/workflows/testing.yml
> similarity index 52%
> rename from .github/workflows/linux.yml
> rename to .github/workflows/testing.yml
> index 125c8708..c4847335 100644
> --- a/.github/workflows/linux.yml
> +++ b/.github/workflows/testing.yml
> @@ -1,4 +1,4 @@
> -name: "Linux test workflow"
> +name: Testing
>
> on:
> push:
> @@ -33,25 +33,38 @@ jobs:
> strategy:
> fail-fast: false
> matrix:
> - ARCH: [aarch64, x86_64]
> + ARCH: [ARM64, x86_64]
> BUILDTYPE: [Debug, Release]
> GC64: [ON, OFF]
> + OS: [Linux, macOS]
> include:
> - - ARCH: aarch64
> + - ARCH: ARM64
> + OS: Linux
> + NAME: Linux/aarch64
> RUNNER: graviton
> - ARCH: x86_64
> + OS: Linux
> + NAME: Linux/x86_64
> RUNNER: ubuntu-20.04-self-hosted
> + - ARCH: ARM64
> + OS: macOS
> + NAME: macOS/M1
> + RUNNER: macos-11-m1
> + - ARCH: x86_64
> + OS: macOS
> + NAME: macOS/x86_64
> + RUNNER: macos-11
> - BUILDTYPE: Debug
> CMAKEFLAGS: -DCMAKE_BUILD_TYPE=Debug -DLUA_USE_ASSERT=ON -DLUA_USE_APICHECK=ON
> - BUILDTYPE: Release
> CMAKEFLAGS: -DCMAKE_BUILD_TYPE=RelWithDebInfo
> exclude:
> - - ARCH: aarch64
> + - ARCH: ARM64
> GC64: OFF
> runs-on: ${{ matrix.RUNNER }}
> name: >
> LuaJIT
> - ${{ matrix.ARCH }}
> + (${{ matrix.NAME }})
> ${{ matrix.BUILDTYPE }}
> GC64:${{ matrix.GC64 }}
> steps:
> @@ -59,12 +72,12 @@ jobs:
> with:
> fetch-depth: 0
> submodules: recursive
> - - name: environment
> - uses: ./.github/actions/environment
> - - name: setup
> - run: |
> - sudo apt -y update
> - sudo apt -y install cmake gcc make perl
> + - name: setup Linux
> + uses: ./.github/actions/setup-linux
> + if: ${{ matrix.OS == 'Linux' }}
> + - name: setup macOS
> + uses: ./.github/actions/setup-macos
> + if: ${{ matrix.OS == 'macOS' }}
> - name: configure
> run: >
> cmake -S . -B ${{ env.BUILDDIR }}
> @@ -77,8 +90,8 @@ jobs:
> run: cmake --build . --parallel --target test
> working-directory: ${{ env.BUILDDIR }}
>
> - test-tarantool-x86_64-debug-wo-GC64:
> - name: Tarantool x86_64 Debug GC64:OFF
> + test-tarantool-linux-x86_64-debug-wo-GC64:
> + name: Tarantool (Linux/x86_64) Debug GC64:OFF
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -86,8 +99,8 @@ jobs:
> buildtype: Debug
> host: ubuntu-20.04-self-hosted
> revision: ${{ github.sha }}
> - test-tarantool-x86_64-debug-w-GC64:
> - name: Tarantool x86_64 Debug GC64:ON
> + test-tarantool-linux-x86_64-debug-w-GC64:
> + name: Tarantool (Linux/x86_64) Debug GC64:ON
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -95,8 +108,8 @@ jobs:
> buildtype: Debug
> host: ubuntu-20.04-self-hosted
> revision: ${{ github.sha }}
> - test-tarantool-x86_64-release-wo-GC64:
> - name: Tarantool x86_64 Release GC64:OFF
> + test-tarantool-linux-x86_64-release-wo-GC64:
> + name: Tarantool (Linux/x86_64) Release GC64:OFF
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -104,8 +117,8 @@ jobs:
> buildtype: RelWithDebInfo
> host: ubuntu-20.04-self-hosted
> revision: ${{ github.sha }}
> - test-tarantool-x86_64-release-w-GC64:
> - name: Tarantool x86_64 Release GC64:ON
> + test-tarantool-linux-x86_64-release-w-GC64:
> + name: Tarantool (Linux/x86_64) Release GC64:ON
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -113,8 +126,8 @@ jobs:
> buildtype: RelWithDebInfo
> host: ubuntu-20.04-self-hosted
> revision: ${{ github.sha }}
> - test-tarantool-aarch64-debug-w-GC64:
> - name: Tarantool aarch64 Debug GC64:ON
> + test-tarantool-linux-aarch64-debug-w-GC64:
> + name: Tarantool (Linux/aarch64) Debug GC64:ON
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -122,8 +135,8 @@ jobs:
> buildtype: Debug
> host: graviton
> revision: ${{ github.sha }}
> - test-tarantool-aarch64-release-w-GC64:
> - name: Tarantool aarch64 Release GC64:ON
> + test-tarantool-linux-aarch64-release-w-GC64:
> + name: Tarantool (Linux/aarch64) Release GC64:ON
> needs: test-luajit
> uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> with:
> @@ -131,3 +144,57 @@ jobs:
> buildtype: RelWithDebInfo
> host: graviton
> revision: ${{ github.sha }}
> + test-tarantool-macos-x86_64-debug-wo-GC64:
> + name: Tarantool (macOS/x86_64) Debug GC64:OFF
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: OFF
> + buildtype: Debug
> + host: macos-11
> + revision: ${{ github.sha }}
> + test-tarantool-macos-x86_64-debug-w-GC64:
> + name: Tarantool (macOS/x86_64) Debug GC64:ON
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: ON
> + buildtype: Debug
> + host: macos-11
> + revision: ${{ github.sha }}
> + test-tarantool-macos-x86_64-release-wo-GC64:
> + name: Tarantool (macOS/x86_64) Release GC64:OFF
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: OFF
> + buildtype: RelWithDebInfo
> + host: macos-11
> + revision: ${{ github.sha }}
> + test-tarantool-macos-x86_64-release-w-GC64:
> + name: Tarantool (macOS/x86_64) Release GC64:ON
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: ON
> + buildtype: RelWithDebInfo
> + host: macos-11
> + revision: ${{ github.sha }}
> + test-tarantool-macos-m1-debug-w-GC64:
> + name: Tarantool (macOS/M1) Debug GC64:ON
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: ON
> + buildtype: Debug
> + host: macos-11-m1
> + revision: ${{ github.sha }}
> + test-tarantool-macos-m1-release-w-GC64:
> + name: Tarantool (macOS/M1) Release GC64:ON
> + needs: test-luajit
> + uses: tarantool/tarantool/.github/workflows/luajit-integration.yml at master
> + with:
> + GC64: ON
> + buildtype: RelWithDebInfo
> + host: macos-11-m1
> + revision: ${{ github.sha }}
More information about the Tarantool-patches
mailing list