Re: [Tarantool-patches] [PATCH v1 luajit 41/41] ci: introduce the performance workflow

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 6844 bytes --]

Hi, Sergey,

thanks for the patch! See my comments.

Sergey

On 10/24/25 14:00, Sergey Kaplun wrote:
> This patch adds the workflow to run benchmarks from various suites,
> aggregate their results, and send statistics to the InfluxDB to be
> processed later.
>
> The workflow contains a matrix to measure GC64 and non-GC64 modes with
> enabled/disabled JIT for x64 architecture.
> ---
>   .github/actions/setup-performance/README.md  |  10 ++
>   .github/actions/setup-performance/action.yml |  18 +++
>   .github/workflows/performance.yml            | 110 +++++++++++++++++++
>   3 files changed, 138 insertions(+)
>   create mode 100644 .github/actions/setup-performance/README.md
>   create mode 100644 .github/actions/setup-performance/action.yml
>   create mode 100644 .github/workflows/performance.yml
>
> diff --git a/.github/actions/setup-performance/README.md b/.github/actions/setup-performance/README.md
> new file mode 100644
> index 00000000..4c4bbdab
> --- /dev/null
> +++ b/.github/actions/setup-performance/README.md
> @@ -0,0 +1,10 @@
> +# Setup performance
> +
> +Action setups the performance on Linux runners.
> +
> +## How to use Github Action from Github workflow
> +
> +Add the following code to the running steps before LuaJIT configuration:
> +```
> +- uses: ./.github/actions/setup-performance
> +```
> diff --git a/.github/actions/setup-performance/action.yml b/.github/actions/setup-performance/action.yml
> new file mode 100644
> index 00000000..24d07440
> --- /dev/null
> +++ b/.github/actions/setup-performance/action.yml
> @@ -0,0 +1,18 @@
> +name: Setup performance
> +description: The Linux machine setup for running LuaJIT benchmarks
> +runs:
> +  using: composite
> +  steps:
> +    - name: Setup CI environment (Linux)
> +      uses: ./.github/actions/setup-linux
> +    - name: Install dependencies for the LuaJIT benchmarks
> +      run: |
> +        apt -y update
> +        apt install -y luarocks curl
> +      shell: bash
> +    - name: Install Lua modules
> +      run: luarocks install lua-cjson
> +      shell: bash
> +    - name: Run script to setup Linux environment
> +      run: sh ./perf/helpers/setup_env.sh
> +      shell: bash
bash or shell is used in the last step? (shebang in setup_env.sh is 
/bin/sh)
> diff --git a/.github/workflows/performance.yml b/.github/workflows/performance.yml
> new file mode 100644
> index 00000000..bfb6be97
> --- /dev/null
> +++ b/.github/workflows/performance.yml
> @@ -0,0 +1,110 @@
> +name: Performance
> +
> +on:
> +  push:
> +    branches-ignore:
> +      - '**-noperf'
> +      - 'tarantool/release/**'
> +      - 'upstream-**'
> +    tags-ignore:
> +      - '**'
> +  schedule:
> +    # Once a day at 03:00 to avoid clashing with runs for the
> +    # Tarantool benchmarks at midnight.
> +    - cron: '0 3 * * *'
> +
> +concurrency:
> +  # An update of a developer branch cancels the previously
> +  # scheduled workflow run for this branch. However, the default
> +  # branch, and long-term branch (tarantool/release/2.11,
> +  # tarantool/release/2.10, etc) workflow runs are never canceled.
> +  #
it is not relevant, right?
> +  # We use a trick here: define the concurrency group as 'workflow
> +  # run ID' + # 'workflow run attempt' because it is a unique
> +  # combination for any run. So it effectively discards grouping.
> +  #
> +  # XXX: we cannot use `github.sha` as a unique identifier because
> +  # pushing a tag may cancel a run that works on a branch push
> +  # event.
> +  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
> +    && format('{0}-{1}', github.run_id, github.run_attempt)
> +    || format('{0}-{1}', github.workflow, github.ref) }}
> +  cancel-in-progress: true
> +
> +jobs:
> +  performance-luajit:
> +    # The 'performance' label _must_ be set only for the single
> +    # runner to guarantee that results are not dependent on the
> +    # machine.
> +    runs-on:
> +      - self-hosted
> +      - Linux
> +      - x86_64
> +      - 'performance'
> +
> +    env:
> +      PERF_BRANCH: ${{ github.ref_name }}
> +      PERF_COMMIT: ${{ github.sha }}
> +
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        GC64: [ON, OFF]
> +        JOFF: [ON, OFF]
> +      # Run each job sequentially.
> +      max-parallel: 1
> +    name: >
> +      LuaJIT
> +GC64:${{ matrix.GC64 }}
> +JOFF:${{ matrix.GC64 }}
> +    steps:
> +      - uses: actions/checkout@v4
> +        with:
> +          fetch-depth: 0
> +          submodules: recursive
> +      - name: setup performance environment
> +        uses: ./.github/actions/setup-performance
> +      - name: configure
> +        # The taskset alone will pin all the process threads
> +        # into a single (random) isolated CPU, see
> +        #https://bugzilla.kernel.org/show_bug.cgi?id=116701.
> +        # The workaround is using realtime scheduler for the
> +        # isolated task using chrt, e. g.:
> +        # sudo taskset 0xef chrt 50.
> +        # But this makes the process use non-standard, real-time
> +        # round-robin scheduling mechanism.
> +        run: >
> +          cmake -S . -B ${{ env.BUILDDIR }}
> +          -DCMAKE_BUILD_TYPE=RelWithDebInfo

RelWithDebInfo is -O2 (moderate optimization), Release is -O3 (high 
optimization).

Do we really need  RelWithDebInfo? I think it deserves a comment.

> +          -DLUAJIT_ENABLE_PERF=ON
> +          -DLUAJIT_BENCH_INIT="taskset 0xfe chrt 50"
> +          -DLUAJIT_DISABLE_JIT=${{ matrix.JOFF }}
> +          -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
> +      - name: build
> +        run: cmake --build . --parallel
> +        working-directory: ${{ env.BUILDDIR }}
> +      - name: perf
> +        run: make LuaJIT-perf
> +        working-directory: ${{ env.BUILDDIR }}
> +      - name: aggregate benchmark results
> +        run: make LuaJIT-perf-aggregate
> +        working-directory: ${{ env.BUILDDIR }}
> +      - name: send statistics to InfluxDB
> +        # --silent -o /dev/null: Prevent dumping any reply part
> +        # in the output in case of an error.
> +        # --fail: Exit with the 22 error code is status >= 400.
> +        # --write-out: See the reason for the failure, if any.
> +        # --retry, --retry-delay: To avoid losing the results of
> +        # running after such a long job, try to retry sending the
> +        # results.
> +        run: >
> +          curl --request POST
> +          "${{ secrets.INFLUXDB_URL }}/api/v2/write?org=tarantool&bucket=luajit-performance&precision=s"
> +          --write-out "%{http_code}"
> +          --retry 5
> +          --retry-delay 5
> +          --connect-timeout 120
> +          --fail --silent -o /dev/null
> +          --header "Authorization: Token ${{ secrets.INFLUXDB_TOKEN }}"
> +          --data-binary @./perf/output/summary.txt
> +        working-directory: ${{ env.BUILDDIR }}

[-- Attachment #2: Type: text/html, Size: 7812 bytes --]