From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6FD9E1656B92; Tue, 18 Nov 2025 16:09:02 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6FD9E1656B92 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1763471342; bh=2tG7ZG6WQrxBZ42bBE1AObwgkeHvY1boFfk6k/T1tyc=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=hPL2GRI2pKAfioseFAcoRRDXUtFJyUqEYybg2PVRr06ipEv0LM8tS9c3/VeVpVWQ3 Kbi8alfARidj4jPiy7Wx/Y8a2W31A3oLfWjergYgM0ThBODQy4d6s//5ozP3eSywQv fVL9XYsx+4yrBwMP7GbDWqdIPqw3oJHEGzWznDBM= Received: from send265.i.mail.ru (send265.i.mail.ru [95.163.59.104]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 518211656B91 for ; Tue, 18 Nov 2025 16:09:01 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 518211656B91 Received: by exim-smtp-88cf54d45-7tl7p with esmtpa (envelope-from ) id 1vLLS8-000000008x5-0Ph7; Tue, 18 Nov 2025 16:09:00 +0300 Content-Type: multipart/alternative; boundary="------------08w0tZ2wcXtJbVeZolFTW6xR" Message-ID: <98db63bf-72e6-490b-833f-caa599e33657@tarantool.org> Date: Tue, 18 Nov 2025 16:08:59 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org References: <74c74521bf5b3d3c629b5e9337c3436d3145f1a4.1761301736.git.skaplun@tarantool.org> In-Reply-To: <74c74521bf5b3d3c629b5e9337c3436d3145f1a4.1761301736.git.skaplun@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9231027FDF5B453B6BAC1C8CCE2995CE7B1433350C4DF093D182A05F538085040BFB2A9D643F3A7123DE06ABAFEAF6705A6E603F6243D6F8E5E9F8D75C37230126C2BE0BE76D801B4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7AEA4A6B3AFC9B957C2099A533E45F2D0395957E7521B51C2CFCAF695D4D8E9FCEA1F7E6F0F101C6759CC434672EE6371C2A783ECEC0211ADC4224003CC836476D5A39DEEDB180909611E41BBFE2FEB2B28F76E9EE58B3048712D6EBEA2921A579A68533E18CFCBC1D4706E98678885959FA2833FD35BB23D9E625A9149C048EE33AC447995A7AD18C26CFBAC0749D213D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8B2EE5AD8F952D28FBA471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FCE3B78367742D7E3B3AA81AA40904B5D9CF19DD082D7633A0C84D3B47A649675F3AA81AA40904B5D98AA50765F79006376A29911348193673D81D268191BDAD3D3666184CF4C3C14F3FC91FA280E0CE3D1A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89FB26E97DCB74E625257739F23D657EF2BB5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A5EA806FF23F0CB8615002B1117B3ED696B71E6D48B7E69A37C66B2B37046EC955823CB91A9FED034534781492E4B8EEAD9F24B12AABCCB078 X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CFEF0865A8E00C9ECAB008750FBF39C80622595BF6C0FBF9E40170E81AEBD3D4E5A0D894938C2B729CDD6EA89E5C47541B2D662D8F0025421AFB84A95C673F4237E652A3ADE77C9233111DC66A97D0BFE2913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVfFX+FmbwDW15JlFpJ1+XRs= X-Mailru-Sender: C4F68CFF4024C8867DFDF7C7F25884581108CEFAACB01BA577EA6416FFE467E16F4C4B34A68EDA9C8F61242874946C06645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v1 luajit 41/41] ci: introduce the performance workflow X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------08w0tZ2wcXtJbVeZolFTW6xR Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, Sergey, thanks for the patch! See my comments. Sergey On 10/24/25 14:00, Sergey Kaplun wrote: > This patch adds the workflow to run benchmarks from various suites, > aggregate their results, and send statistics to the InfluxDB to be > processed later. > > The workflow contains a matrix to measure GC64 and non-GC64 modes with > enabled/disabled JIT for x64 architecture. > --- > .github/actions/setup-performance/README.md | 10 ++ > .github/actions/setup-performance/action.yml | 18 +++ > .github/workflows/performance.yml | 110 +++++++++++++++++++ > 3 files changed, 138 insertions(+) > create mode 100644 .github/actions/setup-performance/README.md > create mode 100644 .github/actions/setup-performance/action.yml > create mode 100644 .github/workflows/performance.yml > > diff --git a/.github/actions/setup-performance/README.md b/.github/actions/setup-performance/README.md > new file mode 100644 > index 00000000..4c4bbdab > --- /dev/null > +++ b/.github/actions/setup-performance/README.md > @@ -0,0 +1,10 @@ > +# Setup performance > + > +Action setups the performance on Linux runners. > + > +## How to use Github Action from Github workflow > + > +Add the following code to the running steps before LuaJIT configuration: > +``` > +- uses: ./.github/actions/setup-performance > +``` > diff --git a/.github/actions/setup-performance/action.yml b/.github/actions/setup-performance/action.yml > new file mode 100644 > index 00000000..24d07440 > --- /dev/null > +++ b/.github/actions/setup-performance/action.yml > @@ -0,0 +1,18 @@ > +name: Setup performance > +description: The Linux machine setup for running LuaJIT benchmarks > +runs: > + using: composite > + steps: > + - name: Setup CI environment (Linux) > + uses: ./.github/actions/setup-linux > + - name: Install dependencies for the LuaJIT benchmarks > + run: | > + apt -y update > + apt install -y luarocks curl > + shell: bash > + - name: Install Lua modules > + run: luarocks install lua-cjson > + shell: bash > + - name: Run script to setup Linux environment > + run: sh ./perf/helpers/setup_env.sh > + shell: bash bash or shell is used in the last step? (shebang in setup_env.sh is /bin/sh) > diff --git a/.github/workflows/performance.yml b/.github/workflows/performance.yml > new file mode 100644 > index 00000000..bfb6be97 > --- /dev/null > +++ b/.github/workflows/performance.yml > @@ -0,0 +1,110 @@ > +name: Performance > + > +on: > + push: > + branches-ignore: > + - '**-noperf' > + - 'tarantool/release/**' > + - 'upstream-**' > + tags-ignore: > + - '**' > + schedule: > + # Once a day at 03:00 to avoid clashing with runs for the > + # Tarantool benchmarks at midnight. > + - cron: '0 3 * * *' > + > +concurrency: > + # An update of a developer branch cancels the previously > + # scheduled workflow run for this branch. However, the default > + # branch, and long-term branch (tarantool/release/2.11, > + # tarantool/release/2.10, etc) workflow runs are never canceled. > + # it is not relevant, right? > + # We use a trick here: define the concurrency group as 'workflow > + # run ID' + # 'workflow run attempt' because it is a unique > + # combination for any run. So it effectively discards grouping. > + # > + # XXX: we cannot use `github.sha` as a unique identifier because > + # pushing a tag may cancel a run that works on a branch push > + # event. > + group: ${{ startsWith(github.ref, 'refs/heads/tarantool/') > + && format('{0}-{1}', github.run_id, github.run_attempt) > + || format('{0}-{1}', github.workflow, github.ref) }} > + cancel-in-progress: true > + > +jobs: > + performance-luajit: > + # The 'performance' label _must_ be set only for the single > + # runner to guarantee that results are not dependent on the > + # machine. > + runs-on: > + - self-hosted > + - Linux > + - x86_64 > + - 'performance' > + > + env: > + PERF_BRANCH: ${{ github.ref_name }} > + PERF_COMMIT: ${{ github.sha }} > + > + strategy: > + fail-fast: false > + matrix: > + GC64: [ON, OFF] > + JOFF: [ON, OFF] > + # Run each job sequentially. > + max-parallel: 1 > + name: > > + LuaJIT > +GC64:${{ matrix.GC64 }} > +JOFF:${{ matrix.GC64 }} > + steps: > + - uses: actions/checkout@v4 > + with: > + fetch-depth: 0 > + submodules: recursive > + - name: setup performance environment > + uses: ./.github/actions/setup-performance > + - name: configure > + # The taskset alone will pin all the process threads > + # into a single (random) isolated CPU, see > + #https://bugzilla.kernel.org/show_bug.cgi?id=116701. > + # The workaround is using realtime scheduler for the > + # isolated task using chrt, e. g.: > + # sudo taskset 0xef chrt 50. > + # But this makes the process use non-standard, real-time > + # round-robin scheduling mechanism. > + run: > > + cmake -S . -B ${{ env.BUILDDIR }} > + -DCMAKE_BUILD_TYPE=RelWithDebInfo RelWithDebInfo is -O2 (moderate optimization), Release is -O3 (high optimization). Do we really need  RelWithDebInfo? I think it deserves a comment. > + -DLUAJIT_ENABLE_PERF=ON > + -DLUAJIT_BENCH_INIT="taskset 0xfe chrt 50" > + -DLUAJIT_DISABLE_JIT=${{ matrix.JOFF }} > + -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }} > + - name: build > + run: cmake --build . --parallel > + working-directory: ${{ env.BUILDDIR }} > + - name: perf > + run: make LuaJIT-perf > + working-directory: ${{ env.BUILDDIR }} > + - name: aggregate benchmark results > + run: make LuaJIT-perf-aggregate > + working-directory: ${{ env.BUILDDIR }} > + - name: send statistics to InfluxDB > + # --silent -o /dev/null: Prevent dumping any reply part > + # in the output in case of an error. > + # --fail: Exit with the 22 error code is status >= 400. > + # --write-out: See the reason for the failure, if any. > + # --retry, --retry-delay: To avoid losing the results of > + # running after such a long job, try to retry sending the > + # results. > + run: > > + curl --request POST > + "${{ secrets.INFLUXDB_URL }}/api/v2/write?org=tarantool&bucket=luajit-performance&precision=s" > + --write-out "%{http_code}" > + --retry 5 > + --retry-delay 5 > + --connect-timeout 120 > + --fail --silent -o /dev/null > + --header "Authorization: Token ${{ secrets.INFLUXDB_TOKEN }}" > + --data-binary @./perf/output/summary.txt > + working-directory: ${{ env.BUILDDIR }} --------------08w0tZ2wcXtJbVeZolFTW6xR Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hi, Sergey,

thanks for the patch! See my comments.

Sergey

On 10/24/25 14:00, Sergey Kaplun wrote:
This patch adds the workflow to run benchmarks from various suites,
aggregate their results, and send statistics to the InfluxDB to be
processed later.

The workflow contains a matrix to measure GC64 and non-GC64 modes with
enabled/disabled JIT for x64 architecture.
---
 .github/actions/setup-performance/README.md  |  10 ++
 .github/actions/setup-performance/action.yml |  18 +++
 .github/workflows/performance.yml            | 110 +++++++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 .github/actions/setup-performance/README.md
 create mode 100644 .github/actions/setup-performance/action.yml
 create mode 100644 .github/workflows/performance.yml

diff --git a/.github/actions/setup-performance/README.md b/.github/actions/setup-performance/README.md
new file mode 100644
index 00000000..4c4bbdab
--- /dev/null
+++ b/.github/actions/setup-performance/README.md
@@ -0,0 +1,10 @@
+# Setup performance
+
+Action setups the performance on Linux runners.
+
+## How to use Github Action from Github workflow
+
+Add the following code to the running steps before LuaJIT configuration:
+```
+- uses: ./.github/actions/setup-performance
+```
diff --git a/.github/actions/setup-performance/action.yml b/.github/actions/setup-performance/action.yml
new file mode 100644
index 00000000..24d07440
--- /dev/null
+++ b/.github/actions/setup-performance/action.yml
@@ -0,0 +1,18 @@
+name: Setup performance
+description: The Linux machine setup for running LuaJIT benchmarks
+runs:
+  using: composite
+  steps:
+    - name: Setup CI environment (Linux)
+      uses: ./.github/actions/setup-linux
+    - name: Install dependencies for the LuaJIT benchmarks
+      run: |
+        apt -y update
+        apt install -y luarocks curl
+      shell: bash
+    - name: Install Lua modules
+      run: luarocks install lua-cjson
+      shell: bash
+    - name: Run script to setup Linux environment
+      run: sh ./perf/helpers/setup_env.sh
+      shell: bash
bash or shell is used in the last step? (shebang in setup_env.sh is /bin/sh) 
diff --git a/.github/workflows/performance.yml b/.github/workflows/performance.yml
new file mode 100644
index 00000000..bfb6be97
--- /dev/null
+++ b/.github/workflows/performance.yml
@@ -0,0 +1,110 @@
+name: Performance
+
+on:
+  push:
+    branches-ignore:
+      - '**-noperf'
+      - 'tarantool/release/**'
+      - 'upstream-**'
+    tags-ignore:
+      - '**'
+  schedule:
+    # Once a day at 03:00 to avoid clashing with runs for the
+    # Tarantool benchmarks at midnight.
+    - cron: '0 3 * * *'
+
+concurrency:
+  # An update of a developer branch cancels the previously
+  # scheduled workflow run for this branch. However, the default
+  # branch, and long-term branch (tarantool/release/2.11,
+  # tarantool/release/2.10, etc) workflow runs are never canceled.
+  #
it is not relevant, right? 
+  # We use a trick here: define the concurrency group as 'workflow
+  # run ID' + # 'workflow run attempt' because it is a unique
+  # combination for any run. So it effectively discards grouping.
+  #
+  # XXX: we cannot use `github.sha` as a unique identifier because
+  # pushing a tag may cancel a run that works on a branch push
+  # event.
+  group: ${{ startsWith(github.ref, 'refs/heads/tarantool/')
+    && format('{0}-{1}', github.run_id, github.run_attempt)
+    || format('{0}-{1}', github.workflow, github.ref) }}
+  cancel-in-progress: true
+
+jobs:
+  performance-luajit:
+    # The 'performance' label _must_ be set only for the single
+    # runner to guarantee that results are not dependent on the
+    # machine.
+    runs-on:
+      - self-hosted
+      - Linux
+      - x86_64
+      - 'performance'
+
+    env:
+      PERF_BRANCH: ${{ github.ref_name }}
+      PERF_COMMIT: ${{ github.sha }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        GC64: [ON, OFF]
+        JOFF: [ON, OFF]
+      # Run each job sequentially.
+      max-parallel: 1
+    name: >
+      LuaJIT
+      GC64:${{ matrix.GC64 }}
+      JOFF:${{ matrix.GC64 }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          submodules: recursive
+      - name: setup performance environment
+        uses: ./.github/actions/setup-performance
+      - name: configure
+        # The taskset alone will pin all the process threads
+        # into a single (random) isolated CPU, see
+        # https://bugzilla.kernel.org/show_bug.cgi?id=116701.
+        # The workaround is using realtime scheduler for the
+        # isolated task using chrt, e. g.:
+        # sudo taskset 0xef chrt 50.
+        # But this makes the process use non-standard, real-time
+        # round-robin scheduling mechanism.
+        run: >
+          cmake -S . -B ${{ env.BUILDDIR }}
+          -DCMAKE_BUILD_TYPE=RelWithDebInfo

RelWithDebInfo is -O2 (moderate optimization), Release is -O3 (high optimization).

Do we really need  RelWithDebInfo? I think it deserves a comment.

+          -DLUAJIT_ENABLE_PERF=ON
+          -DLUAJIT_BENCH_INIT="taskset 0xfe chrt 50"
+          -DLUAJIT_DISABLE_JIT=${{ matrix.JOFF }}
+          -DLUAJIT_ENABLE_GC64=${{ matrix.GC64 }}
+      - name: build
+        run: cmake --build . --parallel
+        working-directory: ${{ env.BUILDDIR }}
+      - name: perf
+        run: make LuaJIT-perf
+        working-directory: ${{ env.BUILDDIR }}
+      - name: aggregate benchmark results
+        run: make LuaJIT-perf-aggregate
+        working-directory: ${{ env.BUILDDIR }}
+      - name: send statistics to InfluxDB
+        # --silent -o /dev/null: Prevent dumping any reply part
+        # in the output in case of an error.
+        # --fail: Exit with the 22 error code is status >= 400.
+        # --write-out: See the reason for the failure, if any.
+        # --retry, --retry-delay: To avoid losing the results of
+        # running after such a long job, try to retry sending the
+        # results.
+        run: >
+          curl --request POST
+          "${{ secrets.INFLUXDB_URL }}/api/v2/write?org=tarantool&bucket=luajit-performance&precision=s"
+          --write-out "%{http_code}"
+          --retry 5
+          --retry-delay 5
+          --connect-timeout 120
+          --fail --silent -o /dev/null
+          --header "Authorization: Token ${{ secrets.INFLUXDB_TOKEN }}"
+          --data-binary @./perf/output/summary.txt
+        working-directory: ${{ env.BUILDDIR }}
--------------08w0tZ2wcXtJbVeZolFTW6xR--