Re: [Tarantool-patches] [PATCH v2 luajit 00/41] LuaJIT performance testing

[Sergey Bronnikov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 9617 bytes --]

Sergey,

thanks for the patch series! LGTM

Sergey

On 12/26/25 12:17, Sergey Kaplun wrote:
> This patchset introduces the performance testing suite for LuaJIT
> benchmarks. It takes the LuaJIT test cleanup benches [1] and adapts them
> to use a custom benchmark module with the GoogleBenchamark-similar
> format. All results are collected and reported to InfluxDB, like it is
> done for the Tarantool's tests.
>
> The results for the following benchmarks are not very stable. It should
> be investigated later (I appreciate any help with this):
>
> * array3d
> * binary-trees
> * euler14-bit
> * k-nukleotide
> * nsieve (most unstable)
> * nsieve-bit
> * spectral-norm
>
> Also, I've added notes to some commits where I'm not sure that the
> implementation/solution is very good. Any insights are welcome :).
>
> [1]:https://github.com/LuaJIT/LuaJIT-test-cleanup/tree/014708b/bench
>
> ---
>
> Changes in the v2:
>
> * Added the comments with the benchmark descriptions.
> * Fixed code style mentioned by Sergey for the chunks that were touched
>    during adjusting.
> * Code style fixes.
> * Fixed <coroutine-ring.lua> and <ray.lua> benchmark performance.
> * Fixed naming of the CI workflow (incorrect JOFF flag).
> * Fixed naming of the <meteor.lua> benchmark "subbench"
>    "meteror" -> "meteor".
> * Decreased number of iterations in array3d from 5 to 4 due to OOM
>    errors.
> * Rebased the branch on the current master branch.
>
> Also, it is worth mentioning that array3d with 5 iterations in JOFF
> non-GC64 mode may lead to the core dump [2]. Since it is rather rare
> flak and the number of iterations is decreased, it can be ignored for
> now, I suppose.
>
> [2]:https://github.com/tarantool/luajit/actions/runs/20506964003/job/58922906212#step:6:24
>
> Sergey Kaplun (41):
>    perf: add LuaJIT-test-cleanup perf suite
>    perf: introduce clock module
>    perf: introduce bench module
>    perf: adjust array3d in LuaJIT-benches
>    perf: adjust binary-trees in LuaJIT-benches
>    perf: adjust chameneos in LuaJIT-benches
>    perf: adjust coroutine-ring in LuaJIT-benches
>    perf: adjust euler14-bit in LuaJIT-benches
>    perf: adjust fannkuch in LuaJIT-benches
>    perf: adjust fasta in LuaJIT-benches
>    perf: adjust k-nucleotide in LuaJIT-benches
>    perf: adjust life in LuaJIT-benches
>    perf: adjust mandelbrot-bit in LuaJIT-benches
>    perf: adjust mandelbrot in LuaJIT-benches
>    perf: adjust md5 in LuaJIT-benches
>    perf: adjust meteor in LuaJIT-benches
>    perf: adjust nbody in LuaJIT-benches
>    perf: adjust nsieve-bit-fp in LuaJIT-benches
>    perf: adjust nsieve-bit in LuaJIT-benches
>    perf: adjust nsieve in LuaJIT-benches
>    perf: adjust partialsums in LuaJIT-benches
>    perf: adjust pidigits-nogmp in LuaJIT-benches
>    perf: adjust ray in LuaJIT-benches
>    perf: adjust recursive-ack in LuaJIT-benches
>    perf: adjust recursive-fib in LuaJIT-benches
>    perf: adjust revcomp in LuaJIT-benches
>    perf: adjust scimark-2010-12-20 in LuaJIT-benches
>    perf: move <scimark_lib.lua> to <libs/> directory
>    perf: adjust scimark-fft in LuaJIT-benches
>    perf: adjust scimark-lu in LuaJIT-benches
>    perf: add scimark-mc in LuaJIT-benches
>    perf: adjust scimark-sor in LuaJIT-benches
>    perf: adjust scimark-sparse in LuaJIT-benches
>    perf: adjust series in LuaJIT-benches
>    perf: adjust spectral-norm in LuaJIT-benches
>    perf: adjust sum-file in LuaJIT-benches
>    perf: add CMake infrastructure
>    perf: add aggregator helper for bench statistics
>    perf: add a script for the environment setup
>    perf: provide CMake option to setup the benchmark
>    ci: introduce the performance workflow
>
>   .github/actions/setup-performance/README.md  |   10 +
>   .github/actions/setup-performance/action.yml |   18 +
>   .github/workflows/performance.yml            |  110 ++
>   .gitignore                                   |    5 +
>   .luacheckrc                                  |    1 +
>   CMakeLists.txt                               |   11 +
>   perf/CMakeLists.txt                          |  119 +++
>   perf/LuaJIT-benches/CMakeLists.txt           |   52 +
>   perf/LuaJIT-benches/PARAM_arm.txt            |   29 +
>   perf/LuaJIT-benches/PARAM_mips.txt           |   29 +
>   perf/LuaJIT-benches/PARAM_ppc.txt            |   29 +
>   perf/LuaJIT-benches/PARAM_x86.txt            |   29 +
>   perf/LuaJIT-benches/SUMCOL_1.txt             | 1000 ++++++++++++++++++
>   perf/LuaJIT-benches/TEST_md5sum.txt          |   20 +
>   perf/LuaJIT-benches/array3d.lua              |   78 ++
>   perf/LuaJIT-benches/binary-trees.lua         |  120 +++
>   perf/LuaJIT-benches/chameneos.lua            |   88 ++
>   perf/LuaJIT-benches/coroutine-ring.lua       |   64 ++
>   perf/LuaJIT-benches/euler14-bit.lua          |   55 +
>   perf/LuaJIT-benches/fannkuch.lua             |   88 ++
>   perf/LuaJIT-benches/fasta.lua                |   35 +
>   perf/LuaJIT-benches/k-nucleotide.lua         |  132 +++
>   perf/LuaJIT-benches/libs/fasta.lua           |  105 ++
>   perf/LuaJIT-benches/libs/scimark_lib.lua     |  297 ++++++
>   perf/LuaJIT-benches/life.lua                 |  194 ++++
>   perf/LuaJIT-benches/mandelbrot-bit.lua       |   68 ++
>   perf/LuaJIT-benches/mandelbrot.lua           |   55 +
>   perf/LuaJIT-benches/md5.lua                  |  201 ++++
>   perf/LuaJIT-benches/meteor.lua               |  251 +++++
>   perf/LuaJIT-benches/nbody.lua                |  146 +++
>   perf/LuaJIT-benches/nsieve-bit-fp.lua        |   69 ++
>   perf/LuaJIT-benches/nsieve-bit.lua           |   58 +
>   perf/LuaJIT-benches/nsieve.lua               |   53 +
>   perf/LuaJIT-benches/partialsums.lua          |   48 +
>   perf/LuaJIT-benches/pidigits-nogmp.lua       |  128 +++
>   perf/LuaJIT-benches/ray.lua                  |  166 +++
>   perf/LuaJIT-benches/recursive-ack.lua        |   28 +
>   perf/LuaJIT-benches/recursive-fib.lua        |   36 +
>   perf/LuaJIT-benches/revcomp.lua              |   71 ++
>   perf/LuaJIT-benches/scimark-2010-12-20.lua   |  412 ++++++++
>   perf/LuaJIT-benches/scimark-fft.lua          |   18 +
>   perf/LuaJIT-benches/scimark-lu.lua           |   19 +
>   perf/LuaJIT-benches/scimark-mc.lua           |   19 +
>   perf/LuaJIT-benches/scimark-sor.lua          |   19 +
>   perf/LuaJIT-benches/scimark-sparse.lua       |   19 +
>   perf/LuaJIT-benches/series.lua               |   46 +
>   perf/LuaJIT-benches/spectral-norm.lua        |   65 ++
>   perf/LuaJIT-benches/sum-file.lua             |   29 +
>   perf/helpers/aggregate.lua                   |  132 +++
>   perf/helpers/setup_env.sh                    |  135 +++
>   perf/utils/bench.lua                         |  511 +++++++++
>   perf/utils/clock.lua                         |   35 +
>   52 files changed, 5555 insertions(+)
>   create mode 100644 .github/actions/setup-performance/README.md
>   create mode 100644 .github/actions/setup-performance/action.yml
>   create mode 100644 .github/workflows/performance.yml
>   create mode 100644 perf/CMakeLists.txt
>   create mode 100644 perf/LuaJIT-benches/CMakeLists.txt
>   create mode 100644 perf/LuaJIT-benches/PARAM_arm.txt
>   create mode 100644 perf/LuaJIT-benches/PARAM_mips.txt
>   create mode 100644 perf/LuaJIT-benches/PARAM_ppc.txt
>   create mode 100644 perf/LuaJIT-benches/PARAM_x86.txt
>   create mode 100644 perf/LuaJIT-benches/SUMCOL_1.txt
>   create mode 100644 perf/LuaJIT-benches/TEST_md5sum.txt
>   create mode 100644 perf/LuaJIT-benches/array3d.lua
>   create mode 100644 perf/LuaJIT-benches/binary-trees.lua
>   create mode 100644 perf/LuaJIT-benches/chameneos.lua
>   create mode 100644 perf/LuaJIT-benches/coroutine-ring.lua
>   create mode 100644 perf/LuaJIT-benches/euler14-bit.lua
>   create mode 100644 perf/LuaJIT-benches/fannkuch.lua
>   create mode 100644 perf/LuaJIT-benches/fasta.lua
>   create mode 100644 perf/LuaJIT-benches/k-nucleotide.lua
>   create mode 100644 perf/LuaJIT-benches/libs/fasta.lua
>   create mode 100644 perf/LuaJIT-benches/libs/scimark_lib.lua
>   create mode 100644 perf/LuaJIT-benches/life.lua
>   create mode 100644 perf/LuaJIT-benches/mandelbrot-bit.lua
>   create mode 100644 perf/LuaJIT-benches/mandelbrot.lua
>   create mode 100644 perf/LuaJIT-benches/md5.lua
>   create mode 100644 perf/LuaJIT-benches/meteor.lua
>   create mode 100644 perf/LuaJIT-benches/nbody.lua
>   create mode 100644 perf/LuaJIT-benches/nsieve-bit-fp.lua
>   create mode 100644 perf/LuaJIT-benches/nsieve-bit.lua
>   create mode 100644 perf/LuaJIT-benches/nsieve.lua
>   create mode 100644 perf/LuaJIT-benches/partialsums.lua
>   create mode 100644 perf/LuaJIT-benches/pidigits-nogmp.lua
>   create mode 100644 perf/LuaJIT-benches/ray.lua
>   create mode 100644 perf/LuaJIT-benches/recursive-ack.lua
>   create mode 100644 perf/LuaJIT-benches/recursive-fib.lua
>   create mode 100644 perf/LuaJIT-benches/revcomp.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-2010-12-20.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-fft.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-lu.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-mc.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-sor.lua
>   create mode 100644 perf/LuaJIT-benches/scimark-sparse.lua
>   create mode 100644 perf/LuaJIT-benches/series.lua
>   create mode 100644 perf/LuaJIT-benches/spectral-norm.lua
>   create mode 100644 perf/LuaJIT-benches/sum-file.lua
>   create mode 100644 perf/helpers/aggregate.lua
>   create mode 100755 perf/helpers/setup_env.sh
>   create mode 100644 perf/utils/bench.lua
>   create mode 100644 perf/utils/clock.lua
>

[-- Attachment #2: Type: text/html, Size: 9837 bytes --]