From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 45462482AA0; Fri, 2 Jan 2026 18:53:20 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 45462482AA0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1767369200; bh=fjDQATxO32zHNV7mKifLQ3YB/QSfQae6ucBluJTXmxo=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=noQrHc2mWA/6vmjwBe0BBJ/m2noDuc6ye9QpiQRU89xm3x9kZ5VN4gtggCTi0o5Hb rLUy3fpl0xL4x5Dp8Vf3MdUWH8FZMoQxvhsXkxhpbwUseBAI7lFTMqCInzR9Qakcu/ Y339P7fVU8npmqHFuiVFb1XjuO+d2xsqpXsLfheU= Received: from send241.i.mail.ru (send241.i.mail.ru [95.163.59.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id E0B1A482AA0 for ; Fri, 2 Jan 2026 18:53:18 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E0B1A482AA0 Received: by exim-smtp-7b4fb89df9-xw5z2 with esmtpa (envelope-from ) id 1vbhSn-00000000DUL-3Z0q; Fri, 02 Jan 2026 18:53:18 +0300 Content-Type: multipart/alternative; boundary="------------AVNq0yu0S7tJFJuyX0iT8z76" Message-ID: <02454c83-21b1-4e0a-9a98-b5a9a060d398@tarantool.org> Date: Fri, 2 Jan 2026 18:53:17 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org References: <53e4ef7ccd9ab4aa9877b53bd66a0cce6d146cb3.1766738771.git.skaplun@tarantool.org> In-Reply-To: <53e4ef7ccd9ab4aa9877b53bd66a0cce6d146cb3.1766738771.git.skaplun@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD979975AF0D777FEBD651A111AECFA267DF4E9497E9DD4D3AA182A05F5380850407A532CA1379576103DE06ABAFEAF67051BD4E442A014E0497ACA87A2557ADB1537AD88C62730AFDD X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7EB05B7739F1E6D56EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637FE9EFE935CD7C6AE8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B20E3370F2A27FE7032E070BE324C7D3C4AF5BA25DB5A9BCB5F6B57BC7E64490618DEB871D839B73339E8FC8737B5C22494854413538E1713FCC7F00164DA146DAFE8445B8C89999729449624AB7ADAF37F6B57BC7E64490611E7FA7ABCAF51C92176DF2183F8FC7C091DAD9F922AA71188941B15DA834481F9449624AB7ADAF37BA3038C0950A5D3613377AFFFEAFD269176DF2183F8FC7C046858BEBD2CECBDC7B076A6E789B0E97A8DF7F3B2552694AD5FFEEA1DED7F25D49FD398EE364050F9647ADFADE5905B14AAC223A686B1DECB3661434B16C20ACC84D3B47A649675FE827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B53A69B3AC30C7B9475ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: 6FEFE4C63DFE2D8532C3C7A134B807951D49B9A4D3932D29834459D11680B505EC02D0365D164B59B3A483C71AF18E35 X-C1DE0DAB: 0D63561A33F958A513BE6FB2906436885002B1117B3ED6966413036A069C47E247A99E6294EE8661823CB91A9FED034534781492E4B8EEAD27E9584FBD6BDD31BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34C2AFAD3E4DDF5968D25B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D6598A1389CF55DF31A7D6F02855C263A75E54AE8172CF683A9C0ADA9026EB3E4699DABE3F7BE09CA5F4B8341EE9D5BE9A0AD5830E59C953D86F31A758C1FC77283FD7D9ABFF66EE92D88CD93680B12512CF4C41F94D744909CE2512F26BEC029E55448553D2254B8D95CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVTZJppT4ZVHRRboOPtH2/lQ= X-Mailru-Sender: C4F68CFF4024C8867DFDF7C7F2588458A25A01B1DF55F6A70045A77C8BEB847DD64BDC364F48AD6C76F9B73EB83D6B8E645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 luajit 27/41] perf: adjust scimark-2010-12-20 in LuaJIT-benches X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------AVNq0yu0S7tJFJuyX0iT8z76 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, Sergey! thanks for the patch! LGTM with a minor comment Sergey On 12/26/25 12:17, Sergey Kaplun wrote: > This patch adjusts the aforementioned test to use the benchmark > framework introduced before. The default arguments are adjusted > according to the file. The arguments to the script still > can be provided in the command line run. > > The time for each subsequent benchmark is increased up to 4 seconds, > accoring the defaults in the "bench" framework. The main difference > between this test and others that will be added in next commits is > the usage of FFI arrays instead of plain Lua tables. > --- > perf/LuaJIT-benches/scimark-2010-12-20.lua | 92 ++++++++++++---------- > 1 file changed, 52 insertions(+), 40 deletions(-) > > diff --git a/perf/LuaJIT-benches/scimark-2010-12-20.lua b/perf/LuaJIT-benches/scimark-2010-12-20.lua > index 353acb7c..4b80ffe2 100644 > --- a/perf/LuaJIT-benches/scimark-2010-12-20.lua > +++ b/perf/LuaJIT-benches/scimark-2010-12-20.lua > @@ -9,25 +9,26 @@ > local SCIMARK_VERSION = "2010-12-10" > local SCIMARK_COPYRIGHT = "Copyright (C) 2006-2010 Mike Pall" > > -local MIN_TIME = 2.0 > +local bench = require("bench").new(arg) > + > local RANDOM_SEED = 101009 -- Must be odd. > local SIZE_SELECT = "small" > > local benchmarks = { > "FFT", "SOR", "MC", "SPARSE", "LU", > small = { > - FFT = { 1024 }, > - SOR = { 100 }, > - MC = { }, > - SPARSE = { 1000, 5000 }, > - LU = { 100 }, > + FFT = { params = { 1024 }, cycles = 50000, }, > + SOR = { params = { 100 }, cycles = 50000, }, > + MC = { params = { }, cycles = 15e7, }, > + SPARSE = { params = { 1000, 5000 }, cycles = 15e4, }, > + LU = { params = { 100 }, cycles = 5000, }, > }, > large = { > - FFT = { 1048576 }, > - SOR = { 1000 }, > - MC = { }, > - SPARSE = { 100000, 1000000 }, > - LU = { 1000 }, > + FFT = { params = { 1048576 }, cycles = 25, }, > + SOR = { params = { 1000 }, cycles = 500, }, > + MC = { params = { }, cycles = 15e7, }, > + SPARSE = { params = { 100000, 1000000 }, cycles = 1500, }, > + LU = { params = { 1000 }, cycles = 50, }, > }, > } > > @@ -342,48 +343,48 @@ local function fmtparams(p1, p2) > return "" > end > > -local function measure(min_time, name, ...) > +local function measure(name, cycles, ...) > array_init() > rand_init(RANDOM_SEED) > local run = benchmarks[name](...) > - local cycles = 1 > - repeat > - local tm = clock() > - local flops = run(cycles, ...) > - tm = clock() - tm > - if tm >= min_time then > - local res = flops / tm * 1.0e-6 > - local p1, p2 = ... > - printf("%-7s %8.2f %s\n", name, res, fmtparams(...)) > - return res > - end > - cycles = cycles * 2 > - until false > + local flops = run(cycles, ...) > + return flops > end > > -printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n", > - SCIMARK_VERSION, SCIMARK_COPYRIGHT) > - > while arg and arg[1] do > local a = table.remove(arg, 1) > - if a == "-noffi" then > + if a == "noffi" then > package.preload.ffi = nil > - elseif a == "-small" then > + elseif a == "small" then > SIZE_SELECT = "small" > - elseif a == "-large" then > + elseif a == "large" then these options are absent in usage (--help), why? > SIZE_SELECT = "large" > elseif benchmarks[a] then > - local p = benchmarks[SIZE_SELECT][a] > - measure(MIN_TIME, a, tonumber(arg[1]) or p[1], tonumber(arg[2]) or p[2]) > + local cycles = benchmarks[SIZE_SELECT][a].cycles > + local p = benchmarks[SIZE_SELECT][a].params > + local b > + b = { > + name = a, > + -- XXX: The description of tests for each function is too > + -- inconvenient. > + skip_check = true, > + payload = function() > + local flops = measure(a, cycles, tonumber(arg[1]) or p[1], > + tonumber(arg[2]) or p[2]) > + b.items = flops > + end, > + } > +bench:add(b) > +bench:run_and_report() > return > else > - printf("Usage: scimark [-noffi] [-small|-large] [BENCH params...]\n\n") > - printf("BENCH -small -large\n") > + printf("Usage: scimark [noffi] [small|large] [BENCH params...]\n\n") > + printf("BENCH small large\n") > printf("---------------------------------------\n") > for _,name in ipairs(benchmarks) do > printf("%-7s %-13s %s\n", name, > - fmtparams(unpack(benchmarks.small[name])), > - fmtparams(unpack(benchmarks.large[name]))) > + fmtparams(unpack(benchmarks.small[name].params)), > + fmtparams(unpack(benchmarks.large[name].params))) > end > printf("\n") > os.exit(1) > @@ -393,8 +394,19 @@ end > local params = benchmarks[SIZE_SELECT] > local sum = 0 > for _,name in ipairs(benchmarks) do > - sum = sum + measure(MIN_TIME, name, unpack(params[name])) > + local cycles = params[name].cycles > + local b > + b = { > + name = name, > + -- XXX: The description of tests for each function is too > + -- inconvenient. > + skip_check = true, > + payload = function() > + local flops = measure(name, cycles, unpack(params[name].params)) > + b.items = flops > + end, > + } > +bench:add(b) > end > -printf("\nSciMark %8.2f [%s problem sizes]\n", sum / #benchmarks, SIZE_SELECT) > -io.flush() > > +bench:run_and_report() --------------AVNq0yu0S7tJFJuyX0iT8z76 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hi, Sergey!

thanks for the patch! LGTM with a minor comment

Sergey

On 12/26/25 12:17, Sergey Kaplun wrote:
This patch adjusts the aforementioned test to use the benchmark
framework introduced before. The default arguments are adjusted
according to the <PARAM_x86.txt> file. The arguments to the script still
can be provided in the command line run.

The time for each subsequent benchmark is increased up to 4 seconds,
accoring the defaults in the "bench" framework. The main difference
between this test and others that will be added in next commits is
the usage of FFI arrays instead of plain Lua tables.
---
 perf/LuaJIT-benches/scimark-2010-12-20.lua | 92 ++++++++++++----------
 1 file changed, 52 insertions(+), 40 deletions(-)

diff --git a/perf/LuaJIT-benches/scimark-2010-12-20.lua b/perf/LuaJIT-benches/scimark-2010-12-20.lua
index 353acb7c..4b80ffe2 100644
--- a/perf/LuaJIT-benches/scimark-2010-12-20.lua
+++ b/perf/LuaJIT-benches/scimark-2010-12-20.lua
@@ -9,25 +9,26 @@
 local SCIMARK_VERSION = "2010-12-10"
 local SCIMARK_COPYRIGHT = "Copyright (C) 2006-2010 Mike Pall"
 
-local MIN_TIME = 2.0
+local bench = require("bench").new(arg)
+
 local RANDOM_SEED = 101009 -- Must be odd.
 local SIZE_SELECT = "small"
 
 local benchmarks = {
   "FFT", "SOR", "MC", "SPARSE", "LU",
   small = {
-    FFT		= { 1024 },
-    SOR		= { 100 },
-    MC		= { },
-    SPARSE	= { 1000, 5000 },
-    LU		= { 100 },
+    FFT		= { params = { 1024 }, cycles = 50000, },
+    SOR		= { params = { 100 }, cycles = 50000, },
+    MC		= { params = { }, cycles = 15e7, },
+    SPARSE	= { params = { 1000, 5000 }, cycles = 15e4, },
+    LU		= { params = { 100 }, cycles = 5000, },
   },
   large = {
-    FFT		= { 1048576 },
-    SOR		= { 1000 },
-    MC		= { },
-    SPARSE	= { 100000, 1000000 },
-    LU		= { 1000 },
+    FFT		= { params = { 1048576 }, cycles = 25, },
+    SOR		= { params = { 1000 }, cycles = 500, },
+    MC		= { params = { }, cycles = 15e7, },
+    SPARSE	= { params = { 100000, 1000000 }, cycles = 1500, },
+    LU		= { params = { 1000 }, cycles = 50, },
   },
 }
 
@@ -342,48 +343,48 @@ local function fmtparams(p1, p2)
   return ""
 end
 
-local function measure(min_time, name, ...)
+local function measure(name, cycles, ...)
   array_init()
   rand_init(RANDOM_SEED)
   local run = benchmarks[name](...)
-  local cycles = 1
-  repeat
-    local tm = clock()
-    local flops = run(cycles, ...)
-    tm = clock() - tm
-    if tm >= min_time then
-      local res = flops / tm * 1.0e-6
-      local p1, p2 = ...
-      printf("%-7s %8.2f  %s\n", name, res, fmtparams(...))
-      return res
-    end
-    cycles = cycles * 2
-  until false
+  local flops = run(cycles, ...)
+  return flops
 end
 
-printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n",
-       SCIMARK_VERSION, SCIMARK_COPYRIGHT)
-
 while arg and arg[1] do
   local a = table.remove(arg, 1)
-  if a == "-noffi" then
+  if a == "noffi" then
     package.preload.ffi = nil
-  elseif a == "-small" then
+  elseif a == "small" then
     SIZE_SELECT = "small"
-  elseif a == "-large" then
+  elseif a == "large" then
these options are absent in usage (--help), why? 
     SIZE_SELECT = "large"
   elseif benchmarks[a] then
-    local p = benchmarks[SIZE_SELECT][a]
-    measure(MIN_TIME, a, tonumber(arg[1]) or p[1], tonumber(arg[2]) or p[2])
+    local cycles = benchmarks[SIZE_SELECT][a].cycles
+    local p = benchmarks[SIZE_SELECT][a].params
+    local b
+    b = {
+      name = a,
+      -- XXX: The description of tests for each function is too
+      -- inconvenient.
+      skip_check = true,
+      payload = function()
+        local flops = measure(a, cycles, tonumber(arg[1]) or p[1],
+                              tonumber(arg[2]) or p[2])
+        b.items = flops
+      end,
+    }
+    bench:add(b)
+    bench:run_and_report()
     return
   else
-    printf("Usage: scimark [-noffi] [-small|-large] [BENCH params...]\n\n")
-    printf("BENCH   -small         -large\n")
+    printf("Usage: scimark [noffi] [small|large] [BENCH params...]\n\n")
+    printf("BENCH   small         large\n")
     printf("---------------------------------------\n")
     for _,name in ipairs(benchmarks) do
       printf("%-7s %-13s %s\n", name,
-	     fmtparams(unpack(benchmarks.small[name])),
-	     fmtparams(unpack(benchmarks.large[name])))
+	     fmtparams(unpack(benchmarks.small[name].params)),
+	     fmtparams(unpack(benchmarks.large[name].params)))
     end
     printf("\n")
     os.exit(1)
@@ -393,8 +394,19 @@ end
 local params = benchmarks[SIZE_SELECT]
 local sum = 0
 for _,name in ipairs(benchmarks) do
-  sum = sum + measure(MIN_TIME, name, unpack(params[name]))
+  local cycles = params[name].cycles
+  local b
+  b = {
+    name = name,
+    -- XXX: The description of tests for each function is too
+    -- inconvenient.
+    skip_check = true,
+    payload = function()
+      local flops = measure(name, cycles, unpack(params[name].params))
+      b.items = flops
+    end,
+  }
+  bench:add(b)
 end
-printf("\nSciMark %8.2f  [%s problem sizes]\n", sum / #benchmarks, SIZE_SELECT)
-io.flush()
 
+bench:run_and_report()
--------------AVNq0yu0S7tJFJuyX0iT8z76--