From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 00B9C158B3AE; Mon, 17 Nov 2025 16:58:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 00B9C158B3AE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1763387929; bh=O69D4RtWL1hOKcibSq2vsf2RAws2TVVnbn3KZGmloPI=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=hB9mMOsK/hsZ0WJbBP2ufILhbL6UKrKGmWrzw7c8jC5DeeBjVQ2tjBfhBXI7ixSyv h6sH/O7W8H/gKhCugQ5K6fG5GVrYWVoUK5tC7BJL4L+fzGVD8aA/5da7+UjLr401fk 6SbDNU/FYFAQsHJHYnGJXkdGmtAcrmKV2XaQe20c= Received: from send265.i.mail.ru (send265.i.mail.ru [95.163.59.104]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 1EDB2158B3A8 for ; Mon, 17 Nov 2025 16:58:47 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1EDB2158B3A8 Received: by exim-smtp-88cf54d45-hj2hr with esmtpa (envelope-from ) id 1vKzkj-00000000CXz-35ml; Mon, 17 Nov 2025 16:58:46 +0300 Content-Type: multipart/alternative; boundary="------------610R6j4kRiXkLvWExD3yC1QS" Message-ID: <617f81c0-e3bb-4ba1-89db-185b7e74fcee@tarantool.org> Date: Mon, 17 Nov 2025 16:56:33 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org References: In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9231027FDF5B453B6816608728D24D50C77B21B7BA0A82510182A05F538085040C1E35FF67AC3319D3DE06ABAFEAF6705AAF399D98DA46668D487FD47310B043B16070DCDF568964E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F9D05773942AAE9CEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB55337566D0BE23D09F2BEA50656772FCF10EEF0BF8C3D20D69E0B1DF1ACC073029C06717389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0D9442B0B5983000E8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6D52CD31C43BF465FCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C224958C1606C78F2434E76E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8B6A4E49BB0F3BA1413AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735BA6625F88748EAEFC4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5A0638285807873FD5002B1117B3ED696A637D27EF4C2D7CED57BAD45EC4C5DE1823CB91A9FED034534781492E4B8EEAD09F854029C6BD0DABDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF1CF13FB8F006A2B3BA8F27B63FACB45B82CD5CDE7F5C30C9F7E0A8DCBB9C882E5896E83BF38A9601DD12B02666E49356EBD5D148A87B402708945569739258E724276C29A5C6B542111DC66A97D0BFE2913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVfFX+FmbwDW1OonJmya+Vms= X-Mailru-Sender: 520A125C2F17F0B1A9638AD358559B59146F403CBB7902E93DE06ABAFEAF6705AAF399D98DA46668B7CBEF92542CD7C8795FA72BAB74744FC77752E0C033A69EA16A481184E8BB1C9B38E6EA4F046BE03A5DB60FBEB33A8A0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v1 luajit 27/41] perf: adjust scimark-2010-12-20 in LuaJIT-benches X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------610R6j4kRiXkLvWExD3yC1QS Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, Sergey, thanks for the patch! See my comments. Sergey On 10/24/25 13:50, Sergey Kaplun wrote: > This patch adjusts the aforementioned test to use the benchmark > framework introduced before. The default arguments are adjusted > according to the file. The arguments to the script still > can be provided in the command line run. > > The time for each subsequent benchmark is increased up to 4 seconds, > accoring the defaults in the "bench" framework. The main difference > between this test and others that will be added in next commits is > the usage of FFI arrays instead of plain Lua tables. > --- > perf/LuaJIT-benches/scimark-2010-12-20.lua | 93 +++++++++++++--------- > 1 file changed, 54 insertions(+), 39 deletions(-) > > diff --git a/perf/LuaJIT-benches/scimark-2010-12-20.lua b/perf/LuaJIT-benches/scimark-2010-12-20.lua > index 353acb7c..3fb627fa 100644 > --- a/perf/LuaJIT-benches/scimark-2010-12-20.lua > +++ b/perf/LuaJIT-benches/scimark-2010-12-20.lua > @@ -9,25 +9,26 @@ > local SCIMARK_VERSION = "2010-12-10" > local SCIMARK_COPYRIGHT = "Copyright (C) 2006-2010 Mike Pall" > > -local MIN_TIME = 2.0 > +local bench = require("bench").new(arg) > + > local RANDOM_SEED = 101009 -- Must be odd. > local SIZE_SELECT = "small" > > local benchmarks = { > "FFT", "SOR", "MC", "SPARSE", "LU", > small = { > - FFT = { 1024 }, > - SOR = { 100 }, > - MC = { }, > - SPARSE = { 1000, 5000 }, > - LU = { 100 }, > + FFT = { params = { 1024 }, cycles = 50000, }, > + SOR = { params = { 100 }, cycles = 50000, }, > + MC = { params = { }, cycles = 15e7, }, > + SPARSE = { params = { 1000, 5000 }, cycles = 15e4, }, > + LU = { params = { 100 }, cycles = 5000, }, > }, > large = { > - FFT = { 1048576 }, > - SOR = { 1000 }, > - MC = { }, > - SPARSE = { 100000, 1000000 }, > - LU = { 1000 }, > + FFT = { params = { 1048576 }, cycles = 25, }, > + SOR = { params = { 1000 }, cycles = 500, }, > + MC = { params = { }, cycles = 15e7, }, > + SPARSE = { params = { 100000, 1000000 }, cycles = 1500, }, > + LU = { params = { 1000 }, cycles = 50, }, > }, > } please add a comment about chosen parameters > > @@ -342,48 +343,51 @@ local function fmtparams(p1, p2) > return "" > end > > -local function measure(min_time, name, ...) > +local function measure(name, cycles, ...) > array_init() > rand_init(RANDOM_SEED) > local run = benchmarks[name](...) > - local cycles = 1 > - repeat > - local tm = clock() > - local flops = run(cycles, ...) > - tm = clock() - tm > - if tm >= min_time then > - local res = flops / tm * 1.0e-6 > - local p1, p2 = ... > - printf("%-7s %8.2f %s\n", name, res, fmtparams(...)) > - return res > - end > - cycles = cycles * 2 > - until false > + local flops = run(cycles, ...) > + return flops > end > > -printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n", > - SCIMARK_VERSION, SCIMARK_COPYRIGHT) > +-- printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n", > +-- SCIMARK_VERSION, SCIMARK_COPYRIGHT) > I propose to move this to a comment with test description. Something like: The test runs the Lua version of SciMark 2.0a, which is a benchmark for scientific and numerical computing developed by programmers at the NIST (National Institute of Standards and Technology). This test is made up of Fast Foruier Transform, Jacobi Successive Over-relaxation, Monte Carlo, Sparse Matrix Multiply, and dense LU matrix factorization benchmarks. plus description of available test-specific options (noffi, small, etc) or just a command-line that will show usage: ./scimark-2010-12-20.lua help > while arg and arg[1] do > local a = table.remove(arg, 1) > - if a == "-noffi" then > + if a == "noffi" then > package.preload.ffi = nil > - elseif a == "-small" then > + elseif a == "small" then > SIZE_SELECT = "small" > - elseif a == "-large" then > + elseif a == "large" then > SIZE_SELECT = "large" > elseif benchmarks[a] then > - local p = benchmarks[SIZE_SELECT][a] > - measure(MIN_TIME, a, tonumber(arg[1]) or p[1], tonumber(arg[2]) or p[2]) > + local cycles = benchmarks[SIZE_SELECT][a].cycles > + local p = benchmarks[SIZE_SELECT][a].params > + local b > + b = { > + name = a, > + -- XXX: The description of tests for each function is too > + -- inconvenient. > + skip_check = true, > + payload = function() > + local flops = measure(a, cycles, tonumber(arg[1]) or p[1], > + tonumber(arg[2]) or p[2]) > + b.items = flops > + end, > + } > +bench:add(b) > +bench:run_and_report() > return > else > - printf("Usage: scimark [-noffi] [-small|-large] [BENCH params...]\n\n") > - printf("BENCH -small -large\n") > + printf("Usage: scimark [noffi] [small|large] [BENCH params...]\n\n") > + printf("BENCH small large\n") > printf("---------------------------------------\n") > for _,name in ipairs(benchmarks) do > printf("%-7s %-13s %s\n", name, > - fmtparams(unpack(benchmarks.small[name])), > - fmtparams(unpack(benchmarks.large[name]))) > + fmtparams(unpack(benchmarks.small[name].params)), > + fmtparams(unpack(benchmarks.large[name].params))) > end > printf("\n") > os.exit(1) > @@ -393,8 +397,19 @@ end > local params = benchmarks[SIZE_SELECT] > local sum = 0 > for _,name in ipairs(benchmarks) do > - sum = sum + measure(MIN_TIME, name, unpack(params[name])) > + local cycles = params[name].cycles > + local b > + b = { > + name = name, > + -- XXX: The description of tests for each function is too > + -- inconvenient. > + skip_check = true, > + payload = function() > + local flops = measure(name, cycles, unpack(params[name].params)) > + b.items = flops > + end, > + } > +bench:add(b) > end > -printf("\nSciMark %8.2f [%s problem sizes]\n", sum / #benchmarks, SIZE_SELECT) > -io.flush() > > +bench:run_and_report() --------------610R6j4kRiXkLvWExD3yC1QS Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hi, Sergey,

thanks for the patch! See my comments.

Sergey

On 10/24/25 13:50, Sergey Kaplun wrote:
This patch adjusts the aforementioned test to use the benchmark
framework introduced before. The default arguments are adjusted
according to the <PARAM_x86.txt> file. The arguments to the script still
can be provided in the command line run.

The time for each subsequent benchmark is increased up to 4 seconds,
accoring the defaults in the "bench" framework. The main difference
between this test and others that will be added in next commits is
the usage of FFI arrays instead of plain Lua tables.
---
 perf/LuaJIT-benches/scimark-2010-12-20.lua | 93 +++++++++++++---------
 1 file changed, 54 insertions(+), 39 deletions(-)

diff --git a/perf/LuaJIT-benches/scimark-2010-12-20.lua b/perf/LuaJIT-benches/scimark-2010-12-20.lua
index 353acb7c..3fb627fa 100644
--- a/perf/LuaJIT-benches/scimark-2010-12-20.lua
+++ b/perf/LuaJIT-benches/scimark-2010-12-20.lua
@@ -9,25 +9,26 @@
 local SCIMARK_VERSION = "2010-12-10"
 local SCIMARK_COPYRIGHT = "Copyright (C) 2006-2010 Mike Pall"
 
-local MIN_TIME = 2.0
+local bench = require("bench").new(arg)
+
 local RANDOM_SEED = 101009 -- Must be odd.
 local SIZE_SELECT = "small"
 
 local benchmarks = {
   "FFT", "SOR", "MC", "SPARSE", "LU",
   small = {
-    FFT		= { 1024 },
-    SOR		= { 100 },
-    MC		= { },
-    SPARSE	= { 1000, 5000 },
-    LU		= { 100 },
+    FFT		= { params = { 1024 }, cycles = 50000, },
+    SOR		= { params = { 100 }, cycles = 50000, },
+    MC		= { params = { }, cycles = 15e7, },
+    SPARSE	= { params = { 1000, 5000 }, cycles = 15e4, },
+    LU		= { params = { 100 }, cycles = 5000, },
   },
   large = {
-    FFT		= { 1048576 },
-    SOR		= { 1000 },
-    MC		= { },
-    SPARSE	= { 100000, 1000000 },
-    LU		= { 1000 },
+    FFT		= { params = { 1048576 }, cycles = 25, },
+    SOR		= { params = { 1000 }, cycles = 500, },
+    MC		= { params = { }, cycles = 15e7, },
+    SPARSE	= { params = { 100000, 1000000 }, cycles = 1500, },
+    LU		= { params = { 1000 }, cycles = 50, },
   },
 }
please add a comment about chosen parameters 
 
@@ -342,48 +343,51 @@ local function fmtparams(p1, p2)
   return ""
 end
 
-local function measure(min_time, name, ...)
+local function measure(name, cycles, ...)
   array_init()
   rand_init(RANDOM_SEED)
   local run = benchmarks[name](...)
-  local cycles = 1
-  repeat
-    local tm = clock()
-    local flops = run(cycles, ...)
-    tm = clock() - tm
-    if tm >= min_time then
-      local res = flops / tm * 1.0e-6
-      local p1, p2 = ...
-      printf("%-7s %8.2f  %s\n", name, res, fmtparams(...))
-      return res
-    end
-    cycles = cycles * 2
-  until false
+  local flops = run(cycles, ...)
+  return flops
 end
 
-printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n",
-       SCIMARK_VERSION, SCIMARK_COPYRIGHT)
+-- printf("Lua SciMark %s based on SciMark 2.0a. %s.\n\n",
+--        SCIMARK_VERSION, SCIMARK_COPYRIGHT)

I propose to move this to a comment with test description.

Something like:

The test runs the Lua version of SciMark 2.0a, which is a benchmark for scientific and numerical computing developed by programmers at the NIST (National Institute of Standards and Technology). This test is made up of Fast Foruier Transform, Jacobi Successive Over-relaxation, Monte Carlo, Sparse Matrix Multiply, and dense LU matrix factorization benchmarks.

plus description of available test-specific options (noffi, small, etc) or just

a command-line that will show usage: ./scimark-2010-12-20.lua help

 while arg and arg[1] do
   local a = table.remove(arg, 1)
-  if a == "-noffi" then
+  if a == "noffi" then
     package.preload.ffi = nil
-  elseif a == "-small" then
+  elseif a == "small" then
     SIZE_SELECT = "small"
-  elseif a == "-large" then
+  elseif a == "large" then
     SIZE_SELECT = "large"
   elseif benchmarks[a] then
-    local p = benchmarks[SIZE_SELECT][a]
-    measure(MIN_TIME, a, tonumber(arg[1]) or p[1], tonumber(arg[2]) or p[2])
+    local cycles = benchmarks[SIZE_SELECT][a].cycles
+    local p = benchmarks[SIZE_SELECT][a].params
+    local b
+    b = {
+      name = a,
+      -- XXX: The description of tests for each function is too
+      -- inconvenient.
+      skip_check = true,
+      payload = function()
+        local flops = measure(a, cycles, tonumber(arg[1]) or p[1],
+                              tonumber(arg[2]) or p[2])
+        b.items = flops
+      end,
+    }
+    bench:add(b)
+    bench:run_and_report()
     return
   else
-    printf("Usage: scimark [-noffi] [-small|-large] [BENCH params...]\n\n")
-    printf("BENCH   -small         -large\n")
+    printf("Usage: scimark [noffi] [small|large] [BENCH params...]\n\n")
+    printf("BENCH   small         large\n")
     printf("---------------------------------------\n")
     for _,name in ipairs(benchmarks) do
       printf("%-7s %-13s %s\n", name,
-	     fmtparams(unpack(benchmarks.small[name])),
-	     fmtparams(unpack(benchmarks.large[name])))
+	     fmtparams(unpack(benchmarks.small[name].params)),
+	     fmtparams(unpack(benchmarks.large[name].params)))
     end
     printf("\n")
     os.exit(1)
@@ -393,8 +397,19 @@ end
 local params = benchmarks[SIZE_SELECT]
 local sum = 0
 for _,name in ipairs(benchmarks) do
-  sum = sum + measure(MIN_TIME, name, unpack(params[name]))
+  local cycles = params[name].cycles
+  local b
+  b = {
+    name = name,
+    -- XXX: The description of tests for each function is too
+    -- inconvenient.
+    skip_check = true,
+    payload = function()
+      local flops = measure(name, cycles, unpack(params[name].params))
+      b.items = flops
+    end,
+  }
+  bench:add(b)
 end
-printf("\nSciMark %8.2f  [%s problem sizes]\n", sum / #benchmarks, SIZE_SELECT)
-io.flush()
 
+bench:run_and_report()
--------------610R6j4kRiXkLvWExD3yC1QS--