From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 0E6FE1652605; Tue, 18 Nov 2025 15:31:06 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0E6FE1652605 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1763469066; bh=GBZy3YTuFmUTG4CjLV15OOu+Sq7/yBxDAw59r6pWkyI=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=CPlHPw5Ds/3hViwFnfVgXqVxYM2TIbdbI/d7VmJ4Z7VQWhCw4gNdorQxjDr2NRgyx GXnI2iyeRtiEvI2cYntFCyI2kkv0EPMJkAyEab+q9KnGu2et5LrEfC4w5mYMQuQipP DCSz38a9vgM4orl618gCvtmBroZ8xD9CK2yGvPqA= Received: from send57.i.mail.ru (send57.i.mail.ru [89.221.237.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id DDDE61652605 for ; Tue, 18 Nov 2025 15:31:04 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org DDDE61652605 Received: by exim-smtp-88cf54d45-kdbb5 with esmtpa (envelope-from ) id 1vLKrP-000000005yJ-3vh0; Tue, 18 Nov 2025 15:31:04 +0300 Content-Type: multipart/alternative; boundary="------------qoBYosKsBo7C5cUn36NXwj0k" Message-ID: <4187d8d3-4d9d-466c-869c-f3dde532c5e5@tarantool.org> Date: Tue, 18 Nov 2025 15:31:02 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org References: In-Reply-To: X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9231027FDF5B453B633231130F8136BF87EDF8928FCC2CA96182A05F538085040A7AEEE9BCAF4163A3DE06ABAFEAF6705E9019130C082C2021FEE2C31F7B931FF261E30AE1A3D80FE X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7FBB2043146276655EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB55337566E92807B878994EEC7CF37C36BD83BA84B3CC358CA7CAC1DC44A6D32F9364E180389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C07E7E81EEA8A9722B8941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6957A4DEDD2346B42CC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB8D32BA5DBAC0009BE9E8FC8737B5C2249BFC5BFC0DB013F7476E601842F6C81A12EF20D2F80756B5FB606B96278B59C4276E601842F6C81A127C277FBC8AE2E8B81832CA52DEE17263AA81AA40904B5D99C9F4D5AE37F343AD1F44FA8B9022EA23BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735BA6625F88748EAEFC4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5817B1839186B26345002B1117B3ED69659C677C53F1816BA4869453249F34FA4823CB91A9FED034534781492E4B8EEADA2D5570B22232E1EBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC7177DD89D51EBB7742424CF958EAFF5D571004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D347324AA9FA07FF01E0C16D231432B17DD7B236EF69FFD7F53C77A1E9090C0FD0E641F59F0281B38011D7E09C32AA3244C80D17E1CB6F648F177DD89D51EBB7742359758B405814491EA455F16B58544A2E30DDF7C44BCB90DA5AE236DF995FB59978A700BF655EAEEED6A17656DB59BCAD427812AF56FC65B X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVfFX+FmbwDW19EeUrwEGEVc= X-Mailru-Sender: C4F68CFF4024C8867DFDF7C7F2588458B86C125A43F25C63E9E07FF329DA7D47E476ACD20C8E6A34D65783B7BED43171645D15D82EE4B272BD6E4642A116CA93524AA66B5ACBE6721EF430B9A63E2A504198E0F3ECE9B5443453F38A29522196 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v1 luajit 38/41] perf: add aggregator helper for bench statistics X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This is a multi-part message in MIME format. --------------qoBYosKsBo7C5cUn36NXwj0k Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi, Sergey, thanks for the patch! See my comments. Sergey On 10/24/25 14:00, Sergey Kaplun wrote: > This patch adds a helper script to aggregate the benchmark results from > JSON files to the format parsable by the InfluxDB line protocol [1]. format cannot be parsed by protocol, please rephrase. Something like "the format compatible with the InfluxDB line protocol" > > All JSON files from each suite in the directory are > considered as the benchmark results and aggregated into the > file that can be posted to the InfluxDB. The > results are aggregated via the new target LuaJIT-perf-aggregate. may be say that cjson is required? > > [1]:https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/ > --- > perf/CMakeLists.txt | 13 ++++ > perf/helpers/aggregate.lua | 124 +++++++++++++++++++++++++++++++++++++ > 2 files changed, 137 insertions(+) > create mode 100644 perf/helpers/aggregate.lua > > diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt > index cc3c312f..68e561fd 100644 > --- a/perf/CMakeLists.txt > +++ b/perf/CMakeLists.txt > @@ -97,3 +97,16 @@ add_custom_target(${PROJECT_NAME}-perf > add_custom_target(${PROJECT_NAME}-perf-console > DEPENDS LuaJIT-benches-console > ) > + > +set(PERF_SUMMARY ${PERF_OUTPUT_DIR}/summary.txt) > +add_custom_target(${PROJECT_NAME}-perf-aggregate > + BYPRODUCTS ${PERF_SUMMARY} > + COMMENT "Aggregate performance test results into ${PERF_SUMMARY}" > + COMMAND ${CMAKE_COMMAND} -E env > + LUA_CPATH="${LUA_CPATH}" > + ${LUAJIT_BINARY} ${CMAKE_CURRENT_SOURCE_DIR}/helpers/aggregate.lua > + ${PERF_SUMMARY} > + ${PERF_OUTPUT_DIR} > + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} > + DEPENDS luajit-main > +) > diff --git a/perf/helpers/aggregate.lua b/perf/helpers/aggregate.lua > new file mode 100644 > index 00000000..12a8ab89 > --- /dev/null > +++ b/perf/helpers/aggregate.lua > @@ -0,0 +1,124 @@ > +local json = require('cjson') What if cjson is absent? Do we want to handle error? > + > +-- File to aggregate the benchmark results from JSON files to the > +-- format parsable by the InfluxDB line protocol [1]: > +-- , > +-- > +-- and have the following format: > +-- =,= > +-- > +-- The reported tag set is a set of values that can be used for > +-- filtering data (i.e., branch or benchmark name). > +-- > +-- luacheck: push no max comment line length > +-- > +-- [1]:https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/ > +-- > +-- luacheck: pop I propose to document command-line options (1st arg is output file, 2nd arg is a dir, "current dir by default"), env variables (PERF_COMMIT, PERF_BRANCH) and requirements (git is an optional requirement, cjson Lua module is mandatory). > + > +local output = assert(arg[1], 'Output file is required as the first argument') > +local input_dir = arg[2] or '.' > + > +local out_fh = assert(io.open(output, 'w+')) > + > +local function exec(cmd) > + return io.popen(cmd):read('*all'):gsub('%s+$', '') > +end > + > +local commit = os.getenv('PERF_COMMIT') or exec('git rev-parse --short HEAD') > +assert(commit, 'can not determine the commit') > + > +local branch = os.getenv('PERF_BRANCH') or > + exec('git rev-parse --abbrev-ref HEAD') > +assert(branch, 'can not determine the branch') > + > +-- Not very robust, but OK for our needs. > +local function listdir(path) > + local handle = io.popen('ls -1 ' .. path) > + > + local files = {} > + for file inhandle:lines() do > + table.insert(files, file) > + end > + > + return files > +end > + > +local tag_set = {branch = branch} > + > +local function table_plain_copy(src) > + local dst = {} > + for k, v in pairs(src) do > + dst[k] = v > + end > + return dst > +end > + > +local function read_all(file) > + local fh = assert(io.open(file, 'rb')) > + local content =fh:read('*all') > +fh:close() > + return content > +end > + > +local REPORTED_FIELDS = { > + 'cpu_time', > + 'items_per_second', > + 'iterations', > + 'real_time', > +} > + > +local function influx_kv(tab) > + local kv_string = {} > + for k, v in pairs(tab) do > + table.insert(kv_string, ('%s=%s'):format(k, v)) > + end > + return table.concat(kv_string, ',') > +end > + > +local time = os.time() > +local function influx_line(measurement, tags, fields) > + return ('%s,%s %s %d\n'):format(measurement, influx_kv(tags), > + influx_kv(fields), time) > +end > + > +for _, suite_name in pairs(listdir(input_dir)) do > + -- May list the report file, but will be ignored by the > + -- condition below. > + local suite_dir = ('%s/%s'):format(input_dir, suite_name) > + for _, file in pairs(listdir(suite_dir)) do > + -- Skip files in which we are not interested. > + if notfile:match('%.json$') then goto continue end > + > + local data = read_all(('%s/%s'):format(suite_dir, file)) > + local bench_name =file:match('([^/]+)%.json') > + local bench_data = json.decode(data) > + local benchmarks = bench_data.benchmarks > + local arch = bench_data.context.arch > + local gc64 = bench_data.context.gc64 > + local jit = bench_data.context.jit > + > + for _, bench in ipairs(benchmarks) do > + local full_tag_set = table_plain_copy(tag_set) > + full_tag_set.name = bench.name > + full_tag_set.suite = suite_name > + full_tag_set.arch = arch > + full_tag_set.gc64 = gc64 > + full_tag_set.jit = jit > + > + -- Save the commit as a field, since we don't want to filter > + -- benchmarks by the commit (one point of data). > + local field_set = {commit = ('"%s"'):format(commit)} > + > + for _, field in ipairs(REPORTED_FIELDS) do > + field_set[field] = bench[field] > + end > + > + local line = influx_line(bench_name, full_tag_set, field_set) > + out_fh:write(line) > + end > + ::continue:: > + end > +end > + > +out_fh:close() --------------qoBYosKsBo7C5cUn36NXwj0k Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit

Hi, Sergey,

thanks for the patch! See my comments.

Sergey

On 10/24/25 14:00, Sergey Kaplun wrote:
This patch adds a helper script to aggregate the benchmark results from
JSON files to the format parsable by the InfluxDB line protocol [1].

format cannot be parsed by protocol, please rephrase.

Something like "the format compatible with the InfluxDB line protocol"


All JSON files from each suite in the <perf/output> directory are
considered as the benchmark results and aggregated into the
<perf/output/summary.txt> file that can be posted to the InfluxDB. The
results are aggregated via the new target LuaJIT-perf-aggregate.
may be say that cjson is required? 

[1]: https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/
---
 perf/CMakeLists.txt        |  13 ++++
 perf/helpers/aggregate.lua | 124 +++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+)
 create mode 100644 perf/helpers/aggregate.lua

diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt
index cc3c312f..68e561fd 100644
--- a/perf/CMakeLists.txt
+++ b/perf/CMakeLists.txt
@@ -97,3 +97,16 @@ add_custom_target(${PROJECT_NAME}-perf
 add_custom_target(${PROJECT_NAME}-perf-console
   DEPENDS LuaJIT-benches-console
 )
+
+set(PERF_SUMMARY ${PERF_OUTPUT_DIR}/summary.txt)
+add_custom_target(${PROJECT_NAME}-perf-aggregate
+  BYPRODUCTS ${PERF_SUMMARY}
+  COMMENT "Aggregate performance test results into ${PERF_SUMMARY}"
+  COMMAND ${CMAKE_COMMAND} -E env
+    LUA_CPATH="${LUA_CPATH}"
+      ${LUAJIT_BINARY} ${CMAKE_CURRENT_SOURCE_DIR}/helpers/aggregate.lua
+        ${PERF_SUMMARY}
+        ${PERF_OUTPUT_DIR}
+  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+  DEPENDS luajit-main
+)
diff --git a/perf/helpers/aggregate.lua b/perf/helpers/aggregate.lua
new file mode 100644
index 00000000..12a8ab89
--- /dev/null
+++ b/perf/helpers/aggregate.lua
@@ -0,0 +1,124 @@
+local json = require('cjson')
What if cjson is absent? Do we want to handle error? 
+
+-- File to aggregate the benchmark results from JSON files to the
+-- format parsable by the InfluxDB line protocol [1]:
+-- <measurement>,<tag_set> <field_set> <timestamp>
+--
+-- <tag_set> and <field_set> have the following format:
+-- <key1>=<value1>,<key2>=<value2>
+--
+-- The reported tag set is a set of values that can be used for
+-- filtering data (i.e., branch or benchmark name).
+--
+-- luacheck: push no max comment line length
+--
+-- [1]: https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/
+--
+-- luacheck: pop

I propose to document command-line options

(1st arg is output file, 2nd arg is a dir, "current dir by default"),

env variables (PERF_COMMIT, PERF_BRANCH) and requirements

(git is an optional requirement, cjson Lua module is mandatory).

+
+local output = assert(arg[1], 'Output file is required as the first argument')
+local input_dir = arg[2] or '.'
+
+local out_fh = assert(io.open(output, 'w+'))
+
+local function exec(cmd)
+  return io.popen(cmd):read('*all'):gsub('%s+$', '')
+end
+
+local commit = os.getenv('PERF_COMMIT') or exec('git rev-parse --short HEAD')
+assert(commit, 'can not determine the commit')
+
+local branch = os.getenv('PERF_BRANCH') or
+  exec('git rev-parse --abbrev-ref HEAD')
+assert(branch, 'can not determine the branch')
+
+-- Not very robust, but OK for our needs.
+local function listdir(path)
+  local handle = io.popen('ls -1 ' .. path)
+
+  local files = {}
+  for file in handle:lines() do
+    table.insert(files, file)
+  end
+
+  return files
+end
+
+local tag_set = {branch = branch}
+
+local function table_plain_copy(src)
+  local dst = {}
+  for k, v in pairs(src) do
+    dst[k] = v
+  end
+  return dst
+end
+
+local function read_all(file)
+  local fh = assert(io.open(file, 'rb'))
+  local content = fh:read('*all')
+  fh:close()
+  return content
+end
+
+local REPORTED_FIELDS = {
+  'cpu_time',
+  'items_per_second',
+  'iterations',
+  'real_time',
+}
+
+local function influx_kv(tab)
+  local kv_string = {}
+  for k, v in pairs(tab) do
+    table.insert(kv_string, ('%s=%s'):format(k, v))
+  end
+  return table.concat(kv_string, ',')
+end
+
+local time = os.time()
+local function influx_line(measurement, tags, fields)
+  return ('%s,%s %s %d\n'):format(measurement, influx_kv(tags),
+          influx_kv(fields), time)
+end
+
+for _, suite_name in pairs(listdir(input_dir)) do
+  -- May list the report file, but will be ignored by the
+  -- condition below.
+  local suite_dir = ('%s/%s'):format(input_dir, suite_name)
+  for _, file in pairs(listdir(suite_dir)) do
+    -- Skip files in which we are not interested.
+    if not file:match('%.json$') then goto continue end
+
+    local data = read_all(('%s/%s'):format(suite_dir, file))
+    local bench_name = file:match('([^/]+)%.json')
+    local bench_data = json.decode(data)
+    local benchmarks = bench_data.benchmarks
+    local arch = bench_data.context.arch
+    local gc64 = bench_data.context.gc64
+    local jit = bench_data.context.jit
+
+    for _, bench in ipairs(benchmarks) do
+      local full_tag_set = table_plain_copy(tag_set)
+      full_tag_set.name = bench.name
+      full_tag_set.suite = suite_name
+      full_tag_set.arch = arch
+      full_tag_set.gc64 = gc64
+      full_tag_set.jit = jit
+
+      -- Save the commit as a field, since we don't want to filter
+      -- benchmarks by the commit (one point of data).
+      local field_set = {commit = ('"%s"'):format(commit)}
+
+      for _, field in ipairs(REPORTED_FIELDS) do
+          field_set[field] = bench[field]
+      end
+
+      local line = influx_line(bench_name, full_tag_set, field_set)
+      out_fh:write(line)
+    end
+    ::continue::
+  end
+end
+
+out_fh:close()
--------------qoBYosKsBo7C5cUn36NXwj0k--