From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 21A7116C8480; Fri, 26 Dec 2025 12:37:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 21A7116C8480 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1766741846; bh=n8tuOjqbTeR8HZTlw7yMiLZBbVk7onYBiNsk/WWUUlI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=fkQBiC+WfrkVdcL19UUvcOl4KzzOn2KCu/4q/ZYR2eLWI7bwSjsaFPtkt+GT52zdz XCoUqxoQY1ytRglXKuuoNzvj/HNnxjoze+R1myv1psPJYdcNwFjD7OL//CHfHkoOFY bHj78gp5KHP5l4W2j1gususq2NV1ROuJcpSBPOgE= Received: from send81.i.mail.ru (send81.i.mail.ru [89.221.237.176]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2DC5716C65BE for ; Fri, 26 Dec 2025 12:20:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2DC5716C65BE Received: by exim-smtp-7b4fb89df9-lft7n with esmtpa (envelope-from ) id 1vZ40C-000000008Cp-0Rkb; Fri, 26 Dec 2025 12:20:53 +0300 To: Sergey Bronnikov Date: Fri, 26 Dec 2025 12:18:09 +0300 Message-ID: <10bf3f3231ee28fa6ce11de0ef8669153f40cef4.1766738771.git.skaplun@tarantool.org> X-Mailer: git-send-email 2.52.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD979975AF0D777FEBDB5A0B7D2FFA411B6E217FB390AD6DF18182A05F53808504013350FED3E18E84C3DE06ABAFEAF6705AB8DA30376D9E0C3B4FA2C6BBBE99EF994B2BCDCC18A84B3 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F6EE1C40B2E8BE15EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC83A81C8FD4AD23D82A6BABE6F325AC2E85FA5F3EDFCBAA7353EFBB55337566657B88B02DF8C869ADC079598865A94C8EDCD39A7D089D1AC0A14D7A192C950C389733CBF5DBD5E913377AFFFEAFD269176DF2183F8FC7C0A29E2F051442AF778941B15DA834481FCF19DD082D7633A0EF3E4896CB9E6436389733CBF5DBD5E9D5E8D9A59859A8B6D52CD31C43BF465FCC7F00164DA146DA6F5DAA56C3B73B237318B6A418E8EAB86D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE76515C59FC18CEA6D731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: 0D63561A33F958A53BFB96D7A6D504875002B1117B3ED696664BFB579D4E567C361FAC1196A180DE823CB91A9FED034534781492E4B8EEAD577AE849BCD98940C79554A2A72441328621D336A7BC284946AD531847A6065A535571D14F44ED41 X-C8649E89: 1C3962B70DF3F0AD73CAD6646DEDE191716CD42B3DD1D34CAB70F9BE574AE9C625B6776AC983F447FC0B9F89525902EE6F57B2FD27647F25E66C117BDB76D6595C68EFED8469F574D0F1F0EBEF1AB5F508D2A00CA703640DDD51D425972D719E8A07714C2307A7E7B8341EE9D5BE9A0A9355529C8A5E871905D8439B09F6DFF0323646C721F2FEF16536EB022892E5344C41F94D744909CECFA6C6B0C050A61A8CAF69B82BA93681CD72808BE417F3B9E0E7457915DAA85F X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu53w8ahmwBjZKM/YPHZyZHvz5uv+WouB9+ObcCpyrx6l7KImUglyhkEat/+ysWwi0gdhEs0JGjl6ggRWTy1haxBpVdbIX1nthFXMZebaIdHP2ghjoIc/363UZI6Kf1ptIMVdtTL5f5BIXbHuBex3Q/TDU= X-DA7885C5: E28B64262C9BEDDEF255D290C0D534F9450753E91E1A116B93DCF91599D20E5D8729452976AB954C5B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393DDD5FD59B456EAD2471B4803F9C48E3D8700636FE984EBB8ECE1EAFE8D8F8B8EE49D44BB4BD9522A059A1ED8796F048DB274557F927329BE89D5A3BC2B10C37545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: [Tarantool-patches] [PATCH v2 luajit 38/41] perf: add aggregator helper for bench statistics X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This patch adds a helper script to aggregate the benchmark results from JSON files to the format compatible with the InfluxDB line protocol [1]. The script takes 2 command line arguments: | luajit aggregate.lua output_file [input_dir] If `input_dir` isn't given, it uses the current directory by default. The script requires the `git` command or specified the `PERF_COMMIT`, `PERF_BRANCH` environment variables. Also, it requires the `cjson` module. All JSON files from each suite in the directory are considered as the benchmark results and aggregated into the file that can be posted to the InfluxDB. The results are aggregated via the new target LuaJIT-perf-aggregate. [1]: https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/ --- perf/CMakeLists.txt | 13 ++++ perf/helpers/aggregate.lua | 132 +++++++++++++++++++++++++++++++++++++ 2 files changed, 145 insertions(+) create mode 100644 perf/helpers/aggregate.lua diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt index 1c2f8e8e..b6e98808 100644 --- a/perf/CMakeLists.txt +++ b/perf/CMakeLists.txt @@ -97,3 +97,16 @@ add_custom_target(${PROJECT_NAME}-perf add_custom_target(${PROJECT_NAME}-perf-console DEPENDS LuaJIT-benches-console ) + +set(PERF_SUMMARY ${PERF_OUTPUT_DIR}/summary.txt) +add_custom_target(${PROJECT_NAME}-perf-aggregate + BYPRODUCTS ${PERF_SUMMARY} + COMMENT "Aggregate performance test results into ${PERF_SUMMARY}" + COMMAND ${CMAKE_COMMAND} -E env + LUA_CPATH="${LUA_CPATH}" + ${LUAJIT_BINARY} ${CMAKE_CURRENT_SOURCE_DIR}/helpers/aggregate.lua + ${PERF_SUMMARY} + ${PERF_OUTPUT_DIR} + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + DEPENDS luajit-main +) diff --git a/perf/helpers/aggregate.lua b/perf/helpers/aggregate.lua new file mode 100644 index 00000000..01410a47 --- /dev/null +++ b/perf/helpers/aggregate.lua @@ -0,0 +1,132 @@ +local json = require('cjson') + +-- File to aggregate the benchmark results from JSON files to the +-- format parsable by the InfluxDB line protocol [1]: +-- , +-- +-- and have the following format: +-- =,= +-- +-- The reported tag set is a set of values that can be used for +-- filtering data (i.e., branch or benchmark name). +-- +-- luacheck: push no max comment line length +-- +-- [1]: https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/ +-- +-- luacheck: pop +-- +-- The script takes 2 command line arguments: +-- | luajit aggregate.lua output_file [input_dir] +-- If `input_dir` isn't given, it uses the current directory by +-- default. +-- The script requires the `git` command or specified +-- `PERF_COMMIT`, `PERF_BRANCH` environment variables. Also, it +-- requires the `cjson` module. + +local output = assert(arg[1], 'Output file is required as the first argument') +local input_dir = arg[2] or '.' + +local out_fh = assert(io.open(output, 'w+')) + +local function exec(cmd) + return io.popen(cmd):read('*all'):gsub('%s+$', '') +end + +local commit = os.getenv('PERF_COMMIT') or exec('git rev-parse --short HEAD') +assert(commit, 'can not determine the commit') + +local branch = os.getenv('PERF_BRANCH') or + exec('git rev-parse --abbrev-ref HEAD') +assert(branch, 'can not determine the branch') + +-- Not very robust, but OK for our needs. +local function listdir(path) + local handle = io.popen('ls -1 ' .. path) + + local files = {} + for file in handle:lines() do + table.insert(files, file) + end + + return files +end + +local tag_set = {branch = branch} + +local function table_plain_copy(src) + local dst = {} + for k, v in pairs(src) do + dst[k] = v + end + return dst +end + +local function read_all(file) + local fh = assert(io.open(file, 'rb')) + local content = fh:read('*all') + fh:close() + return content +end + +local REPORTED_FIELDS = { + 'cpu_time', + 'items_per_second', + 'iterations', + 'real_time', +} + +local function influx_kv(tab) + local kv_string = {} + for k, v in pairs(tab) do + table.insert(kv_string, ('%s=%s'):format(k, v)) + end + return table.concat(kv_string, ',') +end + +local time = os.time() +local function influx_line(measurement, tags, fields) + return ('%s,%s %s %d\n'):format(measurement, influx_kv(tags), + influx_kv(fields), time) +end + +for _, suite_name in pairs(listdir(input_dir)) do + -- May list the report file, but will be ignored by the + -- condition below. + local suite_dir = ('%s/%s'):format(input_dir, suite_name) + for _, file in pairs(listdir(suite_dir)) do + -- Skip files in which we are not interested. + if not file:match('%.json$') then goto continue end + + local data = read_all(('%s/%s'):format(suite_dir, file)) + local bench_name = file:match('([^/]+)%.json') + local bench_data = json.decode(data) + local benchmarks = bench_data.benchmarks + local arch = bench_data.context.arch + local gc64 = bench_data.context.gc64 + local jit = bench_data.context.jit + + for _, bench in ipairs(benchmarks) do + local full_tag_set = table_plain_copy(tag_set) + full_tag_set.name = bench.name + full_tag_set.suite = suite_name + full_tag_set.arch = arch + full_tag_set.gc64 = gc64 + full_tag_set.jit = jit + + -- Save the commit as a field, since we don't want to filter + -- benchmarks by the commit (one point of data). + local field_set = {commit = ('"%s"'):format(commit)} + + for _, field in ipairs(REPORTED_FIELDS) do + field_set[field] = bench[field] + end + + local line = influx_line(bench_name, full_tag_set, field_set) + out_fh:write(line) + end + ::continue:: + end +end + +out_fh:close() -- 2.52.0