From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 1D1A16ECE8; Wed, 20 Apr 2022 16:01:16 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1D1A16ECE8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1650459676; bh=sgcnkcSWooo+8BDzVqqaaDuL9bpDsDEl6yqCfyuSR90=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=xEaxGo1JsxayNX2nsmJp0YzcGYdM9fUea/iDr0lS50gdBaml55N4E2jdX5dltLdbH Jh29kIn6Hvmr6OPnNsZkVHadg11QS8F8oy78EG2+hCwr4qFnPz2p2JLvpS3YkTyzFK GjbJjyPxiQXR183ewIWd7OL36stwhkguz1D8Huk0= Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id E95276F179 for ; Wed, 20 Apr 2022 15:58:19 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E95276F179 Received: by mail-lf1-f49.google.com with SMTP id y32so2837270lfa.6 for ; Wed, 20 Apr 2022 05:58:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kAcuQmm8Uq9u+LjH9P1JohEX977YYWNdv9lxs8T2cCQ=; b=1cbkAflSgETdS60C1Xs7FVFTpLx8h9ubFhUPJpLpu/MMxHuZZLvBTU/U4wa7WpA1B8 qgSh4K7JYYEfz5kdWqm3BykcNsrGAuqJmQxfQwtMhwRMfj5I/jMN/0u4EVy4hqpNJ52K jvVR8UcpTkR8VeSmNRX+AyJBRngCArvLifVC9boSmMmjjcTQJYt7R2Z4+jRBU9iuofYt M9Dwkgs5sw+FwnfMVNdU8gRhEU3Q3Uc++fYdPIshh936J/q53dewO6cCuW3Xl2TOXajj YSb4n0+PPGNt9g9In6Hju1eEakWsmBh//UTag2pHh4mIj7XDsvw/RGxNam/q4PwaBCMw NObg== X-Gm-Message-State: AOAM530pOM2r4QA0CyvlHgFgXk3NJl5I0osGOEX+wXNHBIxPGrhTF7wW XKorublGJCYa4v92fFPvT5zyD6AIUfpNyA== X-Google-Smtp-Source: ABdhPJw8w6TEscSYbm1MDXfcgcIPVlNRYw7BLlt89HGwr8FQXropUY//iF/o31ZoRRPdQU7oCOMWKQ== X-Received: by 2002:a05:6512:139d:b0:46b:af15:a15b with SMTP id p29-20020a056512139d00b0046baf15a15bmr15254070lfa.36.1650459499272; Wed, 20 Apr 2022 05:58:19 -0700 (PDT) Received: from localhost.localdomain ([93.175.28.57]) by smtp.gmail.com with ESMTPSA id o3-20020a198c03000000b00448b7b1780csm1824076lfd.63.2022.04.20.05.58.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Apr 2022 05:58:19 -0700 (PDT) X-Google-Original-From: Maxim Kokryashkin To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Wed, 20 Apr 2022 15:58:02 +0300 Message-Id: <66c12af5025e5491186393a18df92ef0c2777ede.1650459246.git.m.kokryashkin@tarantool.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit v5 6/7] sysprof: introduce Lua API X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maxim Kokryashkin via Tarantool-patches Reply-To: Maxim Kokryashkin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This commit introduces Lua API for sysprof, so it is now possible to call it from anywhere in a Lua script. All of the parameters are passed inside a single Lua table. The following options are supported: - mode: "D" (Default mode, only virtual machine state counters) "L" (Leaf mode, shows the last frame on the stack) "C" (Callchain mode, full stack dump) - interval: time between samples in milliseconds - path: Path to file in which profiling data should be stored Part of tarantool/tarantool#781 --- src/lib_misc.c | 230 +++++++++++++++++- .../misclib-sysprof-lapi.test.lua | 120 +++++++++ 2 files changed, 344 insertions(+), 6 deletions(-) create mode 100644 test/tarantool-tests/misclib-sysprof-lapi.test.lua diff --git a/src/lib_misc.c b/src/lib_misc.c index bfeddc6a..e07f1b32 100644 --- a/src/lib_misc.c +++ b/src/lib_misc.c @@ -109,14 +109,14 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, lua_assert(len <= STREAM_BUFFER_SIZE); for (;;) { - const size_t written = write(fd, data, len - write_total); + const ssize_t written = write(fd, data, len - write_total); if (LJ_UNLIKELY(written == -1)) { /* Re-tries write in case of EINTR. */ if (errno != EINTR) { - /* Will be freed as whole chunk later. */ - *buf_addr = NULL; - return write_total; + /* Will be freed as whole chunk later. */ + *buf_addr = NULL; + return write_total; } errno = 0; @@ -139,16 +139,232 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, /* Default on stop callback. Just close the corresponding descriptor. */ static int on_stop_cb_default(void *opt, uint8_t *buf) { - struct profile_ctx *ctx = opt; - const int fd = ctx->fd; + struct profile_ctx *ctx = NULL; + int fd = 0; + + if (opt == NULL) { + /* Nothing to do. */ + return 0; + } + + ctx = opt; + fd = ctx->fd; UNUSED(buf); lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return close(fd); } +/* ----- misc.sysprof module ---------------------------------------------- */ + +#define LJLIB_MODULE_misc_sysprof + +/* The default profiling interval equals to 11 ms. */ +#define SYSPROF_DEFAULT_INTERVAL 10 +#define SYSPROF_DEFAULT_OUTPUT "sysprof.bin" + +int set_output_path(const char *path, struct luam_Sysprof_Options *opt) { + struct profile_ctx *ctx = opt->ctx; + int fd = 0; + lua_assert(path != NULL); + fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0644); + if(fd == -1) { + return PROFILE_ERRIO; + } + ctx->fd = fd; + return PROFILE_SUCCESS; +} + +int parse_sysprof_opts(lua_State *L, struct luam_Sysprof_Options *opt, int idx) { + GCtab *options = lj_lib_checktab(L, idx); + + /* Get profiling mode. */ + { + const char *mode = NULL; + + cTValue *mode_opt = lj_tab_getstr(options, lj_str_newlit(L, "mode")); + if (!mode_opt || !tvisstr(mode_opt)) { + return PROFILE_ERRUSE; + } + + mode = strVdata(mode_opt); + if (mode[1] != '\0') + return PROFILE_ERRUSE; + + switch (*mode) { + case 'D': + opt->mode = LUAM_SYSPROF_DEFAULT; + break; + case 'L': + opt->mode = LUAM_SYSPROF_LEAF; + break; + case 'C': + opt->mode = LUAM_SYSPROF_CALLGRAPH; + break; + default: + return PROFILE_ERRUSE; + } + } + + /* Get profiling interval. */ + { + cTValue *interval = lj_tab_getstr(options, lj_str_newlit(L, "interval")); + opt->interval = SYSPROF_DEFAULT_INTERVAL; + if (interval && tvisnum(interval)) { + int32_t signed_interval = numberVint(interval); + if (signed_interval < 1) + return PROFILE_ERRUSE; + opt->interval = signed_interval; + } + } + + /* Get output path. */ + if (opt->mode != LUAM_SYSPROF_DEFAULT) + { + const char *path = NULL; + struct profile_ctx *ctx = NULL; + int status = 0; + + cTValue *pathtv = lj_tab_getstr(options, lj_str_newlit(L, "path")); + if (!pathtv) + path = SYSPROF_DEFAULT_OUTPUT; + else if (!tvisstr(pathtv)) + return PROFILE_ERRUSE; + else + path = strVdata(pathtv); + + ctx = lj_mem_new(L, sizeof(*ctx)); + ctx->g = G(L); + opt->ctx = ctx; + opt->buf = ctx->buf; + opt->len = STREAM_BUFFER_SIZE; + + status = set_output_path(path, opt); + if (status != PROFILE_SUCCESS) { + lj_mem_free(ctx->g, ctx, sizeof(*ctx)); + return status; + } + } + + return PROFILE_SUCCESS; +} + +int parse_options(lua_State *L, struct luam_Sysprof_Options *opt) +{ + if (lua_gettop(L) != 1) + return PROFILE_ERRUSE; + + if (!lua_istable(L, 1)) + return PROFILE_ERRUSE; + + return parse_sysprof_opts(L, opt, 1); +} + +int sysprof_error(lua_State *L, int status) +{ + switch (status) { + case PROFILE_ERRUSE: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_MISUSE)); + lua_pushinteger(L, EINVAL); + return 3; +#if LJ_HASSYSPROF + case PROFILE_ERRRUN: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_ISRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case PROFILE_ERRIO: + return luaL_fileresult(L, 0, NULL); +#endif + default: + lua_assert(0); + return 0; + } +} + +/* local res, err, errno = sysprof.start(options) */ +LJLIB_CF(misc_sysprof_start) +{ + int status = PROFILE_SUCCESS; + + struct luam_Sysprof_Options opt = {}; + struct luam_Sysprof_Config conf = {}; + + + status = parse_options(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + return sysprof_error(L, status); + } + + conf.writer = buffer_writer_default; + conf.on_stop = on_stop_cb_default; + conf.backtracer = NULL; + + status = luaM_sysprof_configure(&conf); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + on_stop_cb_default(opt.ctx, opt.buf); + return sysprof_error(L, status); + } + + status = luaM_sysprof_start(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + /* Allocated memory will be freed in on_stop callback. */ + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local res, err, errno = profile.sysprof_stop() */ +LJLIB_CF(misc_sysprof_stop) +{ + int status = luaM_sysprof_stop(L); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local counters, err, errno = sysprof.report() */ +LJLIB_CF(misc_sysprof_report) +{ + struct luam_Sysprof_Counters counters = {}; + GCtab *data_tab = NULL; + GCtab *count_tab = NULL; + + int status = luaM_sysprof_report(&counters); + if (status != PROFILE_SUCCESS) + return sysprof_error(L, status); + + lua_createtable(L, 0, 3); + data_tab = tabV(L->top - 1); + + setnumfield(L, data_tab, "samples", counters.samples); + + lua_createtable(L, 0, LJ_VMST__MAX + 1); + count_tab = tabV(L->top - 1); + + setnumfield(L, count_tab, "INTERP", counters.vmst_interp); + setnumfield(L, count_tab, "LFUNC", counters.vmst_lfunc); + setnumfield(L, count_tab, "FFUNC", counters.vmst_ffunc); + setnumfield(L, count_tab, "CFUNC", counters.vmst_cfunc); + setnumfield(L, count_tab, "GC", counters.vmst_gc); + setnumfield(L, count_tab, "EXIT", counters.vmst_exit); + setnumfield(L, count_tab, "RECORD", counters.vmst_record); + setnumfield(L, count_tab, "OPT", counters.vmst_opt); + setnumfield(L, count_tab, "ASM", counters.vmst_asm); + setnumfield(L, count_tab, "TRACE", counters.vmst_trace); + + lua_setfield(L, -2, "vmstate"); + + return 1; +} + /* ----- misc.memprof module ---------------------------------------------- */ #define LJLIB_MODULE_misc_memprof + /* local started, err, errno = misc.memprof.start(fname) */ LJLIB_CF(misc_memprof_start) { @@ -240,5 +456,7 @@ LUALIB_API int luaopen_misc(struct lua_State *L) { LJ_LIB_REG(L, LUAM_MISCLIBNAME, misc); LJ_LIB_REG(L, LUAM_MISCLIBNAME ".memprof", misc_memprof); + LJ_LIB_REG(L, LUAM_MISCLIBNAME ".sysprof", misc_sysprof); + return 1; } diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua new file mode 100644 index 00000000..49c92bf9 --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua @@ -0,0 +1,120 @@ +-- Sysprof is implemented for x86 and x64 architectures only. +require("utils").skipcond( + (jit.arch ~= "x86" and jit.arch ~= "x64") + or jit.os == "OSX", + jit.arch.." architecture or "..jit.os.. + " OS is NIY for sysprof" +) + +local tap = require("tap") + +local test = tap.test("misc-sysprof-lapi") +test:plan(14) + +jit.off() +jit.flush() + +local bufread = require("utils.bufread") +local symtab = require("utils.symtab") + +local TMP_BINFILE = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%.%1.sysprofdata.tmp.bin") +local BAD_PATH = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%1/sysprofdata.tmp.bin") + +local function payload() + local function fib(n) + if n <= 1 then + return n + end + return fib(n - 1) + fib(n - 2) + end + return fib(32) +end + +local function generate_output(opts) + local res, err = misc.sysprof.start(opts) + assert(res, err) + + payload() + + res,err = misc.sysprof.stop() + assert(res, err) +end + +local function check_mode(mode, interval) + local res = pcall( + generate_output, + { mode = mode, interval = interval, path = TMP_BINFILE } + ) + + if not res then + test:fail(mode .. ' mode with interval ' .. interval) + os.remove(TMP_BINFILE) + end + + local reader = bufread.new(TMP_BINFILE) + symtab.parse(reader) +end + +-- GENERAL + +-- Wrong profiling mode. +local res, err, errno = misc.sysprof.start{ mode = "A" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- Already running. +res, err = misc.sysprof.start{ mode = "D" } +assert(res, err) + +res, err, errno = misc.sysprof.start{ mode = "D" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +res, err = misc.sysprof.stop() +assert(res, err) + +-- Not running. +res, err, errno = misc.sysprof.stop() +test:ok(res == nil and err) +test:ok(type(errno) == "number") + +-- Bad path. +res, err, errno = misc.sysprof.start({ mode = "C", path = BAD_PATH }) +test:ok(res == nil and err:match("No such file or directory")) +test:ok(type(errno) == "number") + +-- Bad interval. +res, err, errno = misc.sysprof.start{ mode = "C", interval = -1 } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- DEFAULT MODE + +if not pcall(generate_output, { mode = "D", interval = 11 }) then + test:fail('`default` mode with interval 11') +end + +local report = misc.sysprof.report() + +test:ok(report.samples > 0) +test:ok(report.vmstate.LFUNC > 0) +test:ok(report.vmstate.TRACE == 0) + +-- With very big interval. +if not pcall(generate_output, { mode = "D", interval = 1000 }) then + test:fail('`default` mode with interval 1000') +end + +report = misc.sysprof.report() +test:ok(report.samples == 0) + +-- LEAF MODE +check_mode("L", 11) + +-- CALL MODE +check_mode("C", 11) + +os.remove(TMP_BINFILE) + +jit.on() +os.exit(test:check() and 0 or 1) -- 2.35.1