From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9EA636ECDB; Wed, 16 Mar 2022 21:57:29 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9EA636ECDB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1647457049; bh=BY5PtlakdZKS4Hi4WlQKWVJnlrVS/DFapYTjhA/TnOc=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=eZzpMS9TfaKEZeB43f8s8hSH95r3in+67caCHSnGRI6dBk5BLNmsxxqfkv3WnxEB8 cB5rey1cTHZC8+X1wqaqUJ7WjqtPpdZjlopu4yafVwWaLWloP6lQ+Y0EdNFYRWiH90 TwKIWns6IZM+pPQrftB0liD2J6vDDe9yGjolqDhE= Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 523686ECDB for ; Wed, 16 Mar 2022 21:57:25 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 523686ECDB Received: by mail-wr1-f48.google.com with SMTP id d7so4287091wrb.7 for ; Wed, 16 Mar 2022 11:57:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=LA/D2GD9xkwp+YFyiQrz+kw5u5YUeWnN600c35y/WLA=; b=N0IV8xX/FgLa5sTVKBwdeQJTdDwBWO9zermlYLSSvXjASS9uLi+GYdr+IusMYktksb lBTJg1Gll3dJQw7YVTCw1sEAP6y+qEmH02DGKm/aJqraDwV1+DIb4DFQ5XH+h5tBgCyo jkfwR39oZ9CvUeIxhcJsWpzU30dnRR66m24xAHwrGuEqqjJCCSS5sdMU96J3FqUocQS5 GRxz0frQZmGzfGmclRKZnXBMccpdaHYSy2kLs98w6wGluexlidiZjE0nEoFPV+oeGVke 3FovBEoDL5RSoQkrpT8HJ4w7aWFq2JtnlzWm8A0OZ+4ZYZJ7ebS7doTSmLeNYjIqaKhr p9Cw== X-Gm-Message-State: AOAM531BtGdcFflhXcmn8IitgRR1uHjX3KGkv1Cr2Y3BkWaRCR28Klnu NbDEjW54kR9I14Q2XguMkkCSlSl5eysvcQ== X-Google-Smtp-Source: ABdhPJyn8lZ1ThT6cI+OsWMEE6dYjmgLMNR7v81rsOMHHR775ofGpN/TTinJwvcvyMnS0PK+ZE0NiA== X-Received: by 2002:adf:ba05:0:b0:1ef:f640:ea0e with SMTP id o5-20020adfba05000000b001eff640ea0emr1012705wrg.59.1647457043642; Wed, 16 Mar 2022 11:57:23 -0700 (PDT) Received: from localhost.localdomain ([152.67.141.72]) by smtp.gmail.com with ESMTPSA id c12-20020a05600c0a4c00b00381141f4967sm3125504wmq.35.2022.03.16.11.57.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Mar 2022 11:57:23 -0700 (PDT) X-Google-Original-From: Maxim Kokryashkin To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Wed, 16 Mar 2022 21:57:19 +0300 Message-Id: <20220316185719.1017050-1-m.kokryashkin@tarantool.org> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit v2 6/7] sysprof: introduce Lua API X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maxim Kokryashkin via Tarantool-patches Reply-To: Maxim Kokryashkin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This commit introduces Lua API for sysprof, so it is now possible to call it from anywhere in a Lua script. All of the parameters are passed inside a single Lua table. The following options are supported: - mode: "D" (Default mode, only virtual machine state counters) "L" (Leaf mode, shows the last frame on the stack) "C" (Callchain mode, full stack dump) - interval: time between samples in milliseconds - path: Path to file in which profiling data should be stored Part of tarantool/tarantool#781 --- >> +#define SYSPROF_DEFAULT_INTERVAL (11) >Why 11 and not 10, 9 or something else? No reason for that. Changed it to 10, so it less questionable now. >> +int sysprof_error(lua_State *L, int status) >> +{ >> + switch (status) { >> + case SYSPROF_ERRUSE: >> + lua_pushnil(L); >> + lua_pushstring(L, err2msg(LJ_ERR_PROF_MISUSE)); >> + lua_pushinteger(L, EINVAL); >> + return 3; >> + case SYSPROF_ERRRUN: > >LJ_ERR_PROF_ISRUNNING and LJ_ERR_PROF_NOTRUNNING is defined only, when >LJ_HASMEMPROF || LJ_HASSYSPROF, so it must be the corresponding ifdef >here. What value should be returned if we don't have neither sysprof nor memprof? src/lib_misc.c | 238 +++++++++++++++++- .../misclib-sysprof-lapi.test.lua | 118 +++++++++ 2 files changed, 348 insertions(+), 8 deletions(-) create mode 100644 test/tarantool-tests/misclib-sysprof-lapi.test.lua diff --git a/src/lib_misc.c b/src/lib_misc.c index 22d29d78..e93cf55b 100644 --- a/src/lib_misc.c +++ b/src/lib_misc.c @@ -109,14 +109,14 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, lua_assert(len <= STREAM_BUFFER_SIZE); for (;;) { - const size_t written = write(fd, data, len - write_total); + const ssize_t written = write(fd, data, len - write_total); if (LJ_UNLIKELY(written == -1)) { /* Re-tries write in case of EINTR. */ if (errno != EINTR) { - /* Will be freed as whole chunk later. */ - *buf_addr = NULL; - return write_total; + /* Will be freed as whole chunk later. */ + *buf_addr = NULL; + return write_total; } errno = 0; @@ -139,16 +139,236 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, /* Default on stop callback. Just close the corresponding descriptor. */ static int on_stop_cb_default(void *opt, uint8_t *buf) { - struct profile_ctx *ctx = opt; - const int fd = ctx->fd; + struct profile_ctx *ctx = NULL; + int fd = 0; + + if (opt == NULL) { + /* Nothing to do. */ + return 0; + } + + ctx = opt; + fd = ctx->fd; UNUSED(buf); lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return close(fd); } +/* ----- misc.sysprof module ---------------------------------------------- */ + +#define LJLIB_MODULE_misc_sysprof + +/* The default profiling interval equals to 11 ms. */ +#define SYSPROF_DEFAULT_INTERVAL 10 +#define SYSPROF_DEFAULT_OUTPUT "sysprof.bin" + +int set_output_path(const char *path, struct luam_sysprof_options *opt) { + struct profile_ctx *ctx = opt->ctx; + int fd = 0; + lua_assert(path != NULL); + fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0644); + if(fd == -1) { + return SYSPROF_ERRIO; + } + ctx->fd = fd; + return SYSPROF_SUCCESS; +} + +int parse_sysprof_opts(lua_State *L, struct luam_sysprof_options *opt, int idx) { + GCtab *options = lj_lib_checktab(L, idx); + + /* Get profiling mode. */ + { + const char *mode = NULL; + + cTValue *mode_opt = lj_tab_getstr(options, lj_str_newlit(L, "mode")); + if (!mode_opt || !tvisstr(mode_opt)) { + return SYSPROF_ERRUSE; + } + + mode = strVdata(mode_opt); + if (mode[1] != '\0') + return SYSPROF_ERRUSE; + + switch (*mode) { + case 'D': + opt->mode = LUAM_SYSPROF_DEFAULT; + break; + case 'L': + opt->mode = LUAM_SYSPROF_LEAF; + break; + case 'C': + opt->mode = LUAM_SYSPROF_CALLGRAPH; + break; + default: + return SYSPROF_ERRUSE; + } + } + + /* Get profiling interval. */ + { + cTValue *interval = lj_tab_getstr(options, lj_str_newlit(L, "interval")); + opt->interval = SYSPROF_DEFAULT_INTERVAL; + if (interval && tvisnum(interval)) { + int32_t signed_interval = numberVint(interval); + if (signed_interval < 1) + return SYSPROF_ERRUSE; + opt->interval = signed_interval; + } + } + + /* Get output path. */ + if (opt->mode != LUAM_SYSPROF_DEFAULT) + { + const char *path = NULL; + struct profile_ctx *ctx = NULL; + int status = 0; + + cTValue *pathtv = lj_tab_getstr(options, lj_str_newlit(L, "path")); + if (!pathtv) + path = SYSPROF_DEFAULT_OUTPUT; + else if (!tvisstr(pathtv)) + return SYSPROF_ERRUSE; + else + path = strVdata(pathtv); + + ctx = lj_mem_new(L, sizeof(*ctx)); + ctx->g = G(L); + opt->ctx = ctx; + opt->buf = ctx->buf; + opt->len = STREAM_BUFFER_SIZE; + + status = set_output_path(path, opt); + if (status != SYSPROF_SUCCESS) { + lj_mem_free(ctx->g, ctx, sizeof(*ctx)); + return status; + } + } + + return SYSPROF_SUCCESS; +} + +int parse_options(lua_State *L, struct luam_sysprof_options *opt) +{ + if (lua_gettop(L) != 1) + return SYSPROF_ERRUSE; + + if (!lua_istable(L, 1)) + return SYSPROF_ERRUSE; + + return parse_sysprof_opts(L, opt, 1); +} + +int sysprof_error(lua_State *L, int status) +{ + switch (status) { + case SYSPROF_ERRUSE: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_MISUSE)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRRUN: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_ISRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRSTOP: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_NOTRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRIO: + return luaL_fileresult(L, 0, NULL); + default: + lua_assert(0); + return 0; + } +} + +/* local res, err, errno = sysprof.start(options) */ +LJLIB_CF(misc_sysprof_start) +{ + int status = SYSPROF_SUCCESS; + + struct luam_sysprof_options opt = {}; + struct luam_sysprof_config conf = {}; + + + status = parse_options(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + return sysprof_error(L, status); + } + + conf.writer = buffer_writer_default; + conf.on_stop = on_stop_cb_default; + conf.backtracer = NULL; + + status = luaM_sysprof_configure(&conf); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + on_stop_cb_default(opt.ctx, opt.buf); + return sysprof_error(L, status); + } + + status = luaM_sysprof_start(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + /* Allocated memory will be freed in on_stop callback. */ + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local res, err, errno = profile.sysprof_stop() */ +LJLIB_CF(misc_sysprof_stop) +{ + int status = luaM_sysprof_stop(L); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local counters, err, errno = sysprof.report() */ +LJLIB_CF(misc_sysprof_report) +{ + struct luam_sysprof_counters counters = {}; + GCtab *data_tab = NULL; + GCtab *count_tab = NULL; + + int status = luaM_sysprof_report(&counters); + if (status != SYSPROF_SUCCESS) + return sysprof_error(L, status); + + lua_createtable(L, 0, 3); + data_tab = tabV(L->top - 1); + + setnumfield(L, data_tab, "samples", counters.samples); + setnumfield(L, data_tab, "overruns", counters.overruns); + + lua_createtable(L, 0, LJ_VMST__MAX + 1); + count_tab = tabV(L->top - 1); + + setnumfield(L, count_tab, "INTERP", counters.vmst_interp); + setnumfield(L, count_tab, "LFUNC", counters.vmst_lfunc); + setnumfield(L, count_tab, "FFUNC", counters.vmst_ffunc); + setnumfield(L, count_tab, "CFUNC", counters.vmst_cfunc); + setnumfield(L, count_tab, "GC", counters.vmst_gc); + setnumfield(L, count_tab, "EXIT", counters.vmst_exit); + setnumfield(L, count_tab, "RECORD", counters.vmst_record); + setnumfield(L, count_tab, "OPT", counters.vmst_opt); + setnumfield(L, count_tab, "ASM", counters.vmst_asm); + setnumfield(L, count_tab, "TRACE", counters.vmst_trace); + + lua_setfield(L, -2, "vmstate"); + + return 1; +} + /* ----- misc.memprof module ---------------------------------------------- */ #define LJLIB_MODULE_misc_memprof + /* local started, err, errno = misc.memprof.start(fname) */ LJLIB_CF(misc_memprof_start) { @@ -169,9 +389,9 @@ LJLIB_CF(misc_memprof_start) opt.len = STREAM_BUFFER_SIZE; ctx->g = G(L); - ctx->stream = fopen(fname, "wb"); + ctx->fd = open(fname, O_CREAT | O_WRONLY | O_TRUNC); - if (ctx->stream == NULL) { + if (ctx->fd == -1) { lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return luaL_fileresult(L, 0, fname); } @@ -240,5 +460,7 @@ LUALIB_API int luaopen_misc(struct lua_State *L) { LJ_LIB_REG(L, LUAM_MISCLIBNAME, misc); LJ_LIB_REG(L, LUAM_MISCLIBNAME ".memprof", misc_memprof); + LJ_LIB_REG(L, LUAM_MISCLIBNAME ".sysprof", misc_sysprof); + return 1; } diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua new file mode 100644 index 00000000..b4c9cb7b --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua @@ -0,0 +1,118 @@ +-- Sysprof is implemented for x86 and x64 architectures only. +require("utils").skipcond( + jit.arch ~= "x86" and jit.arch ~= "x64", + jit.arch.." architecture is NIY for sysprof" +) + +local tap = require("tap") + +local test = tap.test("misc-sysprof-lapi") +test:plan(14) + +jit.off() +jit.flush() + +local bufread = require("utils.bufread") +local symtab = require("utils.symtab") + +local TMP_BINFILE = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%.%1.sysprofdata.tmp.bin") +local BAD_PATH = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%1/sysprofdata.tmp.bin") + +local function payload() + local function fib(n) + if n <= 1 then + return n + end + return fib(n - 1) + fib(n - 2) + end + return fib(32) +end + +local function generate_output(opts) + local res, err = misc.sysprof.start(opts) + assert(res, err) + + payload() + + res,err = misc.sysprof.stop() + assert(res, err) +end + +local function check_mode(mode, interval) + local res = pcall( + generate_output, + { mode = mode, interval = interval, path = TMP_BINFILE } + ) + + if not res then + test:fail(mode .. ' mode with interval ' .. interval) + os.remove(TMP_BINFILE) + end + + local reader = bufread.new(TMP_BINFILE) + symtab.parse(reader) +end + +-- GENERAL + +-- Wrong profiling mode. +local res, err, errno = misc.sysprof.start{ mode = "A" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- Already running. +res, err = misc.sysprof.start{ mode = "D" } +assert(res, err) + +res, err, errno = misc.sysprof.start{ mode = "D" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +res, err = misc.sysprof.stop() +assert(res, err) + +-- Not running. +res, err, errno = misc.sysprof.stop() +test:ok(res == nil and err:match("profiler is not running")) +test:ok(type(errno) == "number") + +-- Bad path. +res, err, errno = misc.sysprof.start({ mode = "C", path = BAD_PATH }) +test:ok(res == nil and err:match("No such file or directory")) +test:ok(type(errno) == "number") + +-- Bad interval. +res, err, errno = misc.sysprof.start{ mode = "C", interval = -1 } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- DEFAULT MODE + +if not pcall(generate_output, { mode = "D", interval = 11 }) then + test:fail('`default` mode with interval 11') +end + +local report = misc.sysprof.report() + +test:ok(report.samples > 0) +test:ok(report.vmstate.LFUNC > 0) +test:ok(report.vmstate.TRACE == 0) + +-- With very big interval. +if not pcall(generate_output, { mode = "D", interval = 1000 }) then + test:fail('`default` mode with interval 1000') +end + +report = misc.sysprof.report() +test:ok(report.samples == 0) + +-- LEAF MODE +check_mode("L", 11) + +-- CALL MODE +check_mode("C", 11) + +os.remove(TMP_BINFILE) + +jit.on() +os.exit(test:check() and 0 or 1) -- 2.35.1