From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6883E6ECC0; Wed, 6 Apr 2022 15:52:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6883E6ECC0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1649249570; bh=vqEYjMaObWj8FiHoPtiVL+Irr3I7lDdAQU2n9QvQTIE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=u2o4n0vEtcjOisqQnH5v5t5PPkBpxQvNNHgLXSOe1eMAjt39oVrxmEfFlyG4e1nfv 237Zkl5qT2wody70CV9ZOyIx4LZMIVPHABZ5+pfK/XK86L93b8DPpf0KSVqJMeXxNu by209Kft72JjhuXiyOeOF8Hz8GJT3s2uQfXIwfdo= Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 533ED6ECC0 for ; Wed, 6 Apr 2022 15:49:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 533ED6ECC0 Received: by mail-lf1-f52.google.com with SMTP id bu29so4005154lfb.0 for ; Wed, 06 Apr 2022 05:49:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aienxHRTt5VcFsXn52A0bCQUEt+RAbJkqIk2CikqUFo=; b=6+pyUyo2A4UNWoEnIuvt43cecomnWqpRn9kBaO/VY41a1lDSz0w93MQn9QNas2FFBX Zcgrk9dB7IVkTmeZJB5SWN9vWN407JFD/anbOtp+Slz8fMlxHT6AjTRFX4MBd4iViMso TaGl0eHci35BgrkTzA33ozKQ5WmVhJ/vrtu4TRhOi+700qEQbqEtXl5yF9o5UqP66hYC qMcazBw2k+Bm4u+nfkTG9gOPnR3SYe2RlFBZnyXyVQoM5TpH+Cb9R9GLdrHgJJNHJzj3 xrE64B5EovDfTztMvtubxSs1vPZATcqDLfNdy4Tjq/un+ECL1HMfyb0T7cCLHn6XCuOD RuRw== X-Gm-Message-State: AOAM533XH19bWNYruorABOnlSnfkLnQShZhpARghvDIHgZw+LLr4amYF VXXgn8aSC5A5hvJV2paslcFfKI7uYndQmA== X-Google-Smtp-Source: ABdhPJz6SRHRB3Nho7YCpUb7zcbR6mcELFVFRSbNJn/vT6v/wyfWAoWlqLUtuLSMeXR8c2TDBgekag== X-Received: by 2002:a05:6512:3b2c:b0:44a:35fd:994c with SMTP id f44-20020a0565123b2c00b0044a35fd994cmr5852868lfv.473.1649249396452; Wed, 06 Apr 2022 05:49:56 -0700 (PDT) Received: from localhost.localdomain ([93.175.28.20]) by smtp.gmail.com with ESMTPSA id i25-20020a0565123e1900b0044a2908b040sm1821375lfv.115.2022.04.06.05.49.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Apr 2022 05:49:56 -0700 (PDT) X-Google-Original-From: Maxim Kokryashkin To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Wed, 6 Apr 2022 15:49:38 +0300 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit v3 6/7] sysprof: introduce Lua API X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maxim Kokryashkin via Tarantool-patches Reply-To: Maxim Kokryashkin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This commit introduces Lua API for sysprof, so it is now possible to call it from anywhere in a Lua script. All of the parameters are passed inside a single Lua table. The following options are supported: - mode: "D" (Default mode, only virtual machine state counters) "L" (Leaf mode, shows the last frame on the stack) "C" (Callchain mode, full stack dump) - interval: time between samples in milliseconds - path: Path to file in which profiling data should be stored Part of tarantool/tarantool#781 --- src/lib_misc.c | 238 +++++++++++++++++- .../misclib-sysprof-lapi.test.lua | 118 +++++++++ 2 files changed, 348 insertions(+), 8 deletions(-) create mode 100644 test/tarantool-tests/misclib-sysprof-lapi.test.lua diff --git a/src/lib_misc.c b/src/lib_misc.c index 22d29d78..e93cf55b 100644 --- a/src/lib_misc.c +++ b/src/lib_misc.c @@ -109,14 +109,14 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, lua_assert(len <= STREAM_BUFFER_SIZE); for (;;) { - const size_t written = write(fd, data, len - write_total); + const ssize_t written = write(fd, data, len - write_total); if (LJ_UNLIKELY(written == -1)) { /* Re-tries write in case of EINTR. */ if (errno != EINTR) { - /* Will be freed as whole chunk later. */ - *buf_addr = NULL; - return write_total; + /* Will be freed as whole chunk later. */ + *buf_addr = NULL; + return write_total; } errno = 0; @@ -139,16 +139,236 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, /* Default on stop callback. Just close the corresponding descriptor. */ static int on_stop_cb_default(void *opt, uint8_t *buf) { - struct profile_ctx *ctx = opt; - const int fd = ctx->fd; + struct profile_ctx *ctx = NULL; + int fd = 0; + + if (opt == NULL) { + /* Nothing to do. */ + return 0; + } + + ctx = opt; + fd = ctx->fd; UNUSED(buf); lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return close(fd); } +/* ----- misc.sysprof module ---------------------------------------------- */ + +#define LJLIB_MODULE_misc_sysprof + +/* The default profiling interval equals to 11 ms. */ +#define SYSPROF_DEFAULT_INTERVAL 10 +#define SYSPROF_DEFAULT_OUTPUT "sysprof.bin" + +int set_output_path(const char *path, struct luam_sysprof_options *opt) { + struct profile_ctx *ctx = opt->ctx; + int fd = 0; + lua_assert(path != NULL); + fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0644); + if(fd == -1) { + return SYSPROF_ERRIO; + } + ctx->fd = fd; + return SYSPROF_SUCCESS; +} + +int parse_sysprof_opts(lua_State *L, struct luam_sysprof_options *opt, int idx) { + GCtab *options = lj_lib_checktab(L, idx); + + /* Get profiling mode. */ + { + const char *mode = NULL; + + cTValue *mode_opt = lj_tab_getstr(options, lj_str_newlit(L, "mode")); + if (!mode_opt || !tvisstr(mode_opt)) { + return SYSPROF_ERRUSE; + } + + mode = strVdata(mode_opt); + if (mode[1] != '\0') + return SYSPROF_ERRUSE; + + switch (*mode) { + case 'D': + opt->mode = LUAM_SYSPROF_DEFAULT; + break; + case 'L': + opt->mode = LUAM_SYSPROF_LEAF; + break; + case 'C': + opt->mode = LUAM_SYSPROF_CALLGRAPH; + break; + default: + return SYSPROF_ERRUSE; + } + } + + /* Get profiling interval. */ + { + cTValue *interval = lj_tab_getstr(options, lj_str_newlit(L, "interval")); + opt->interval = SYSPROF_DEFAULT_INTERVAL; + if (interval && tvisnum(interval)) { + int32_t signed_interval = numberVint(interval); + if (signed_interval < 1) + return SYSPROF_ERRUSE; + opt->interval = signed_interval; + } + } + + /* Get output path. */ + if (opt->mode != LUAM_SYSPROF_DEFAULT) + { + const char *path = NULL; + struct profile_ctx *ctx = NULL; + int status = 0; + + cTValue *pathtv = lj_tab_getstr(options, lj_str_newlit(L, "path")); + if (!pathtv) + path = SYSPROF_DEFAULT_OUTPUT; + else if (!tvisstr(pathtv)) + return SYSPROF_ERRUSE; + else + path = strVdata(pathtv); + + ctx = lj_mem_new(L, sizeof(*ctx)); + ctx->g = G(L); + opt->ctx = ctx; + opt->buf = ctx->buf; + opt->len = STREAM_BUFFER_SIZE; + + status = set_output_path(path, opt); + if (status != SYSPROF_SUCCESS) { + lj_mem_free(ctx->g, ctx, sizeof(*ctx)); + return status; + } + } + + return SYSPROF_SUCCESS; +} + +int parse_options(lua_State *L, struct luam_sysprof_options *opt) +{ + if (lua_gettop(L) != 1) + return SYSPROF_ERRUSE; + + if (!lua_istable(L, 1)) + return SYSPROF_ERRUSE; + + return parse_sysprof_opts(L, opt, 1); +} + +int sysprof_error(lua_State *L, int status) +{ + switch (status) { + case SYSPROF_ERRUSE: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_MISUSE)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRRUN: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_ISRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRSTOP: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_NOTRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case SYSPROF_ERRIO: + return luaL_fileresult(L, 0, NULL); + default: + lua_assert(0); + return 0; + } +} + +/* local res, err, errno = sysprof.start(options) */ +LJLIB_CF(misc_sysprof_start) +{ + int status = SYSPROF_SUCCESS; + + struct luam_sysprof_options opt = {}; + struct luam_sysprof_config conf = {}; + + + status = parse_options(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + return sysprof_error(L, status); + } + + conf.writer = buffer_writer_default; + conf.on_stop = on_stop_cb_default; + conf.backtracer = NULL; + + status = luaM_sysprof_configure(&conf); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + on_stop_cb_default(opt.ctx, opt.buf); + return sysprof_error(L, status); + } + + status = luaM_sysprof_start(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + /* Allocated memory will be freed in on_stop callback. */ + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local res, err, errno = profile.sysprof_stop() */ +LJLIB_CF(misc_sysprof_stop) +{ + int status = luaM_sysprof_stop(L); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local counters, err, errno = sysprof.report() */ +LJLIB_CF(misc_sysprof_report) +{ + struct luam_sysprof_counters counters = {}; + GCtab *data_tab = NULL; + GCtab *count_tab = NULL; + + int status = luaM_sysprof_report(&counters); + if (status != SYSPROF_SUCCESS) + return sysprof_error(L, status); + + lua_createtable(L, 0, 3); + data_tab = tabV(L->top - 1); + + setnumfield(L, data_tab, "samples", counters.samples); + setnumfield(L, data_tab, "overruns", counters.overruns); + + lua_createtable(L, 0, LJ_VMST__MAX + 1); + count_tab = tabV(L->top - 1); + + setnumfield(L, count_tab, "INTERP", counters.vmst_interp); + setnumfield(L, count_tab, "LFUNC", counters.vmst_lfunc); + setnumfield(L, count_tab, "FFUNC", counters.vmst_ffunc); + setnumfield(L, count_tab, "CFUNC", counters.vmst_cfunc); + setnumfield(L, count_tab, "GC", counters.vmst_gc); + setnumfield(L, count_tab, "EXIT", counters.vmst_exit); + setnumfield(L, count_tab, "RECORD", counters.vmst_record); + setnumfield(L, count_tab, "OPT", counters.vmst_opt); + setnumfield(L, count_tab, "ASM", counters.vmst_asm); + setnumfield(L, count_tab, "TRACE", counters.vmst_trace); + + lua_setfield(L, -2, "vmstate"); + + return 1; +} + /* ----- misc.memprof module ---------------------------------------------- */ #define LJLIB_MODULE_misc_memprof + /* local started, err, errno = misc.memprof.start(fname) */ LJLIB_CF(misc_memprof_start) { @@ -169,9 +389,9 @@ LJLIB_CF(misc_memprof_start) opt.len = STREAM_BUFFER_SIZE; ctx->g = G(L); - ctx->stream = fopen(fname, "wb"); + ctx->fd = open(fname, O_CREAT | O_WRONLY | O_TRUNC); - if (ctx->stream == NULL) { + if (ctx->fd == -1) { lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return luaL_fileresult(L, 0, fname); } @@ -240,5 +460,7 @@ LUALIB_API int luaopen_misc(struct lua_State *L) { LJ_LIB_REG(L, LUAM_MISCLIBNAME, misc); LJ_LIB_REG(L, LUAM_MISCLIBNAME ".memprof", misc_memprof); + LJ_LIB_REG(L, LUAM_MISCLIBNAME ".sysprof", misc_sysprof); + return 1; } diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua new file mode 100644 index 00000000..b4c9cb7b --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua @@ -0,0 +1,118 @@ +-- Sysprof is implemented for x86 and x64 architectures only. +require("utils").skipcond( + jit.arch ~= "x86" and jit.arch ~= "x64", + jit.arch.." architecture is NIY for sysprof" +) + +local tap = require("tap") + +local test = tap.test("misc-sysprof-lapi") +test:plan(14) + +jit.off() +jit.flush() + +local bufread = require("utils.bufread") +local symtab = require("utils.symtab") + +local TMP_BINFILE = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%.%1.sysprofdata.tmp.bin") +local BAD_PATH = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%1/sysprofdata.tmp.bin") + +local function payload() + local function fib(n) + if n <= 1 then + return n + end + return fib(n - 1) + fib(n - 2) + end + return fib(32) +end + +local function generate_output(opts) + local res, err = misc.sysprof.start(opts) + assert(res, err) + + payload() + + res,err = misc.sysprof.stop() + assert(res, err) +end + +local function check_mode(mode, interval) + local res = pcall( + generate_output, + { mode = mode, interval = interval, path = TMP_BINFILE } + ) + + if not res then + test:fail(mode .. ' mode with interval ' .. interval) + os.remove(TMP_BINFILE) + end + + local reader = bufread.new(TMP_BINFILE) + symtab.parse(reader) +end + +-- GENERAL + +-- Wrong profiling mode. +local res, err, errno = misc.sysprof.start{ mode = "A" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- Already running. +res, err = misc.sysprof.start{ mode = "D" } +assert(res, err) + +res, err, errno = misc.sysprof.start{ mode = "D" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +res, err = misc.sysprof.stop() +assert(res, err) + +-- Not running. +res, err, errno = misc.sysprof.stop() +test:ok(res == nil and err:match("profiler is not running")) +test:ok(type(errno) == "number") + +-- Bad path. +res, err, errno = misc.sysprof.start({ mode = "C", path = BAD_PATH }) +test:ok(res == nil and err:match("No such file or directory")) +test:ok(type(errno) == "number") + +-- Bad interval. +res, err, errno = misc.sysprof.start{ mode = "C", interval = -1 } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- DEFAULT MODE + +if not pcall(generate_output, { mode = "D", interval = 11 }) then + test:fail('`default` mode with interval 11') +end + +local report = misc.sysprof.report() + +test:ok(report.samples > 0) +test:ok(report.vmstate.LFUNC > 0) +test:ok(report.vmstate.TRACE == 0) + +-- With very big interval. +if not pcall(generate_output, { mode = "D", interval = 1000 }) then + test:fail('`default` mode with interval 1000') +end + +report = misc.sysprof.report() +test:ok(report.samples == 0) + +-- LEAF MODE +check_mode("L", 11) + +-- CALL MODE +check_mode("C", 11) + +os.remove(TMP_BINFILE) + +jit.on() +os.exit(test:check() and 0 or 1) -- 2.35.1