From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 3FD036ECC0; Wed, 13 Apr 2022 23:39:23 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3FD036ECC0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1649882363; bh=D0bgD7l7iAu7bw9wEHgybWCd9VWTpIqhTW0RWkS3yyQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=k+qHY4Vv5s0GnzwXYh962HGK6pwGfLc/59Nuq3MPzYohh0CM6Jz/N6H6GoNyUpf69 mLHU/QCCoJ4ft6g0qFW60IJ0/PYeYGhVLz0/oLLGviygVRwCd06H48z6YgS2plIDWP ljpChKXrXth72i97YO467kbDyy2TwGi6RZhzfBLM= Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 419C06EFF5 for ; Wed, 13 Apr 2022 23:36:25 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 419C06EFF5 Received: by mail-lj1-f169.google.com with SMTP id r2so3614069ljd.10 for ; Wed, 13 Apr 2022 13:36:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vll6xjkpuJREkuuuT2TmEHAyoK7LINp3n6nC/1s61l8=; b=i3uNoklJe/FbTQ7jBOorNGvJlslQ1aa3MuAl2hCETNIHIbcDqI2ynNWDqQEbJ1p77o ThtLHj7hk2MvbBtkInHvokDh0mThIzBSTLtczeEoHVA6TXrS5i6Jhyu0eG6kjThUkmO5 1VWmIgzBQfHtb3/agZg0Kyetv2m6odB8KNEcCNqxlFnHSkqixf5W3FU26q4KfxQ6S5Da Oi2D2eej7+fvJxCuJopVnsQ+MdsMiyL/zAH9wNooDMuUSw8fLIWlqNyQeP6jFxRHXgHe hF3jzuGCcoG4kMz5ajGa/hwridk6+7uqjGqLOqWHarn6kSYKDGXjwxam8tsQfTd6LAaW NtfQ== X-Gm-Message-State: AOAM533+85hWOMXO9uDiPbw73TWC9/lLe/UY/K6uL4QvAb0gJ9knTRlU HG2evFwzEpxczG1YKlO3pf9K65Kc1KafJg== X-Google-Smtp-Source: ABdhPJxs22G7sxeFD3By67utfHhBBwVi39mvpdAgzDr4U06WZMQzd2uBssx0ThI/u3fczaX7gwyplw== X-Received: by 2002:a2e:9797:0:b0:24b:693c:91ea with SMTP id y23-20020a2e9797000000b0024b693c91eamr8014245lji.400.1649882184529; Wed, 13 Apr 2022 13:36:24 -0700 (PDT) Received: from localhost.localdomain ([93.175.11.199]) by smtp.gmail.com with ESMTPSA id e15-20020a19500f000000b0046bb76678bcsm2389lfb.131.2022.04.13.13.36.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 13:36:24 -0700 (PDT) X-Google-Original-From: Maxim Kokryashkin To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Wed, 13 Apr 2022 23:36:12 +0300 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit v4 6/7] sysprof: introduce Lua API X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maxim Kokryashkin via Tarantool-patches Reply-To: Maxim Kokryashkin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" This commit introduces Lua API for sysprof, so it is now possible to call it from anywhere in a Lua script. All of the parameters are passed inside a single Lua table. The following options are supported: - mode: "D" (Default mode, only virtual machine state counters) "L" (Leaf mode, shows the last frame on the stack) "C" (Callchain mode, full stack dump) - interval: time between samples in milliseconds - path: Path to file in which profiling data should be stored Part of tarantool/tarantool#781 --- src/lib_misc.c | 233 +++++++++++++++++- .../misclib-sysprof-lapi.test.lua | 118 +++++++++ 2 files changed, 343 insertions(+), 8 deletions(-) create mode 100644 test/tarantool-tests/misclib-sysprof-lapi.test.lua diff --git a/src/lib_misc.c b/src/lib_misc.c index 22d29d78..0fe6b708 100644 --- a/src/lib_misc.c +++ b/src/lib_misc.c @@ -109,14 +109,14 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, lua_assert(len <= STREAM_BUFFER_SIZE); for (;;) { - const size_t written = write(fd, data, len - write_total); + const ssize_t written = write(fd, data, len - write_total); if (LJ_UNLIKELY(written == -1)) { /* Re-tries write in case of EINTR. */ if (errno != EINTR) { - /* Will be freed as whole chunk later. */ - *buf_addr = NULL; - return write_total; + /* Will be freed as whole chunk later. */ + *buf_addr = NULL; + return write_total; } errno = 0; @@ -139,16 +139,231 @@ static size_t buffer_writer_default(const void **buf_addr, size_t len, /* Default on stop callback. Just close the corresponding descriptor. */ static int on_stop_cb_default(void *opt, uint8_t *buf) { - struct profile_ctx *ctx = opt; - const int fd = ctx->fd; + struct profile_ctx *ctx = NULL; + int fd = 0; + + if (opt == NULL) { + /* Nothing to do. */ + return 0; + } + + ctx = opt; + fd = ctx->fd; UNUSED(buf); lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return close(fd); } +/* ----- misc.sysprof module ---------------------------------------------- */ + +#define LJLIB_MODULE_misc_sysprof + +/* The default profiling interval equals to 11 ms. */ +#define SYSPROF_DEFAULT_INTERVAL 10 +#define SYSPROF_DEFAULT_OUTPUT "sysprof.bin" + +int set_output_path(const char *path, struct luam_Sysprof_Options *opt) { + struct profile_ctx *ctx = opt->ctx; + int fd = 0; + lua_assert(path != NULL); + fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0644); + if(fd == -1) { + return PROFILE_ERRIO; + } + ctx->fd = fd; + return PROFILE_SUCCESS; +} + +int parse_sysprof_opts(lua_State *L, struct luam_Sysprof_Options *opt, int idx) { + GCtab *options = lj_lib_checktab(L, idx); + + /* Get profiling mode. */ + { + const char *mode = NULL; + + cTValue *mode_opt = lj_tab_getstr(options, lj_str_newlit(L, "mode")); + if (!mode_opt || !tvisstr(mode_opt)) { + return PROFILE_ERRUSE; + } + + mode = strVdata(mode_opt); + if (mode[1] != '\0') + return PROFILE_ERRUSE; + + switch (*mode) { + case 'D': + opt->mode = LUAM_SYSPROF_DEFAULT; + break; + case 'L': + opt->mode = LUAM_SYSPROF_LEAF; + break; + case 'C': + opt->mode = LUAM_SYSPROF_CALLGRAPH; + break; + default: + return PROFILE_ERRUSE; + } + } + + /* Get profiling interval. */ + { + cTValue *interval = lj_tab_getstr(options, lj_str_newlit(L, "interval")); + opt->interval = SYSPROF_DEFAULT_INTERVAL; + if (interval && tvisnum(interval)) { + int32_t signed_interval = numberVint(interval); + if (signed_interval < 1) + return PROFILE_ERRUSE; + opt->interval = signed_interval; + } + } + + /* Get output path. */ + if (opt->mode != LUAM_SYSPROF_DEFAULT) + { + const char *path = NULL; + struct profile_ctx *ctx = NULL; + int status = 0; + + cTValue *pathtv = lj_tab_getstr(options, lj_str_newlit(L, "path")); + if (!pathtv) + path = SYSPROF_DEFAULT_OUTPUT; + else if (!tvisstr(pathtv)) + return PROFILE_ERRUSE; + else + path = strVdata(pathtv); + + ctx = lj_mem_new(L, sizeof(*ctx)); + ctx->g = G(L); + opt->ctx = ctx; + opt->buf = ctx->buf; + opt->len = STREAM_BUFFER_SIZE; + + status = set_output_path(path, opt); + if (status != PROFILE_SUCCESS) { + lj_mem_free(ctx->g, ctx, sizeof(*ctx)); + return status; + } + } + + return PROFILE_SUCCESS; +} + +int parse_options(lua_State *L, struct luam_Sysprof_Options *opt) +{ + if (lua_gettop(L) != 1) + return PROFILE_ERRUSE; + + if (!lua_istable(L, 1)) + return PROFILE_ERRUSE; + + return parse_sysprof_opts(L, opt, 1); +} + +int sysprof_error(lua_State *L, int status) +{ + switch (status) { + case PROFILE_ERRUSE: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_MISUSE)); + lua_pushinteger(L, EINVAL); + return 3; + case PROFILE_ERRRUN: + lua_pushnil(L); + lua_pushstring(L, err2msg(LJ_ERR_PROF_ISRUNNING)); + lua_pushinteger(L, EINVAL); + return 3; + case PROFILE_ERRIO: + return luaL_fileresult(L, 0, NULL); + default: + lua_assert(0); + return 0; + } +} + +/* local res, err, errno = sysprof.start(options) */ +LJLIB_CF(misc_sysprof_start) +{ + int status = PROFILE_SUCCESS; + + struct luam_Sysprof_Options opt = {}; + struct luam_Sysprof_Config conf = {}; + + + status = parse_options(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + return sysprof_error(L, status); + } + + conf.writer = buffer_writer_default; + conf.on_stop = on_stop_cb_default; + conf.backtracer = NULL; + + status = luaM_sysprof_configure(&conf); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) { + on_stop_cb_default(opt.ctx, opt.buf); + return sysprof_error(L, status); + } + + status = luaM_sysprof_start(L, &opt); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + /* Allocated memory will be freed in on_stop callback. */ + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local res, err, errno = profile.sysprof_stop() */ +LJLIB_CF(misc_sysprof_stop) +{ + int status = luaM_sysprof_stop(L); + if (LJ_UNLIKELY(status != PROFILE_SUCCESS)) + return sysprof_error(L, status); + + lua_pushboolean(L, 1); + return 1; +} + +/* local counters, err, errno = sysprof.report() */ +LJLIB_CF(misc_sysprof_report) +{ + struct luam_Sysprof_Counters counters = {}; + GCtab *data_tab = NULL; + GCtab *count_tab = NULL; + + int status = luaM_sysprof_report(&counters); + if (status != PROFILE_SUCCESS) + return sysprof_error(L, status); + + lua_createtable(L, 0, 3); + data_tab = tabV(L->top - 1); + + setnumfield(L, data_tab, "samples", counters.samples); + setnumfield(L, data_tab, "overruns", counters.overruns); + + lua_createtable(L, 0, LJ_VMST__MAX + 1); + count_tab = tabV(L->top - 1); + + setnumfield(L, count_tab, "INTERP", counters.vmst_interp); + setnumfield(L, count_tab, "LFUNC", counters.vmst_lfunc); + setnumfield(L, count_tab, "FFUNC", counters.vmst_ffunc); + setnumfield(L, count_tab, "CFUNC", counters.vmst_cfunc); + setnumfield(L, count_tab, "GC", counters.vmst_gc); + setnumfield(L, count_tab, "EXIT", counters.vmst_exit); + setnumfield(L, count_tab, "RECORD", counters.vmst_record); + setnumfield(L, count_tab, "OPT", counters.vmst_opt); + setnumfield(L, count_tab, "ASM", counters.vmst_asm); + setnumfield(L, count_tab, "TRACE", counters.vmst_trace); + + lua_setfield(L, -2, "vmstate"); + + return 1; +} + /* ----- misc.memprof module ---------------------------------------------- */ #define LJLIB_MODULE_misc_memprof + /* local started, err, errno = misc.memprof.start(fname) */ LJLIB_CF(misc_memprof_start) { @@ -169,9 +384,9 @@ LJLIB_CF(misc_memprof_start) opt.len = STREAM_BUFFER_SIZE; ctx->g = G(L); - ctx->stream = fopen(fname, "wb"); + ctx->fd = open(fname, O_CREAT | O_WRONLY | O_TRUNC); - if (ctx->stream == NULL) { + if (ctx->fd == -1) { lj_mem_free(ctx->g, ctx, sizeof(*ctx)); return luaL_fileresult(L, 0, fname); } @@ -240,5 +455,7 @@ LUALIB_API int luaopen_misc(struct lua_State *L) { LJ_LIB_REG(L, LUAM_MISCLIBNAME, misc); LJ_LIB_REG(L, LUAM_MISCLIBNAME ".memprof", misc_memprof); + LJ_LIB_REG(L, LUAM_MISCLIBNAME ".sysprof", misc_sysprof); + return 1; } diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua new file mode 100644 index 00000000..625bdf55 --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua @@ -0,0 +1,118 @@ +-- Sysprof is implemented for x86 and x64 architectures only. +require("utils").skipcond( + jit.arch ~= "x86" and jit.arch ~= "x64", + jit.arch.." architecture is NIY for sysprof" +) + +local tap = require("tap") + +local test = tap.test("misc-sysprof-lapi") +test:plan(14) + +jit.off() +jit.flush() + +local bufread = require("utils.bufread") +local symtab = require("utils.symtab") + +local TMP_BINFILE = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%.%1.sysprofdata.tmp.bin") +local BAD_PATH = arg[0]:gsub(".+/([^/]+)%.test%.lua$", "%1/sysprofdata.tmp.bin") + +local function payload() + local function fib(n) + if n <= 1 then + return n + end + return fib(n - 1) + fib(n - 2) + end + return fib(32) +end + +local function generate_output(opts) + local res, err = misc.sysprof.start(opts) + assert(res, err) + + payload() + + res,err = misc.sysprof.stop() + assert(res, err) +end + +local function check_mode(mode, interval) + local res = pcall( + generate_output, + { mode = mode, interval = interval, path = TMP_BINFILE } + ) + + if not res then + test:fail(mode .. ' mode with interval ' .. interval) + os.remove(TMP_BINFILE) + end + + local reader = bufread.new(TMP_BINFILE) + symtab.parse(reader) +end + +-- GENERAL + +-- Wrong profiling mode. +local res, err, errno = misc.sysprof.start{ mode = "A" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- Already running. +res, err = misc.sysprof.start{ mode = "D" } +assert(res, err) + +res, err, errno = misc.sysprof.start{ mode = "D" } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +res, err = misc.sysprof.stop() +assert(res, err) + +-- Not running. +res, err, errno = misc.sysprof.stop() +test:ok(res == nil and err) +test:ok(type(errno) == "number") + +-- Bad path. +res, err, errno = misc.sysprof.start({ mode = "C", path = BAD_PATH }) +test:ok(res == nil and err:match("No such file or directory")) +test:ok(type(errno) == "number") + +-- Bad interval. +res, err, errno = misc.sysprof.start{ mode = "C", interval = -1 } +test:ok(res == nil and err:match("profiler misuse")) +test:ok(type(errno) == "number") + +-- DEFAULT MODE + +if not pcall(generate_output, { mode = "D", interval = 11 }) then + test:fail('`default` mode with interval 11') +end + +local report = misc.sysprof.report() + +test:ok(report.samples > 0) +test:ok(report.vmstate.LFUNC > 0) +test:ok(report.vmstate.TRACE == 0) + +-- With very big interval. +if not pcall(generate_output, { mode = "D", interval = 1000 }) then + test:fail('`default` mode with interval 1000') +end + +report = misc.sysprof.report() +test:ok(report.samples == 0) + +-- LEAF MODE +check_mode("L", 11) + +-- CALL MODE +check_mode("C", 11) + +os.remove(TMP_BINFILE) + +jit.on() +os.exit(test:check() and 0 or 1) -- 2.35.1