From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 3EA696F3FC; Wed, 8 Sep 2021 20:53:00 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 3EA696F3FC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1631123580; bh=NlAchLYFTBH8yRBKHulhSz8rtN4Vb3U64u+3H7+Tfkg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=RUR4v6WQfxUKt/c6/wvYP9GQRS2+3sARAdMeCKSIj508ypcjxc55k5PxWQ0qUkthd Xy0UQS88M24Vy2HrHQHIN/CPEwfKWRTwZPVh6kxLA4ov8Sqx8hQY5De1dPAynZZzr9 jBWkCWngY3JsS/dIcDoOkAuFfrRsbZAO6oZg63oc= Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 334676E468 for ; Wed, 8 Sep 2021 20:51:02 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 334676E468 Received: by mail-lf1-f47.google.com with SMTP id c8so6515178lfi.3 for ; Wed, 08 Sep 2021 10:51:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AvFFWtQo3qVufX3xVpxpxlAbuWGpvEpCJdhRip0xOcQ=; b=ahbXyhqHSkQUmcaceUobWBDEQbMIHC9yuCOtrXlLng7gGcXTp8d9wwAqyUDgQgIeQb ei1fhUZsPXPGpLq1CYfOiqLmTE63AFP/XTTV8Y08Iv2T3U7zuodUzHcSjjBYsAA4SYjD Nr5QEz1geQt15i9YCi54KLYgkbZYuhL9991tvT35RlNMfD7zp1itEbbwh71CEiurLtB8 askGAYIfyCbhuMsjoFk5SWriheT232O5in/I8WWX5DMTx87PYnUls4fZD5IMCUGLW87h uz9hN/0iHVeUjO149f3lu/eFVsBInEIay38/SvorViH9DMEAKvEwk5tgWo4o8XEwxZi2 oUvg== X-Gm-Message-State: AOAM532zouS+HNiV4zLMdJ5ADmk8xFRjcN6KQoake5M/iyZ36pgotltJ +NllG09uLtKdOQPULJBRsc0z8CNNoL5G95CN X-Google-Smtp-Source: ABdhPJxjxIXYVtHHDlZAPQsXcbkCQi57AO0rXyEF85+pN07PsmheLmO47OdfA3eJJh2Kbzus1eaF4g== X-Received: by 2002:ac2:5456:: with SMTP id d22mr3372462lfn.139.1631123460926; Wed, 08 Sep 2021 10:51:00 -0700 (PDT) Received: from localhost.localdomain ([93.175.11.199]) by smtp.gmail.com with ESMTPSA id x13sm249434lfq.262.2021.09.08.10.51.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 10:51:00 -0700 (PDT) X-Google-Original-From: Maxim Kokryashkin To: tarantool-patches@dev.tarantool.org, imun@tarantool.org, skaplun@tarantool.org Date: Wed, 8 Sep 2021 20:50:47 +0300 Message-Id: X-Mailer: git-send-email 2.33.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH luajit 4/7] core: introduce lua and platform profiler X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Maxim Kokryashkin via Tarantool-patches Reply-To: Maxim Kokryashkin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" From: Mikhail Shishatskiy This patch introduces a sampling platform profiler for the Lua machine. The profiler uses the signal sampling backend from the low-level profiler built in vanilla LuaJIT, which was put into a separate module in one of the previous patches. Thus, one cannot use both profilers at the same time. First of all profiler dumps the definitions of all loaded Lua functions and all loaded shared libraries (symtab) via the write buffer introduced in one of the previous patches. As the profiling signal may occur at any time, we need to provide some guarantees for the sampling profiler to unwind consistent stack frames. So, the VM is adjusted to save stack's current top frame into a dedicated variable `guesttop` in the virtual machine global state. When signal occurs, the profiler dumps current VM state and additional info about about stacks that can be considered consistent in a given state. Also, profiling can be done without writing to file: vmstate counters are accumulated in static memory and can be received via the `luaM_sysprof_report` function. For more details see the header file. When profiling is over, the old signal handler is restored, and a special epilogue header is written. Part of tarantool/tarantool#781 --- CMakeLists.txt | 6 + src/CMakeLists.txt | 1 + src/Makefile.dep.original | 8 +- src/lj_arch.h | 11 + src/lj_errmsg.h | 2 +- src/lj_mapi.c | 26 + src/lj_state.c | 7 + src/lj_sysprof.c | 483 ++++++++++++++++++ src/lj_sysprof.h | 94 ++++ src/lmisclib.h | 93 ++++ test/tarantool-tests/CMakeLists.txt | 1 + .../misclib-sysprof-capi.test.lua | 53 ++ .../misclib-sysprof-capi/CMakeLists.txt | 1 + .../misclib-sysprof-capi/testsysprof.c | 269 ++++++++++ 14 files changed, 1052 insertions(+), 3 deletions(-) create mode 100644 src/lj_sysprof.c create mode 100644 src/lj_sysprof.h create mode 100644 test/tarantool-tests/misclib-sysprof-capi.test.lua create mode 100644 test/tarantool-tests/misclib-sysprof-capi/CMakeLists.txt create mode 100644 test/tarantool-tests/misclib-sysprof-capi/testsysprof.c diff --git a/CMakeLists.txt b/CMakeLists.txt index 5348e043..0c702093 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -184,6 +184,12 @@ if(LUAJIT_DISABLE_MEMPROF) AppendFlags(TARGET_C_FLAGS -DLUAJIT_DISABLE_MEMPROF) endif() +# Disable platform and lua profiler. +option(LUAJIT_DISABLE_SYSPROF "LuaJIT platform and lua profiler support" OFF) +if(LUAJIT_DISABLE_SYSPROF) + AppendFlags(TARGET_C_FLAGS -DLUAJIT_DISABLE_SYSPROF) +endif() + # Switch to harder (and slower) hash function when a collision # chain in the string hash table exceeds a certain length. option(LUAJIT_SMART_STRINGS "Harder string hashing function" ON) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 0f3d888f..c19d8b54 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -69,6 +69,7 @@ make_source_list(SOURCES_UTILS make_source_list(SOURCES_PROFILER SOURCES lj_memprof.c + lj_sysprof.c lj_profile_timer.c lj_profile.c ) diff --git a/src/Makefile.dep.original b/src/Makefile.dep.original index b7d82719..503817bb 100644 --- a/src/Makefile.dep.original +++ b/src/Makefile.dep.original @@ -140,8 +140,8 @@ lj_lib.o: lj_lib.c lauxlib.h lua.h luaconf.h lj_obj.h lj_def.h lj_arch.h \ lj_load.o: lj_load.c lua.h luaconf.h lauxlib.h lj_obj.h lj_def.h \ lj_arch.h lj_gc.h lj_err.h lj_errmsg.h lj_buf.h lj_str.h lj_func.h \ lj_frame.h lj_bc.h lj_vm.h lj_lex.h lj_bcdump.h lj_parse.h -lj_mapi.o: lj_mapi.c lua.h luaconf.h lmisclib.h lj_obj.h lj_def.h lj_arch.h \ - lj_dispatch.h lj_bc.h lj_jit.h lj_ir.h +lj_mapi.o: lj_mapi.c lua.h luaconf.h lmisclib.h lj_obj.h lj_def.h \ + lj_arch.h lj_dispatch.h lj_bc.h lj_jit.h lj_ir.h lj_sysprof.h lj_mcode.o: lj_mcode.c lj_obj.h lua.h luaconf.h lj_def.h lj_arch.h \ lj_gc.h lj_err.h lj_errmsg.h lj_jit.h lj_ir.h lj_mcode.h lj_trace.h \ lj_dispatch.h lj_bc.h lj_traceerr.h lj_vm.h @@ -204,6 +204,10 @@ lj_strscan.o: lj_strscan.c lj_obj.h lua.h luaconf.h lj_def.h lj_arch.h \ lj_char.h lj_strscan.h lj_symtab.o: lj_symtab.c lj_symtab.h lj_wbuf.h lj_def.h lua.h luaconf.h \ lj_obj.h lj_arch.h +lj_sysprof.o: lj_sysprof.c lj_arch.h lua.h luaconf.h lj_sysprof.h \ + lj_obj.h lj_def.h lmisclib.h lj_debug.h lj_dispatch.h lj_bc.h lj_jit.h \ + lj_ir.h lj_frame.h lj_trace.h lj_traceerr.h lj_wbuf.h lj_profile_timer.h \ + lj_symtab.h lj_tab.o: lj_tab.c lj_obj.h lua.h luaconf.h lj_def.h lj_arch.h lj_gc.h \ lj_err.h lj_errmsg.h lj_tab.h lj_trace.o: lj_trace.c lj_obj.h lua.h luaconf.h lj_def.h lj_arch.h \ diff --git a/src/lj_arch.h b/src/lj_arch.h index 5bf0afb8..f0b60092 100644 --- a/src/lj_arch.h +++ b/src/lj_arch.h @@ -220,6 +220,7 @@ #endif #define LJ_ARCH_NOMEMPROF 1 +#define LJ_ARCH_NOSYSPROF 1 #elif LUAJIT_TARGET == LUAJIT_ARCH_ARM64 @@ -243,6 +244,7 @@ #define LJ_ARCH_VERSION 80 #define LJ_ARCH_NOMEMPROF 1 +#define LJ_ARCH_NOSYSPROF 1 #elif LUAJIT_TARGET == LUAJIT_ARCH_PPC @@ -310,6 +312,7 @@ #endif #define LJ_ARCH_NOMEMPROF 1 +#define LJ_ARCH_NOSYSPROF 1 #elif LUAJIT_TARGET == LUAJIT_ARCH_MIPS32 || LUAJIT_TARGET == LUAJIT_ARCH_MIPS64 @@ -371,6 +374,7 @@ #endif #define LJ_ARCH_NOMEMPROF 1 +#define LJ_ARCH_NOSYSPROF 1 #else #error "No target architecture defined" @@ -585,4 +589,11 @@ #define LJ_HASMEMPROF 1 #endif +/* Disable or enable the platform and lua profiler. */ +#if defined(LUAJIT_DISABLE_SYSPROF) || defined(LJ_ARCH_NOSYSPROF) || LJ_TARGET_WINDOWS || LJ_TARGET_CYGWIN || LJ_TARGET_PS3 || LJ_TARGET_PS4 || LJ_TARGET_XBOX360 +#define LJ_HASSYSPROF 0 +#else +#define LJ_HASSYSPROF 1 +#endif + #endif diff --git a/src/lj_errmsg.h b/src/lj_errmsg.h index ae0a18c0..77a08cb0 100644 --- a/src/lj_errmsg.h +++ b/src/lj_errmsg.h @@ -187,7 +187,7 @@ ERRDEF(FFI_NYICALL, "NYI: cannot call this C function (yet)") /* Profiler errors. */ ERRDEF(PROF_MISUSE, "profiler misuse") -#if LJ_HASMEMPROF +#if LJ_HASMEMPROF || LJ_HASSYSPROF ERRDEF(PROF_ISRUNNING, "profiler is running already") ERRDEF(PROF_NOTRUNNING, "profiler is not running") #endif diff --git a/src/lj_mapi.c b/src/lj_mapi.c index b2b35a17..0b2284f0 100644 --- a/src/lj_mapi.c +++ b/src/lj_mapi.c @@ -18,6 +18,8 @@ #include "lj_jit.h" #endif +#include "lj_sysprof.h" + LUAMISC_API void luaM_metrics(lua_State *L, struct luam_Metrics *metrics) { global_State *g = G(L); @@ -63,3 +65,27 @@ LUAMISC_API void luaM_metrics(lua_State *L, struct luam_Metrics *metrics) metrics->jit_trace_num = 0; #endif } + +/* --- Platform and Lua profiler ------------------------------------------ */ + +LUAMISC_API int luaM_sysprof_configure(const struct luam_sysprof_config *config) +{ + return lj_sysprof_configure(config); +} + +LUAMISC_API int luaM_sysprof_start(lua_State *L, + const struct luam_sysprof_options *opt) +{ + return lj_sysprof_start(L, opt); +} + +/* Stop profiling. */ +LUAMISC_API int luaM_sysprof_stop(lua_State *L) +{ + return lj_sysprof_stop(L); +} + +LUAMISC_API int luaM_sysprof_report(struct luam_sysprof_counters *counters) +{ + return lj_sysprof_report(counters); +} diff --git a/src/lj_state.c b/src/lj_state.c index f82b1b5b..cc6f92f1 100644 --- a/src/lj_state.c +++ b/src/lj_state.c @@ -33,6 +33,10 @@ #include "lj_memprof.h" #endif +#if LJ_HASSYSPROF +#include "lj_sysprof.h" +#endif + /* -- Stack handling ------------------------------------------------------ */ /* Stack sizes. */ @@ -267,6 +271,9 @@ LUA_API void lua_close(lua_State *L) #if LJ_HASMEMPROF lj_memprof_stop(L); #endif +#if LJ_HASSYSPROF + lj_sysprof_stop(L); +#endif #if LJ_HASPROFILE luaJIT_profile_stop(L); #endif diff --git a/src/lj_sysprof.c b/src/lj_sysprof.c new file mode 100644 index 00000000..c0c83fa9 --- /dev/null +++ b/src/lj_sysprof.c @@ -0,0 +1,483 @@ +#define lj_sysprof_c +#define LUA_CORE + +#include "lj_arch.h" +#include "lj_sysprof.h" + +#if LJ_HASSYSPROF + +#include "lj_obj.h" +#include "lj_debug.h" +#include "lj_dispatch.h" +#include "lj_frame.h" + +#if LJ_HASJIT +#include "lj_jit.h" +#include "lj_trace.h" +#endif + +#include "lj_wbuf.h" +#include "lj_profile_timer.h" +#include "lj_symtab.h" + +#include +#include +#include + +#define SYSPROF_HANDLER_STACK_DEPTH 4 +#define SYSPROF_BACKTRACE_BUF_SIZE 4096 + +enum sysprof_state { + /* Profiler needs to be configured. */ + SPS_UNCONFIGURED, + /* Profiler is not running. */ + SPS_IDLE, + /* Profiler is running. */ + SPS_PROFILE, + /* + ** Stopped in case of stopped or failed stream. + ** Saved errno is set at luaM_sysprof_stop. + */ + SPS_HALT +}; + +struct sysprof { + global_State *g; /* Profiled VM. */ + pthread_t thread; /* Profiled thread. */ + enum sysprof_state state; /* Internal state. */ + struct lj_wbuf out; /* Output accumulator. */ + struct luam_sysprof_counters counters; /* Profiling counters. */ + struct luam_sysprof_options opt; /* Profiling options. */ + struct luam_sysprof_config config; /* Profiler configurations. */ + lj_profile_timer timer; /* Profiling timer. */ + int saved_errno; /* Saved errno when profiler failed. */ +}; + +static struct sysprof sysprof = {0}; + +/* --- Stream ------------------------------------------------------------- */ + +static const uint8_t ljp_header[] = {'l', 'j', 'p', LJP_FORMAT_VERSION, + 0x0, 0x0, 0x0}; + +static int stream_is_needed(struct sysprof *sp) +{ + return LUAM_SYSPROF_DEFAULT != sp->opt.mode; +} + +static void stream_prologue(struct sysprof *sp) +{ + lj_symtab_dump(&sp->out, sp->g); + lj_wbuf_addn(&sp->out, ljp_header, sizeof(ljp_header)); +} + +static void stream_epilogue(struct sysprof *sp) +{ + lj_wbuf_addbyte(&sp->out, LJP_EPILOGUE_BYTE); +} + +static void stream_lfunc(struct lj_wbuf *buf, GCfunc *func) +{ + GCproto *pt = funcproto(func); + lua_assert(pt != NULL); + lj_wbuf_addbyte(buf, LJP_FRAME_LFUNC); + lj_wbuf_addu64(buf, (uintptr_t)pt); + lj_wbuf_addu64(buf, (uint64_t)pt->firstline); +} + +static void stream_cfunc(struct lj_wbuf *buf, GCfunc *func) +{ + lj_wbuf_addbyte(buf, LJP_FRAME_CFUNC); + lj_wbuf_addu64(buf, (uintptr_t)func->c.f); +} + +static void stream_ffunc(struct lj_wbuf *buf, GCfunc *func) +{ + lj_wbuf_addbyte(buf, LJP_FRAME_FFUNC); + lj_wbuf_addu64(buf, func->c.ffid); +} + +static void stream_frame_lua(struct lj_wbuf *buf, cTValue *frame) +{ + GCfunc *func = frame_func(frame); + lua_assert(NULL != func); + if (isluafunc(func)) { + stream_lfunc(buf, func); + } else if (isffunc(func)) { + stream_ffunc(buf, func); + } else { + stream_cfunc(buf, func); + } +} + +static void stream_backtrace_lua(struct sysprof *sp) +{ + global_State *g = sp->g; + struct lj_wbuf *buf = &sp->out; + cTValue *top_frame = NULL, *frame = NULL, *bot = NULL; + lua_State *L = NULL; + + lua_assert(g != NULL); + L = gco2th(gcref(g->cur_L)); + lua_assert(L != NULL); + + top_frame = sp->g->top_frame.guesttop.interp_base - 1; + bot = tvref(L->stack) + LJ_FR2; + /* Traverse frames backwards */ + for (frame = top_frame; frame > bot; frame = frame_prev(frame)) { + if (frame_gc(frame) == obj2gco(L)) { + continue; /* Skip dummy frames. See lj_err_optype_call(). */ + } + stream_frame_lua(buf, frame); + } + + lj_wbuf_addbyte(buf, LJP_FRAME_LUA_LAST); +} + +static void *stream_frame_host(int frame_no, void *addr) +{ + struct sysprof *sp = &sysprof; + + if (LJ_UNLIKELY(frame_no <= SYSPROF_HANDLER_STACK_DEPTH)) { + /* + ** We don't want the profiler stack to be streamed, as it will + ** burden the profile with unnecessary information. + */ + return addr; + } else if (LJ_UNLIKELY(sp->opt.mode == LUAM_SYSPROF_LEAF && + frame_no > SYSPROF_HANDLER_STACK_DEPTH + 1)) { + return NULL; + } + + lj_wbuf_addu64(&sp->out, (uintptr_t)addr); + return addr; +} + +static void default_backtrace_host(void *(writer)(int frame_no, void *addr)) +{ + static void* backtrace_buf[SYSPROF_BACKTRACE_BUF_SIZE] = {}; + + struct sysprof *sp = &sysprof; + const int depth = backtrace(backtrace_buf, + sp->opt.mode == LUAM_SYSPROF_LEAF + ? SYSPROF_HANDLER_STACK_DEPTH + 1 + : SYSPROF_BACKTRACE_BUF_SIZE); + lua_assert(depth >= SYSPROF_HANDLER_STACK_DEPTH); + + for (int i = SYSPROF_HANDLER_STACK_DEPTH; i < depth; ++i) { + if (!writer(i - SYSPROF_HANDLER_STACK_DEPTH + 1, backtrace_buf[i])) { + return; + } + } +} + +static void stream_backtrace_host(struct sysprof *sp) +{ + lua_assert(sp->config.backtracer != NULL); + sp->config.backtracer(stream_frame_host); + lj_wbuf_addu64(&sp->out, (uintptr_t)LJP_FRAME_HOST_LAST); +} + +static void stream_trace(struct sysprof *sp) +{ + struct lj_wbuf *out = &sp->out; + + uint32_t traceno = sp->g->vmstate; + jit_State *J = G2J(sp->g); + + lj_wbuf_addu64(out, traceno); + lj_wbuf_addu64(out, (uintptr_t)J->prev_pt); + lj_wbuf_addu64(out, J->prev_line); +} + +static void stream_guest(struct sysprof *sp) +{ + stream_backtrace_lua(sp); + stream_backtrace_host(sp); +} + +static void stream_host(struct sysprof *sp) +{ + stream_backtrace_host(sp); +} + +typedef void (*event_streamer)(struct sysprof *sp); + +static event_streamer event_streamers[] = { + /* XXX: order is important */ + stream_host, /* LJ_VMST_INTERP */ + stream_guest, /* LJ_VMST_LFUNC */ + stream_guest, /* LJ_VMST_FFUNC */ + stream_guest, /* LJ_VMST_CFUNC */ + stream_host, /* LJ_VMST_GC */ + stream_host, /* LJ_VMST_EXIT */ + stream_host, /* LJ_VMST_RECORD */ + stream_host, /* LJ_VMST_OPT */ + stream_host, /* LJ_VMST_ASM */ + stream_trace /* LJ_VMST_TRACE */ +}; + +static void stream_event(struct sysprof *sp, uint32_t vmstate) +{ + event_streamer stream = NULL; + + /* Check that vmstate fits in 4 bits (see streaming format) */ + lua_assert((vmstate & ~(uint32_t)((1 << 4) - 1)) == 0); + lj_wbuf_addbyte(&sp->out, (uint8_t)vmstate); + + stream = event_streamers[vmstate]; + lua_assert(NULL != stream); + stream(sp); +} + +/* -- Signal handler ------------------------------------------------------ */ + +static void sysprof_record_sample(struct sysprof *sp, siginfo_t *info) +{ + global_State *g = sp->g; + uint32_t _vmstate = ~(uint32_t)(g->vmstate); + uint32_t vmstate = _vmstate < LJ_VMST_TRACE ? _vmstate : LJ_VMST_TRACE; + + lua_assert(pthread_self() == sp->thread); + + /* Caveat: order of counters must match vmstate order in . */ + ((uint64_t *)&sp->counters)[vmstate]++; + + sp->counters.samples++; + sp->counters.overruns += info->si_overrun; + + if (stream_is_needed(sp)) { + stream_event(sp, vmstate); + if (LJ_UNLIKELY(lj_wbuf_test_flag(&sp->out, STREAM_ERRIO|STREAM_STOP))) { + sp->saved_errno = lj_wbuf_errno(&sp->out); + lj_wbuf_terminate(&sp->out); + sp->state = SPS_HALT; + } + } +} + +static void sysprof_signal_handler(int sig, siginfo_t *info, void *ctx) +{ + struct sysprof *sp = &sysprof; + UNUSED(sig); + UNUSED(ctx); + + switch (sp->state) { + case SPS_PROFILE: + sysprof_record_sample(sp, info); + break; + + case SPS_IDLE: + case SPS_HALT: + /* noop */ + break; + + default: + lua_assert(0); + break; + } +} + +/* -- Internal ------------------------------------------------------------ */ + +static int sysprof_validate(struct sysprof *sp, + const struct luam_sysprof_options *opt) +{ + switch (sp->state) { + case SPS_UNCONFIGURED: + return SYSPROF_ERRUSE; + + case SPS_IDLE: { + if (opt->mode > LUAM_SYSPROF_CALLGRAPH) { + return SYSPROF_ERRUSE; + } else if (opt->mode != LUAM_SYSPROF_DEFAULT && + (opt->buf == NULL || opt->len == 0 || + sp->config.writer == NULL || sp->config.on_stop == NULL)) { + return SYSPROF_ERRUSE; + } else if (opt->interval == 0) { + return SYSPROF_ERRUSE; + } + break; + } + + case SPS_PROFILE: + case SPS_HALT: + return SYSPROF_ERRRUN; + + default: + lua_assert(0); + } + + return SYSPROF_SUCCESS; +} + +static int sysprof_init(struct sysprof *sp, lua_State *L, + const struct luam_sysprof_options *opt) +{ + int status = sysprof_validate(sp, opt); + if (SYSPROF_SUCCESS != status) { + return status; + } + + /* Copy validated options to sysprof state. */ + memcpy(&sp->opt, opt, sizeof(sp->opt)); + + /* Init general fields. */ + sp->g = G(L); + sp->thread = pthread_self(); + + /* Reset counters. */ + memset(&sp->counters, 0, sizeof(sp->counters)); + + /* Reset saved errno. */ + sp->saved_errno = 0; + + if (stream_is_needed(sp)) { + lj_wbuf_init(&sp->out, sp->config.writer, opt->ctx, opt->buf, opt->len); + } + + return SYSPROF_SUCCESS; +} + +/* -- Public profiling API ------------------------------------------------ */ + +int lj_sysprof_configure(const struct luam_sysprof_config *config) +{ + struct sysprof *sp = &sysprof; + lua_assert(config != NULL); + if (sp->state != SPS_UNCONFIGURED && sp->state != SPS_IDLE) { + return SYSPROF_ERRUSE; + } + + memcpy(&sp->config, config, sizeof(*config)); + + if (sp->config.backtracer == NULL) { + sp->config.backtracer = default_backtrace_host; + } + + sp->state = SPS_IDLE; + + return SYSPROF_SUCCESS; +} + +int lj_sysprof_start(lua_State *L, const struct luam_sysprof_options *opt) +{ + struct sysprof *sp = &sysprof; + + int status = sysprof_init(sp, L, opt); + if (SYSPROF_SUCCESS != status) { + if (NULL != sp->config.on_stop) { + /* + ** Initialization may fail in case of unconfigured sysprof, + ** so we cannot guarantee cleaning up resources in this case. + */ + sp->config.on_stop(opt->ctx, opt->buf); + } + return status; + } + + sp->state = SPS_PROFILE; + + if (stream_is_needed(sp)) { + stream_prologue(sp); + if (LJ_UNLIKELY(lj_wbuf_test_flag(&sp->out, STREAM_ERRIO|STREAM_STOP))) { + /* on_stop call may change errno value. */ + int saved_errno = lj_wbuf_errno(&sp->out); + /* Ignore possible errors. mp->out.buf may be NULL here. */ + sp->config.on_stop(opt->ctx, sp->out.buf); + lj_wbuf_terminate(&sp->out); + sp->state = SPS_IDLE; + errno = saved_errno; + return SYSPROF_ERRIO; + } + } + + sp->timer.opt.interval_msec = opt->interval; + sp->timer.opt.handler = sysprof_signal_handler; + lj_profile_timer_start(&sp->timer); + + return SYSPROF_SUCCESS; +} + +int lj_sysprof_stop(lua_State *L) +{ + struct sysprof *sp = &sysprof; + global_State *g = sp->g; + struct lj_wbuf *out = &sp->out; + + if (SPS_IDLE == sp->state) { + return SYSPROF_ERRSTOP; + } else if (G(L) != g) { + return SYSPROF_ERRUSE; + } + + lj_profile_timer_stop(&sp->timer); + + if (SPS_HALT == sp->state) { + errno = sp->saved_errno; + sp->state = SPS_IDLE; + /* wbuf was terminated when error occured. */ + return SYSPROF_ERRIO; + } + + sp->state = SPS_IDLE; + + if (stream_is_needed(sp)) { + int cb_status = 0; + + stream_epilogue(sp); + lj_wbuf_flush(out); + + cb_status = sp->config.on_stop(sp->opt.ctx, out->buf); + if (LJ_UNLIKELY(lj_wbuf_test_flag(out, STREAM_ERRIO|STREAM_STOP)) || + 0 != cb_status) { + errno = lj_wbuf_errno(out); + lj_wbuf_terminate(out); + return SYSPROF_ERRIO; + } + + lj_wbuf_terminate(out); + } + + return SYSPROF_SUCCESS; +} + +int lj_sysprof_report(struct luam_sysprof_counters *counters) +{ + const struct sysprof *sp = &sysprof; + if (sp->state != SPS_IDLE) { + return SYSPROF_ERRUSE; + } + memcpy(counters, &sp->counters, sizeof(sp->counters)); + return SYSPROF_SUCCESS; +} + +#else /* LJ_HASSYSPROF */ + +int lj_sysprof_configure(const struct luam_sysprof_config *config) +{ + UNUSED(config); + return SYSPROF_ERRUSE; +} + +int lj_sysprof_start(lua_State *L, const struct luam_sysprof_options *opt) +{ + UNUSED(L); + opt->on_stop(opt->ctx, opt->buf); + return SYSPROF_ERRUSE; +} + +int lj_sysprof_stop(lua_State *L) +{ + UNUSED(L); + return SYSPROF_ERRUSE; +} + +int lj_sysprof_report(struct luam_sysprof_counters *counters) +{ + UNUSED(counters); + return SYSPROF_ERRUSE; +} + +#endif /* LJ_HASSYSPROF */ + diff --git a/src/lj_sysprof.h b/src/lj_sysprof.h new file mode 100644 index 00000000..515bc08c --- /dev/null +++ b/src/lj_sysprof.h @@ -0,0 +1,94 @@ +/* +** Sysprof - platform and Lua profiler +*/ + +/* +** XXX: Platform profiler is not thread safe. Please, don't try to +** use it inside several VM, you can profile only one at a time. +*/ + +/* +** XXX: Platform profiler uses the same signal backend as lj_profile. Please, +** don't use both at the same time. +*/ + +#ifndef _LJ_SYSPROF_H +#define _LJ_SYSPROF_H + +#include "lj_obj.h" +#include "lmisclib.h" + +#define LJP_FORMAT_VERSION 0x1 + +/* +** Event stream format: +** +** stream := symtab sysprof +** symtab := see symtab description +** sysprof := prologue sample* epilogue +** prologue := 'l' 'j' 'p' version reserved +** version := +** reserved := +** sample := sample-guest | sample-host | sample-trace +** sample-guest := sample-header stack-lua stack-host +** sample-host := sample-header stack-host +** sample-trace := sample-header traceno sym-addr line-no +** sample-header := +** stack-lua := frame-lua* frame-lua-last +** stack-host := frame-host* frame-host-last +** frame-lua := frame-lfunc | frame-cfunc | frame-ffunc +** frame-lfunc := frame-header sym-addr line-no +** frame-cfunc := frame-header exec-addr +** frame-ffunc := frame-header ffid +** frame-lua-last := frame-header +** frame-header := +** frame-host := exec-addr +** frame-host-last := +** line-no := +** traceno := +** ffid := +** sym-addr := +** exec-addr := +** epilogue := sample-header +** +** : A single byte (no surprises here) +** : Unsigned integer represented in ULEB128 encoding +** +** (Order of bits below is hi -> lo) +** +** version: [VVVVVVVV] +** * VVVVVVVV: Byte interpreted as a plain integer version number +** +** sample-header: [FUUUUEEE] +** * EEE : 3 bits for representing vmstate (LJ_VMST_*) +** * UUUU : 4 unused bits +** * F : 0 for regular samples, 1 for epilogue's Final header +** (if F is set to 1, all other bits are currently ignored) +** +** frame-header: [FUUUUUEE] +** * EE : 2 bits for representing frame type (FRAME_*) +** * UUUUU : 5 unused bits +** * F : 0 for regular frames, 1 for final frame +** (if F is set to 1, all other bits are currently ignored) +** +** frame-host-last = NULL +*/ + +#define LJP_FRAME_LFUNC ((uint8_t)1) +#define LJP_FRAME_CFUNC ((uint8_t)2) +#define LJP_FRAME_FFUNC ((uint8_t)3) +#define LJP_FRAME_LUA_LAST 0x80 +#define LJP_FRAME_HOST_LAST NULL + +#define LJP_EPILOGUE_BYTE 0x80 + +int lj_sysprof_configure(const struct luam_sysprof_config *config); + +int lj_sysprof_start(lua_State *L, const struct luam_sysprof_options *opt); + +int lj_sysprof_stop(lua_State *L); + +int lj_sysprof_report(struct luam_sysprof_counters *counters); + +#endif + diff --git a/src/lmisclib.h b/src/lmisclib.h index 0c07707e..3545ff47 100644 --- a/src/lmisclib.h +++ b/src/lmisclib.h @@ -60,6 +60,99 @@ struct luam_Metrics { LUAMISC_API void luaM_metrics(lua_State *L, struct luam_Metrics *metrics); +/* --- Sysprof - platform and lua profiler -------------------------------- */ + +/* Profiler configurations */ +struct luam_sysprof_config { + /* + ** Writer function for profile events. + ** Should return amount of written bytes on success or zero in case of error. + ** Setting *data to NULL means end of profiling. + ** For details see . + */ + size_t (*writer)(const void **data, size_t len, void *ctx); + /* + ** Callback on profiler stopping. Required for correctly cleaning + ** at VM finalization when profiler is still running. + ** Returns zero on success. + */ + int (*on_stop)(void *ctx, uint8_t *buf); + /* + ** Backtracing function for the host stack. Should call `frame_writer` on + ** each frame in the stack in the order from the stack top to the stack + ** bottom. The `frame_writer` function is implemented inside the sysprof + ** and will be passed to the `backtracer` function. If `frame_writer` returns + ** NULL, backtracing should be stopped. If `frame_writer` returns not NULL, + ** the backtracing should be continued if there are frames left. + */ + void (*backtracer)(void *(*frame_writer)(int frame_no, void *addr)); +}; + +enum luam_sysprof_mode { + /* + ** DEFAULT mode collects only data for luam_sysprof_counters, which is stored + ** in memory and can be collected with luaM_sysprof_report after profiler + ** stops. + */ + LUAM_SYSPROF_DEFAULT, + /* + ** LEAF mode = DEFAULT + streams samples with only top frames of host and + ** guests stacks in format described in + */ + LUAM_SYSPROF_LEAF, + /* + ** CALLGRAPH mode = DEFAULT + streams samples with full callchains of host + ** and guest stacks in format described in + */ + LUAM_SYSPROF_CALLGRAPH +}; + +struct luam_sysprof_counters { + uint64_t vmst_interp; + uint64_t vmst_lfunc; + uint64_t vmst_ffunc; + uint64_t vmst_cfunc; + uint64_t vmst_gc; + uint64_t vmst_exit; + uint64_t vmst_record; + uint64_t vmst_opt; + uint64_t vmst_asm; + uint64_t vmst_trace; + /* XXX: order of vmst counters is important */ + uint64_t samples; + uint64_t overruns; +}; + +/* Profiler options */ +struct luam_sysprof_options { + /* Profiling mode */ + enum luam_sysprof_mode mode; + /* Sampling interval in msec */ + uint64_t interval; + /* Custom buffer to write data. */ + uint8_t *buf; + /* The buffer's size. */ + size_t len; + /* Context for the profile writer and final callback. */ + void *ctx; +}; + +#define SYSPROF_SUCCESS (0) +#define SYSPROF_ERRUSE (1) +#define SYSPROF_ERRRUN (2) +#define SYSPROF_ERRSTOP (3) +#define SYSPROF_ERRIO (4) + +LUAMISC_API int luaM_sysprof_configure(const struct luam_sysprof_config *config); + +LUAMISC_API int luaM_sysprof_start(lua_State *L, + const struct luam_sysprof_options *opt); + +LUAMISC_API int luaM_sysprof_stop(lua_State *L); + +LUAMISC_API int luaM_sysprof_report(struct luam_sysprof_counters *counters); + + #define LUAM_MISCLIBNAME "misc" LUALIB_API int luaopen_misc(lua_State *L); diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt index a872fa5e..cf47cc67 100644 --- a/test/tarantool-tests/CMakeLists.txt +++ b/test/tarantool-tests/CMakeLists.txt @@ -62,6 +62,7 @@ add_subdirectory(gh-6189-cur_L) add_subdirectory(lj-49-bad-lightuserdata) add_subdirectory(lj-flush-on-trace) add_subdirectory(misclib-getmetrics-capi) +add_subdirectory(misclib-sysprof-capi) # The part of the memory profiler toolchain is located in tools # directory, jit, profiler, and bytecode toolchains are located diff --git a/test/tarantool-tests/misclib-sysprof-capi.test.lua b/test/tarantool-tests/misclib-sysprof-capi.test.lua new file mode 100644 index 00000000..d468572d --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-capi.test.lua @@ -0,0 +1,53 @@ +-- Sysprof is implemented for x86 and x64 architectures only. +require("utils").skipcond( + jit.arch ~= "x86" and jit.arch ~= "x64", + jit.arch.." architecture is NIY for memprof" +) + +local testsysprof = require("testsysprof") + +local tap = require("tap") + +local test = tap.test("clib-misc-sysprof") +test:plan(4) + +test:ok(testsysprof.base()) +test:ok(testsysprof.validation()) + +local function lua_payload(n) + if n <= 1 then + return n + end + return lua_payload(n - 1) + lua_payload(n - 2) +end + +local function payload() + local n_iterations = 500000 + + local co = coroutine.create(function () + for i = 1, n_iterations do + if i % 2 == 0 then + testsysprof.c_payload(10) + else + lua_payload(10) + end + coroutine.yield() + end + end) + + for _ = 1, n_iterations do + coroutine.resume(co) + end +end + +local jit = require('jit') + +jit.off() + +test:ok(testsysprof.profile_func(payload)) + +jit.on() +jit.flush() + +test:ok(testsysprof.profile_func(payload)) + diff --git a/test/tarantool-tests/misclib-sysprof-capi/CMakeLists.txt b/test/tarantool-tests/misclib-sysprof-capi/CMakeLists.txt new file mode 100644 index 00000000..d9fb1a1a --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-capi/CMakeLists.txt @@ -0,0 +1 @@ +BuildTestCLib(testsysprof testsysprof.c) diff --git a/test/tarantool-tests/misclib-sysprof-capi/testsysprof.c b/test/tarantool-tests/misclib-sysprof-capi/testsysprof.c new file mode 100644 index 00000000..46970a72 --- /dev/null +++ b/test/tarantool-tests/misclib-sysprof-capi/testsysprof.c @@ -0,0 +1,269 @@ +#include +#include +#include + +#include + +#include +#include + +#undef NDEBUG +#include + + +/* --- utils -------------------------------------------------------------- */ + +#define SYSPROF_INTERVAL_DEFAULT 11 + +/* +** Yep, 8Mb. Tuned in order not to bother the platform with too often flushes. +*/ +#define STREAM_BUFFER_SIZE (8 * 1024 * 1024) + +/* Structure given as ctx to memprof writer and on_stop callback. */ +struct sysprof_ctx { + /* Output file stream for data. */ + FILE *stream; + /* Buffer for data. */ + uint8_t buf[STREAM_BUFFER_SIZE]; +}; + +/* +** Default buffer writer function. +** Just call fwrite to the corresponding FILE. +*/ +static size_t buffer_writer_default(const void **buf_addr, size_t len, + void *opt) +{ + struct sysprof_ctx *ctx = opt; + FILE *stream = ctx->stream; + const void * const buf_start = *buf_addr; + const void *data = *buf_addr; + size_t write_total = 0; + + assert(len <= STREAM_BUFFER_SIZE); + + for (;;) { + const size_t written = fwrite(data, 1, len - write_total, stream); + + if (written == 0) { + /* Re-tries write in case of EINTR. */ + if (errno != EINTR) { + /* Will be freed as whole chunk later. */ + *buf_addr = NULL; + return write_total; + } + errno = 0; + continue; + } + + write_total += written; + assert(write_total <= len); + + if (write_total == len) + break; + + data = (uint8_t *)data + (ptrdiff_t)written; + } + + *buf_addr = buf_start; + return write_total; +} + +/* Default on stop callback. Just close the corresponding stream. */ +static int on_stop_cb_default(void *opt, uint8_t *buf) +{ + struct sysprof_ctx *ctx = opt; + FILE *stream = ctx->stream; + free(ctx); + return fclose(stream); +} + +static int stream_init(struct luam_sysprof_options *opt) +{ + struct sysprof_ctx *ctx = calloc(1, sizeof(struct sysprof_ctx)); + if (NULL == ctx) { + return SYSPROF_ERRIO; + } + + ctx->stream = fopen("/dev/null", "wb"); + if (NULL == ctx->stream) { + free(ctx); + return SYSPROF_ERRIO; + } + + opt->ctx = ctx; + opt->buf = ctx->buf; + opt->len = STREAM_BUFFER_SIZE; + + return SYSPROF_SUCCESS; +} + +/* --- Payload ------------------------------------------------------------ */ + +static double fib(double n) +{ + if (n <= 1) { + return n; + } + return fib(n - 1) + fib(n - 2); +} + +static int c_payload(lua_State *L) +{ + fib(luaL_checknumber(L, 1)); +} + +/* --- sysprof C API tests ------------------------------------------------ */ + +static int base(lua_State *L) +{ + struct luam_sysprof_config config = {}; + (void)config.writer; + (void)config.on_stop; + (void)config.backtracer; + + struct luam_sysprof_options opt = {}; + (void)opt.interval; + (void)opt.mode; + (void)opt.ctx; + (void)opt.buf; + (void)opt.len; + + struct luam_sysprof_counters cnt = {}; + luaM_sysprof_report(&cnt); + + (void)cnt.samples; + (void)cnt.overruns; + (void)cnt.vmst_interp; + (void)cnt.vmst_lfunc; + (void)cnt.vmst_ffunc; + (void)cnt.vmst_cfunc; + (void)cnt.vmst_gc; + (void)cnt.vmst_exit; + (void)cnt.vmst_record; + (void)cnt.vmst_opt; + (void)cnt.vmst_asm; + (void)cnt.vmst_trace; + + lua_pushboolean(L, 1); + return 1; +} + +static int validation(lua_State *L) +{ + struct luam_sysprof_config config = {}; + struct luam_sysprof_options opt = {}; + int status = SYSPROF_SUCCESS; + + status = luaM_sysprof_configure(&config); + assert(SYSPROF_SUCCESS == status); + + /* Unknown mode */ + opt.mode = 0x40; + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_ERRUSE == status); + + /* Buffer not configured */ + opt.mode = LUAM_SYSPROF_CALLGRAPH; + opt.buf = NULL; + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_ERRUSE == status); + + /* Bad interval */ + opt.mode = LUAM_SYSPROF_DEFAULT; + opt.interval = 0; + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_ERRUSE == status); + + /* Check if profiling started */ + opt.mode = LUAM_SYSPROF_DEFAULT; + opt.interval = SYSPROF_INTERVAL_DEFAULT; + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_SUCCESS == status); + + /* Already running */ + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_ERRRUN == status); + + /* Profiler stopping */ + status = luaM_sysprof_stop(L); + assert(SYSPROF_SUCCESS == status); + + /* Stopping profiler which is not running */ + status = luaM_sysprof_stop(L); + assert(SYSPROF_ERRSTOP == status); + + lua_pushboolean(L, 1); + return 1; +} + +static int profile_func(lua_State *L) +{ + struct luam_sysprof_config config = {}; + struct luam_sysprof_options opt = {}; + struct luam_sysprof_counters cnt = {}; + int status = SYSPROF_ERRUSE; + + int n = lua_gettop(L); + if (n != 1 || !lua_isfunction(L, 1)) { + luaL_error(L, "incorrect argument: 1 function is required"); + } + + /* + ** Since all the other modes functionality is the + ** subset of CALLGRAPH mode, run this mode to test + ** the profiler's behavior. + */ + opt.mode = LUAM_SYSPROF_CALLGRAPH; + opt.interval = SYSPROF_INTERVAL_DEFAULT; + stream_init(&opt); + + config.on_stop = on_stop_cb_default; + config.writer = buffer_writer_default; + status = luaM_sysprof_configure(&config); + assert(SYSPROF_SUCCESS == status); + + status = luaM_sysprof_start(L, &opt); + assert(SYSPROF_SUCCESS == status); + + /* Run payload. */ + if (lua_pcall(L, 0, 0, 0) != 0) { + luaL_error(L, "error running payload: %s", lua_tostring(L, -1)); + } + + status = luaM_sysprof_stop(L); + assert(SYSPROF_SUCCESS == status); + + status = luaM_sysprof_report(&cnt); + assert(SYSPROF_SUCCESS == status); + + assert(cnt.samples > 1); + assert(cnt.samples == cnt.vmst_asm + + cnt.vmst_cfunc + + cnt.vmst_exit + + cnt.vmst_ffunc + + cnt.vmst_gc + + cnt.vmst_interp + + cnt.vmst_lfunc + + cnt.vmst_opt + + cnt.vmst_record + + cnt.vmst_trace); + + lua_pushboolean(L, 1); + return 1; +} + +static const struct luaL_Reg testsysprof[] = { + {"c_payload", c_payload}, + {"base", base}, + {"validation", validation}, + {"profile_func", profile_func}, + {NULL, NULL} +}; + +LUA_API int luaopen_testsysprof(lua_State *L) +{ + luaL_register(L, "testsysprof", testsysprof); + return 1; +} -- 2.33.0