From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 41BAE71725F; Tue, 5 Dec 2023 15:05:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 41BAE71725F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1701777948; bh=d7epYSWENVrnTJq2D9hFwsjDJrQncEFuXqyoeJrEW3A=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=jg/cCKQwqQdYq4XFZ0h7EHfWngd3G/Clq7gJUpPE5BbuOwZ0xnAO/AhT5UzjbwN1b MuoTRpzWp/OvD66tpa0XL+EnHDLAdSpESxpi4/S7LfKm6diFLktcsQq8ykQ8/e/eca VGpoolYDGoAHSMTsp7n6CIwgngczqlA1VQ34QuhU= Received: from smtp61.i.mail.ru (smtp61.i.mail.ru [95.163.41.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 6ADAB716A53 for ; Tue, 5 Dec 2023 15:05:47 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6ADAB716A53 Received: by smtp61.i.mail.ru with esmtpa (envelope-from ) id 1rAUBN-008AA9-0h; Tue, 05 Dec 2023 15:05:45 +0300 Date: Tue, 5 Dec 2023 12:04:57 +0000 To: Sergey Kaplun Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD91486559A3ABF84E7CB0A098B493E0346EE397C45C3170F78182A05F538085040001B1A6EF31A30C221E10C39BE5FA76D393797E4EF3670CD3AAEAD6689879B79 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE70ED3881ADD6CEF6AEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637BF3E16F0C87855308638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D809EA7125DBCAABB57E167497100F6446117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC8C7ADC89C2F0B2A5A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD186FD1C55BDD38FC3FD2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EEB1593CA6EC85F86D96C9B5BF839F39F6D8FC6C240DEA76429C9F4D5AE37F343AA9539A8B242431040A6AB1C7CE11FEE3457EE4B4996FC546040F9FF01DFDA4A8C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947CB861051D4BA689FC2E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89FB26E97DCB74E6252C6EABA9B74D0DA47B5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A5842F0E11063D72694B9ADBB2953C390EE8C381C0FDC58A04F87CCE6106E1FC07E67D4AC08A07B9B064E7220B7C550592CB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CFE0A15EE7CDADA5E71438DB589A9EBE236C939789A3F09A94B6EBC476B740E7FCD9E67331CBA3B6E49157FEB932BFB23DCBEE0AD8044DA6E849B517B7245C6DB2A74DFFEFA5DC0E7F02C26D483E81D6BEECAEF3E2CCC1ED8C383653B6C8D9AE0FD16FCAA6493B703A X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojJEmydpeiNq/9Wj/2g+XScA== X-Mailru-Sender: 2FEBA92C8E508479FE7B9A1DF348D53119CEA2682A1799D642D49DFE90960A5D1B51B3791EA374152326FE6F2A341ACE0FB9F97486540B4CD9E8847AB8CFED4D9ABF8A61C016C2CFB0DAF586E7D11B3E67EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 2/2] test: rewrite sysprof test using managed execution X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Igor Munkin via Tarantool-patches Reply-To: Igor Munkin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Sergey, Thanks for your review! On 05.12.23, Sergey Kaplun wrote: > Hi, Igor! > Thanks for the patch! > Really awesome improvement for our sysprof testing! > Please consider my comments below. > > On 03.12.23, Maxim Kokryashkin wrote: > > > --- > > > .../gh-8594-sysprof-ffunc-crash.test.c | 269 ++++++++++++++++++ > > > .../gh-8594-sysprof-ffunc-crash.test.lua | 55 ---- > > > 2 files changed, 269 insertions(+), 55 deletions(-) > > > create mode 100644 test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c > > > delete mode 100644 test/tarantool-tests/gh-8594-sysprof-ffunc-crash.test.lua > > > > > > diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c > > > new file mode 100644 > > > index 00000000..4caed8cb > > > --- /dev/null > > > +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c > > > @@ -0,0 +1,269 @@ > > > + > > > +#if LUAJIT_OS == LUAJIT_OS_LINUX && LUAJIT_TARGET == LUAJIT_ARCH_X64 > > Why not use the `LJ_HASSYSPROF` flag? > > I agree here. `LJ_HASSYSPROF` is more foolproof -- if we will port > sysprof for arm64 (for example) we will not forget to update this test. See my arguments for this in my previous reply. > > > Minor: This link may be useful too: > https://c9x.me/x86/html/file_module_x86_id_142.html Thanks, added: ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index d17386dd..d9d2e481 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -41,6 +41,7 @@ * * See more info here: * * https://man7.org/linux/man-pages/man2/ptrace.2.html + * * https://c9x.me/x86/html/file_module_x86_id_142.html * * https://github.com/tarantool/tarantool/issues/8594 * * https://github.com/tarantool/tarantool/issues/9387 */ ================================================================================ > > > > + */ > > > + > > > +#define MESSAGE "Canary is alive" > > > +#define LUACALL "local a = tostring('" MESSAGE "') return a" > > Nit: I suggest adding an excess empty line here to separate extern asm functions > from Lua code defines. OK, added. > > > > +/* XXX: Resolve the necessary addresses from VM engine. */ > > > +extern void *lj_ff_tostring(void); > > > +extern void *lj_fff_res1(void); > > > + > > > +/* Sysprof "/dev/null" stream helpers. {{{ */ > > > + > > > +/* > > > + * Yep, 8Mb. Tuned in order not to bother the platform with too > > > + * often flushes. > > > + */ > > This comment is misleading since there is no any flushes in the writer > function. Well, it's not about I/O flushes, but rather about sysprof write-buffer pressure, isn't it? Hence, 8Mb is needed to make the test omit almost all buffer-flush related callbacks. > Also, do we need such big buffer? Maybe `PATH_MAX` * 2 (or something > like that) should be enough? Could you elaborate the reason for such a strange constant, please? > > > > +#define STREAM_BUFFER_SIZE (8 * 1024 * 1024) > > > +#define DEVNULL -1 > > `DEVNULL` name is misleading (like we've actually opened `/dev/null`). > Maybe `DUMMY_FD` is better? Also, I suppose we can drop the `fd` field > at all. We can just use static structure in this translation unit and > compare pointer against it, so there is no need for an additional field > fd in the structure itself. I see little reasoning in this suggestion: the approach with the structure is quite straight-forward and copied from another sysprof test implemented in C. Re naming: I prefer to leave it as is. In general, your nit is nice, but I believe it should be applied for the patch moving tricks to utils library. > > > > + > > > + > > > +static int stream_new(struct luam_Sysprof_Options *options) { > > Please, use separate line for block start `{`. > Here and below. OK, fixed everywhere in test. ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 942787a4..884c62cc 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -72,7 +72,8 @@ struct devnull_ctx { uint8_t buf[STREAM_BUFFER_SIZE]; }; -static int stream_new(struct luam_Sysprof_Options *options) { +static int stream_new(struct luam_Sysprof_Options *options) +{ struct devnull_ctx *ctx = calloc(1, sizeof(struct devnull_ctx)); if (ctx == NULL) return PROFILE_ERRIO; @@ -86,14 +87,16 @@ static int stream_new(struct luam_Sysprof_Options *options) { return PROFILE_SUCCESS; } -static int stream_delete(void *rawctx, uint8_t *buf) { +static int stream_delete(void *rawctx, uint8_t *buf) +{ struct devnull_ctx *ctx = rawctx; assert(ctx->fd == DEVNULL); free(ctx); return PROFILE_SUCCESS; } -static size_t stream_writer(const void **buf_addr, size_t len, void *rawctx) { +static size_t stream_writer(const void **buf_addr, size_t len, void *rawctx) +{ struct devnull_ctx *ctx = rawctx; assert(ctx->fd == DEVNULL); /* Do nothing, just return back to the profiler. */ @@ -102,7 +105,8 @@ static size_t stream_writer(const void **buf_addr, size_t len, void *rawctx) { /* }}} Sysprof "/dev/null" stream helpers. */ -static int tracee(const char *luacode) { +static int tracee(const char *luacode) +{ struct luam_Sysprof_Counters counters = {}; struct luam_Sysprof_Options opt = { /* Collect full backtraces per event. */ @@ -162,7 +166,8 @@ static int tracee(const char *luacode) { return EXIT_SUCCESS; } -static void wait_alive(pid_t chpid) { +static void wait_alive(pid_t chpid) +{ int wstatus; /* Wait tracee signal-delivery-stop. */ @@ -175,13 +180,15 @@ static void wait_alive(pid_t chpid) { } const uint8_t INT3 = 0xCC; -static inline unsigned long int3poison(unsigned long instruction) { +static inline unsigned long int3poison(unsigned long instruction) +{ const size_t int3bits = sizeof(INT3) * 8; const unsigned long int3mask = -1UL >> int3bits << int3bits; return (instruction & int3mask) | INT3; } -static void continue_until(pid_t chpid, void *addr) { +static void continue_until(pid_t chpid, void *addr) +{ struct user_regs_struct regs; /* Obtain the instructions at the . */ @@ -220,7 +227,8 @@ static void continue_until(pid_t chpid, void *addr) { ptrace(PTRACE_POKETEXT, chpid, addr, data); } -static int tracer(pid_t chpid) { +static int tracer(pid_t chpid) +{ int wstatus; /* Wait until tracee is ready. */ @@ -250,7 +258,8 @@ static int tracer(pid_t chpid) { return TEST_EXIT_SUCCESS; } -static int test_tostring_call(void *ctx) { +static int test_tostring_call(void *ctx) +{ pid_t chpid = fork(); switch(chpid) { case -1: @@ -271,13 +280,15 @@ static int test_tostring_call(void *ctx) { #else /* LUAJIT_OS == LUAJIT_OS_LINUX && LUAJIT_TARGET == LUAJIT_ARCH_X64 */ -static int test_tostring_call(void *ctx) { +static int test_tostring_call(void *ctx) +{ return skip_all("sysprof is implemented for Linux/x86_64 only"); } #endif /* LUAJIT_OS == LUAJIT_OS_LINUX && LUAJIT_TARGET == LUAJIT_ARCH_X64 */ -int main(void) { +int main(void) +{ const struct test_unit tgroup[] = { test_unit_def(test_tostring_call), }; ================================================================================ > > > > +static int tracee(const char *luacode) { > > > + struct luam_Sysprof_Counters counters = {}; > > > + struct luam_Sysprof_Options opt = { > > > + /* Collect full backtraces per event. */ > > > + .mode = LUAM_SYSPROF_CALLGRAPH, > > > + /* > > > + * XXX: Setting the "endless timer". The test > > > + * requires the single event to be streamed at > > > + * instruction, so to avoid spoiling > > > + * the stream with other unwanted events, the > > > + * timer is set to some unreachable point, so the > > > + * profiler will be guaranteed to stop before any > > > + * event is emitted. > > > + */ > > > + .interval = -1ULL, > > > + }; > > > + > > > + /* Allow tracing for this process */ > > Nit: Missed dot at the end of the sentence. Fixed. ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 884c62cc..40926a50 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -123,7 +123,7 @@ static int tracee(const char *luacode) .interval = -1ULL, }; - /* Allow tracing for this process */ + /* Allow tracing for this process. */ if (ptrace(PTRACE_TRACEME, 0, 0, 0) < 0) { perror("Failed to turn the calling process into a tracee"); return EXIT_FAILURE; ================================================================================ > > > > + if (ptrace(PTRACE_TRACEME, 0, 0, 0) < 0) { > > Nit: It's more readable to use `ptrace(PTRACE_TRACEME, 0, NULL, NULL)` > to match arguments types. OK, as you wish. ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 40926a50..6e5a9255 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -124,7 +124,7 @@ static int tracee(const char *luacode) }; /* Allow tracing for this process. */ - if (ptrace(PTRACE_TRACEME, 0, 0, 0) < 0) { + if (ptrace(PTRACE_TRACEME, 0, NULL, NULL) < 0) { perror("Failed to turn the calling process into a tracee"); return EXIT_FAILURE; } ================================================================================ > > > > + perror("Failed to turn the calling thread into a tracee"); > > Typo?: s/thread/process/ > > Also, it's better to use `test_comment()` format here (i.e., with "#") > to be TAP-compatible. No, it's not. Tracee output is not TAP compatible, since the tracer is the only TAP output emitter. > > > > + return EXIT_FAILURE; > > > + } > > > + > > > + /* > > > + * XXX: Allow parent (which is our tracer now) to observe > > > + * our signal-delivery-stop (i.e. the tracee is ready). > > > + * For more info see ptrace(2), "Attaching and detaching". > > > + */ > > > + raise(SIGSTOP); > > > + > > > + lua_State *L = utils_lua_init(); > > > + > > > + /* Customize and start profiler. */ > > > + assert(stream_new(&opt) == PROFILE_SUCCESS); > > > + assert(luaM_sysprof_set_writer(stream_writer) == PROFILE_SUCCESS); > > > + assert(luaM_sysprof_set_on_stop(stream_delete) == PROFILE_SUCCESS); > > > + assert(luaM_sysprof_start(L, &opt) == PROFILE_SUCCESS); > > > + > > > + /* FIXME: Make this part test agnostic. */ > > Typo: s/test agnostic/test-agnostic/ > > Nit: TODO looks more valid for me (since it's working). > Feel free to ignore. Fixed. ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 6e5a9255..44421812 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -144,7 +144,7 @@ static int tracee(const char *luacode) assert(luaM_sysprof_set_on_stop(stream_delete) == PROFILE_SUCCESS); assert(luaM_sysprof_start(L, &opt) == PROFILE_SUCCESS); - /* FIXME: Make this part test agnostic. */ + /* TODO: Make this part test-agnostic. */ assert(luaL_dostring(L, luacode) == LUA_OK); assert(strcmp(lua_tostring(L, -1), MESSAGE) == 0); ================================================================================ > > > > + assert(luaL_dostring(L, luacode) == LUA_OK); > > > + assert(strcmp(lua_tostring(L, -1), MESSAGE) == 0); > > > + > > > + /* Terminate profiler and Lua universe. */ > > > + assert(luaM_sysprof_stop(L) == PROFILE_SUCCESS); > > > + utils_lua_close(L); > > It doesn't seem to be semantically correct to terminate Lua universe > > before obtaining the counters, even though there is no obstacles to do > > so in the API implementation itself. > > > + > > > + /* > > > + * XXX: The only event to be streamed must be FFUNC at > > > + * instruction. > > Typo: s//the / > > > > + * FIXME: Make this part test agnostic. > > Typo: s/test agnostic/test-agnostic/ > > Nit: TODO looks more valid for me (since it's working). > Feel free to ignore. Fixed. ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 44421812..036f8141 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -153,8 +153,8 @@ static int tracee(const char *luacode) /* * XXX: The only event to be streamed must be FFUNC at - * instruction. - * FIXME: Make this part test agnostic. + * the instruction. + * TODO: Make this part test agnostic. */ assert(luaM_sysprof_report(&counters) == PROFILE_SUCCESS); assert(counters.samples == 1); ================================================================================ > > > > +const uint8_t INT3 = 0xCC; > > Maybe it should be moved up to other constants and defines? I'd rather prefer to leave it here to localize the scope of its usage. > > > > +static inline unsigned long int3poison(unsigned long instruction) { > > > + const size_t int3bits = sizeof(INT3) * 8; > > > + const unsigned long int3mask = -1UL >> int3bits << int3bits; > > Minor: It's better to use parentesises here. I just want to do tuda-suda to nil eight LSBs. Since operations has the same prio, it will look like (1 + 2) + 3. I see little enhancements, so ignoring. > > > > + return (instruction & int3mask) | INT3; > > > +} > > > + > > > +static int continue_until(pid_t chpid, void *addr) { > > > + int wstatus; > > > + struct user_regs_struct regs; > > > + > > > + /* Obtain the instructions at the . */ > > > + unsigned long data = ptrace(PTRACE_PEEKTEXT, chpid, addr, 0); > > Nit: s/0/NULL/. > Here and below. Yes, honey :( ================================================================================ diff --git a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c index 036f8141..6bf7562f 100644 --- a/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c +++ b/test/tarantool-c-tests/gh-8594-sysprof-ffunc-crash.test.c @@ -192,7 +192,7 @@ static void continue_until(pid_t chpid, void *addr) struct user_regs_struct regs; /* Obtain the instructions at the . */ - unsigned long data = ptrace(PTRACE_PEEKTEXT, chpid, addr, 0); + unsigned long data = ptrace(PTRACE_PEEKTEXT, chpid, addr, NULL); /* * Emit the instruction to the . * XXX: is poisoned as the LSB to the @@ -201,7 +201,7 @@ static void continue_until(pid_t chpid, void *addr) ptrace(PTRACE_POKETEXT, chpid, addr, int3poison(data)); /* Resume tracee until SIGTRAP occurs. */ - ptrace(PTRACE_CONT, chpid, 0, 0); + ptrace(PTRACE_CONT, chpid, NULL, NULL); /* * Wait tracee signal-delivery-stop and check @@ -210,7 +210,7 @@ static void continue_until(pid_t chpid, void *addr) wait_alive(chpid); /* Obtain GPR set to tweak RIP for further execution. */ - ptrace(PTRACE_GETREGS, chpid, 0, ®s); + ptrace(PTRACE_GETREGS, chpid, NULL, ®s); /* * Make sure we indeed are stopped at . * XXX: RIP points right after instruction. @@ -223,7 +223,7 @@ static void continue_until(pid_t chpid, void *addr) * instruction at the . */ regs.rip -= sizeof(INT3); - ptrace(PTRACE_SETREGS, chpid, 0, ®s); + ptrace(PTRACE_SETREGS, chpid, NULL, ®s); ptrace(PTRACE_POKETEXT, chpid, addr, data); } ================================================================================ > > > > + /* The tracee has to be alive and stopped by SIGTRAP. */ > > > + assert_false(WIFEXITED(wstatus)); > > > + assert_true(WIFSTOPPED(wstatus)); > > This part is duplicated, let's move it to a separate `healthcheck` > > function, or something like that. > > May be we can move these checks inside the `continue_until()` routine? > Do we want to check something except the status is `WIFSTOPPED(status)`? See the updated hunk in the previous reply. > > > -- > Best regards, > Sergey Kaplun -- Best regards, IM