From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 14018469FF8; Wed, 24 May 2023 12:50:43 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 14018469FF8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1684921843; bh=NKiaBTBmcTd7JiyasAVxDxpPmciWXl4ErN6FxFJ/EOI=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=vfH5TKeRuQktsif1LzyJR7747xOz7AjKCOon8Iy1b0/BAiFmaX2faLusY+KbloUyI B1F38sptG0rHFh6wYoy6rpN3F8Y9VLDC8GfBrk2iJ4H7CJacvNRAvfgDqkUrkr/Uo6 RA6slCtkXZgwMfP/JDn71tGajlDqWErE5nqC6wv4= Received: from smtpng3.i.mail.ru (smtpng3.i.mail.ru [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 6AA054695E0 for ; Wed, 24 May 2023 12:50:41 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6AA054695E0 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1q1l8i-0005xw-Ck; Wed, 24 May 2023 12:50:41 +0300 Date: Wed, 24 May 2023 12:46:34 +0300 To: Maxim Kokryashkin Message-ID: References: <20230523152748.203212-1-m.kokryashkin@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230523152748.203212-1-m.kokryashkin@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD9318AAE2601AA39B8CDDE39D04FBBDEACD358682397C8999C00894C459B0CD1B9AD7832FD8C5FAF6B60B62119E294A8FA00A491649C02A0306AE5DE23F762A4BF X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE77D3344F2D0C9F5BEEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637E5A8E5ECB3FF24018638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D826F94B9D5FFD46DC619B821254AB780B117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC8E8F974BDF881913A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520C24E1E72F37C03A0F6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B9F5955FECEF5819E75ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: 0D63561A33F958A514120B1587B6AF38CBDFE1F2CD4894CEF01277A428187216F87CCE6106E1FC07E67D4AC08A07B9B0DB8A315C1FF4794DBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF3D5B36023A2EDA4DBD91AC06A73AF3BFC49083459C3970496D4F7ABA481AE2EA192AF36DABE85845D5191E2618EE498C5198EB5AAFB590B276B44660069A6A36F4E8A8FB6BF8EBF5 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj7RK+RruQK1d0Nhm88EhOgQ== X-DA7885C5: 59E5B863EDA370E9E3166EC3F2F1C0BA6DE395A9A6F15C59305AA820D38A3478262E2D401490A4A0DB037EFA58388B346E8BC1A9835FDE71 X-Mailru-Sender: 689FA8AB762F73930F533AC2B33E986B84185EAA3A79AA2CF45D2544AB345FA10FBE9A32752B8C9C2AA642CC12EC09F1FB559BB5D741EB962F61BD320559CF1EFD657A8799238ED55FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit] sysprof: move symtab update into profile hook X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Maxim! Thanks for the patch! Please, consider my comments below. On 23.05.23, Maxim Kokryashkin wrote: > Before the patch, the symtab update was done in the signal > handler. That update requires memory allocation, which > can't be done safely in a signal handler. This patch reuses > LuaJIT's HOOK_PROFILE for symtab update routine execution, so > it is now possible to update it safely after the signal > handler exit. > > Resolves tarantool/tarantool#8140 > --- > Branch: https://github.com/tarantool/luajit/tree/fckxorg/gh-8140-sysprof-allocator-crash > Issue: https://github.com/tarantool/tarantool/issues/8140 > PR: https://github.com/tarantool/tarantool/pull/8691 > > src/lj_dispatch.c | 10 ++++ > src/lj_sysprof.c | 34 +++++++++++++- > src/lj_sysprof.h | 4 ++ > test/tarantool-tests/CMakeLists.txt | 1 + > .../gh-8140-sysprof-allocator-crash.test.lua | 46 +++++++++++++++++++ > .../CMakeLists.txt | 9 ++++ > .../sysprofalloc.c | 17 +++++++ > 7 files changed, 119 insertions(+), 2 deletions(-) > create mode 100644 test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua > create mode 100644 test/tarantool-tests/gh-8140-sysprof-allocator-crash/CMakeLists.txt > create mode 100644 test/tarantool-tests/gh-8140-sysprof-allocator-crash/sysprofalloc.c > > diff --git a/src/lj_dispatch.c b/src/lj_dispatch.c > index ee735450..65948a0c 100644 > --- a/src/lj_dispatch.c > +++ b/src/lj_dispatch.c > diff --git a/src/lj_sysprof.c b/src/lj_sysprof.c > index 2e9ed9b3..5b5cbe2a 100644 > --- a/src/lj_sysprof.c > +++ b/src/lj_sysprof.c > @@ -76,6 +76,7 @@ struct sysprof { > lj_profile_timer timer; /* Profiling timer. */ > int saved_errno; /* Saved errno when profiler failed. */ > uint32_t lib_adds; /* Number of libs loaded. Monotonic. */ > + volatile sig_atomic_t symtab_update_needed; /* Symtab update request flag. */ > }; > /* > ** XXX: Only one VM can be profiled at a time. > @@ -88,6 +89,36 @@ static struct sysprof sysprof = {0}; > static const uint8_t ljp_header[] = {'l', 'j', 'p', LJP_FORMAT_VERSION, > 0x0, 0x0, 0x0}; > > +void lj_symtab_update_hook(lua_State *L) { > + struct sysprof *sp = &sysprof; > + global_State *g = G(L); > + uint8_t mask; > + mask = (g->hookmask & ~HOOK_PROFILE); Nit: I suppose, that two lines above may be joined. > + sp->symtab_update_needed = 0; > + if (!(mask & HOOK_VMEVENT)) { Does it mean, that if we have signal inside VMEVENT we don't update our symtab table (since flag is reset) (*)? Should we move the line | sp->symtab_update_needed = 0; inside if condition? > + g->hookmask = HOOK_VMEVENT; > + lj_dispatch_update(g); > + lj_symtab_dump_newc(&sp->lib_adds, &sp->out, LJP_SYMTAB_CFUNC_EVENT, L); > + } > + g->hookmask = mask; > + lj_dispatch_update(g); > +} > + > +int lj_symtab_update_requested() { > + struct sysprof *sp = &sysprof; > + return sp->symtab_update_needed; > +} > + > +static void setup_symtab_update_hook(struct sysprof *sp) { > + global_State *g = sp->g; > + uint8_t mask = g->hookmask; > + if (!(mask & (HOOK_PROFILE|HOOK_VMEVENT|HOOK_GC))) { Why GC hook is mentioned here? > + sp->symtab_update_needed = 1; > + g->hookmask = (mask | HOOK_PROFILE); > + lj_dispatch_update(g); > + } The similar to (*) question here: If we have a signal inside VMEVENT, does this mean that there is no symbols to load for sure? If so, coment is highly desired. > +} > + > static int stream_is_needed(struct sysprof *sp) > { > return sp->opt.mode != LUAM_SYSPROF_DEFAULT; > @@ -240,8 +271,7 @@ static void stream_guest(struct sysprof *sp, uint32_t vmstate) > diff --git a/src/lj_sysprof.h b/src/lj_sysprof.h > index 7e8c2e6e..456ca76f 100644 > --- a/src/lj_sysprof.h > +++ b/src/lj_sysprof.h > @@ -108,4 +108,8 @@ void lj_sysprof_add_proto(const struct GCproto *pt); > diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt > index a428d009..25244733 100644 > --- a/test/tarantool-tests/CMakeLists.txt > +++ b/test/tarantool-tests/CMakeLists.txt > diff --git a/test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua b/test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua > new file mode 100644 > index 00000000..afce83e2 > --- /dev/null > +++ b/test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua > @@ -0,0 +1,46 @@ > +local tap = require('tap') > +local test = tap.test('gh-8140-sysprof-allocator-crash'):skipcond({ > + ["Sysprof is implemented for x86_64 only"] = jit.arch ~= "x86" and > + jit.arch ~= "x64", > + ["Sysprof is implemented for Linux only"] = jit.os ~= "Linux", > +}) I've reverted your changes and test still passes before the patch: | /home/burii/reviews/luajit/gh-8140-sysprof-in-alloc/test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua ............. ok | All tests successful. I've restored the sysprof.[ch] and dispatch changes, and leave tests as is. | gst | On branch fckxorg/gh-8140-sysprof-allocator-crash | Your branch is behind 'origin/fckxorg/gh-8140-sysprof-allocator-crash' by 1 commit, and can be fast-forwarded. | (use "git pull" to update your local branch) | | Changes to be committed: | (use "git restore --staged ..." to unstage) | modified: test/tarantool-tests/CMakeLists.txt | new file: test/tarantool-tests/gh-8140-sysprof-allocator-crash.test.lua | new file: test/tarantool-tests/gh-8140-sysprof-allocator-crash/CMakeLists.txt | new file: test/tarantool-tests/gh-8140-sysprof-allocator-crash/sysprofalloc.c I've tried several times and test still passes. > +test:plan(2) > + > +local profilename = require("utils").profilename > +local profile = require('jit.profile') > + > +local TMP_BINFILE = profilename("sysprofdata.tmp.bin") > +local callback_called = false > + > +local function payload() > + local r = 0 > + for i = 1, 1e8 do > + r = r + i > + end > + return r > +end May be we should use more GC-specific payload? > + > +local function callback(_, _, _) > + callback_called = true > +end > + > +profile.start('f', callback) > +payload() > +profile.stop() > + > +test:ok(callback_called, 'LuaJIT profiler callback was not called.') Looks like it should be "was called"? > + > +jit.off() > +misc.sysprof.start{mode='C', interval=1, path=TMP_BINFILE} > +for _ = 1, 1e4 do > + require('sysprofalloc').get_string() Why do you try to require the module several times? Is it saved in `package.loaded` anyway? > + -- Make sure that C library is collected, so it will be loaded > + -- again on the next iteration. > + collectgarbage() > +end > +misc.sysprof.stop() > + > +test:ok(true, 'Sysprof has crashed.') Looks like it should be "has not crashed"? > + > +os.remove(TMP_BINFILE) > +os.exit(test:check() and 0 or 1) > diff --git a/test/tarantool-tests/gh-8140-sysprof-allocator-crash/CMakeLists.txt b/test/tarantool-tests/gh-8140-sysprof-allocator-crash/CMakeLists.txt > new file mode 100644 > index 00000000..dbe60219 > --- /dev/null > +++ b/test/tarantool-tests/gh-8140-sysprof-allocator-crash/CMakeLists.txt > @@ -0,0 +1,9 @@ > +if (NOT(CMAKE_SYSTEM_NAME STREQUAL "Darwin")) > + BuildTestCLib(sysprofalloc sysprofalloc.c) > + # Unfortunately, command is introduced > + # since CMake 3.13, so we can't use it now considering ancient > + # distros support. Just build linker flags by hands. > + set(CMAKE_SHARED_LINKER_FLAGS > + "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--hash-style=both" Do we need both hashes here? > + ) > +endif() > diff --git a/test/tarantool-tests/gh-8140-sysprof-allocator-crash/sysprofalloc.c b/test/tarantool-tests/gh-8140-sysprof-allocator-crash/sysprofalloc.c > new file mode 100644 > index 00000000..d3c41d42 > --- /dev/null > +++ b/test/tarantool-tests/gh-8140-sysprof-allocator-crash/sysprofalloc.c > @@ -0,0 +1,17 @@ > +#include > +#include > + > +int get_string(lua_State *L) { > + lua_pushstring(L, "test string"); > + return 1; Something strange with indentation: please, use tabs for __our__ C code in tests. > +} > + > +static const struct luaL_Reg sysprofalloc [] = { > + {"get_string", get_string}, > + {NULL, NULL} > +}; > + > +int luaopen_sysprofalloc(lua_State *L) { > + luaL_register(L, "sysprofalloc", sysprofalloc); > + return 1; > +} > -- > 2.40.1 > -- Best regards, Sergey Kaplun