From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id C36FA299719; Tue, 7 Feb 2023 10:27:11 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C36FA299719 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1675754831; bh=00zLNDxqMYyy1k2V4m0VWYiRjVXDjeVXdn527b78eD0=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FkwfAjzJQBhSSGSzbMc8WjF0kZUbffcM8YUedvo9ULKx0o23TOF9D2tgikt5Z+2C8 HqxH+qJ6Tdwy86JgJzWwIVW5N456PGbW4neMJQzrPVfmwpF/sIRwGhv+xdUSGlWoNL A6+PhBRDhrgXHsMt48AVrBIeXrXSiC/sv7C5aMa0= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 844857031B for ; Tue, 7 Feb 2023 10:27:10 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 844857031B Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1pPINh-0003ht-Jt; Tue, 07 Feb 2023 10:27:10 +0300 Date: Tue, 7 Feb 2023 10:23:40 +0300 To: Maxim Kokryashkin Message-ID: References: <20221205180151.417737-1-m.kokryashkin@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221205180151.417737-1-m.kokryashkin@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9D919194CF4FC660488DBEAA600CFE3E713BC9BB5F2D7A95A182A05F5380850408F640B55DAA2936284F54BBCA1E1E910C63A0EAFBE45F3E66C3BCC28E311DEAA X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7E3137013C338FE3BEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637DB9D17925D4663DB8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8E6DFEA73607CED081F4C8F18CAF50E80117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCF1175FABE1C0F9B6A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F4460429728776938767073520CCD848CCB6FE560C618001F51B5FD3F9D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B65D56369A3576CBA5089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34C53D1911AD7E41992F088D44FB4FDAAED958345C478855E5D57E844912AA208713A8A6E44EDF0CC11D7E09C32AA3244C703E05CBEF3EA9D056DF6507B3F89B8BBBA718C7E6A9E042927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojCpmAPesMHueoNiLAKYa8IA== X-DA7885C5: D38B236BAA7557C888A89D8F8AFEBD3D5668C8597C83AF6EE7F1A6AA2D5BA3DA262E2D401490A4A0DB037EFA58388B346E8BC1A9835FDE71 X-Mailru-Sender: 689FA8AB762F73933AF1F914F131DBF514902FF8B05FCE4FC550C7D812CF5F6C0FBE9A32752B8C9C2AA642CC12EC09F1FB559BB5D741EB962F61BD320559CF1EFD657A8799238ED55FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit v3] memprof: introduce cli flag to run dump parser X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Maxim! Thanks for the fixes! I belive that this is the last iteration of the review, so LGTM, after you'll fix some comments below. On 05.12.22, Maxim Kokryashkin wrote: > It is really inconvenient to use a standalone shell script to parse > memprof dump. That is why this commit introduces a CLI flag for tools > to the LuaJIT, so now it is possible to parse memprof dump > as simple as: > ``` > luajit -tm memprof.bin > ``` > > Closes tarantool/tarantool#5688 > --- > Changes in v3: > - Fixed comments as per review by Sergey > > Branch: https://github.com/tarantool/luajit/tree/fckxorg/gh-5688-cli-for-memprof-parse > PR: https://github.com/tarantool/tarantool/pull/8002 > Issue: https://github.com/tarantool/tarantool/issues/5688 > > CMakeLists.txt | 9 +-- > Makefile.original | 7 +- > src/CMakeLists.txt | 5 ++ > src/lj_tools_conf.h.in | 7 ++ > src/luajit.c | 63 ++++++++++++++++-- > .../gh-5688-memprof-cli-flag.test.lua | 64 +++++++++++++++++++ > tools/CMakeLists.txt | 2 + > 7 files changed, 145 insertions(+), 12 deletions(-) > create mode 100644 src/lj_tools_conf.h.in > create mode 100644 test/tarantool-tests/gh-5688-memprof-cli-flag.test.lua > > diff --git a/CMakeLists.txt b/CMakeLists.txt > index c870cce2..97d0d42f 100644 > --- a/CMakeLists.txt > +++ b/CMakeLists.txt > diff --git a/Makefile.original b/Makefile.original > index 0c92df9e..bb0ab73d 100644 > --- a/Makefile.original > +++ b/Makefile.original > diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt > index dffc0a4d..17674a41 100644 > --- a/src/CMakeLists.txt > +++ b/src/CMakeLists.txt > diff --git a/src/lj_tools_conf.h.in b/src/lj_tools_conf.h.in > new file mode 100644 > index 00000000..9f9a2e49 > --- /dev/null > +++ b/src/lj_tools_conf.h.in > diff --git a/src/luajit.c b/src/luajit.c > index 1ca24301..bd9ae8f3 100644 > --- a/src/luajit.c > +++ b/src/luajit.c > > +/* > +** On most Linux distros, it is the default value for the > +** maximum length of a string passed to `execve`. > +** However, there is no common value for other OSes, so > +** the size of 32 default memory pages is adopted. > +**/ Typo: s<**/><*/> > +#define MAX_ENV_VAR 32 * 4096 > + > +static int update_env_var(const char *name, const char *value) > +{ > + char env_buf[MAX_ENV_VAR] = ""; > + const char *env = getenv(name); > + if (env == NULL) { > + return setenv(name, value, 0); > + } else { > + strcpy(env_buf, env); > + return setenv(name, strcat(env_buf, value), 0); You shold use non-zero value here, if you want to rewrite the enviroment variable. | LUA_PATH=";;" src/luajit -tm /tmp/tmp_memprof.bin | src/luajit: /home/burii/reviews/luajit/cli-flags/tools/memprof.lua:13: module 'utils.bufread' not found: | no field package.preload['utils.bufread'] | ... But after the following patch it works fine: =================================================================== diff --git a/src/luajit.c b/src/luajit.c index bd9ae8f3..e40a4d30 100644 --- a/src/luajit.c +++ b/src/luajit.c @@ -424,7 +424,7 @@ static int update_env_var(const char *name, const char *value) return setenv(name, value, 0); } else { strcpy(env_buf, env); - return setenv(name, strcat(env_buf, value), 0); + return setenv(name, strcat(env_buf, value), 1); } } =================================================================== | LUA_PATH="./?.lua;;" src/luajit -tm /tmp/tmp_memprof.bin | ALLOCATIONS | =(command line):1: 174 events +6096 bytes -0 bytes | ... Looks like a good testcase to add. | + return setenv(name, strcat(env_buf, value), 0); Minor: I suggest to add a check length for total buffer length. (Yes, it's highly unlikely, that someone has env var >= 128 KB, but we still can use ENOMEM in such case.) Feel free to ignore. > + } > +} > + > /* check that argument has no extra characters at the end */ > #define notail(x) {if ((x)[2] != '\0') return -1;} > > @@ -398,6 +436,7 @@ static int dobytecode(lua_State *L, char **argv) > #define FLAGS_EXEC 4 > #define FLAGS_OPTION 8 > #define FLAGS_NOENV 16 > +#define FLAGS_TOOL 32 > > static int collectargs(char **argv, int *flags) > { > @@ -419,6 +458,12 @@ static int collectargs(char **argv, int *flags) > notail(argv[i]); > *flags |= FLAGS_VERSION; > break; > + case 't': > + *flags |= FLAGS_TOOL; > + if (argv[i][2] == '\0') return -1; > + if (argv[i + 1] == NULL) return -1; > + update_env_var("LUA_PATH", TOOLS_PATH); Should we check the return value of the call here (EINVAL, or ENOMEM)? If we don't worry about ENOMEM, feel free to ignore. > + return i + 1; > case 'e': > *flags |= FLAGS_EXEC; > case 'j': /* LuaJIT extension */ > @@ -474,6 +519,10 @@ static int runargs(lua_State *L, char **argv, int argn) > diff --git a/test/tarantool-tests/gh-5688-memprof-cli-flag.test.lua b/test/tarantool-tests/gh-5688-memprof-cli-flag.test.lua > new file mode 100644 > index 00000000..ba2d0219 > --- /dev/null > +++ b/test/tarantool-tests/gh-5688-memprof-cli-flag.test.lua > @@ -0,0 +1,64 @@ > +local utils = require('utils') > + > +-- XXX: The patch is for luajit only, and it doesn't Typo: s/luajit/LuaJIT/ > +-- work on Tarantool. > +-- luacheck: no global Nit: can we move luacheck comment one line below? (Firstly don't get is it `utils` global) > +utils.skipcond( > + (jit.arch ~= 'x86' and jit.arch ~= 'x64') or _TARANTOOL, > + jit.arch..' architecture is NIY for memprof' > +) > + > +local tap = require('tap') > + > +local test = tap.test('gh-5688-memprof-cli-flag') > +test:plan(2) > + > +jit.off() > +jit.flush() > + > +local table_new = require 'table.new' > + > +local TMP_BINFILE = utils.profilename('memprofdata.tmp.bin') > +local BAD_PATH = utils.profilename('bad-path-tmp.bin') > +local EXECUTABLE = utils.luacmd(arg) > + > +local function default_payload() > + -- Preallocate table to avoid table array part reallocations. > + local _ = table_new(20, 0) Minor: Should it be a constant as far as it is used twice? Also, do we need this monkey business about table size and so on as far as we don't check exactly values? > + > + -- Want too see 20 objects here. > + for i = 1, 20 do > + -- Try to avoid crossing with "test" module objects. > + _[i] = 'memprof-str-'..i > + end > + > + _ = nil > + -- VMSTATE == GC, reported as INTERNAL. > + collectgarbage() > +end > diff --git a/tools/CMakeLists.txt b/tools/CMakeLists.txt > index dd7ec6bd..e2e97b63 100644 > --- a/tools/CMakeLists.txt > +++ b/tools/CMakeLists.txt > -- > 2.38.1 > -- Best regards, Sergey Kaplun