From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id C1CB863B671; Thu, 7 Sep 2023 18:21:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C1CB863B671 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1694100112; bh=ikadDx98y8Bry7Hw68ABUG0+2tXQk2zO153kVL7l92A=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=DK842IBVfQmtg5+97COUcHLCVnNT5w+SdhhpIhWZ/fWGvmCa9GkL/LIMZRreRIgOq ihvp9/5PKEh9eaEFCbObjykL9nFNOgUvlI0CYIO82FvYmf6WdzqV8wsF83giUjIOgU kwYn6DD3qo/3NG1nxMabfoHHyoWCFxRTQa6x+Ok4= Received: from smtp43.i.mail.ru (smtp43.i.mail.ru [95.163.41.66]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id D52B95CE1A1 for ; Thu, 7 Sep 2023 18:21:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D52B95CE1A1 Received: by smtp43.i.mail.ru with esmtpa (envelope-from ) id 1qeGpL-002jf5-04; Thu, 07 Sep 2023 18:21:51 +0300 Message-ID: <272d18bd-5333-1224-31c3-8d59475f6afc@tarantool.org> Date: Thu, 7 Sep 2023 18:21:50 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Content-Language: en-US To: Sergey Kaplun , Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org, max.kokryashkin@gmail.com References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD96201AD55A1C8F7DB4B94316895B7D143959E6F65F85AE25C182A05F5380850404C228DA9ACA6FE276DC8FF4014F1413799A37398BB5DEA4221FB6D76241ABF45142629AA6E676541 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE72424932217190DC2EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006372D22CC9620B7D5068638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8FB0410995022EAE80CE915DD31A80B3A117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC00E8CE3DD197987DA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD186FD1C55BDD38FC3FD2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE9647ADFADE5905B19100238FE36DC7A2D8FC6C240DEA76429C9F4D5AE37F343AA9539A8B242431040A6AB1C7CE11FEE34E7D9683544204AF2D242C3BD2E3F4C6C4224003CC836476E2F48590F00D11D6E2021AF6380DFAD1A18204E546F3947C2FFDA4F57982C5F42E808ACE2090B5E1725E5C173C3A84C317B107DEF921CE79089D37D7C0E48F6C8AA50765F79006373568875668442E59EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C1DE0DAB: 0D63561A33F958A5D40078F05CAC9822A2292F5AF40DDD95CBDB9E86D3B0545BF87CCE6106E1FC07E67D4AC08A07B9B065B78C30F681404DCB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC7177DD89D51EBB7742424CF958EAFF5D571004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D34AC6E62257D6CD1C9E29DE1ADA50360FE793ABED4E11E7B8932E300A96C875F133970789C93A97D1E1D7E09C32AA3244CD86F32499502537D729D0455BD2A0C5F7101BF96129E4011BAD658CF5C8AB4025DA084F8E80FEBD3FFA33E6B6B2F82C47A83BD0C44CE203720ABEDE4BBDD9CDD X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojHV97et/Y5fzwS8IHlSAjEA== X-Mailru-Sender: 11C2EC085EDE56FAC07928AF2646A76958476436DFE495F699A37398BB5DEA42816609AA8FC7A761EBA65886582A37BD66FEC6BF5C9C28D98A98C1125256619760D574B6FC815AB872D6B4FCE48DF648AE208404248635DF X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2][v2] Fix embedded bytecode loader. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey On 9/5/23 17:10, Sergey Kaplun wrote: > Hi, Sergey! > Thanks for the patch! > Please, consider my comments below. > > On 31.08.23, Sergey Bronnikov wrote: >> From: Sergey Bronnikov >> >> (cherry-picked from commit 820339960123dc78a7ce03edf53fcf4fdae0e55d) >> >> The original problem is specific to x32 and is as follows: when a chunk >> with a bytecode library is loaded into memory, and the address is higher >> than 0x80000100, the `LexState->pe`, that contains an address of the end >> of the bytecode chunk in the memory, will wrap around and become smaller >> than the address in `LexState->p`, that contains an address of the >> beginning of bytecode chunk in the memory. In `bcread_fill()` called by >> `bcread_want()`, `memcpy()` is called with a very large size and causes >> bus error on x86 and segmentation fault on ARM Android. > Typo: s/bus error/the bus error/ > Typo: s/segmentation fault/the segmentation fault/ Fixed. > >> The problem cannot be reproduced on platforms supported by Tarantool >> (ARM64, x86_64), so test doesn't reproduce a problem without a patch and >> tests the patch partially. >> >> Sergey Bronnikov: >> * added the description and the test >> --- >> src/lib_package.c | 4 +- >> src/lj_bcread.c | 10 +- >> src/lj_lex.c | 6 ++ >> src/lj_lex.h | 1 + >> .../lj-549-bytecode-loader.test.lua | 96 +++++++++++++++++++ >> 5 files changed, 110 insertions(+), 7 deletions(-) >> create mode 100644 test/tarantool-tests/lj-549-bytecode-loader.test.lua >> > > >> diff --git a/test/tarantool-tests/lj-549-bytecode-loader.test.lua b/test/tarantool-tests/lj-549-bytecode-loader.test.lua >> new file mode 100644 >> index 00000000..889be80a >> --- /dev/null >> +++ b/test/tarantool-tests/lj-549-bytecode-loader.test.lua >> @@ -0,0 +1,96 @@ >> +local tap = require('tap') >> +local ffi = require('ffi') >> +local utils = require('utils') >> +local test = tap.test('lj-549-bytecode-loader'):skipcond({ >> + -- ['Test requires GC64 mode enabled'] = not require('ffi').abi('gc64'), >> +}) > Minor: It's better to require ffi and utils after test initialization > via `tap.test()`, see other tests for example. > Also, I suppose that we don't need `utils` itself, but > `utils.exec.makecmd`. Fixed. >> + >> +test:plan(1) >> + >> +-- Test creates a shared library with LuaJIT bytecode, >> +-- loads shared library as a Lua module and checks, >> +-- that no crashes eliminated. >> +-- >> +-- $ make HOST_CC='gcc -m32' TARGET_CFLAGS='-m32' \ >> +-- TARGET_LDFLAGS='-m32' \ >> +-- TARGET_SHLDFLAGS='-m32' \ >> +-- -f Makefile.original >> +-- $ echo 'print("test")' > a.lua >> +-- $ LUA_PATH="src/?.lua;;" luajit -b a.lua a.c >> +-- $ gcc -m32 -fPIC -shared a.c -o a.so >> +-- $ luajit -e "require('a')" >> +-- Program received signal SIGBUS, Bus error >> + >> +local function file_exists(fname) >> + return io.open(fname, 'r') or true and false > OK, this is a little bit confusing: > If file doesn't exists we go to `or true` and after check `and false` > which is always false. Tricky, but works. > > Also, here we don't close file handler. > I suggest it is better to rewrite this as the following: > | local fh = io.open(name, 'r') > | return fh and io.close(fh) > > It is simplier to read, and fixes problem with leaking handler. Updated. > >> +end >> + >> +local function get_file_name(file) >> + return file:match("[^/]*$") > Minor: it may match the empty string for a directory occasionally: > | src/luajit -e 'print([["]]..("/tmp/"):match("[^/]*$")..[["]])' Fixed. > | "" > > Nit: use single quotes instead of double quotes if possible. Without context it is difficult to get what is line you talk about. As I see everything is fine with quotes in version on the branch. > > Nit: `[^/\\]` is better since it also covers Windows. > See > | local dirname = arg[0]:gsub('([^/\\]+)$', '') > Since we don't support Windows feel free to ignore. > >> +end >> + >> +local stdout_msg = 'Lango team' >> +local lua_code = ('print(%q)'):format(stdout_msg) >> +local fpath = os.tmpname() >> +local path_lua = ('%s.lua'):format(fpath) >> +local path_c = ('%s.c'):format(fpath) >> +local path_so = ('%s.so'):format(fpath) > Minor: I suppose it should be renamed to `path_shared`, since on macOS > they have the ".dyld" suffix for shared libs. Hence, we need to use the > suffix in format of the shared library name too. You may take some > inspiration from here [1]. Fixed. >> + >> +-- Create a file with a minimal Lua code. >> +local fh = assert(io.open(path_lua, 'w')) >> +fh:write(lua_code) >> +fh:close() >> + >> +local module_name = assert(get_file_name(fpath)) >> + >> +local basedir = function(path) >> + local sep = '/' > Why do we need an additional variable here? For clarity. > > Nit: Indent is 4 spaces instead of 2. > >> + return path:match('(.*' .. sep .. ')') or './' > It's better to mention that the pattern matching is greedy, so we match > until the last separator. Updated. >> +end >> + >> +-- Create a C file with LuaJIT bytecode. >> +-- We cannot use utils.makecmd, because command-line generated >> +-- by `makecmd` contains `-e` that is incompatible with option `-b`. > Nit: comment line width is more than 66 symbols Fixed. >> +local function create_c_file(pathlua, pathc) >> + local lua_path = os.getenv('LUA_PATH') >> + local lua_bin = require('utils').exec.luacmd(arg):match('%S+') >> + local cmd_fmt = 'LUA_PATH="%s" %s -b %s %s' >> + local cmd = (cmd_fmt):format(lua_path, lua_bin, pathlua, pathc) >> + local ret = os.execute(cmd) >> + assert(ret == 0, 'create a C file with bytecode') >> +end >> + >> +create_c_file(path_lua, path_c) >> +assert(file_exists(path_c)) > Minor: The test flow is a little bit hard to read due to function > declarations. Maybe it is better to declare all utility functions first > and then use them one by one? This makes control flow easier to read. Rearranged, take a look please. > >> + >> +-- Compile C source code with LuaJIT bytecode to a shared library. >> +-- `-m64` is not available on ARM64, see >> +-- "3.18.1 AArch64 Options in the manual", >> +-- https://gcc.gnu.org/onlinedocs/gcc/AArch64-Options.html >> +local cflags_64 = jit.arch == 'arm64' and '-march=armv8-a' or '-m64' >> +local cflags = ffi.abi('32bit') and '-m32' or cflags_64 >> +local cc_cmd = ('cc %s -fPIC -shared %s -o %s'):format(cflags, path_c, path_so) >> +local ph = io.popen(cc_cmd) >> +ph:close() >> +assert(file_exists(path_so)) >> + >> +-- Load shared library as a Lua module. >> +local lua_cpath = ('"/tmp/?.so;"'):format(basedir(fpath)) >> +assert(file_exists(path_so)) >> +local cmd = utils.exec.makecmd(arg, { >> + script = ('-e "require([[%s]])"'):format(module_name), > Nit: Indent is 4 spaces instead of 2. Fixed. > >> + env = { >> + LUA_CPATH = lua_cpath, > Nit: Indent is 4 spaces instead of 2. Fixed. > >> + -- It is required to cleanup LUA_PATH, otherwise >> + -- LuaJIT loads Lua module, see tarantool-tests/utils/init.lua. > Nit: comment line width is more than 66 symbols Fixed too. > > Actually I don't understand from the comment what Lua module exactly is > loaded. Maybe it's better to fix this behaviour? What behaviour you want to fix? > Feel free to ignore. > >> + LUA_PATH = '', >> + }, >> +}) >> +local res = cmd() >> +test:ok(res == stdout_msg, 'bytecode loader works') >> + >> +os.remove(path_lua) >> +os.remove(path_c) >> +os.remove(path_so) >> + >> +os.exit(test:check() and 0 or 1) >> -- >> 2.34.1 >> > [1]: https://github.com/tarantool/tarantool/blob/dc8973c3de6311ab11df8d43520e1d40de4b9c7b/test/box/func_reload.test.lua#L5 >