From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 2EFFB6EC5D; Wed, 7 Apr 2021 19:39:30 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2EFFB6EC5D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617813570; bh=/NQlPPqBlLMGr7f9AgiTu1azKHuOk6BzKekGLNmZgiI=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ScfKYVGHBP1ieFANDqG0nm2njexgM/4dalP3Dh+Zmqqo/xOM2tqpixzLS6jxxccJv Ar+PGeVt37RoriXOQ4S+EUWYlXB7fm7snIBO7hMEMXaOsDSrG7J6XTYgAaoPTqAu2I xlxnRc7CGyZ/X4DOfTmqWQJ01EBIQF2epeslydSk= Received: from smtp63.i.mail.ru (smtp63.i.mail.ru [217.69.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id CB1066EC5D for ; Wed, 7 Apr 2021 19:39:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org CB1066EC5D Received: by smtp63.i.mail.ru with esmtpa (envelope-from ) id 1lUBDD-0002Cd-Ns; Wed, 07 Apr 2021 19:39:28 +0300 Date: Wed, 7 Apr 2021 19:38:32 +0300 To: Igor Munkin Message-ID: References: <6ca8010540a67a92b36327abf44b489ebddc5054.1617641697.git.imun@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6ca8010540a67a92b36327abf44b489ebddc5054.1617641697.git.imun@tarantool.org> X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD912A3E3D5D4B49FC12DF659087DAACB92BFBA00DE2463A66B00894C459B0CD1B92CE503B5C0C9889D336760CF82D453C6264CEB74278BE38AAF7879D52E6103F3 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7081BBE264C6D7F42EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006374FBED2B83F5E00CA8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2C424FD20AE59F916337B8C722B4C105F8F4427C359EA06A0D2E47CDBA5A96583C09775C1D3CA48CF17B107DEF921CE79117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE7328B01A8D746D8839FA2833FD35BB23DF004C906525384302BEBFE083D3B9BA71A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89F83C798A30B85E16B5E1C53F199C2BB95B5C8C57E37DE458BEDA766A37F9254B7 X-C1DE0DAB: 0D63561A33F958A58FB64B4D30189468AC6504866711EC91082F5DAC259BE3F5D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3435B1F14A93C4073FD545C0FCC1A7665C3739F1B0DDC8ED068607C31E4D88396F019E74292762E4711D7E09C32AA3244C817A012B4319D77E3D30D3C06F323403B018FE5BB746DCD1FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioju+jaMfvANXoZg6BsXw/ePQ== X-Mailru-Sender: 3B9A0136629DC91206CBC582EFEF4CB4DD24B941467DFD948874614ABB2BD2771D49A047E6701026F2400F607609286E924004A7DEC283833C7120B22964430C52B393F8C72A41A89437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 3/3] test: fix dynamic modules loading on MacOS X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Igor, Thanks for the patch! LGTM, except a few nitpicks below. On 05.04.21, Igor Munkin wrote: > This patch resolves the issue with running the tests with auxiliary > dynamically loaded modules in case of out-of-source build. > > The first issue relates to the way modules loaded at runtime are built > on MacOS. Since the auxiliary libraries are built as a dynamically > loaded modules on MacOS instead of shared libraries as it is done on > Linux and BSD, another environment variable should be used to guide > while searching the extension. Hence the values collected in > scope of macro need to be set to DYLD_LIBRARY_PATH > variable instead of LD_LIBRARY_PATH on MacOS. > > Unfortunately, this rather small change doesn't resolve the problem at > all and the root cause lies much deeper than it seems at the beginning. This paragraph looks uninformative, looks like you can just use "Moreover, " for the next sentence instead. > > Apple tries their best to "protect their users from malicious software". > As a result SIP[1] has been designed and released. Now, Apple developers > are *so protected*, that they can load nothing being not installed in > the system, since some programs sanitize the environment before they > start child processes. Specifically, environment variables starting with > DYLD_ and LD_ are unset for child process started by system programs[2]. Sorry, but this paragraph is too wordy. I propose to reformulate it like the following: | Moreover, Apple tries their best to "protect their users from malicious | software". As a result, SIP[1] has been designed and released. As a part | of it, environment variables starting with DYLD_ and LD_ are unset | for child process started by system programs[2]. Feel free to ignore. > > That which does not kill us makes us stronger: fortunately, these I suppose you can just drop this preamble. | These environment variables are used by FFI machinery to find the proper | shared library, hence we can still tweak testing environment before | calling . > environment variables are used by FFI machinery to find the proper > shared library, hence we can still tweak testing environment before > calling . However, the value can't be passed via the standard > environment variable, so we prepend TEST_ prefix to its name to get > around SIP magic tricks. Finally, to set the variable required by FFI > machinery the introduced routine is used. PROFIT! > Your move, Cupertino geniuses. The last two sentences are redundant. Remove them please. > > [1]: https://support.apple.com/en-us/HT204899 > [2]: https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html > > Resolves tarantool/tarantool#5959 > Follows up tarantool/tarantool#4862 > > Co-authored-by: Sergey Ostanevich > Co-authored-by: Mons Anderson > Signed-off-by: Igor Munkin > --- > test/tarantool-tests/CMakeLists.txt | 39 +++++++++++++++++-- > .../gh-4427-ffi-sandwich.test.lua | 4 ++ > .../lj-flush-on-trace.test.lua | 4 ++ > test/tarantool-tests/utils.lua | 39 ++++++++++++++++--- > 4 files changed, 77 insertions(+), 9 deletions(-) > > diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt > index 3e36ff86..c793ad60 100644 > --- a/test/tarantool-tests/CMakeLists.txt > +++ b/test/tarantool-tests/CMakeLists.txt > @@ -69,11 +69,46 @@ set(LUA_PATH > ) > set(LUA_TEST_SUFFIX .test.lua) > set(LUA_TEST_FLAGS --failures --shuffle) > +set(LUA_TEST_ENV > + "LUA_PATH=\"${LUA_PATH}\;\;\"" > + "LUA_CPATH=\"${LUA_CPATH}\;\;\"" > +) > > if(CMAKE_VERBOSE_MAKEFILE) > list(APPEND LUA_TEST_FLAGS --verbose) > endif() > > +# XXX: Since the auxiliary libraries are built as a dynamically > +# loaded modules on MacOS instead of shared libraries as it is > +# done on Linux and BSD, another environment variable should be > +# used to guide while searching the extension. > +if(CMAKE_SYSTEM_NAME STREQUAL "Darwin") > + # XXX: Apple tries their best to "protect their users from > + # malware". As a result SIP (see the link[1] below) has been > + # designed and released. Now, Apple developers are so protected, > + # that they can load nothing being not installed in the system, > + # since some programs sanitize the environment before they start > + # child processes. Specifically, environment variables starting > + # with DYLD_ and LD_ are unset for child process started by > + # system programs (like /usr/bin/env used for preparing testing > + # environment). For more info, see the docs[2] below. > + # > + # That which does not kill us makes us stronger: fortunately, > + # these environment variables are used by FFI machinery to find > + # the proper shared library, hence we can still tweak testing > + # environment before calling . However, the value > + # can't be passed via the standard environment variable, so we > + # prepend TEST_ prefix to its name to get around SIP magic > + # tricks. Finally, to set the variable required by FFI machinery > + # the introduced routine is used. Same comments as for the commit message. > + # > + # [1]: https://support.apple.com/en-us/HT204899 > + # [2]: https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html > + list(APPEND LUA_TEST_ENV TEST_DYLD_LIBRARY_PATH="${LD_LIBRARY_PATH}") > +else() > + list(APPEND LUA_TEST_ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}") > +endif() > + > # LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list > # with dependecies are set in scope of BuildTestLib macro. > add_custom_target(tarantool-tests > @@ -83,9 +118,7 @@ add_custom_command(TARGET tarantool-tests > COMMENT "Running Tarantool tests" > COMMAND > env > - LUA_PATH="${LUA_PATH}\;\;" > - LUA_CPATH="${LUA_CPATH}\;\;" > - LD_LIBRARY_PATH="${LD_LIBRARY_PATH}" > + ${LUA_TEST_ENV} > ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR} > --exec '${LUAJIT_TEST_COMMAND}' > --ext ${LUA_TEST_SUFFIX} > diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua > index 64df5dbd..651dc3f4 100644 > --- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua > +++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua > @@ -3,6 +3,10 @@ local utils = require('utils') > -- Disabled on *BSD due to #4819. > utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819') > > +-- XXX: Tweak the process environment to get around SIP. > +-- See the comment in suite CMakeLists.txt for more info. > +utils.tweakenv(jit.os == 'OSX', 'DYLD_LIBRARY_PATH') > + > utils.selfrun(arg, { > { > arg = { > diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua > index edc5cf61..1ad4f832 100644 > --- a/test/tarantool-tests/lj-flush-on-trace.test.lua > +++ b/test/tarantool-tests/lj-flush-on-trace.test.lua > @@ -3,6 +3,10 @@ local utils = require('utils') > -- Disabled on *BSD due to #4819. > utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819') > > +-- XXX: Tweak the process environment to get around SIP. > +-- See the comment in suite CMakeLists.txt for more info. > +utils.tweakenv(jit.os == 'OSX', 'DYLD_LIBRARY_PATH') > + > utils.selfrun(arg, { > { > arg = { > diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua > index d2dd71b0..9bdb71ec 100644 > --- a/test/tarantool-tests/utils.lua > +++ b/test/tarantool-tests/utils.lua > @@ -1,7 +1,12 @@ > local M = {} > > +local ffi = require('ffi') > local tap = require('tap') > > +ffi.cdef([[ > + int setenv(const char *name, const char *value, int overwrite); > +]]) Side note: looks like we need to move `os.setenv()` fucntion from the Tarantool's built-ins onboard as `misc.setenv()`. > + > -- > 2.25.0 > -- Best regards, Sergey Kaplun