Re: [Tarantool-patches] [PATCH luajit 2/8] test: introduce MakeLuaPath.cmake helper

[Igor Munkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Sergey,

Thanks for your review!

On 18.08.22, Sergey Kaplun wrote:
> Hi, Igor!
> 
> Thanks for the patch!
> 
> Please consider my comments below.
> 
> On 11.08.22, Igor Munkin wrote:
> > While extending test suites it is often required to append additional
> > path where Lua or Lua-C auxiliary modules are located to LUA_PATH or
> > LUA_CPATH environment variables. Due to insane semicolon interpolation
> > in CMake strings (that converts such string to a list as a result), we
> > need to escape semicolon in LUA_PATH/LUA_CPATH strings while building
> > the resulting value.
> > 
> > After the years of struggling MakeLuaPath.cmake module is introduced to
> 
> Minor: s/After/Over/

No it's not. I mean that as a result of the years of struggling the
module is introduced. It was not being introduced over the years of
struggling.

> 
> > make LUA_PATH and LUA_CPATH definition convenient with <make_lua_path>
> > helper. This function takes all paths given as a variable list argument,
> > joins them in a reverse order by a semicolon and yields the resulting
> > string to a specified CMake variable.
> > 
> > Signed-off-by: Igor Munkin <imun@tarantool.org>
> > ---
> >  cmake/MakeLuaPath.cmake                   | 46 +++++++++++++++++++++++
> >  test/CMakeLists.txt                       |  2 +
> >  test/PUC-Rio-Lua-5.1-tests/CMakeLists.txt |  8 +++-
> >  test/lua-Harness-tests/CMakeLists.txt     | 16 +++++---
> >  test/tarantool-tests/CMakeLists.txt       | 28 ++++++++------
> >  5 files changed, 81 insertions(+), 19 deletions(-)
> >  create mode 100644 cmake/MakeLuaPath.cmake
> > 
> > diff --git a/cmake/MakeLuaPath.cmake b/cmake/MakeLuaPath.cmake
> > new file mode 100644
> > index 00000000..9a5a3bb8
> > --- /dev/null
> > +++ b/cmake/MakeLuaPath.cmake
> > @@ -0,0 +1,46 @@
> > +# make_lua_path provides a convenient way to define LUA_PATH and
> > +# LUA_CPATH variables.
> > +#
> > +# Example usage:
> > +#
> > +#   make_lua_path(LUA_PATH
> > +#     PATH
> > +#       ${CMAKE_CURRENT_SOURCE_DIR}/?.lua
> > +#       ${CMAKE_BINARY_DIR}/?.lua
> > +#       ./?.lua
> > +#   )
> > +#
> > +# This will give you the string:
> > +#    "./?.lua;${CMAKE_BINARY_DIR}/?.lua;${CMAKE_CURRENT_SOURCE_DIR}/?.lua;;"
> > +# XXX: Mind the reverse order of the entries in the result string.
> 
> Why do we need this behaviour? May be it is better to save strict order
> of entries?

The code will become more complex as a result and in the 99.999% of the
cases the order of LUA_PATH entries doesn't bother you. So I don't see
we should make the code more complex to save the order of the entries.

> 
> > +
> > +function(make_lua_path path)
> > +  set(prefix ARG)
> > +  set(noValues)
> > +  set(singleValues)
> > +  set(multiValues PATHS)
> > +
> > +  # FIXME: if we update to CMake >= 3.5, can remove this line.
> 
> So, we can just check CMake version as Sergey suggested.

No, we can't (see the comment in the Sergey's thread).

> 
> > +  include(CMakeParseArguments)
> > +  cmake_parse_arguments(${prefix}
> > +                        "${noValues}"
> > +                        "${singleValues}"
> > +                        "${multiValues}"
> > +                        ${ARGN})
> > +
> > +  # XXX: This is the sentinel semicolon having special meaning
> 
> Nit: It's not single semicolon, but two of them. This ";;" special value
> is created during concatenation at the next lines.

Yes, it is. But strictly saying only the second semicolon has a special
meaning, the first one is a separator; otherwise it should be three of
them :)

> 
> > +  # for LUA_PATH and LUA_CPATH variables. For more info, see the
> > +  # link below:
> > +  # https://www.lua.org/manual/5.1/manual.html#pdf-LUA_PATH
> > +  set(result "\;")
> > +
> > +  foreach(inc ${ARG_PATHS})
> > +    # XXX: If one joins two strings with semicolon, the value
> 
> Typo: s/semicolon/the semicolon/

Fixed.

> 
> > +    # automatically becomes a list. I found a single working
> > +    # solution to make result variable be a string via "escaping"
> 
> Minor: s/be/stay/ or just s/be//
> Feel free to ignore.

I don't get why. Ignoring.

> 
> > +    # the semicolon right in string interpolation.
> > +    set(result "${inc}\;${result}")
> > +  endforeach()
> > +
> > +  set(${path} "${result}" PARENT_SCOPE)
> > +endfunction()
> > diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
> > index ba25af54..a8262b12 100644
> > --- a/test/CMakeLists.txt
> > +++ b/test/CMakeLists.txt
> > @@ -3,6 +3,8 @@
> >  # See the rationale in the root CMakeLists.txt.
> >  cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
> >  
> > +include(MakeLuaPath)
> > +
> 
> Minor: It would be nice to drop the comment that this will be used in
> all test suites by transitivity.
> Feel free to ignore.

There are some includes made in the root CMakeLists.txt that are used
project-wide. There are no comments nearby. Ignoring for this case too.

> 
> >  find_program(LUACHECK luacheck)
> >  if(LUACHECK)
> >    # XXX: The tweak below relates to luacheck problem with paths.

<snipped>

> > diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
> > index ecda2e63..27866869 100644
> > --- a/test/tarantool-tests/CMakeLists.txt
> > +++ b/test/tarantool-tests/CMakeLists.txt
> > @@ -43,14 +43,11 @@ macro(BuildTestCLib lib sources)
> >    # semicolon. If one finds the normal way to make it work, feel
> >    # free to reach me.
> >    set(TESTLIBS "${lib};${TESTLIBS}" PARENT_SCOPE)
> > -  # Add the directory where the lib is built to the LUA_CPATH
> > -  # environment variable, so LuaJIT can find and load it.
> > -  # XXX: Here we see the other side of the coin. If one joins two
> > -  # strings with semicolon, the value automatically becomes a
> > -  # list. I found a single working solution to make LUA_CPATH be
> > -  # a string via "escaping" the semicolon right in string
> > -  # interpolation.
> > -  set(LUA_CPATH "${CMAKE_CURRENT_BINARY_DIR}/?${CMAKE_SHARED_LIBRARY_SUFFIX}\;${LUA_CPATH}" PARENT_SCOPE)
> > +  # Add the directory where the lib is built to the list with
> > +  # entries for LUA_CPATH environment variable, so LuaJIT can find
> > +  # and load it. See the comment about extending the list in the
> > +  # parent scope few lines above.
> > +  set(LUA_CPATHS "${CMAKE_CURRENT_BINARY_DIR}/?${CMAKE_SHARED_LIBRARY_SUFFIX};${LUA_CPATHS}" PARENT_SCOPE)
> >    # Also add this directory to LD_LIBRARY_PATH environment
> >    # variable, so FFI machinery can find and load it.
> >    set(LD_LIBRARY_PATH "${CMAKE_CURRENT_BINARY_DIR}:${LD_LIBRARY_PATH}" PARENT_SCOPE)
> > @@ -76,14 +73,21 @@ add_subdirectory(misclib-sysprof-capi)
> >  # in src/ directory and auxiliary tests-related modules are
> >  # located in the current directory (but tests are run in the
> >  # binary directory), so LUA_PATH need to be updated.
> > -set(LUA_PATH
> > -  "${CMAKE_CURRENT_SOURCE_DIR}/?.lua\;${PROJECT_SOURCE_DIR}/tools/?.lua\;${PROJECT_SOURCE_DIR}/src/?.lua"
> > +make_lua_path(LUA_PATH
> > +  PATHS
> > +    ${CMAKE_CURRENT_SOURCE_DIR}/?.lua
> > +    ${PROJECT_SOURCE_DIR}/tools/?.lua
> > +    ${PROJECT_SOURCE_DIR}/src/?.lua
> >  )
> > +# Update LUA_CPATH with the library paths collected within
> > +# <BuildTestLib> macro.
> > +make_lua_path(LUA_CPATH PATHS ${LUA_CPATHS})
> 
> Side note: Looks like this part of patching is unnecessary -- we can
> just leave it as is (isntead manipulation with ; later in
> `make_lua_path()`). But it is good for consistency of working with env
> variables, so feel free to ignore.

Yes, I did it for consistency of working with env variables (otherwise
there would be a mess with semicolons everywhere). Ignoring.

> 
> > +
> >  set(LUA_TEST_SUFFIX .test.lua)
> >  set(LUA_TEST_FLAGS --failures --shuffle)
> >  set(LUA_TEST_ENV
> > -  "LUA_PATH=\"${LUA_PATH}\;\;\""
> > -  "LUA_CPATH=\"${LUA_CPATH}\;\;\""
> > +  "LUA_PATH=\"${LUA_PATH}\""
> > +  "LUA_CPATH=\"${LUA_CPATH}\""
> >  )
> >  
> >  if(CMAKE_VERBOSE_MAKEFILE)
> > -- 
> > 2.34.0
> > 
> 
> -- 
> Best regards,
> Sergey Kaplun

-- 
Best regards,
IM