[Tarantool-patches] [PATCH] build: refactor static build process
Alexandr Barulev
huston.mavr at gmail.com
Mon Aug 24 11:44:02 MSK 2020
I've modified this patch: disabled building libunwind with minidebuginfo
to prevent linking libunwind with liblzma.
Here is a diff:
diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
index 53ceb609c..d90a642e6 100644
--- a/static-build/CMakeLists.txt
+++ b/static-build/CMakeLists.txt
@@ -185,6 +185,7 @@ else()
--prefix=<INSTALL_DIR>
--disable-shared
--enable-static
+ --disable-minidebuginfo # to prevent linking with liblzma
STEP_TARGETS download
)
endif()
чт, 6 авг. 2020 г. в 16:32, Alexandr Barulev <huston.mavr at gmail.com>:
> I’ve squashed commit and changed it’s message;
> Also I’ve sended diff at previous answer
>
> https://github.com/tarantool/tarantool/tree/rosik/refactor-static-build
>
> ср, 5 авг. 2020 г. в 20:08, Mavr Huston <huston.mavr at gmail.com>:
>
>> Hi, thanks for the review!
>>
>> libicu installs as ExternalProject_Add too, its missed in commit message;
>>
>> Problem with curses and ncurses was on macOS and linux, because libcurses
>> is an entire copy of libncurses, and tarantool links with system
>> libcurses instead of libncurses installed as tarantool dependency, but
>> module FindCurses.cmkae provides workaround for this problem -
>> CURSES_NEED_NCURSES flag.- to use ncurses instead of curses. (i will fix
>> this part at commit message)
>>
>> About disable-shred flag, used at libcurl building - we want to link only
>> with
>> static libraries, so we prevent creating unused .so.
>>
>> I've renamed static_build_no_deps_* jobs after review to
>> static_build_cmake_*
>>
>> Also about such path tarantool-prefix/* - it's a cmake
>> ExternalProject_Add()
>> default path (i've also added comment at .travis.mk)
>>
>> Useless comments "Init macOS test env" deleted.
>>
>> > if (BUILD_STATIC)
>> > - set(LIBZ_LIB_NAME libz.a)
>> > + find_library(LIBZ_LIBRARY NAMES libz.a)
>> > else()
>> > - set(LIBZ_LIB_NAME z)
>> > + find_library(LIBZ_LIBRARY NAMES z)
>> > endif()
>> > - find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME})
>> Here we simplified code, by deleting useless variable.
>>
>> I've added commentaries to cmake/compiler.cmake about libunwind on macOS
>> and about ignoring flag -static-libstdc++ on macOS
>>
>> I've fixed static-build for using system compiler: gcc/g++ on linux
>> and clang/clang++ on macOS
>>
>> I've refactored IF (NOT APPLE) condition to IF (APPLE) at
>> static-build/CMakeLists.txt
>>
>> I've mentioned macOS dependencies at static-build/README.md xcode-tools
>> and
>> others, also I've added example with CMAKE_TARANTOOL_ARGS.
>>
>> Added commentaries about _EP_INSTALL_DIR at static-build/CMakeLists.txt
>>
>> Also deleted unused use_unix_sockets_iproto = True
>>
>> Also deleted curl-features.test.lua, because after rebase on master it
>> fails,
>> due to missing curl_version_info symbol at tarantool binary. This symbol
>> lost
>> after #807c7fa584f21ee955b2a14623d70f7510a3650d (build: update curl
>> submodule
>> to 7.71.1 version )
>>
>>
>> After pass the review I'll squash this changes to base commit and update
>> commit
>> message.
>>
>> Here is a diff of changes:
>> =====================================
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 5ec5dd9b9..c9aef3dc7 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -534,14 +534,14 @@ static_build:
>> script:
>> - ${GITLAB_MAKE} test_static_build
>>
>> -static_build_no_deps_linux:
>> +static_build_cmake_linux:
>> <<: *docker_test_definition
>> script:
>> - - ${GITLAB_MAKE} test_static_build_no_deps_linux
>> + - ${GITLAB_MAKE} test_static_build_cmake_linux
>>
>> -static_build_no_deps_osx_15:
>> +static_build_cmake_osx_15:
>> stage: test
>> tags:
>> - osx_15
>> script:
>> - - ${GITLAB_MAKE} test_static_build_no_deps_osx
>> + - ${GITLAB_MAKE} test_static_build_cmake_osx
>> diff --git a/.travis.mk b/.travis.mk
>> index 64862348f..482672429 100644
>> --- a/.travis.mk
>> +++ b/.travis.mk
>> @@ -149,8 +149,8 @@ test_static_build: deps_debian_static
>> CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk
>> test_debian_no_deps
>>
>> # New static build
>> -
>> -test_static_build_no_deps_linux:
>> +# builddir used in this target - is a default build path from cmake
>> ExternalProject_Add()
>> +test_static_build_cmake_linux:
>> cd static-build && cmake . && make -j && ctest -V
>> cd test && /usr/bin/python test-run.py --force \
>> --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build
>> $(TEST_RUN_EXTRA_PARAMS)
>> @@ -218,7 +218,6 @@ INIT_TEST_ENV_OSX=\
>> rm -rf /tmp/tnt
>>
>> test_osx_no_deps: build_osx
>> - # Init macOS test env
>> ${INIT_TEST_ENV_OSX}; \
>> cd test && ./test-run.py --vardir /tmp/tnt --force
>> $(TEST_RUN_EXTRA_PARAMS)
>>
>> @@ -233,9 +232,9 @@ base_deps_osx:
>> brew install --force ${STATIC_OSX_PKGS} || brew upgrade
>> ${STATIC_OSX_PKGS}
>> pip install --force-reinstall -r test-run/requirements.txt
>>
>> -test_static_build_no_deps_osx: base_deps_osx
>> +# builddir used in this target - is a default build path from cmake
>> ExternalProject_Add()
>> +test_static_build_cmake_osx: base_deps_osx
>> cd static-build && cmake . && make -j && ctest -V
>> - # Init macOS test env
>> ${INIT_TEST_ENV_OSX}; \
>> cd test && ./test-run.py --vardir /tmp/tnt \
>> --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \
>> diff --git a/cmake/FindReadline.cmake b/cmake/FindReadline.cmake
>> index 8422181d6..afe480679 100644
>> --- a/cmake/FindReadline.cmake
>> +++ b/cmake/FindReadline.cmake
>> @@ -14,6 +14,14 @@ if(BUILD_STATIC)
>> if (NOT CURSES_INFO_LIBRARY)
>> set(CURSES_INFO_LIBRARY "")
>> endif()
>> +
>> + # From Modules/FindCurses.cmake:
>> + # Set ``CURSES_NEED_NCURSES`` to ``TRUE`` before the
>> + # ``find_package(Curses)`` call if NCurses functionality is required.
>> + # This flag is set for linking with required library (installed
>> + # via static-build/CMakeLists.txt). If this variable won't be set
>> + # then tarantool binary links with system library curses which is an
>> + # entire copy of ncurses
>> set(CURSES_NEED_NCURSES TRUE)
>> endif()
>>
>> diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake
>> index 14f1e1186..db2ae6227 100644
>> --- a/cmake/compiler.cmake
>> +++ b/cmake/compiler.cmake
>> @@ -131,6 +131,8 @@ set(CMAKE_REQUIRED_INCLUDES "")
>> if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
>> set(UNWIND_LIB_NAME libunwind.a)
>> else()
>> + # libunwind can't be compiled on macOS.
>> + # But there exists libunwind.dylib as a part of MacOSSDK
>> set(UNWIND_LIB_NAME unwind)
>> endif()
>> find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES
>> ${UNWIND_LIB_NAME})
>> @@ -192,6 +194,9 @@ if (ENABLE_BACKTRACE)
>> find_package_message(UNWIND_LIBRARIES "Found unwind"
>> "${UNWIND_LIBRARIES}")
>> endif()
>>
>> +# On macOS there is no '-static-libstdc++' flag and it's use will
>> +# raise following error:
>> +# error: argument unused during compilation: '-static-libstdc++'
>> if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
>> # Static linking for c++ routines
>> add_compile_flags("C;CXX" "-static-libstdc++")
>> diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
>> index 86582af0a..53ceb609c 100644
>> --- a/static-build/CMakeLists.txt
>> +++ b/static-build/CMakeLists.txt
>> @@ -9,11 +9,18 @@ set(NCURSES_VERSION 6.2)
>> set(READLINE_VERSION 8.0)
>> set(UNWIND_VERSION 1.3-rc1)
>>
>> -find_program(C_COMPILER gcc)
>> -find_program(CXX_COMPILER g++)
>> +if (APPLE)
>> + find_program(C_COMPILER clang)
>> + find_program(CXX_COMPILER clang++)
>> +else()
>> + find_program(C_COMPILER gcc)
>> + find_program(CXX_COMPILER g++)
>> +endif()
>> set(CMAKE_C_COMPILER ${C_COMPILER})
>> set(CMAKE_CXX_COMPILER ${CXX_COMPILER})
>>
>> +# Install all libraries required by tarantool at current build dir
>> +
>> #
>> # OpenSSL
>> #
>> @@ -80,7 +87,18 @@ ExternalProject_Add(readline
>> #
>> # ICONV
>> #
>> -if (NOT APPLE)
>> +if (APPLE)
>> + ExternalProject_Add(iconv
>> + URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
>> + CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
>> + <SOURCE_DIR>/configure
>> + --prefix=<INSTALL_DIR>
>> + --disable-shared
>> + --enable-static
>> + --with-gnu-ld
>> + STEP_TARGETS download
>> + )
>> +else()
>> # In linux iconv is embedded into glibc
>> # So we find system header and copy it locally
>> find_path(ICONV_INCLUDE_DIR iconv.h)
>> @@ -101,20 +119,11 @@ if (NOT APPLE)
>> add_custom_target(iconv
>> DEPENDS
>> "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h"
>> )
>> + # This is a hack for further getting install directory of library
>> + # by ExternalProject_Get_Property
>> set_target_properties(iconv
>> PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX}
>> )
>> -else()
>> - ExternalProject_Add(iconv
>> - URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
>> - CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
>> - <SOURCE_DIR>/configure
>> - --prefix=<INSTALL_DIR>
>> - --disable-shared
>> - --enable-static
>> - --with-gnu-ld
>> - STEP_TARGETS download
>> - )
>> endif()
>>
>> #
>> @@ -162,6 +171,8 @@ if (APPLE)
>> endif()
>>
>> add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES})
>> + # This is a hack for further getting install directory of library
>> + # by ExternalProject_Get_Property
>> set_target_properties(unwind
>> PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX}
>> )
>> @@ -178,6 +189,8 @@ else()
>> )
>> endif()
>>
>> +# Get install directories of builded libraries for building
>> +# tarantool with custon CMAKE_PREFIX_PATH
>> foreach(PROJ openssl icu zlib ncurses readline iconv unwind)
>> ExternalProject_Get_Property(${PROJ} install_dir)
>> set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir})
>> @@ -197,16 +210,14 @@ ExternalProject_Add(tarantool
>> -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}
>> -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE
>> -DOPENSSL_USE_STATIC_LIBS=TRUE
>> - -DCMAKE_BUILD_TYPE=Debug
>> -DBUILD_STATIC=TRUE
>> -DENABLE_DIST=TRUE
>> -DENABLE_BACKTRACE=TRUE
>> - -DPACKAGE:STRING=${PACKAGE_NAME}
>> -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
>> -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
>> ${CMAKE_TARANTOOL_ARGS}
>> - BUILD_COMMAND ${CMAKE_MAKE_PROGRAM} -j
>> STEP_TARGETS build
>> + BUILD_COMMAND $(MAKE)
>> )
>>
>> enable_testing()
>> diff --git a/static-build/README.md b/static-build/README.md
>> index 29fe085c3..0019e963f 100644
>> --- a/static-build/README.md
>> +++ b/static-build/README.md
>> @@ -13,6 +13,24 @@ yum install -y \
>> python-msgpack python-yaml python-argparse python-six python-gevent
>> ```
>>
>> +MacOS:
>> +
>> +Before you start please install default Xcode Tools by Apple:
>> +
>> +```
>> +sudo xcode-select --install
>> +sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
>> +```
>> +
>> +Install brew using command from
>> +[Homebrew repository instructions](https://github.com/Homebrew/inst)
>> +
>> +After that run next script:
>> +
>> +```bash
>> + brew install autoconf automake libtool cmake
>> file://$${PWD}/tools/brew_taps/tntpython2.rbs
>> + pip install --force-reinstall -r test-run/requirements.txt
>> +```
>>
>> ### Usage
>>
>> @@ -21,3 +39,20 @@ cmake .
>> make -j
>> ctest -V
>> ```
>> +
>> +## Customize your build
>> +
>> +If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS`
>> variable
>> +
>> +### Usage
>> +
>> +There is three types of `CMAKE_BUILD_TYPE`:
>> +* Debug - default
>> +* Release
>> +* RelWithDebInfo
>> +
>> +And you want to build tarantool with RelWithDebInfo:
>> +
>> +```bash
>> +cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" .
>> +```
>> diff --git a/static-build/test/static-build/curl-features.test.lua
>> b/static-build/test/static-build/curl-features.test.lua
>> deleted file mode 100755
>> index 57b1c4306..000000000
>> --- a/static-build/test/static-build/curl-features.test.lua
>> +++ /dev/null
>> @@ -1,67 +0,0 @@
>> -#!/usr/bin/env tarantool
>> -
>> -local tap = require('tap')
>> -local ffi = require('ffi')
>> -ffi.cdef([[
>> - struct curl_version_info_data {
>> - int age; /* see description below */
>> - const char *version; /* human readable string */
>> - unsigned int version_num; /* numeric representation */
>> - const char *host; /* human readable string */
>> - int features; /* bitmask, see below */
>> - char *ssl_version; /* human readable string */
>> - long ssl_version_num; /* not used, always zero */
>> - const char *libz_version; /* human readable string */
>> - const char * const *protocols; /* protocols */
>> -
>> - /* when 'age' is CURLVERSION_SECOND or higher, the members below
>> exist */
>> - const char *ares; /* human readable string */
>> - int ares_num; /* number */
>> -
>> - /* when 'age' is CURLVERSION_THIRD or higher, the members below
>> exist */
>> - const char *libidn; /* human readable string */
>> -
>> - /* when 'age' is CURLVERSION_FOURTH or higher (>= 7.16.1), the
>> members
>> - below exist */
>> - int iconv_ver_num; /* '_libiconv_version' if iconv support
>> enabled */
>> -
>> - const char *libssh_version; /* human readable string */
>> -
>> - /* when 'age' is CURLVERSION_FIFTH or higher (>= 7.57.0), the
>> members
>> - below exist */
>> - unsigned int brotli_ver_num; /* Numeric Brotli version
>> - (MAJOR << 24) | (MINOR << 12) |
>> PATCH */
>> - const char *brotli_version; /* human readable string. */
>> -
>> - /* when 'age' is CURLVERSION_SIXTH or higher (>= 7.66.0), the
>> members
>> - below exist */
>> - unsigned int nghttp2_ver_num; /* Numeric nghttp2 version
>> - (MAJOR << 16) | (MINOR << 8) |
>> PATCH */
>> - const char *nghttp2_version; /* human readable string. */
>> -
>> - const char *quic_version; /* human readable quic (+ HTTP/3)
>> library +
>> - version or NULL */
>> -
>> - /* when 'age' is CURLVERSION_SEVENTH or higher (>= 7.70.0), the
>> members
>> - below exist */
>> - const char *cainfo; /* the built-in default
>> CURLOPT_CAINFO, might
>> - be NULL */
>> - const char *capath; /* the built-in default
>> CURLOPT_CAPATH, might
>> - be NULL */
>> - };
>> -
>> - struct curl_version_info_data *curl_version_info(int age);
>> -]])
>> -
>> -local info = ffi.C.curl_version_info(7)
>> -local test = tap.test('curl-features')
>> -test:plan(2)
>> -
>> -if test:ok(info.ssl_version ~= nil, 'Curl built with SSL support') then
>> - test:diag('ssl_version: ' .. ffi.string(info.ssl_version))
>> -end
>> -if test:ok(info.libz_version ~= nil, 'Curl built with LIBZ') then
>> - test:diag('libz_version: ' .. ffi.string(info.libz_version))
>> -end
>> -
>> -os.exit(test:check() and 0 or 1)
>> diff --git a/static-build/test/static-build/suite.ini
>> b/static-build/test/static-build/suite.ini
>> index 4da3d5d2f..92e349466 100644
>> --- a/static-build/test/static-build/suite.ini
>> +++ b/static-build/test/static-build/suite.ini
>> @@ -3,4 +3,3 @@ core = app
>> description = Static build tests
>> script = box.lua
>> is_parallel = True
>> -use_unix_sockets_iproto = True
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.tarantool.org/pipermail/tarantool-patches/attachments/20200824/10fc6118/attachment.html>
More information about the Tarantool-patches
mailing list