Hello! Here is a new commit message(as discussed at telegram chat): Author: HustonMmmavr Date: Thu Sep 10 12:43:20 2020 +0300 build: refactor static build process Refactored static build process to use static-build/CMakeLists.txt instead of Dockerfile.staticbuild (this allows to support static build on macOS). Following third-party dependencies for static build are installed via cmake `ExternalProject_Add`: - OpenSSL - Zlib - Ncurses - Readline - Unwind - ICU * Added support static build for macOS * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static build at file cmake/BuildLibCURL.cmake: - disable building shared libcurl libraries (by setting `--disable-shared` option) - disable hiding libcurl symbols (by setting `--disable-symbol-hiding` option) - prevent linking libcurl with system libz (by setting `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option) * Removed Dockerfile.staticbuild * Added new gitlab.ci jobs to test new style static build: - static_build_cmake_linux - static_build_cmake_osx_15 * Removed static_docker_build gitlab.ci job Closes #5095 Co-authored-by: Yaroslav Dynnikov ср, 9 сент. 2020 г. в 18:45, HustonMmmavr : > From: Yaroslav Dynnikov > > Refactored static build process to use static-build/CMakeLists.txt > instead of Dockerfile.staticbuild (this allows to support static > build on macOS). Following third-party dependencies for static build > are installed via cmake `ExternalProject_Add`: > - OpenSSL > - Zlib > - Ncurses > - Readline > - Unwind > - ICU > > * Added support static build for macOS > * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static > build at file cmake/BuildLibCURL.cmake: > - disable building shared libcurl libraries (by setting > `--disable-shared` option) > - disable hiding libcurl symbols (by setting > `--disable-symbol-hiding` option) > - prevent linking libcurl with system libz (by setting > `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option) > * Removed Dockerfile.staticbuild > * Added new gitlab.ci jobs to test new style static build: > - static_build_cmake_linux > - static_build_cmake_osx_15 > * Removed static_docker_build gitlab.ci job > > Closes #5095 > --- > > Branch: > https://github.com/tarantool/tarantool/tree/rosik/refactor-static-build > Issue: https://github.com/tarantool/tarantool/issues/5095 > > Changes in v3: > 1. Fixed typos at commit message > 2. Fixed commentaries at .travis.mk > 3. Fixed finding libz library at cmake/BuildLibCurl.cmake as at master > branch > 4. Instead of using static-build/.gitignore use the root .gitignore file > 5. Refactored static-build/CMakeLists.txt: > * Project name is set to tarantool-static > * Instead of explicitly finding of c/c++ compilers current version uses > cmake `project(tarantool-static C CXX)` command. This change requires > to manually set CFLAGS/CPPFLAGS/LDFLAGS for all static-build > dependencies (because building dependencies at macOS requires path > to macOS SDK) the idea was taken from cmake/BuildLibCurl.cmake > * Added commentaries about problem with libicu > * Removed unused TEST_COMMAND at ExternalProject_Add(zlib ...) also > removed unnecessary # STEP_TARGETS at ExternalProject_Add(readline > ...) > * Deleted doubled whitespaces > 6. Fixed static-build/README.md: > * Added list of required tools > * Added example for ubuntu/debian > * Fixed indentation and typos > 7. Refactored tests: > * Got rid of test-run > * Deleted doubled whitespaces at exports.test.lua > * Fixed defining of RTLD_DEFAULT > > > .gitignore | 8 + > .gitlab-ci.yml | 11 +- > .travis.mk | 56 +++- > Dockerfile.staticbuild | 98 ------ > cmake/BuildLibCURL.cmake | 13 +- > cmake/compiler.cmake | 24 +- > cmake/os.cmake | 5 +- > static-build/CMakeLists.txt | 311 ++++++++++++++++++ > static-build/README.md | 90 +++++ > static-build/test/CheckDependencies.cmake | 43 +++ > .../test/static-build/exports.test.lua | 142 ++++++++ > .../test/static-build/traceback.test.lua | 15 + > 12 files changed, 692 insertions(+), 124 deletions(-) > delete mode 100644 Dockerfile.staticbuild > create mode 100644 static-build/CMakeLists.txt > create mode 100644 static-build/README.md > create mode 100644 static-build/test/CheckDependencies.cmake > create mode 100755 static-build/test/static-build/exports.test.lua > create mode 100755 static-build/test/static-build/traceback.test.lua > > diff --git a/.gitignore b/.gitignore > index a42c7db35..c6e261e18 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -151,3 +151,11 @@ src/box/sql/parse.h > src/box/sql/parse.c > src/box/sql/opcodes.h > src/box/sql/opcodes.c > + > +# static-build > +static-build/*-prefix > +static-build/Makefile > +static-build/build > + > +# macOS files > +.DS_Store > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 0ead08711..c9aef3dc7 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -534,9 +534,14 @@ static_build: > script: > - ${GITLAB_MAKE} test_static_build > > -static_docker_build: > +static_build_cmake_linux: > + <<: *docker_test_definition > + script: > + - ${GITLAB_MAKE} test_static_build_cmake_linux > + > +static_build_cmake_osx_15: > stage: test > tags: > - - deploy_test > + - osx_15 > script: > - - ${GITLAB_MAKE} test_static_docker_build > + - ${GITLAB_MAKE} test_static_build_cmake_osx > diff --git a/.travis.mk b/.travis.mk > index efc05cf05..58d0c1596 100644 > --- a/.travis.mk > +++ b/.travis.mk > @@ -148,8 +148,14 @@ deps_debian_static: > test_static_build: deps_debian_static > CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk > test_debian_no_deps > > -test_static_docker_build: > - docker build --no-cache --network=host --build-arg RUN_TESTS=ON -f > Dockerfile.staticbuild . > +# New static build > +# builddir used in this target - is a default build path from cmake > +# ExternalProject_Add() > +test_static_build_cmake_linux: > + cd static-build && cmake > -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" > . && \ > + make -j && ctest -V > + cd test && /usr/bin/python test-run.py --force \ > + --builddir > ${PWD}/static-build/tarantool-prefix/src/tarantool-build > $(TEST_RUN_EXTRA_PARAMS) > > # ################### > # Static Analysis > @@ -193,15 +199,16 @@ build_osx: > cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON > ${CMAKE_EXTRA_PARAMS} > make -j > > -test_osx_no_deps: build_osx > - # Limits: Increase the maximum number of open file descriptors on > macOS: > - # Travis-ci needs the "ulimit -n " call > - # Gitlab-ci needs the "launchctl limit maxfiles " call > - # Also gitlib-ci needs the password to change the limits, while > - # travis-ci runs under root user. Limit setup must be in the same > - # call as tests runs call. > - # Tests: Temporary excluded replication/ suite with some tests > - # from other suites by issues #4357 and #4370 > + > +# Limits: Increase the maximum number of open file descriptors on macOS: > +# Travis-ci needs the "ulimit -n " call > +# Gitlab-ci needs the "launchctl limit maxfiles " call > +# Also gitlib-ci needs the password to change the limits, while > +# travis-ci runs under root user. Limit setup must be in the same > +# call as tests runs call. > +# Tests: Temporary excluded replication/ suite with some tests > +# from other suites by issues #4357 and #4370 > +INIT_TEST_ENV_OSX=\ > sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \ > launchctl limit maxfiles || : ; \ > ulimit -n ${MAX_FILES} || : ; \ > @@ -210,11 +217,34 @@ test_osx_no_deps: build_osx > launchctl limit maxproc || : ; \ > ulimit -u ${MAX_PROC} || : ; \ > ulimit -u ; \ > - rm -rf /tmp/tnt ; \ > - cd test && ./test-run.py --vardir /tmp/tnt --force > $(TEST_RUN_EXTRA_PARAMS) > + rm -rf /tmp/tnt > + > +test_osx_no_deps: build_osx > + ${INIT_TEST_ENV_OSX}; \ > + cd test && ./test-run.py --vardir /tmp/tnt --force > $(TEST_RUN_EXTRA_PARAMS) > > test_osx: deps_osx test_osx_no_deps > > +# Static macOS build > + > +STATIC_OSX_PKGS=autoconf automake libtool cmake > file://$${PWD}/tools/brew_taps/tntpython2.rb > +base_deps_osx: > + brew update || echo | /usr/bin/ruby -e \ > + "$$(curl -fsSL > https://raw.githubusercontent.com/Homebrew/install/master/install)" > + brew install --force ${STATIC_OSX_PKGS} || brew upgrade > ${STATIC_OSX_PKGS} > + pip install --force-reinstall -r test-run/requirements.txt > + > +# builddir used in this target - is a default build path from cmake > +# ExternalProject_Add() > +test_static_build_cmake_osx: base_deps_osx > + cd static-build && cmake > -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" > . && \ > + make -j && ctest -V > + ${INIT_TEST_ENV_OSX}; \ > + cd test && ./test-run.py --vardir /tmp/tnt \ > + --builddir > ${PWD}/static-build/tarantool-prefix/src/tarantool-build \ > + --force $(TEST_RUN_EXTRA_PARAMS) > + > + > ########### > # FreeBSD # > ########### > diff --git a/Dockerfile.staticbuild b/Dockerfile.staticbuild > deleted file mode 100644 > index f67f46f5e..000000000 > --- a/Dockerfile.staticbuild > +++ /dev/null > @@ -1,98 +0,0 @@ > -FROM centos:7 > - > -RUN yum install -y epel-release > -RUN yum install -y yum install > https://centos7.iuscommunity.org/ius-release.rpm > - > -RUN set -x \ > - && yum -y install \ > - libstdc++ \ > - libstdc++-static \ > - readline \ > - openssl \ > - lz4 \ > - binutils \ > - ncurses \ > - libgomp \ > - lua \ > - curl \ > - tar \ > - zip \ > - unzip \ > - libunwind \ > - zlib \ > - && yum -y install \ > - perl \ > - gcc-c++ \ > - cmake \ > - lz4-devel \ > - binutils-devel \ > - lua-devel \ > - make \ > - git \ > - autoconf \ > - automake \ > - libtool \ > - wget > - > -RUN yum -y install ncurses-static readline-static zlib-static pcre-static > glibc-static > - > -RUN yum -y install python-devel python-pip > - > -RUN set -x && \ > - cd / && \ > - curl -O -L https://www.openssl.org/source/openssl-1.1.1f.tar.gz && \ > - tar -xvf openssl-1.1.1f.tar.gz && \ > - cd openssl-1.1.1f && \ > - ./config --libdir=lib && \ > - make -j && make install > - > -RUN set -x && \ > - cd / && \ > - curl -O -L > https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz > && \ > - tar -xvf icu4c-62_1-src.tgz && \ > - cd icu/source && \ > - ./configure --with-data-packaging=static --enable-static > --enable-shared && \ > - make -j && make install > - > -RUN set -x && \ > - cd / && \ > - curl -O -L > http://download.savannah.nongnu.org/releases/libunwind/libunwind-1.3-rc1.tar.gz > && \ > - tar -xvf libunwind-1.3-rc1.tar.gz && \ > - cd libunwind-1.3-rc1 && \ > - ./configure --enable-static --enable-shared && \ > - make -j && make install > - > -COPY . /tarantool > - > -WORKDIR /tarantool > - > -RUN set -x && \ > - git submodule init && \ > - git submodule update > - > -# Cleanup for 'build' directory added, because it purges all artefacts > -# produced for curl build, including the old configuration in build/curl > -RUN set -x && \ > - find . -name 'CMakeFiles' -type d -exec rm -rf {} + && \ > - find . -name 'CMakeCache.txt' -type f -delete && \ > - rm -rf build test/small test/luajit-tap > - > -RUN pip install -r /tarantool/test-run/requirements.txt > - > -RUN set -x && \ > - cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \ > - -DENABLE_DIST:BOOL=ON \ > - -DBUILD_STATIC=ON \ > - -DOPENSSL_USE_STATIC_LIBS=ON \ > - -DOPENSSL_ROOT_DIR=/usr/local \ > - . && \ > - make -j && make install > - > -ARG RUN_TESTS > -RUN if [ -n "${RUN_TESTS}" ]; then \ > - set -x && \ > - cd test && \ > - /usr/bin/python test-run.py --force; \ > - fi > - > -ENTRYPOINT ["/bin/bash"] > diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake > index 5f8b15a63..86fec39e9 100644 > --- a/cmake/BuildLibCURL.cmake > +++ b/cmake/BuildLibCURL.cmake > @@ -4,15 +4,21 @@ macro(curl_build) > set(LIBCURL_BINARY_DIR ${PROJECT_BINARY_DIR}/build/curl/work) > set(LIBCURL_INSTALL_DIR ${PROJECT_BINARY_DIR}/build/curl/dest) > > + message(STATUS "Looking for zlib") > + find_path(ZLIB_INCLUDE_DIR zlib.h) > + message(STATUS "Looking for zlib.h - ${ZLIB_INCLUDE_DIR}") > if (BUILD_STATIC) > set(LIBZ_LIB_NAME libz.a) > else() > set(LIBZ_LIB_NAME z) > endif() > find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME}) > - if ("${LIBZ_LIBRARY}" STREQUAL "LIBZ_LIBRARY-NOTFOUND") > + message(STATUS "Looking for libz - ${LIBZ_LIBRARY}") > + > + if (NOT ZLIB_INCLUDE_DIR OR NOT LIBZ_LIBRARY) > message(FATAL_ERROR "Unable to find zlib") > endif() > + get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} > DIRECTORY) > > # Use the same OpenSSL library for libcurl as is used for > # tarantool itself. > @@ -88,9 +94,10 @@ macro(curl_build) > > --prefix > --enable-static > - --enable-shared > + --disable-shared > + --disable-symbol-hiding > > - --with-zlib > + --with-zlib=${FOUND_ZLIB_ROOT_DIR} > ${LIBCURL_OPENSSL_OPT} > --with-ca-fallback > > diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake > index 5a1141ebd..db2ae6227 100644 > --- a/cmake/compiler.cmake > +++ b/cmake/compiler.cmake > @@ -33,10 +33,6 @@ if (CMAKE_COMPILER_IS_GNUCC) > Your GCC version is ${CMAKE_CXX_COMPILER_VERSION}, please > update > ") > endif() > -else() > - if (BUILD_STATIC) > - message(FATAL_ERROR "Static build is supported for GCC only") > - endif() > endif() > > # > @@ -120,10 +116,23 @@ set (CMAKE_CXX_FLAGS_RELWITHDEBINFO > > unset(CC_DEBUG_OPT) > > +message(STATUS "Looking for libunwind.h") > +find_path(UNWIND_INCLUDE_DIR libunwind.h) > +message(STATUS "Looking for libunwind.h - ${UNWIND_INCLUDE_DIR}") > + > +if (UNWIND_INCLUDE_DIR) > + include_directories(${UNWIND_INCLUDE_DIR}) > +endif() > + > +set(CMAKE_REQUIRED_INCLUDES ${UNWIND_INCLUDE_DIR}) > check_include_file(libunwind.h HAVE_LIBUNWIND_H) > -if(BUILD_STATIC) > +set(CMAKE_REQUIRED_INCLUDES "") > + > +if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) > set(UNWIND_LIB_NAME libunwind.a) > else() > + # libunwind can't be compiled on macOS. > + # But there exists libunwind.dylib as a part of MacOSSDK > set(UNWIND_LIB_NAME unwind) > endif() > find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME}) > @@ -185,7 +194,10 @@ if (ENABLE_BACKTRACE) > find_package_message(UNWIND_LIBRARIES "Found unwind" > "${UNWIND_LIBRARIES}") > endif() > > -if(BUILD_STATIC) > +# On macOS there is no '-static-libstdc++' flag and it's use will > +# raise following error: > +# error: argument unused during compilation: '-static-libstdc++' > +if(BUILD_STATIC AND NOT TARGET_OS_DARWIN) > # Static linking for c++ routines > add_compile_flags("C;CXX" "-static-libstdc++") > endif() > diff --git a/cmake/os.cmake b/cmake/os.cmake > index 905be61df..276a79b42 100644 > --- a/cmake/os.cmake > +++ b/cmake/os.cmake > @@ -107,7 +107,10 @@ elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Darwin") > > # Latest versions of Homebrew wont 'link --force' for libraries, that > were > # preinstalled in system. So we'll use this dirty hack > - find_program(HOMEBREW_EXECUTABLE brew) > + > + if (NOT BUILD_STATIC) > + find_program(HOMEBREW_EXECUTABLE brew) > + endif() > if(EXISTS ${HOMEBREW_EXECUTABLE}) > execute_process(COMMAND ${HOMEBREW_EXECUTABLE} --prefix > OUTPUT_VARIABLE HOMEBREW_PREFIX > diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt > new file mode 100644 > index 000000000..90029fdd8 > --- /dev/null > +++ b/static-build/CMakeLists.txt > @@ -0,0 +1,311 @@ > +cmake_minimum_required(VERSION 2.8) > + > +# Detect system compilers for further configuring dependencies to be > +# builded with these compilers. This is used to build tarantool and > +# it's dependencies by usign one compiler system (for example libicu > +# by default uses clang if it exists when others uses gcc/g++ on > +# linux machine). > +project(tarantool-static C CXX) > + > +include(ExternalProject) > +set(OPENSSL_VERSION 1.1.1f) > +set(ZLIB_VERSION 1.2.11) > +set(NCURSES_VERSION 6.2) > +set(READLINE_VERSION 8.0) > +set(UNWIND_VERSION 1.3-rc1) > + > +# Pass -isysroot= option on Mac OS to a preprocessor and a C > +# compiler to find header files installed with an SDK. > +# > +# The idea is to set these (DEPENDENCY_*) variables to corresponding > +# environment variables at each depenency configure script. > +# > +# Note: Passing of CPPFLAGS / CFLAGS explicitly discards using of > +# corresponsing environment variables. So pass empty LDFLAGS to discard > +# using of corresponding environment variable. It is possible that a > +# linker flag assumes that some compilation flag is set. We don't pass > +# CFLAGS from environment, so we should not do it for LDFLAGS too. > +set(DEPENDENCY_CFLAGS "") > +set(DEPENDENCY_CPPFLAGS "") > +set(DEPENDENCY_LDFLAGS) > +if (APPLE) > + set(DEPENDENCY_CFLAGS "${CMAKE_C_SYSROOT_FLAG} > ${CMAKE_OSX_SYSROOT}") > + set(DEPENDENCY_CPPFLAGS "${CMAKE_C_SYSROOT_FLAG} > ${CMAKE_OSX_SYSROOT}") > +endif() > + > +# Install all libraries required by tarantool at current build dir > + > +# > +# OpenSSL > +# > +ExternalProject_Add(openssl > + URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz > + CONFIGURE_COMMAND /config > + CC=${CMAKE_C_COMPILER} > + CXX=${CMAKE_CXX_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + > + --prefix= > + --libdir=lib > + no-shared > + INSTALL_COMMAND ${CMAKE_MAKE_PROGRAM} install_sw > +) > + > +# > +# ICU > +# > +ExternalProject_Add(icu > + URL > https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz > + # By default libicu is built by using clang/clang++ compiler (if it > + # exists). Here is a link for detecting compilers at libicu configure > + # script: > https://github.com/unicode-org/icu/blob/7c7b8bd5702310b972f888299169bc3cc88bf0a6/icu4c/source/configure.ac#L135 > + # This will cause the problem on linux machine: tarantool is built > + # with gcc/g++ and libicu is built with clang/clang++ (if it exists) > + # so at linking stage `rellocation` errors will occur. To solve this, > + # we can set CC/CXX to CMAKE_C_COMPILER/CMAKE_CXX_COMPILER variables > + # manually which are detected above (by cmake `project()` command) > + CONFIGURE_COMMAND /source/configure > + CC=${CMAKE_C_COMPILER} > + CXX=${CMAKE_CXX_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + > + --with-data-packaging=static > + --prefix= > + --disable-shared > + --enable-static > +) > + > +# > +# ZLIB > +# > +ExternalProject_Add(zlib > + URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz > + CONFIGURE_COMMAND env > + CC=${CMAKE_C_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + /configure > + --prefix= > + --static > +) > + > +# > +# Ncurses > +# > +ExternalProject_Add(ncurses > + URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz > + CONFIGURE_COMMAND /configure > + CC=${CMAKE_C_COMPILER} > + CXX=${CMAKE_CXX_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + > + --prefix= > + > + # This flag enables creation of libcurses.a as a symlink to > libncurses.a > + # and disables subdir creation `ncurses` at > /include. It is > + # necessary for correct work of FindCurses.cmake module (this > module is > + # builtin at cmake package) which used in cmake/FindReadline.cmake > + --enable-overwrite > + > + # enable building libtinfo to prevent linking with libtinfo from > system > + # directories > + --with-termlib > + > + # set search paths for terminfo db > + > --with-terminfo-dirs=/lib/terminfo:/usr/share/terminfo:/etc/terminfo > + > + # disable install created terminfo db, use db from system > + --disable-db-install > + --without-progs > + --without-manpages > +) > + > +# > +# ReadLine > +# > +ExternalProject_Add(readline > + URL > https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz > + CONFIGURE_COMMAND /configure > + CC=${CMAKE_C_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + > + --prefix= > + --disable-shared > +) > + > +# > +# ICONV > +# > +if (APPLE) > + ExternalProject_Add(iconv > + URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz > + CONFIGURE_COMMAND /configure > + CC=${CMAKE_C_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + > + --prefix= > + --disable-shared > + --enable-static > + --with-gnu-ld > + STEP_TARGETS download > + ) > +else() > + # In linux iconv is embedded into glibc > + # So we find system header and copy it locally > + find_path(ICONV_INCLUDE_DIR iconv.h) > + if(NOT ICONV_INCLUDE_DIR) > + message(FATAL_ERROR "iconv include header not found") > + endif() > + > + set(ICONV_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix") > + > + add_custom_command( > + OUTPUT "${ICONV_INSTALL_PREFIX}/include/iconv.h" > + COMMAND ${CMAKE_COMMAND} -E make_directory > + "${ICONV_INSTALL_PREFIX}/include" > + COMMAND ${CMAKE_COMMAND} -E copy > + "${ICONV_INCLUDE_DIR}/iconv.h" > + "${ICONV_INSTALL_PREFIX}/include/iconv.h" > + ) > + add_custom_target(iconv > + DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h" > + ) > + # This is a hack for further getting install directory of library > + # by ExternalProject_Get_Property > + set_target_properties(iconv > + PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX} > + ) > +endif() > + > +# > +# Unwind > +# > +if (APPLE) > + # On macOS libunwind is a part of MacOSX.sdk > + # So we need to find library and header and > + # copy it locally > + find_path(UNWIND_INCLUDE_DIR libunwind.h) > + find_library(UNWIND_LIBRARY libunwind.dylib > + PATH_SUFFIXES system > + ) > + > + set(UNWIND_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/unwind-prefix") > + > + set(UNWIND_DEPENDENCIES) > + > + if (UNWIND_INCLUDE_DIR AND UNWIND_LIBRARY) > + add_custom_command( > + OUTPUT "${UNWIND_INSTALL_PREFIX}/include/unwind.h" > + OUTPUT "${UNWIND_INSTALL_PREFIX}/include/libunwind.h" > + COMMAND ${CMAKE_COMMAND} -E make_directory > + "${UNWIND_INSTALL_PREFIX}/include" > + COMMAND ${CMAKE_COMMAND} -E copy > + "${UNWIND_INCLUDE_DIR}/libunwind.h" > + "${UNWIND_INCLUDE_DIR}/unwind.h" > + "${UNWIND_INSTALL_PREFIX}/include/" > + ) > + add_custom_command( > + OUTPUT "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib" > + COMMAND ${CMAKE_COMMAND} -E make_directory > + "${UNWIND_INSTALL_PREFIX}/lib" > + COMMAND ${CMAKE_COMMAND} -E copy > + "${UNWIND_LIBRARY}" > + "${UNWIND_INSTALL_PREFIX}/lib/" > + ) > + set(UNWIND_DEPENDENCIES > + ${UNWIND_DEPENDENCIES} > + "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib" > + "${UNWIND_INSTALL_PREFIX}/include/libunwind.h" > + ) > + else() > + message(STATUS "Unwind not found") > + endif() > + > + add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES}) > + # This is a hack for further getting install directory of library > + # by ExternalProject_Get_Property > + set_target_properties(unwind > + PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX} > + ) > +else() > + ExternalProject_Add(unwind > + URL > https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz > + CONFIGURE_COMMAND /configure > + CC=${CMAKE_C_COMPILER} > + CXX=${CMAKE_CXX_COMPILER} > + CFLAGS=${DEPENDENCY_CFLAGS} > + CPPFLAGS=${DEPENDENCY_CPPFLAGS} > + LDFLAGS=${DEPENDENCY_LDFLAGS} > + --prefix= > + --disable-shared > + --enable-static > + --disable-minidebuginfo # to prevent linking with liblzma > + STEP_TARGETS download > + ) > +endif() > + > +# Get install directories of builded libraries for building > +# tarantool with custon CMAKE_PREFIX_PATH > +foreach(PROJ openssl icu zlib ncurses readline iconv unwind) > + ExternalProject_Get_Property(${PROJ} install_dir) > + set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir}) > + set(TARANTOOL_DEPENDS ${PROJ} ${TARANTOOL_DEPENDS}) > + message(STATUS "Add external project ${PROJ} in ${install_dir}") > +endforeach() > + > +ExternalProject_Add(tarantool > + DEPENDS ${TARANTOOL_DEPENDS} > + SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/.. > + LIST_SEPARATOR : > + CMAKE_ARGS > + # Override LOCALSTATEDIR to avoid cmake "special" cases: > + # > https://cmake.org/cmake/help/v3.4/module/GNUInstallDirs.html#special-cases > + -DCMAKE_INSTALL_LOCALSTATEDIR=/var > + -DCMAKE_INSTALL_PREFIX= > + -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH} > + -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE > + -DOPENSSL_USE_STATIC_LIBS=TRUE > + -DBUILD_STATIC=TRUE > + -DENABLE_DIST=TRUE > + -DENABLE_BACKTRACE=TRUE > + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} > + -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} > + ${CMAKE_TARANTOOL_ARGS} > + STEP_TARGETS build > + BUILD_COMMAND $(MAKE) > +) > + > +enable_testing() > +ExternalProject_Get_Property(tarantool binary_dir) > +SET(TARANTOOL_BINARY_DIR ${binary_dir}) > + > +add_test( > + NAME check-dependencies > + COMMAND ${CMAKE_COMMAND} > + -D FILE=${TARANTOOL_BINARY_DIR}/src/tarantool > + -P CheckDependencies.cmake > + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test > +) > + > +add_test( > + NAME check-exports > + COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool > + ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/exports.test.lua > +) > + > +add_test( > + NAME check-traceback > + COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool > + > ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/traceback.test.lua > +) > diff --git a/static-build/README.md b/static-build/README.md > new file mode 100644 > index 000000000..2219b5026 > --- /dev/null > +++ b/static-build/README.md > @@ -0,0 +1,90 @@ > +# Tarantool static build tooling > + > +These files help to prepare environment for building Tarantool > +statically. And builds it. > + > +## Prerequisites > + > +Please install following tools and libraries that will > +be necessary for building and testing: > +* git > +* A C/C++ compiler. > + > + Ordinarily, this is gcc and g++ version 4.6 or later. On Mac OS X, this > + is Clang version 3.2+. > +* cmake > +* autoconf automake libtool > +* make > +* Python and modules. > + > + Python interpreter is not necessary for building Tarantool itself, > unless you > + intend to use the “Run the test suite". For all platforms, this is > python > + version 2.7+ (but not 3.x). You need the following Python modules: > + * pyyaml > + * argparse > + * msgpack-python > + * gevent > + * six > + > +### Here is an examples for your OS: > + > +CentOS: > + > +```bash > +yum install -y \ > + git perl gcc cmake make gcc-c++ libstdc++-static autoconf automake > libtool \ > + python-msgpack python-yaml python-argparse python-six python-gevent > +``` > + > +Ubuntu/Debian: > + > +```bash > +apt-get install -y \ > + build-essential cmake make coreutils autoconf automake libtool sed \ > + python python-pip python-setuptools python-dev \ > + python-msgpack python-yaml python-argparse python-six python-gevent > +``` > + > +MacOS: > + > +Before you start please install default Xcode Tools by Apple: > + > +```bash > +sudo xcode-select --install > +sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer > +``` > + > +Install brew using command from > +[Homebrew repository instructions](https://github.com/Homebrew/inst) > + > +After that run next script: > + > +```bash > +brew install autoconf automake libtool cmake > file://$${PWD}/tools/brew_taps/tntpython2.rbs > +pip install --force-reinstall -r test-run/requirements.txt > +``` > + > +## Usage > + > +```bash > +cmake . > +make -j > +ctest -V > +``` > + > +## Customize your build > + > +If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS` > variable > + > +### Usage > + > +There are three types of `CMAKE_BUILD_TYPE`: > +* Debug - default > +* Release > +* RelWithDebInfo > + > +And you want to build tarantool with RelWithDebInfo: > + > +```bash > +cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" . > +``` > diff --git a/static-build/test/CheckDependencies.cmake > b/static-build/test/CheckDependencies.cmake > new file mode 100644 > index 000000000..49e91e7fa > --- /dev/null > +++ b/static-build/test/CheckDependencies.cmake > @@ -0,0 +1,43 @@ > +## This is a cmake-based test, it checks that tarantool static binary > +# has no dependencies except allowed ones. > + > +include(GetPrerequisites) > +if(NOT FILE) > + message(FATAL_ERROR "Usage: " > + "${CMAKE_COMMAND} -DFILE= -P CheckDependencies.cmake") > +elseif(NOT EXISTS ${FILE}) > + message(FATAL_ERROR "${FILE}: No such file") > +endif() > + > +get_prerequisites(${FILE} DEPENDENCIES 0 0 "" "") > + > +if (APPLE) > + set(ALLOWLIST > + libSystem > + CoreFoundation > + libc++ > + ) > +elseif(UNIX) > + set(ALLOWLIST > + libdl > + librt > + libc > + libm > + libgcc_s > + libpthread > + ) > +else() > + message(FATAL_ERROR "Unknown platform") > +endif() > + > +foreach(DEPENDENCY_FILE ${DEPENDENCIES}) > + message("Dependency: ${DEPENDENCY_FILE}") > +endforeach() > + > +foreach(DEPENDENCY_FILE ${DEPENDENCIES}) > + get_filename_component(libname ${DEPENDENCY_FILE} NAME_WE) > + list (FIND ALLOWLIST ${libname} _index) > + if (_index EQUAL -1) > + message(FATAL_ERROR "Blocklisted dependency: ${DEPENDENCY_FILE}") > + endif() > +endforeach() > diff --git a/static-build/test/static-build/exports.test.lua > b/static-build/test/static-build/exports.test.lua > new file mode 100755 > index 000000000..de54973d8 > --- /dev/null > +++ b/static-build/test/static-build/exports.test.lua > @@ -0,0 +1,142 @@ > +#!/usr/bin/env tarantool > + > +local tap = require('tap') > +local ffi = require('ffi') > +ffi.cdef([[ > + void *dlsym(void *handle, const char *symbol); > +]]) > + > +local test = tap.test('exports') > + > +-- See `man 3 dlsym`: > +-- RTLD_DEFAULT > +-- Find the first occurrence of the desired symbol using the default > +-- shared object search order. The search will include global symbols > +-- in the executable and its dependencies, as well as symbols in shared > +-- objects that were dynamically loaded with the RTLD_GLOBAL flag. > +local RTLD_DEFAULT = ffi.cast("void *", jit.os == "OSX" and -2LL or 0LL) > + > +local function check_symbol(sym) > + test:ok(ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, ('Symbol %q > found'):format(sym)) > +end > + > +local check_symbols = { > + -- FFI > + > + 'guava', > + 'base64_decode', > + 'base64_encode', > + 'SHA1internal', > + 'random_bytes', > + 'fiber_time', > + 'ibuf_create', > + 'ibuf_destroy', > + 'port_destroy', > + 'csv_create', > + 'csv_destroy', > + 'title_get', > + 'title_update', > + 'tnt_iconv', > + 'tnt_iconv_open', > + 'tnt_iconv_close', > + 'exception_get_int', > + 'exception_get_string', > + > + 'tarantool_lua_ibuf', > + 'uuid_nil', > + 'tt_uuid_create', > + 'tt_uuid_str', > + 'tt_uuid_is_equal', > + 'tt_uuid_is_nil', > + 'tt_uuid_bswap', > + 'tt_uuid_from_string', > + 'log_level', > + 'log_format', > + 'uri_parse', > + 'uri_format', > + 'PMurHash32', > + 'PMurHash32_Process', > + 'PMurHash32_Result', > + 'crc32_calc', > + 'mp_encode_double', > + 'mp_encode_float', > + 'mp_encode_decimal', > + 'mp_decode_double', > + 'mp_decode_float', > + 'mp_decode_extl', > + 'mp_sizeof_decimal', > + 'decimal_unpack', > + > + 'log_type', > + 'say_set_log_level', > + 'say_logrotate', > + 'say_set_log_format', > + 'tarantool_uptime', > + 'tarantool_exit', > + 'log_pid', > + 'space_by_id', > + 'space_run_triggers', > + 'space_bsize', > + 'box_schema_version', > + > + 'crypto_EVP_MD_CTX_new', > + 'crypto_EVP_MD_CTX_free', > + 'crypto_HMAC_CTX_new', > + 'crypto_HMAC_CTX_free', > + 'crypto_stream_new', > + 'crypto_stream_begin', > + 'crypto_stream_append', > + 'crypto_stream_commit', > + 'crypto_stream_delete', > + > + -- Module API > + > + '_say', > + 'swim_cfg', > + 'swim_quit', > + 'fiber_new', > + 'fiber_cancel', > + 'coio_wait', > + 'coio_close', > + 'coio_call', > + 'coio_getaddrinfo', > + 'luaT_call', > + 'box_txn', > + 'box_select', > + 'clock_realtime', > + 'string_strip_helper', > + > + -- Lua / LuaJIT > + > + 'lua_newstate', > + 'lua_close', > + 'luaL_loadstring', > + 'luaJIT_profile_start', > + 'luaJIT_profile_stop', > + 'luaJIT_profile_dumpstack', > + > + 'ERR_error_string', > + 'ERR_get_error', > + > + 'EVP_get_digestbyname', > + 'EVP_get_cipherbyname', > + 'EVP_CIPHER_CTX_new', > + 'EVP_CIPHER_CTX_free', > + 'EVP_CIPHER_block_size', > + 'HMAC_Init_ex', > + 'HMAC_Update', > + 'HMAC_Final', > + > + 'ZSTD_compress', > + 'ZSTD_decompress', > + 'ZSTD_free', > + 'ZSTD_malloc', > + 'ZSTD_versionString', > +} > + > +test:plan(#check_symbols) > +for _, sym in ipairs(check_symbols) do > + check_symbol(sym) > +end > + > +os.exit(test:check() and 0 or 1) > diff --git a/static-build/test/static-build/traceback.test.lua > b/static-build/test/static-build/traceback.test.lua > new file mode 100755 > index 000000000..32e1ed8b5 > --- /dev/null > +++ b/static-build/test/static-build/traceback.test.lua > @@ -0,0 +1,15 @@ > +#!/usr/bin/env tarantool > + > +local tap = require('tap') > +local fiber = require('fiber') > + > +local test = tap.test('traceback') > +test:plan(1) > + > +local info = fiber.info()[fiber.id()] > +test:ok(info.backtrace ~= nil, 'fiber.info() has backtrace') > +for _, l in pairs(info.backtrace or {}) do > + test:diag('%s: %s', next(l)) > +end > + > +os.exit(test:check() and 0 or 1) > -- > 2.26.2 > >