Re: [Tarantool-patches] [PATCH v3] build: refactor static build process

[Alexandr Barulev <huston.mavr@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 37923 bytes --]

Hello!

Here is a new commit message(as discussed at telegram chat):

Author: HustonMmmavr <huston.mavr@gmail.com>
Date:   Thu Sep 10 12:43:20 2020 +0300

build: refactor static build process

Refactored static build process to use static-build/CMakeLists.txt
instead of Dockerfile.staticbuild (this allows to support static
build on macOS). Following third-party dependencies for static build
are installed via cmake `ExternalProject_Add`:
    - OpenSSL
    - Zlib
    - Ncurses
    - Readline
    - Unwind
    - ICU

* Added support static build for macOS
* Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
    build at file cmake/BuildLibCURL.cmake:
    - disable building shared libcurl libraries (by setting
        `--disable-shared` option)
    - disable hiding libcurl symbols (by setting
        `--disable-symbol-hiding` option)
    - prevent linking libcurl with system libz (by setting
        `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
* Removed Dockerfile.staticbuild
* Added new gitlab.ci jobs to test new style static build:
    - static_build_cmake_linux
    - static_build_cmake_osx_15
* Removed static_docker_build gitlab.ci job

Closes #5095

Co-authored-by: Yaroslav Dynnikov <yaroslav.dynnikov@gmail.com>

ср, 9 сент. 2020 г. в 18:45, HustonMmmavr <huston.mavr@gmail.com>:

> From: Yaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
>
> Refactored static build process to use static-build/CMakeLists.txt
> instead of Dockerfile.staticbuild (this allows to support static
> build on macOS). Following third-party dependencies for static build
> are installed via cmake `ExternalProject_Add`:
>   - OpenSSL
>   - Zlib
>   - Ncurses
>   - Readline
>   - Unwind
>   - ICU
>
> * Added support static build for macOS
> * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
>   build at file cmake/BuildLibCURL.cmake:
>     - disable building shared libcurl libraries (by setting
>       `--disable-shared` option)
>     - disable hiding libcurl symbols (by setting
>       `--disable-symbol-hiding` option)
>     - prevent linking libcurl with system libz (by setting
>       `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
> * Removed Dockerfile.staticbuild
> * Added new gitlab.ci jobs to test new style static build:
>   - static_build_cmake_linux
>   - static_build_cmake_osx_15
> * Removed static_docker_build gitlab.ci job
>
> Closes #5095
> ---
>
> Branch:
> https://github.com/tarantool/tarantool/tree/rosik/refactor-static-build
> Issue: https://github.com/tarantool/tarantool/issues/5095
>
> Changes in v3:
> 1. Fixed typos at commit message
> 2. Fixed commentaries at .travis.mk
> 3. Fixed finding libz library at cmake/BuildLibCurl.cmake as at master
>    branch
> 4. Instead of using static-build/.gitignore use the root .gitignore file
> 5. Refactored static-build/CMakeLists.txt:
>    * Project name is set to tarantool-static
>    * Instead of explicitly finding of c/c++ compilers current version uses
>      cmake `project(tarantool-static C CXX)` command. This change requires
>      to manually set CFLAGS/CPPFLAGS/LDFLAGS for all static-build
>      dependencies (because building dependencies at macOS requires path
>      to macOS SDK) the idea was taken from cmake/BuildLibCurl.cmake
>    * Added commentaries about problem with libicu
>    * Removed unused TEST_COMMAND at ExternalProject_Add(zlib ...) also
>      removed unnecessary # STEP_TARGETS at ExternalProject_Add(readline
> ...)
>    * Deleted doubled whitespaces
> 6. Fixed static-build/README.md:
>    * Added list of required tools
>    * Added example for ubuntu/debian
>    * Fixed indentation and typos
> 7. Refactored tests:
>    * Got rid of test-run
>    * Deleted doubled whitespaces at exports.test.lua
>    * Fixed defining of RTLD_DEFAULT
>
>
>  .gitignore                                    |   8 +
>  .gitlab-ci.yml                                |  11 +-
>  .travis.mk                                    |  56 +++-
>  Dockerfile.staticbuild                        |  98 ------
>  cmake/BuildLibCURL.cmake                      |  13 +-
>  cmake/compiler.cmake                          |  24 +-
>  cmake/os.cmake                                |   5 +-
>  static-build/CMakeLists.txt                   | 311 ++++++++++++++++++
>  static-build/README.md                        |  90 +++++
>  static-build/test/CheckDependencies.cmake     |  43 +++
>  .../test/static-build/exports.test.lua        | 142 ++++++++
>  .../test/static-build/traceback.test.lua      |  15 +
>  12 files changed, 692 insertions(+), 124 deletions(-)
>  delete mode 100644 Dockerfile.staticbuild
>  create mode 100644 static-build/CMakeLists.txt
>  create mode 100644 static-build/README.md
>  create mode 100644 static-build/test/CheckDependencies.cmake
>  create mode 100755 static-build/test/static-build/exports.test.lua
>  create mode 100755 static-build/test/static-build/traceback.test.lua
>
> diff --git a/.gitignore b/.gitignore
> index a42c7db35..c6e261e18 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -151,3 +151,11 @@ src/box/sql/parse.h
>  src/box/sql/parse.c
>  src/box/sql/opcodes.h
>  src/box/sql/opcodes.c
> +
> +# static-build
> +static-build/*-prefix
> +static-build/Makefile
> +static-build/build
> +
> +# macOS files
> +.DS_Store
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 0ead08711..c9aef3dc7 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -534,9 +534,14 @@ static_build:
>    script:
>      - ${GITLAB_MAKE} test_static_build
>
> -static_docker_build:
> +static_build_cmake_linux:
> +  <<: *docker_test_definition
> +  script:
> +    - ${GITLAB_MAKE} test_static_build_cmake_linux
> +
> +static_build_cmake_osx_15:
>    stage: test
>    tags:
> -    - deploy_test
> +    - osx_15
>    script:
> -    - ${GITLAB_MAKE} test_static_docker_build
> +    - ${GITLAB_MAKE} test_static_build_cmake_osx
> diff --git a/.travis.mk b/.travis.mk
> index efc05cf05..58d0c1596 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -148,8 +148,14 @@ deps_debian_static:
>  test_static_build: deps_debian_static
>         CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk
> test_debian_no_deps
>
> -test_static_docker_build:
> -       docker build --no-cache --network=host --build-arg RUN_TESTS=ON -f
> Dockerfile.staticbuild .
> +# New static build
> +# builddir used in this target - is a default build path from cmake
> +# ExternalProject_Add()
> +test_static_build_cmake_linux:
> +       cd static-build && cmake
> -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON"
> . && \
> +       make -j && ctest -V
> +       cd test && /usr/bin/python test-run.py --force \
> +               --builddir
> ${PWD}/static-build/tarantool-prefix/src/tarantool-build
> $(TEST_RUN_EXTRA_PARAMS)
>
>  # ###################
>  # Static Analysis
> @@ -193,15 +199,16 @@ build_osx:
>         cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
> ${CMAKE_EXTRA_PARAMS}
>         make -j
>
> -test_osx_no_deps: build_osx
> -       # Limits: Increase the maximum number of open file descriptors on
> macOS:
> -       #   Travis-ci needs the "ulimit -n <value>" call
> -       #   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
> -       # Also gitlib-ci needs the password to change the limits, while
> -       # travis-ci runs under root user. Limit setup must be in the same
> -       # call as tests runs call.
> -       # Tests: Temporary excluded replication/ suite with some tests
> -       #        from other suites by issues #4357 and #4370
> +
> +# Limits: Increase the maximum number of open file descriptors on macOS:
> +#   Travis-ci needs the "ulimit -n <value>" call
> +#   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
> +# Also gitlib-ci needs the password to change the limits, while
> +# travis-ci runs under root user. Limit setup must be in the same
> +# call as tests runs call.
> +# Tests: Temporary excluded replication/ suite with some tests
> +#        from other suites by issues #4357 and #4370
> +INIT_TEST_ENV_OSX=\
>         sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>                 launchctl limit maxfiles || : ; \
>                 ulimit -n ${MAX_FILES} || : ; \
> @@ -210,11 +217,34 @@ test_osx_no_deps: build_osx
>                 launchctl limit maxproc || : ; \
>                 ulimit -u ${MAX_PROC} || : ; \
>                 ulimit -u ; \
> -               rm -rf /tmp/tnt ; \
> -               cd test && ./test-run.py --vardir /tmp/tnt --force
> $(TEST_RUN_EXTRA_PARAMS)
> +               rm -rf /tmp/tnt
> +
> +test_osx_no_deps: build_osx
> +       ${INIT_TEST_ENV_OSX}; \
> +       cd test && ./test-run.py --vardir /tmp/tnt --force
> $(TEST_RUN_EXTRA_PARAMS)
>
>  test_osx: deps_osx test_osx_no_deps
>
> +# Static macOS build
> +
> +STATIC_OSX_PKGS=autoconf automake libtool cmake
> file://$${PWD}/tools/brew_taps/tntpython2.rb
> +base_deps_osx:
> +       brew update || echo | /usr/bin/ruby -e \
> +               "$$(curl -fsSL
> https://raw.githubusercontent.com/Homebrew/install/master/install)"
> +       brew install --force ${STATIC_OSX_PKGS} || brew upgrade
> ${STATIC_OSX_PKGS}
> +       pip install --force-reinstall -r test-run/requirements.txt
> +
> +# builddir used in this target - is a default build path from cmake
> +# ExternalProject_Add()
> +test_static_build_cmake_osx: base_deps_osx
> +       cd static-build && cmake
> -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON"
> . && \
> +       make -j && ctest -V
> +       ${INIT_TEST_ENV_OSX}; \
> +       cd test && ./test-run.py --vardir /tmp/tnt \
> +               --builddir
> ${PWD}/static-build/tarantool-prefix/src/tarantool-build \
> +               --force $(TEST_RUN_EXTRA_PARAMS)
> +
> +
>  ###########
>  # FreeBSD #
>  ###########
> diff --git a/Dockerfile.staticbuild b/Dockerfile.staticbuild
> deleted file mode 100644
> index f67f46f5e..000000000
> --- a/Dockerfile.staticbuild
> +++ /dev/null
> @@ -1,98 +0,0 @@
> -FROM centos:7
> -
> -RUN yum install -y epel-release
> -RUN yum install -y yum install
> https://centos7.iuscommunity.org/ius-release.rpm
> -
> -RUN set -x \
> -    && yum -y install \
> -        libstdc++ \
> -        libstdc++-static \
> -        readline \
> -        openssl \
> -        lz4 \
> -        binutils \
> -        ncurses \
> -        libgomp \
> -        lua \
> -        curl \
> -        tar \
> -        zip \
> -        unzip \
> -        libunwind \
> -        zlib \
> -    && yum -y install \
> -        perl \
> -        gcc-c++ \
> -        cmake \
> -        lz4-devel \
> -        binutils-devel \
> -        lua-devel \
> -        make \
> -        git \
> -        autoconf \
> -        automake \
> -        libtool \
> -        wget
> -
> -RUN yum -y install ncurses-static readline-static zlib-static pcre-static
> glibc-static
> -
> -RUN yum -y install python-devel python-pip
> -
> -RUN set -x && \
> -    cd / && \
> -    curl -O -L https://www.openssl.org/source/openssl-1.1.1f.tar.gz && \
> -    tar -xvf openssl-1.1.1f.tar.gz && \
> -    cd openssl-1.1.1f && \
> -    ./config --libdir=lib && \
> -    make -j && make install
> -
> -RUN set -x && \
> -    cd / && \
> -    curl -O -L
> https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz
> && \
> -    tar -xvf icu4c-62_1-src.tgz && \
> -    cd icu/source && \
> -    ./configure --with-data-packaging=static --enable-static
> --enable-shared && \
> -    make -j && make install
> -
> -RUN set -x && \
> -    cd / && \
> -    curl -O -L
> http://download.savannah.nongnu.org/releases/libunwind/libunwind-1.3-rc1.tar.gz
> && \
> -    tar -xvf libunwind-1.3-rc1.tar.gz && \
> -    cd libunwind-1.3-rc1 && \
> -    ./configure --enable-static --enable-shared && \
> -    make -j && make install
> -
> -COPY . /tarantool
> -
> -WORKDIR /tarantool
> -
> -RUN set -x && \
> -    git submodule init && \
> -    git submodule update
> -
> -# Cleanup for 'build' directory added, because it purges all artefacts
> -# produced for curl build, including the old configuration in build/curl
> -RUN set -x && \
> -    find . -name 'CMakeFiles' -type d -exec rm -rf {} + && \
> -    find . -name 'CMakeCache.txt' -type f -delete && \
> -    rm -rf build test/small test/luajit-tap
> -
> -RUN pip install -r /tarantool/test-run/requirements.txt
> -
> -RUN set -x && \
> -    cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
> -         -DENABLE_DIST:BOOL=ON \
> -         -DBUILD_STATIC=ON \
> -         -DOPENSSL_USE_STATIC_LIBS=ON \
> -         -DOPENSSL_ROOT_DIR=/usr/local \
> -         . && \
> -    make -j && make install
> -
> -ARG RUN_TESTS
> -RUN if [ -n "${RUN_TESTS}" ]; then \
> -        set -x && \
> -        cd test && \
> -        /usr/bin/python test-run.py --force; \
> -    fi
> -
> -ENTRYPOINT ["/bin/bash"]
> diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
> index 5f8b15a63..86fec39e9 100644
> --- a/cmake/BuildLibCURL.cmake
> +++ b/cmake/BuildLibCURL.cmake
> @@ -4,15 +4,21 @@ macro(curl_build)
>      set(LIBCURL_BINARY_DIR ${PROJECT_BINARY_DIR}/build/curl/work)
>      set(LIBCURL_INSTALL_DIR ${PROJECT_BINARY_DIR}/build/curl/dest)
>
> +    message(STATUS "Looking for zlib")
> +    find_path(ZLIB_INCLUDE_DIR zlib.h)
> +    message(STATUS "Looking for zlib.h - ${ZLIB_INCLUDE_DIR}")
>      if (BUILD_STATIC)
>          set(LIBZ_LIB_NAME libz.a)
>      else()
>          set(LIBZ_LIB_NAME z)
>      endif()
>      find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME})
> -    if ("${LIBZ_LIBRARY}" STREQUAL "LIBZ_LIBRARY-NOTFOUND")
> +    message(STATUS "Looking for libz - ${LIBZ_LIBRARY}")
> +
> +    if (NOT ZLIB_INCLUDE_DIR OR NOT LIBZ_LIBRARY)
>          message(FATAL_ERROR "Unable to find zlib")
>      endif()
> +    get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR}
> DIRECTORY)
>
>      # Use the same OpenSSL library for libcurl as is used for
>      # tarantool itself.
> @@ -88,9 +94,10 @@ macro(curl_build)
>
>                  --prefix <INSTALL_DIR>
>                  --enable-static
> -                --enable-shared
> +                --disable-shared
> +                --disable-symbol-hiding
>
> -                --with-zlib
> +                --with-zlib=${FOUND_ZLIB_ROOT_DIR}
>                  ${LIBCURL_OPENSSL_OPT}
>                  --with-ca-fallback
>
> diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake
> index 5a1141ebd..db2ae6227 100644
> --- a/cmake/compiler.cmake
> +++ b/cmake/compiler.cmake
> @@ -33,10 +33,6 @@ if (CMAKE_COMPILER_IS_GNUCC)
>             Your GCC version is ${CMAKE_CXX_COMPILER_VERSION}, please
> update
>                     ")
>         endif()
> -else()
> -     if (BUILD_STATIC)
> -           message(FATAL_ERROR "Static build is supported for GCC only")
> -     endif()
>  endif()
>
>  #
> @@ -120,10 +116,23 @@ set (CMAKE_CXX_FLAGS_RELWITHDEBINFO
>
>  unset(CC_DEBUG_OPT)
>
> +message(STATUS "Looking for libunwind.h")
> +find_path(UNWIND_INCLUDE_DIR libunwind.h)
> +message(STATUS "Looking for libunwind.h - ${UNWIND_INCLUDE_DIR}")
> +
> +if (UNWIND_INCLUDE_DIR)
> +    include_directories(${UNWIND_INCLUDE_DIR})
> +endif()
> +
> +set(CMAKE_REQUIRED_INCLUDES ${UNWIND_INCLUDE_DIR})
>  check_include_file(libunwind.h HAVE_LIBUNWIND_H)
> -if(BUILD_STATIC)
> +set(CMAKE_REQUIRED_INCLUDES "")
> +
> +if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
>      set(UNWIND_LIB_NAME libunwind.a)
>  else()
> +    # libunwind can't be compiled on macOS.
> +    # But there exists libunwind.dylib as a part of MacOSSDK
>      set(UNWIND_LIB_NAME unwind)
>  endif()
>  find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME})
> @@ -185,7 +194,10 @@ if (ENABLE_BACKTRACE)
>      find_package_message(UNWIND_LIBRARIES "Found unwind"
> "${UNWIND_LIBRARIES}")
>  endif()
>
> -if(BUILD_STATIC)
> +# On macOS there is no '-static-libstdc++' flag and it's use will
> +# raise following error:
> +# error: argument unused during compilation: '-static-libstdc++'
> +if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
>      # Static linking for c++ routines
>      add_compile_flags("C;CXX" "-static-libstdc++")
>  endif()
> diff --git a/cmake/os.cmake b/cmake/os.cmake
> index 905be61df..276a79b42 100644
> --- a/cmake/os.cmake
> +++ b/cmake/os.cmake
> @@ -107,7 +107,10 @@ elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Darwin")
>
>      # Latest versions of Homebrew wont 'link --force' for libraries, that
> were
>      # preinstalled in system. So we'll use this dirty hack
> -    find_program(HOMEBREW_EXECUTABLE brew)
> +
> +    if (NOT BUILD_STATIC)
> +        find_program(HOMEBREW_EXECUTABLE brew)
> +    endif()
>      if(EXISTS ${HOMEBREW_EXECUTABLE})
>          execute_process(COMMAND ${HOMEBREW_EXECUTABLE} --prefix
>                          OUTPUT_VARIABLE HOMEBREW_PREFIX
> diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
> new file mode 100644
> index 000000000..90029fdd8
> --- /dev/null
> +++ b/static-build/CMakeLists.txt
> @@ -0,0 +1,311 @@
> +cmake_minimum_required(VERSION 2.8)
> +
> +# Detect system compilers for further configuring dependencies to be
> +# builded with these compilers. This is used to build tarantool and
> +# it's dependencies by usign one compiler system (for example libicu
> +# by default uses clang if it exists when others uses gcc/g++ on
> +# linux machine).
> +project(tarantool-static C CXX)
> +
> +include(ExternalProject)
> +set(OPENSSL_VERSION 1.1.1f)
> +set(ZLIB_VERSION 1.2.11)
> +set(NCURSES_VERSION 6.2)
> +set(READLINE_VERSION 8.0)
> +set(UNWIND_VERSION 1.3-rc1)
> +
> +# Pass -isysroot=<SDK_PATH> option on Mac OS to a preprocessor and a C
> +# compiler to find header files installed with an SDK.
> +#
> +# The idea is to set these (DEPENDENCY_*) variables to corresponding
> +# environment variables at each depenency configure script.
> +#
> +# Note: Passing of CPPFLAGS / CFLAGS explicitly discards using of
> +# corresponsing environment variables. So pass empty LDFLAGS to discard
> +# using of corresponding environment variable. It is possible that a
> +# linker flag assumes that some compilation flag is set. We don't pass
> +# CFLAGS from environment, so we should not do it for LDFLAGS too.
> +set(DEPENDENCY_CFLAGS "")
> +set(DEPENDENCY_CPPFLAGS "")
> +set(DEPENDENCY_LDFLAGS)
> +if (APPLE)
> +    set(DEPENDENCY_CFLAGS   "${CMAKE_C_SYSROOT_FLAG}
> ${CMAKE_OSX_SYSROOT}")
> +    set(DEPENDENCY_CPPFLAGS "${CMAKE_C_SYSROOT_FLAG}
> ${CMAKE_OSX_SYSROOT}")
> +endif()
> +
> +# Install all libraries required by tarantool at current build dir
> +
> +#
> +# OpenSSL
> +#
> +ExternalProject_Add(openssl
> +    URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
> +    CONFIGURE_COMMAND <SOURCE_DIR>/config
> +        CC=${CMAKE_C_COMPILER}
> +        CXX=${CMAKE_CXX_COMPILER}
> +        CFLAGS=${DEPENDENCY_CFLAGS}
> +        CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +        LDFLAGS=${DEPENDENCY_LDFLAGS}
> +
> +        --prefix=<INSTALL_DIR>
> +        --libdir=lib
> +        no-shared
> +    INSTALL_COMMAND ${CMAKE_MAKE_PROGRAM} install_sw
> +)
> +
> +#
> +# ICU
> +#
> +ExternalProject_Add(icu
> +    URL
> https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz
> +    # By default libicu is built by using clang/clang++ compiler (if it
> +    # exists). Here is a link for detecting compilers at libicu configure
> +    # script:
> https://github.com/unicode-org/icu/blob/7c7b8bd5702310b972f888299169bc3cc88bf0a6/icu4c/source/configure.ac#L135
> +    # This will cause the problem on linux machine: tarantool is built
> +    # with gcc/g++ and libicu is built with clang/clang++ (if it exists)
> +    # so at linking stage `rellocation` errors will occur. To solve this,
> +    # we can set CC/CXX to CMAKE_C_COMPILER/CMAKE_CXX_COMPILER variables
> +    # manually which are detected above (by cmake `project()` command)
> +    CONFIGURE_COMMAND <SOURCE_DIR>/source/configure
> +        CC=${CMAKE_C_COMPILER}
> +        CXX=${CMAKE_CXX_COMPILER}
> +        CFLAGS=${DEPENDENCY_CFLAGS}
> +        CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +        LDFLAGS=${DEPENDENCY_LDFLAGS}
> +
> +        --with-data-packaging=static
> +        --prefix=<INSTALL_DIR>
> +        --disable-shared
> +        --enable-static
> +)
> +
> +#
> +# ZLIB
> +#
> +ExternalProject_Add(zlib
> +    URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
> +    CONFIGURE_COMMAND env
> +        CC=${CMAKE_C_COMPILER}
> +        CFLAGS=${DEPENDENCY_CFLAGS}
> +        CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +        LDFLAGS=${DEPENDENCY_LDFLAGS}
> +        <SOURCE_DIR>/configure
> +        --prefix=<INSTALL_DIR>
> +        --static
> +)
> +
> +#
> +# Ncurses
> +#
> +ExternalProject_Add(ncurses
> +    URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz
> +    CONFIGURE_COMMAND <SOURCE_DIR>/configure
> +        CC=${CMAKE_C_COMPILER}
> +        CXX=${CMAKE_CXX_COMPILER}
> +        CFLAGS=${DEPENDENCY_CFLAGS}
> +        CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +        LDFLAGS=${DEPENDENCY_LDFLAGS}
> +
> +        --prefix=<INSTALL_DIR>
> +
> +        # This flag enables creation of libcurses.a as a symlink to
> libncurses.a
> +        # and disables subdir creation `ncurses` at
> <install_dir>/include. It is
> +        # necessary for correct work of FindCurses.cmake module (this
> module is
> +        # builtin at cmake package) which used in cmake/FindReadline.cmake
> +        --enable-overwrite
> +
> +        # enable building libtinfo to prevent linking with libtinfo from
> system
> +        # directories
> +        --with-termlib
> +
> +        # set search paths for terminfo db
> +
> --with-terminfo-dirs=/lib/terminfo:/usr/share/terminfo:/etc/terminfo
> +
> +        # disable install created terminfo db, use db from system
> +        --disable-db-install
> +        --without-progs
> +        --without-manpages
> +)
> +
> +#
> +# ReadLine
> +#
> +ExternalProject_Add(readline
> +    URL
> https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz
> +    CONFIGURE_COMMAND <SOURCE_DIR>/configure
> +        CC=${CMAKE_C_COMPILER}
> +        CFLAGS=${DEPENDENCY_CFLAGS}
> +        CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +        LDFLAGS=${DEPENDENCY_LDFLAGS}
> +
> +        --prefix=<INSTALL_DIR>
> +        --disable-shared
> +)
> +
> +#
> +# ICONV
> +#
> +if (APPLE)
> +    ExternalProject_Add(iconv
> +        URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
> +        CONFIGURE_COMMAND <SOURCE_DIR>/configure
> +            CC=${CMAKE_C_COMPILER}
> +            CFLAGS=${DEPENDENCY_CFLAGS}
> +            CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +            LDFLAGS=${DEPENDENCY_LDFLAGS}
> +
> +            --prefix=<INSTALL_DIR>
> +            --disable-shared
> +            --enable-static
> +            --with-gnu-ld
> +        STEP_TARGETS download
> +    )
> +else()
> +    # In linux iconv is embedded into glibc
> +    # So we find system header and copy it locally
> +    find_path(ICONV_INCLUDE_DIR iconv.h)
> +    if(NOT ICONV_INCLUDE_DIR)
> +        message(FATAL_ERROR "iconv include header not found")
> +    endif()
> +
> +    set(ICONV_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix")
> +
> +    add_custom_command(
> +        OUTPUT "${ICONV_INSTALL_PREFIX}/include/iconv.h"
> +        COMMAND ${CMAKE_COMMAND} -E make_directory
> +            "${ICONV_INSTALL_PREFIX}/include"
> +        COMMAND ${CMAKE_COMMAND} -E copy
> +            "${ICONV_INCLUDE_DIR}/iconv.h"
> +            "${ICONV_INSTALL_PREFIX}/include/iconv.h"
> +    )
> +    add_custom_target(iconv
> +        DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h"
> +    )
> +    # This is a hack for further getting install directory of library
> +    # by ExternalProject_Get_Property
> +    set_target_properties(iconv
> +        PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX}
> +    )
> +endif()
> +
> +#
> +# Unwind
> +#
> +if (APPLE)
> +    # On macOS libunwind is a part of MacOSX.sdk
> +    # So we need to find library and header and
> +    # copy it locally
> +    find_path(UNWIND_INCLUDE_DIR libunwind.h)
> +    find_library(UNWIND_LIBRARY libunwind.dylib
> +        PATH_SUFFIXES system
> +    )
> +
> +    set(UNWIND_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/unwind-prefix")
> +
> +    set(UNWIND_DEPENDENCIES)
> +
> +    if (UNWIND_INCLUDE_DIR AND UNWIND_LIBRARY)
> +        add_custom_command(
> +            OUTPUT "${UNWIND_INSTALL_PREFIX}/include/unwind.h"
> +            OUTPUT "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
> +            COMMAND ${CMAKE_COMMAND} -E make_directory
> +                "${UNWIND_INSTALL_PREFIX}/include"
> +            COMMAND ${CMAKE_COMMAND} -E copy
> +                "${UNWIND_INCLUDE_DIR}/libunwind.h"
> +                "${UNWIND_INCLUDE_DIR}/unwind.h"
> +                "${UNWIND_INSTALL_PREFIX}/include/"
> +        )
> +        add_custom_command(
> +            OUTPUT "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
> +            COMMAND ${CMAKE_COMMAND} -E make_directory
> +                "${UNWIND_INSTALL_PREFIX}/lib"
> +            COMMAND ${CMAKE_COMMAND} -E copy
> +                "${UNWIND_LIBRARY}"
> +                "${UNWIND_INSTALL_PREFIX}/lib/"
> +        )
> +        set(UNWIND_DEPENDENCIES
> +            ${UNWIND_DEPENDENCIES}
> +            "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
> +            "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
> +        )
> +    else()
> +        message(STATUS "Unwind not found")
> +    endif()
> +
> +    add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES})
> +    # This is a hack for further getting install directory of library
> +    # by ExternalProject_Get_Property
> +    set_target_properties(unwind
> +        PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX}
> +    )
> +else()
> +    ExternalProject_Add(unwind
> +        URL
> https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz
> +        CONFIGURE_COMMAND <SOURCE_DIR>/configure
> +            CC=${CMAKE_C_COMPILER}
> +            CXX=${CMAKE_CXX_COMPILER}
> +            CFLAGS=${DEPENDENCY_CFLAGS}
> +            CPPFLAGS=${DEPENDENCY_CPPFLAGS}
> +            LDFLAGS=${DEPENDENCY_LDFLAGS}
> +            --prefix=<INSTALL_DIR>
> +            --disable-shared
> +            --enable-static
> +            --disable-minidebuginfo # to prevent linking with liblzma
> +        STEP_TARGETS download
> +    )
> +endif()
> +
> +# Get install directories of builded libraries for building
> +# tarantool with custon CMAKE_PREFIX_PATH
> +foreach(PROJ openssl icu zlib ncurses readline iconv unwind)
> +    ExternalProject_Get_Property(${PROJ} install_dir)
> +    set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir})
> +    set(TARANTOOL_DEPENDS ${PROJ} ${TARANTOOL_DEPENDS})
> +    message(STATUS "Add external project ${PROJ} in ${install_dir}")
> +endforeach()
> +
> +ExternalProject_Add(tarantool
> +    DEPENDS ${TARANTOOL_DEPENDS}
> +    SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/..
> +    LIST_SEPARATOR :
> +    CMAKE_ARGS
> +        # Override LOCALSTATEDIR to avoid cmake "special" cases:
> +        #
> https://cmake.org/cmake/help/v3.4/module/GNUInstallDirs.html#special-cases
> +        -DCMAKE_INSTALL_LOCALSTATEDIR=<INSTALL_DIR>/var
> +        -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
> +        -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}
> +        -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE
> +        -DOPENSSL_USE_STATIC_LIBS=TRUE
> +        -DBUILD_STATIC=TRUE
> +        -DENABLE_DIST=TRUE
> +        -DENABLE_BACKTRACE=TRUE
> +        -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
> +        -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
> +        ${CMAKE_TARANTOOL_ARGS}
> +    STEP_TARGETS build
> +    BUILD_COMMAND $(MAKE)
> +)
> +
> +enable_testing()
> +ExternalProject_Get_Property(tarantool binary_dir)
> +SET(TARANTOOL_BINARY_DIR ${binary_dir})
> +
> +add_test(
> +    NAME check-dependencies
> +    COMMAND ${CMAKE_COMMAND}
> +        -D FILE=${TARANTOOL_BINARY_DIR}/src/tarantool
> +        -P CheckDependencies.cmake
> +    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
> +)
> +
> +add_test(
> +    NAME check-exports
> +    COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool
> +            ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/exports.test.lua
> +)
> +
> +add_test(
> +    NAME check-traceback
> +    COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool
> +
> ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/traceback.test.lua
> +)
> diff --git a/static-build/README.md b/static-build/README.md
> new file mode 100644
> index 000000000..2219b5026
> --- /dev/null
> +++ b/static-build/README.md
> @@ -0,0 +1,90 @@
> +# Tarantool static build tooling
> +
> +These files help to prepare environment for building Tarantool
> +statically. And builds it.
> +
> +## Prerequisites
> +
> +Please install following tools and libraries that will
> +be necessary for building and testing:
> +* git
> +* A C/C++ compiler.
> +
> +  Ordinarily, this is gcc and g++ version 4.6 or later. On Mac OS X, this
> +  is Clang version 3.2+.
> +* cmake
> +* autoconf automake libtool
> +* make
> +* Python and modules.
> +
> +  Python interpreter is not necessary for building Tarantool itself,
> unless you
> +  intend to use the “Run the test suite". For all platforms, this is
> python
> +  version 2.7+ (but not 3.x). You need the following Python modules:
> +  * pyyaml
> +  * argparse
> +  * msgpack-python
> +  * gevent
> +  * six
> +
> +### Here is an examples for your OS:
> +
> +CentOS:
> +
> +```bash
> +yum install -y \
> +    git perl gcc cmake make gcc-c++ libstdc++-static autoconf automake
> libtool \
> +    python-msgpack python-yaml python-argparse python-six python-gevent
> +```
> +
> +Ubuntu/Debian:
> +
> +```bash
> +apt-get install -y \
> +    build-essential cmake make coreutils autoconf automake libtool sed \
> +    python python-pip python-setuptools python-dev \
> +    python-msgpack python-yaml python-argparse python-six python-gevent
> +```
> +
> +MacOS:
> +
> +Before you start please install default Xcode Tools by Apple:
> +
> +```bash
> +sudo xcode-select --install
> +sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
> +```
> +
> +Install brew using command from
> +[Homebrew repository instructions](https://github.com/Homebrew/inst)
> +
> +After that run next script:
> +
> +```bash
> +brew install autoconf automake libtool cmake
> file://$${PWD}/tools/brew_taps/tntpython2.rbs
> +pip install --force-reinstall -r test-run/requirements.txt
> +```
> +
> +## Usage
> +
> +```bash
> +cmake .
> +make -j
> +ctest -V
> +```
> +
> +## Customize your build
> +
> +If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS`
> variable
> +
> +### Usage
> +
> +There are three types of `CMAKE_BUILD_TYPE`:
> +* Debug - default
> +* Release
> +* RelWithDebInfo
> +
> +And you want to build tarantool with RelWithDebInfo:
> +
> +```bash
> +cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" .
> +```
> diff --git a/static-build/test/CheckDependencies.cmake
> b/static-build/test/CheckDependencies.cmake
> new file mode 100644
> index 000000000..49e91e7fa
> --- /dev/null
> +++ b/static-build/test/CheckDependencies.cmake
> @@ -0,0 +1,43 @@
> +## This is a cmake-based test, it checks that tarantool static binary
> +# has no dependencies except allowed ones.
> +
> +include(GetPrerequisites)
> +if(NOT FILE)
> +    message(FATAL_ERROR "Usage: "
> +        "${CMAKE_COMMAND} -DFILE=<FILENAME> -P CheckDependencies.cmake")
> +elseif(NOT EXISTS ${FILE})
> +    message(FATAL_ERROR "${FILE}: No such file")
> +endif()
> +
> +get_prerequisites(${FILE} DEPENDENCIES 0 0 "" "")
> +
> +if (APPLE)
> +    set(ALLOWLIST
> +        libSystem
> +        CoreFoundation
> +        libc++
> +    )
> +elseif(UNIX)
> +    set(ALLOWLIST
> +        libdl
> +        librt
> +        libc
> +        libm
> +        libgcc_s
> +        libpthread
> +    )
> +else()
> +    message(FATAL_ERROR "Unknown platform")
> +endif()
> +
> +foreach(DEPENDENCY_FILE ${DEPENDENCIES})
> +    message("Dependency: ${DEPENDENCY_FILE}")
> +endforeach()
> +
> +foreach(DEPENDENCY_FILE ${DEPENDENCIES})
> +    get_filename_component(libname ${DEPENDENCY_FILE} NAME_WE)
> +    list (FIND ALLOWLIST ${libname} _index)
> +    if (_index EQUAL -1)
> +        message(FATAL_ERROR "Blocklisted dependency: ${DEPENDENCY_FILE}")
> +    endif()
> +endforeach()
> diff --git a/static-build/test/static-build/exports.test.lua
> b/static-build/test/static-build/exports.test.lua
> new file mode 100755
> index 000000000..de54973d8
> --- /dev/null
> +++ b/static-build/test/static-build/exports.test.lua
> @@ -0,0 +1,142 @@
> +#!/usr/bin/env tarantool
> +
> +local tap = require('tap')
> +local ffi = require('ffi')
> +ffi.cdef([[
> +    void *dlsym(void *handle, const char *symbol);
> +]])
> +
> +local test = tap.test('exports')
> +
> +-- See `man 3 dlsym`:
> +-- RTLD_DEFAULT
> +--   Find the first occurrence of the desired symbol using the default
> +--   shared object search order. The search will include global symbols
> +--   in the executable and its dependencies, as well as symbols in shared
> +--   objects that were dynamically loaded with the RTLD_GLOBAL flag.
> +local RTLD_DEFAULT = ffi.cast("void *", jit.os == "OSX" and -2LL or 0LL)
> +
> +local function check_symbol(sym)
> +    test:ok(ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, ('Symbol %q
> found'):format(sym))
> +end
> +
> +local check_symbols = {
> +    -- FFI
> +
> +    'guava',
> +    'base64_decode',
> +    'base64_encode',
> +    'SHA1internal',
> +    'random_bytes',
> +    'fiber_time',
> +    'ibuf_create',
> +    'ibuf_destroy',
> +    'port_destroy',
> +    'csv_create',
> +    'csv_destroy',
> +    'title_get',
> +    'title_update',
> +    'tnt_iconv',
> +    'tnt_iconv_open',
> +    'tnt_iconv_close',
> +    'exception_get_int',
> +    'exception_get_string',
> +
> +    'tarantool_lua_ibuf',
> +    'uuid_nil',
> +    'tt_uuid_create',
> +    'tt_uuid_str',
> +    'tt_uuid_is_equal',
> +    'tt_uuid_is_nil',
> +    'tt_uuid_bswap',
> +    'tt_uuid_from_string',
> +    'log_level',
> +    'log_format',
> +    'uri_parse',
> +    'uri_format',
> +    'PMurHash32',
> +    'PMurHash32_Process',
> +    'PMurHash32_Result',
> +    'crc32_calc',
> +    'mp_encode_double',
> +    'mp_encode_float',
> +    'mp_encode_decimal',
> +    'mp_decode_double',
> +    'mp_decode_float',
> +    'mp_decode_extl',
> +    'mp_sizeof_decimal',
> +    'decimal_unpack',
> +
> +    'log_type',
> +    'say_set_log_level',
> +    'say_logrotate',
> +    'say_set_log_format',
> +    'tarantool_uptime',
> +    'tarantool_exit',
> +    'log_pid',
> +    'space_by_id',
> +    'space_run_triggers',
> +    'space_bsize',
> +    'box_schema_version',
> +
> +    'crypto_EVP_MD_CTX_new',
> +    'crypto_EVP_MD_CTX_free',
> +    'crypto_HMAC_CTX_new',
> +    'crypto_HMAC_CTX_free',
> +    'crypto_stream_new',
> +    'crypto_stream_begin',
> +    'crypto_stream_append',
> +    'crypto_stream_commit',
> +    'crypto_stream_delete',
> +
> +    -- Module API
> +
> +    '_say',
> +    'swim_cfg',
> +    'swim_quit',
> +    'fiber_new',
> +    'fiber_cancel',
> +    'coio_wait',
> +    'coio_close',
> +    'coio_call',
> +    'coio_getaddrinfo',
> +    'luaT_call',
> +    'box_txn',
> +    'box_select',
> +    'clock_realtime',
> +    'string_strip_helper',
> +
> +    -- Lua / LuaJIT
> +
> +    'lua_newstate',
> +    'lua_close',
> +    'luaL_loadstring',
> +    'luaJIT_profile_start',
> +    'luaJIT_profile_stop',
> +    'luaJIT_profile_dumpstack',
> +
> +    'ERR_error_string',
> +    'ERR_get_error',
> +
> +    'EVP_get_digestbyname',
> +    'EVP_get_cipherbyname',
> +    'EVP_CIPHER_CTX_new',
> +    'EVP_CIPHER_CTX_free',
> +    'EVP_CIPHER_block_size',
> +    'HMAC_Init_ex',
> +    'HMAC_Update',
> +    'HMAC_Final',
> +
> +    'ZSTD_compress',
> +    'ZSTD_decompress',
> +    'ZSTD_free',
> +    'ZSTD_malloc',
> +    'ZSTD_versionString',
> +}
> +
> +test:plan(#check_symbols)
> +for _, sym in ipairs(check_symbols) do
> +    check_symbol(sym)
> +end
> +
> +os.exit(test:check() and 0 or 1)
> diff --git a/static-build/test/static-build/traceback.test.lua
> b/static-build/test/static-build/traceback.test.lua
> new file mode 100755
> index 000000000..32e1ed8b5
> --- /dev/null
> +++ b/static-build/test/static-build/traceback.test.lua
> @@ -0,0 +1,15 @@
> +#!/usr/bin/env tarantool
> +
> +local tap = require('tap')
> +local fiber = require('fiber')
> +
> +local test = tap.test('traceback')
> +test:plan(1)
> +
> +local info = fiber.info()[fiber.id()]
> +test:ok(info.backtrace ~= nil, 'fiber.info() has backtrace')
> +for _, l in pairs(info.backtrace or {}) do
> +    test:diag('%s: %s', next(l))
> +end
> +
> +os.exit(test:check() and 0 or 1)
> --
> 2.26.2
>
>

[-- Attachment #2: Type: text/html, Size: 47266 bytes --]