[Tarantool-patches] [PATCH v3] build: refactor static build process
HustonMmmavr
huston.mavr at gmail.com
Wed Sep 9 18:45:46 MSK 2020
From: Yaroslav Dynnikov <yaroslav.dynnikov at gmail.com>
Refactored static build process to use static-build/CMakeLists.txt
instead of Dockerfile.staticbuild (this allows to support static
build on macOS). Following third-party dependencies for static build
are installed via cmake `ExternalProject_Add`:
- OpenSSL
- Zlib
- Ncurses
- Readline
- Unwind
- ICU
* Added support static build for macOS
* Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
build at file cmake/BuildLibCURL.cmake:
- disable building shared libcurl libraries (by setting
`--disable-shared` option)
- disable hiding libcurl symbols (by setting
`--disable-symbol-hiding` option)
- prevent linking libcurl with system libz (by setting
`--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
* Removed Dockerfile.staticbuild
* Added new gitlab.ci jobs to test new style static build:
- static_build_cmake_linux
- static_build_cmake_osx_15
* Removed static_docker_build gitlab.ci job
Closes #5095
---
Branch: https://github.com/tarantool/tarantool/tree/rosik/refactor-static-build
Issue: https://github.com/tarantool/tarantool/issues/5095
Changes in v3:
1. Fixed typos at commit message
2. Fixed commentaries at .travis.mk
3. Fixed finding libz library at cmake/BuildLibCurl.cmake as at master
branch
4. Instead of using static-build/.gitignore use the root .gitignore file
5. Refactored static-build/CMakeLists.txt:
* Project name is set to tarantool-static
* Instead of explicitly finding of c/c++ compilers current version uses
cmake `project(tarantool-static C CXX)` command. This change requires
to manually set CFLAGS/CPPFLAGS/LDFLAGS for all static-build
dependencies (because building dependencies at macOS requires path
to macOS SDK) the idea was taken from cmake/BuildLibCurl.cmake
* Added commentaries about problem with libicu
* Removed unused TEST_COMMAND at ExternalProject_Add(zlib ...) also
removed unnecessary # STEP_TARGETS at ExternalProject_Add(readline ...)
* Deleted doubled whitespaces
6. Fixed static-build/README.md:
* Added list of required tools
* Added example for ubuntu/debian
* Fixed indentation and typos
7. Refactored tests:
* Got rid of test-run
* Deleted doubled whitespaces at exports.test.lua
* Fixed defining of RTLD_DEFAULT
.gitignore | 8 +
.gitlab-ci.yml | 11 +-
.travis.mk | 56 +++-
Dockerfile.staticbuild | 98 ------
cmake/BuildLibCURL.cmake | 13 +-
cmake/compiler.cmake | 24 +-
cmake/os.cmake | 5 +-
static-build/CMakeLists.txt | 311 ++++++++++++++++++
static-build/README.md | 90 +++++
static-build/test/CheckDependencies.cmake | 43 +++
.../test/static-build/exports.test.lua | 142 ++++++++
.../test/static-build/traceback.test.lua | 15 +
12 files changed, 692 insertions(+), 124 deletions(-)
delete mode 100644 Dockerfile.staticbuild
create mode 100644 static-build/CMakeLists.txt
create mode 100644 static-build/README.md
create mode 100644 static-build/test/CheckDependencies.cmake
create mode 100755 static-build/test/static-build/exports.test.lua
create mode 100755 static-build/test/static-build/traceback.test.lua
diff --git a/.gitignore b/.gitignore
index a42c7db35..c6e261e18 100644
--- a/.gitignore
+++ b/.gitignore
@@ -151,3 +151,11 @@ src/box/sql/parse.h
src/box/sql/parse.c
src/box/sql/opcodes.h
src/box/sql/opcodes.c
+
+# static-build
+static-build/*-prefix
+static-build/Makefile
+static-build/build
+
+# macOS files
+.DS_Store
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 0ead08711..c9aef3dc7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -534,9 +534,14 @@ static_build:
script:
- ${GITLAB_MAKE} test_static_build
-static_docker_build:
+static_build_cmake_linux:
+ <<: *docker_test_definition
+ script:
+ - ${GITLAB_MAKE} test_static_build_cmake_linux
+
+static_build_cmake_osx_15:
stage: test
tags:
- - deploy_test
+ - osx_15
script:
- - ${GITLAB_MAKE} test_static_docker_build
+ - ${GITLAB_MAKE} test_static_build_cmake_osx
diff --git a/.travis.mk b/.travis.mk
index efc05cf05..58d0c1596 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -148,8 +148,14 @@ deps_debian_static:
test_static_build: deps_debian_static
CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk test_debian_no_deps
-test_static_docker_build:
- docker build --no-cache --network=host --build-arg RUN_TESTS=ON -f Dockerfile.staticbuild .
+# New static build
+# builddir used in this target - is a default build path from cmake
+# ExternalProject_Add()
+test_static_build_cmake_linux:
+ cd static-build && cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" . && \
+ make -j && ctest -V
+ cd test && /usr/bin/python test-run.py --force \
+ --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build $(TEST_RUN_EXTRA_PARAMS)
# ###################
# Static Analysis
@@ -193,15 +199,16 @@ build_osx:
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
make -j
-test_osx_no_deps: build_osx
- # Limits: Increase the maximum number of open file descriptors on macOS:
- # Travis-ci needs the "ulimit -n <value>" call
- # Gitlab-ci needs the "launchctl limit maxfiles <value>" call
- # Also gitlib-ci needs the password to change the limits, while
- # travis-ci runs under root user. Limit setup must be in the same
- # call as tests runs call.
- # Tests: Temporary excluded replication/ suite with some tests
- # from other suites by issues #4357 and #4370
+
+# Limits: Increase the maximum number of open file descriptors on macOS:
+# Travis-ci needs the "ulimit -n <value>" call
+# Gitlab-ci needs the "launchctl limit maxfiles <value>" call
+# Also gitlib-ci needs the password to change the limits, while
+# travis-ci runs under root user. Limit setup must be in the same
+# call as tests runs call.
+# Tests: Temporary excluded replication/ suite with some tests
+# from other suites by issues #4357 and #4370
+INIT_TEST_ENV_OSX=\
sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
launchctl limit maxfiles || : ; \
ulimit -n ${MAX_FILES} || : ; \
@@ -210,11 +217,34 @@ test_osx_no_deps: build_osx
launchctl limit maxproc || : ; \
ulimit -u ${MAX_PROC} || : ; \
ulimit -u ; \
- rm -rf /tmp/tnt ; \
- cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
+ rm -rf /tmp/tnt
+
+test_osx_no_deps: build_osx
+ ${INIT_TEST_ENV_OSX}; \
+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
test_osx: deps_osx test_osx_no_deps
+# Static macOS build
+
+STATIC_OSX_PKGS=autoconf automake libtool cmake file://$${PWD}/tools/brew_taps/tntpython2.rb
+base_deps_osx:
+ brew update || echo | /usr/bin/ruby -e \
+ "$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+ brew install --force ${STATIC_OSX_PKGS} || brew upgrade ${STATIC_OSX_PKGS}
+ pip install --force-reinstall -r test-run/requirements.txt
+
+# builddir used in this target - is a default build path from cmake
+# ExternalProject_Add()
+test_static_build_cmake_osx: base_deps_osx
+ cd static-build && cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo;-DENABLE_WERROR=ON" . && \
+ make -j && ctest -V
+ ${INIT_TEST_ENV_OSX}; \
+ cd test && ./test-run.py --vardir /tmp/tnt \
+ --builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \
+ --force $(TEST_RUN_EXTRA_PARAMS)
+
+
###########
# FreeBSD #
###########
diff --git a/Dockerfile.staticbuild b/Dockerfile.staticbuild
deleted file mode 100644
index f67f46f5e..000000000
--- a/Dockerfile.staticbuild
+++ /dev/null
@@ -1,98 +0,0 @@
-FROM centos:7
-
-RUN yum install -y epel-release
-RUN yum install -y yum install https://centos7.iuscommunity.org/ius-release.rpm
-
-RUN set -x \
- && yum -y install \
- libstdc++ \
- libstdc++-static \
- readline \
- openssl \
- lz4 \
- binutils \
- ncurses \
- libgomp \
- lua \
- curl \
- tar \
- zip \
- unzip \
- libunwind \
- zlib \
- && yum -y install \
- perl \
- gcc-c++ \
- cmake \
- lz4-devel \
- binutils-devel \
- lua-devel \
- make \
- git \
- autoconf \
- automake \
- libtool \
- wget
-
-RUN yum -y install ncurses-static readline-static zlib-static pcre-static glibc-static
-
-RUN yum -y install python-devel python-pip
-
-RUN set -x && \
- cd / && \
- curl -O -L https://www.openssl.org/source/openssl-1.1.1f.tar.gz && \
- tar -xvf openssl-1.1.1f.tar.gz && \
- cd openssl-1.1.1f && \
- ./config --libdir=lib && \
- make -j && make install
-
-RUN set -x && \
- cd / && \
- curl -O -L https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz && \
- tar -xvf icu4c-62_1-src.tgz && \
- cd icu/source && \
- ./configure --with-data-packaging=static --enable-static --enable-shared && \
- make -j && make install
-
-RUN set -x && \
- cd / && \
- curl -O -L http://download.savannah.nongnu.org/releases/libunwind/libunwind-1.3-rc1.tar.gz && \
- tar -xvf libunwind-1.3-rc1.tar.gz && \
- cd libunwind-1.3-rc1 && \
- ./configure --enable-static --enable-shared && \
- make -j && make install
-
-COPY . /tarantool
-
-WORKDIR /tarantool
-
-RUN set -x && \
- git submodule init && \
- git submodule update
-
-# Cleanup for 'build' directory added, because it purges all artefacts
-# produced for curl build, including the old configuration in build/curl
-RUN set -x && \
- find . -name 'CMakeFiles' -type d -exec rm -rf {} + && \
- find . -name 'CMakeCache.txt' -type f -delete && \
- rm -rf build test/small test/luajit-tap
-
-RUN pip install -r /tarantool/test-run/requirements.txt
-
-RUN set -x && \
- cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
- -DENABLE_DIST:BOOL=ON \
- -DBUILD_STATIC=ON \
- -DOPENSSL_USE_STATIC_LIBS=ON \
- -DOPENSSL_ROOT_DIR=/usr/local \
- . && \
- make -j && make install
-
-ARG RUN_TESTS
-RUN if [ -n "${RUN_TESTS}" ]; then \
- set -x && \
- cd test && \
- /usr/bin/python test-run.py --force; \
- fi
-
-ENTRYPOINT ["/bin/bash"]
diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
index 5f8b15a63..86fec39e9 100644
--- a/cmake/BuildLibCURL.cmake
+++ b/cmake/BuildLibCURL.cmake
@@ -4,15 +4,21 @@ macro(curl_build)
set(LIBCURL_BINARY_DIR ${PROJECT_BINARY_DIR}/build/curl/work)
set(LIBCURL_INSTALL_DIR ${PROJECT_BINARY_DIR}/build/curl/dest)
+ message(STATUS "Looking for zlib")
+ find_path(ZLIB_INCLUDE_DIR zlib.h)
+ message(STATUS "Looking for zlib.h - ${ZLIB_INCLUDE_DIR}")
if (BUILD_STATIC)
set(LIBZ_LIB_NAME libz.a)
else()
set(LIBZ_LIB_NAME z)
endif()
find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME})
- if ("${LIBZ_LIBRARY}" STREQUAL "LIBZ_LIBRARY-NOTFOUND")
+ message(STATUS "Looking for libz - ${LIBZ_LIBRARY}")
+
+ if (NOT ZLIB_INCLUDE_DIR OR NOT LIBZ_LIBRARY)
message(FATAL_ERROR "Unable to find zlib")
endif()
+ get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} DIRECTORY)
# Use the same OpenSSL library for libcurl as is used for
# tarantool itself.
@@ -88,9 +94,10 @@ macro(curl_build)
--prefix <INSTALL_DIR>
--enable-static
- --enable-shared
+ --disable-shared
+ --disable-symbol-hiding
- --with-zlib
+ --with-zlib=${FOUND_ZLIB_ROOT_DIR}
${LIBCURL_OPENSSL_OPT}
--with-ca-fallback
diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake
index 5a1141ebd..db2ae6227 100644
--- a/cmake/compiler.cmake
+++ b/cmake/compiler.cmake
@@ -33,10 +33,6 @@ if (CMAKE_COMPILER_IS_GNUCC)
Your GCC version is ${CMAKE_CXX_COMPILER_VERSION}, please update
")
endif()
-else()
- if (BUILD_STATIC)
- message(FATAL_ERROR "Static build is supported for GCC only")
- endif()
endif()
#
@@ -120,10 +116,23 @@ set (CMAKE_CXX_FLAGS_RELWITHDEBINFO
unset(CC_DEBUG_OPT)
+message(STATUS "Looking for libunwind.h")
+find_path(UNWIND_INCLUDE_DIR libunwind.h)
+message(STATUS "Looking for libunwind.h - ${UNWIND_INCLUDE_DIR}")
+
+if (UNWIND_INCLUDE_DIR)
+ include_directories(${UNWIND_INCLUDE_DIR})
+endif()
+
+set(CMAKE_REQUIRED_INCLUDES ${UNWIND_INCLUDE_DIR})
check_include_file(libunwind.h HAVE_LIBUNWIND_H)
-if(BUILD_STATIC)
+set(CMAKE_REQUIRED_INCLUDES "")
+
+if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
set(UNWIND_LIB_NAME libunwind.a)
else()
+ # libunwind can't be compiled on macOS.
+ # But there exists libunwind.dylib as a part of MacOSSDK
set(UNWIND_LIB_NAME unwind)
endif()
find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME})
@@ -185,7 +194,10 @@ if (ENABLE_BACKTRACE)
find_package_message(UNWIND_LIBRARIES "Found unwind" "${UNWIND_LIBRARIES}")
endif()
-if(BUILD_STATIC)
+# On macOS there is no '-static-libstdc++' flag and it's use will
+# raise following error:
+# error: argument unused during compilation: '-static-libstdc++'
+if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
# Static linking for c++ routines
add_compile_flags("C;CXX" "-static-libstdc++")
endif()
diff --git a/cmake/os.cmake b/cmake/os.cmake
index 905be61df..276a79b42 100644
--- a/cmake/os.cmake
+++ b/cmake/os.cmake
@@ -107,7 +107,10 @@ elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Darwin")
# Latest versions of Homebrew wont 'link --force' for libraries, that were
# preinstalled in system. So we'll use this dirty hack
- find_program(HOMEBREW_EXECUTABLE brew)
+
+ if (NOT BUILD_STATIC)
+ find_program(HOMEBREW_EXECUTABLE brew)
+ endif()
if(EXISTS ${HOMEBREW_EXECUTABLE})
execute_process(COMMAND ${HOMEBREW_EXECUTABLE} --prefix
OUTPUT_VARIABLE HOMEBREW_PREFIX
diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
new file mode 100644
index 000000000..90029fdd8
--- /dev/null
+++ b/static-build/CMakeLists.txt
@@ -0,0 +1,311 @@
+cmake_minimum_required(VERSION 2.8)
+
+# Detect system compilers for further configuring dependencies to be
+# builded with these compilers. This is used to build tarantool and
+# it's dependencies by usign one compiler system (for example libicu
+# by default uses clang if it exists when others uses gcc/g++ on
+# linux machine).
+project(tarantool-static C CXX)
+
+include(ExternalProject)
+set(OPENSSL_VERSION 1.1.1f)
+set(ZLIB_VERSION 1.2.11)
+set(NCURSES_VERSION 6.2)
+set(READLINE_VERSION 8.0)
+set(UNWIND_VERSION 1.3-rc1)
+
+# Pass -isysroot=<SDK_PATH> option on Mac OS to a preprocessor and a C
+# compiler to find header files installed with an SDK.
+#
+# The idea is to set these (DEPENDENCY_*) variables to corresponding
+# environment variables at each depenency configure script.
+#
+# Note: Passing of CPPFLAGS / CFLAGS explicitly discards using of
+# corresponsing environment variables. So pass empty LDFLAGS to discard
+# using of corresponding environment variable. It is possible that a
+# linker flag assumes that some compilation flag is set. We don't pass
+# CFLAGS from environment, so we should not do it for LDFLAGS too.
+set(DEPENDENCY_CFLAGS "")
+set(DEPENDENCY_CPPFLAGS "")
+set(DEPENDENCY_LDFLAGS)
+if (APPLE)
+ set(DEPENDENCY_CFLAGS "${CMAKE_C_SYSROOT_FLAG} ${CMAKE_OSX_SYSROOT}")
+ set(DEPENDENCY_CPPFLAGS "${CMAKE_C_SYSROOT_FLAG} ${CMAKE_OSX_SYSROOT}")
+endif()
+
+# Install all libraries required by tarantool at current build dir
+
+#
+# OpenSSL
+#
+ExternalProject_Add(openssl
+ URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
+ CONFIGURE_COMMAND <SOURCE_DIR>/config
+ CC=${CMAKE_C_COMPILER}
+ CXX=${CMAKE_CXX_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+
+ --prefix=<INSTALL_DIR>
+ --libdir=lib
+ no-shared
+ INSTALL_COMMAND ${CMAKE_MAKE_PROGRAM} install_sw
+)
+
+#
+# ICU
+#
+ExternalProject_Add(icu
+ URL https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz
+ # By default libicu is built by using clang/clang++ compiler (if it
+ # exists). Here is a link for detecting compilers at libicu configure
+ # script: https://github.com/unicode-org/icu/blob/7c7b8bd5702310b972f888299169bc3cc88bf0a6/icu4c/source/configure.ac#L135
+ # This will cause the problem on linux machine: tarantool is built
+ # with gcc/g++ and libicu is built with clang/clang++ (if it exists)
+ # so at linking stage `rellocation` errors will occur. To solve this,
+ # we can set CC/CXX to CMAKE_C_COMPILER/CMAKE_CXX_COMPILER variables
+ # manually which are detected above (by cmake `project()` command)
+ CONFIGURE_COMMAND <SOURCE_DIR>/source/configure
+ CC=${CMAKE_C_COMPILER}
+ CXX=${CMAKE_CXX_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+
+ --with-data-packaging=static
+ --prefix=<INSTALL_DIR>
+ --disable-shared
+ --enable-static
+)
+
+#
+# ZLIB
+#
+ExternalProject_Add(zlib
+ URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
+ CONFIGURE_COMMAND env
+ CC=${CMAKE_C_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+ <SOURCE_DIR>/configure
+ --prefix=<INSTALL_DIR>
+ --static
+)
+
+#
+# Ncurses
+#
+ExternalProject_Add(ncurses
+ URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz
+ CONFIGURE_COMMAND <SOURCE_DIR>/configure
+ CC=${CMAKE_C_COMPILER}
+ CXX=${CMAKE_CXX_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+
+ --prefix=<INSTALL_DIR>
+
+ # This flag enables creation of libcurses.a as a symlink to libncurses.a
+ # and disables subdir creation `ncurses` at <install_dir>/include. It is
+ # necessary for correct work of FindCurses.cmake module (this module is
+ # builtin at cmake package) which used in cmake/FindReadline.cmake
+ --enable-overwrite
+
+ # enable building libtinfo to prevent linking with libtinfo from system
+ # directories
+ --with-termlib
+
+ # set search paths for terminfo db
+ --with-terminfo-dirs=/lib/terminfo:/usr/share/terminfo:/etc/terminfo
+
+ # disable install created terminfo db, use db from system
+ --disable-db-install
+ --without-progs
+ --without-manpages
+)
+
+#
+# ReadLine
+#
+ExternalProject_Add(readline
+ URL https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz
+ CONFIGURE_COMMAND <SOURCE_DIR>/configure
+ CC=${CMAKE_C_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+
+ --prefix=<INSTALL_DIR>
+ --disable-shared
+)
+
+#
+# ICONV
+#
+if (APPLE)
+ ExternalProject_Add(iconv
+ URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
+ CONFIGURE_COMMAND <SOURCE_DIR>/configure
+ CC=${CMAKE_C_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+
+ --prefix=<INSTALL_DIR>
+ --disable-shared
+ --enable-static
+ --with-gnu-ld
+ STEP_TARGETS download
+ )
+else()
+ # In linux iconv is embedded into glibc
+ # So we find system header and copy it locally
+ find_path(ICONV_INCLUDE_DIR iconv.h)
+ if(NOT ICONV_INCLUDE_DIR)
+ message(FATAL_ERROR "iconv include header not found")
+ endif()
+
+ set(ICONV_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix")
+
+ add_custom_command(
+ OUTPUT "${ICONV_INSTALL_PREFIX}/include/iconv.h"
+ COMMAND ${CMAKE_COMMAND} -E make_directory
+ "${ICONV_INSTALL_PREFIX}/include"
+ COMMAND ${CMAKE_COMMAND} -E copy
+ "${ICONV_INCLUDE_DIR}/iconv.h"
+ "${ICONV_INSTALL_PREFIX}/include/iconv.h"
+ )
+ add_custom_target(iconv
+ DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h"
+ )
+ # This is a hack for further getting install directory of library
+ # by ExternalProject_Get_Property
+ set_target_properties(iconv
+ PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX}
+ )
+endif()
+
+#
+# Unwind
+#
+if (APPLE)
+ # On macOS libunwind is a part of MacOSX.sdk
+ # So we need to find library and header and
+ # copy it locally
+ find_path(UNWIND_INCLUDE_DIR libunwind.h)
+ find_library(UNWIND_LIBRARY libunwind.dylib
+ PATH_SUFFIXES system
+ )
+
+ set(UNWIND_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/unwind-prefix")
+
+ set(UNWIND_DEPENDENCIES)
+
+ if (UNWIND_INCLUDE_DIR AND UNWIND_LIBRARY)
+ add_custom_command(
+ OUTPUT "${UNWIND_INSTALL_PREFIX}/include/unwind.h"
+ OUTPUT "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
+ COMMAND ${CMAKE_COMMAND} -E make_directory
+ "${UNWIND_INSTALL_PREFIX}/include"
+ COMMAND ${CMAKE_COMMAND} -E copy
+ "${UNWIND_INCLUDE_DIR}/libunwind.h"
+ "${UNWIND_INCLUDE_DIR}/unwind.h"
+ "${UNWIND_INSTALL_PREFIX}/include/"
+ )
+ add_custom_command(
+ OUTPUT "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
+ COMMAND ${CMAKE_COMMAND} -E make_directory
+ "${UNWIND_INSTALL_PREFIX}/lib"
+ COMMAND ${CMAKE_COMMAND} -E copy
+ "${UNWIND_LIBRARY}"
+ "${UNWIND_INSTALL_PREFIX}/lib/"
+ )
+ set(UNWIND_DEPENDENCIES
+ ${UNWIND_DEPENDENCIES}
+ "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
+ "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
+ )
+ else()
+ message(STATUS "Unwind not found")
+ endif()
+
+ add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES})
+ # This is a hack for further getting install directory of library
+ # by ExternalProject_Get_Property
+ set_target_properties(unwind
+ PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX}
+ )
+else()
+ ExternalProject_Add(unwind
+ URL https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz
+ CONFIGURE_COMMAND <SOURCE_DIR>/configure
+ CC=${CMAKE_C_COMPILER}
+ CXX=${CMAKE_CXX_COMPILER}
+ CFLAGS=${DEPENDENCY_CFLAGS}
+ CPPFLAGS=${DEPENDENCY_CPPFLAGS}
+ LDFLAGS=${DEPENDENCY_LDFLAGS}
+ --prefix=<INSTALL_DIR>
+ --disable-shared
+ --enable-static
+ --disable-minidebuginfo # to prevent linking with liblzma
+ STEP_TARGETS download
+ )
+endif()
+
+# Get install directories of builded libraries for building
+# tarantool with custon CMAKE_PREFIX_PATH
+foreach(PROJ openssl icu zlib ncurses readline iconv unwind)
+ ExternalProject_Get_Property(${PROJ} install_dir)
+ set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir})
+ set(TARANTOOL_DEPENDS ${PROJ} ${TARANTOOL_DEPENDS})
+ message(STATUS "Add external project ${PROJ} in ${install_dir}")
+endforeach()
+
+ExternalProject_Add(tarantool
+ DEPENDS ${TARANTOOL_DEPENDS}
+ SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/..
+ LIST_SEPARATOR :
+ CMAKE_ARGS
+ # Override LOCALSTATEDIR to avoid cmake "special" cases:
+ # https://cmake.org/cmake/help/v3.4/module/GNUInstallDirs.html#special-cases
+ -DCMAKE_INSTALL_LOCALSTATEDIR=<INSTALL_DIR>/var
+ -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
+ -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}
+ -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE
+ -DOPENSSL_USE_STATIC_LIBS=TRUE
+ -DBUILD_STATIC=TRUE
+ -DENABLE_DIST=TRUE
+ -DENABLE_BACKTRACE=TRUE
+ -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+ -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+ ${CMAKE_TARANTOOL_ARGS}
+ STEP_TARGETS build
+ BUILD_COMMAND $(MAKE)
+)
+
+enable_testing()
+ExternalProject_Get_Property(tarantool binary_dir)
+SET(TARANTOOL_BINARY_DIR ${binary_dir})
+
+add_test(
+ NAME check-dependencies
+ COMMAND ${CMAKE_COMMAND}
+ -D FILE=${TARANTOOL_BINARY_DIR}/src/tarantool
+ -P CheckDependencies.cmake
+ WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
+)
+
+add_test(
+ NAME check-exports
+ COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool
+ ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/exports.test.lua
+)
+
+add_test(
+ NAME check-traceback
+ COMMAND ${TARANTOOL_BINARY_DIR}/src/tarantool
+ ${CMAKE_CURRENT_SOURCE_DIR}/test/static-build/traceback.test.lua
+)
diff --git a/static-build/README.md b/static-build/README.md
new file mode 100644
index 000000000..2219b5026
--- /dev/null
+++ b/static-build/README.md
@@ -0,0 +1,90 @@
+# Tarantool static build tooling
+
+These files help to prepare environment for building Tarantool
+statically. And builds it.
+
+## Prerequisites
+
+Please install following tools and libraries that will
+be necessary for building and testing:
+* git
+* A C/C++ compiler.
+
+ Ordinarily, this is gcc and g++ version 4.6 or later. On Mac OS X, this
+ is Clang version 3.2+.
+* cmake
+* autoconf automake libtool
+* make
+* Python and modules.
+
+ Python interpreter is not necessary for building Tarantool itself, unless you
+ intend to use the “Run the test suite". For all platforms, this is python
+ version 2.7+ (but not 3.x). You need the following Python modules:
+ * pyyaml
+ * argparse
+ * msgpack-python
+ * gevent
+ * six
+
+### Here is an examples for your OS:
+
+CentOS:
+
+```bash
+yum install -y \
+ git perl gcc cmake make gcc-c++ libstdc++-static autoconf automake libtool \
+ python-msgpack python-yaml python-argparse python-six python-gevent
+```
+
+Ubuntu/Debian:
+
+```bash
+apt-get install -y \
+ build-essential cmake make coreutils autoconf automake libtool sed \
+ python python-pip python-setuptools python-dev \
+ python-msgpack python-yaml python-argparse python-six python-gevent
+```
+
+MacOS:
+
+Before you start please install default Xcode Tools by Apple:
+
+```bash
+sudo xcode-select --install
+sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
+```
+
+Install brew using command from
+[Homebrew repository instructions](https://github.com/Homebrew/inst)
+
+After that run next script:
+
+```bash
+brew install autoconf automake libtool cmake file://$${PWD}/tools/brew_taps/tntpython2.rbs
+pip install --force-reinstall -r test-run/requirements.txt
+```
+
+## Usage
+
+```bash
+cmake .
+make -j
+ctest -V
+```
+
+## Customize your build
+
+If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS` variable
+
+### Usage
+
+There are three types of `CMAKE_BUILD_TYPE`:
+* Debug - default
+* Release
+* RelWithDebInfo
+
+And you want to build tarantool with RelWithDebInfo:
+
+```bash
+cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" .
+```
diff --git a/static-build/test/CheckDependencies.cmake b/static-build/test/CheckDependencies.cmake
new file mode 100644
index 000000000..49e91e7fa
--- /dev/null
+++ b/static-build/test/CheckDependencies.cmake
@@ -0,0 +1,43 @@
+## This is a cmake-based test, it checks that tarantool static binary
+# has no dependencies except allowed ones.
+
+include(GetPrerequisites)
+if(NOT FILE)
+ message(FATAL_ERROR "Usage: "
+ "${CMAKE_COMMAND} -DFILE=<FILENAME> -P CheckDependencies.cmake")
+elseif(NOT EXISTS ${FILE})
+ message(FATAL_ERROR "${FILE}: No such file")
+endif()
+
+get_prerequisites(${FILE} DEPENDENCIES 0 0 "" "")
+
+if (APPLE)
+ set(ALLOWLIST
+ libSystem
+ CoreFoundation
+ libc++
+ )
+elseif(UNIX)
+ set(ALLOWLIST
+ libdl
+ librt
+ libc
+ libm
+ libgcc_s
+ libpthread
+ )
+else()
+ message(FATAL_ERROR "Unknown platform")
+endif()
+
+foreach(DEPENDENCY_FILE ${DEPENDENCIES})
+ message("Dependency: ${DEPENDENCY_FILE}")
+endforeach()
+
+foreach(DEPENDENCY_FILE ${DEPENDENCIES})
+ get_filename_component(libname ${DEPENDENCY_FILE} NAME_WE)
+ list (FIND ALLOWLIST ${libname} _index)
+ if (_index EQUAL -1)
+ message(FATAL_ERROR "Blocklisted dependency: ${DEPENDENCY_FILE}")
+ endif()
+endforeach()
diff --git a/static-build/test/static-build/exports.test.lua b/static-build/test/static-build/exports.test.lua
new file mode 100755
index 000000000..de54973d8
--- /dev/null
+++ b/static-build/test/static-build/exports.test.lua
@@ -0,0 +1,142 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local ffi = require('ffi')
+ffi.cdef([[
+ void *dlsym(void *handle, const char *symbol);
+]])
+
+local test = tap.test('exports')
+
+-- See `man 3 dlsym`:
+-- RTLD_DEFAULT
+-- Find the first occurrence of the desired symbol using the default
+-- shared object search order. The search will include global symbols
+-- in the executable and its dependencies, as well as symbols in shared
+-- objects that were dynamically loaded with the RTLD_GLOBAL flag.
+local RTLD_DEFAULT = ffi.cast("void *", jit.os == "OSX" and -2LL or 0LL)
+
+local function check_symbol(sym)
+ test:ok(ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, ('Symbol %q found'):format(sym))
+end
+
+local check_symbols = {
+ -- FFI
+
+ 'guava',
+ 'base64_decode',
+ 'base64_encode',
+ 'SHA1internal',
+ 'random_bytes',
+ 'fiber_time',
+ 'ibuf_create',
+ 'ibuf_destroy',
+ 'port_destroy',
+ 'csv_create',
+ 'csv_destroy',
+ 'title_get',
+ 'title_update',
+ 'tnt_iconv',
+ 'tnt_iconv_open',
+ 'tnt_iconv_close',
+ 'exception_get_int',
+ 'exception_get_string',
+
+ 'tarantool_lua_ibuf',
+ 'uuid_nil',
+ 'tt_uuid_create',
+ 'tt_uuid_str',
+ 'tt_uuid_is_equal',
+ 'tt_uuid_is_nil',
+ 'tt_uuid_bswap',
+ 'tt_uuid_from_string',
+ 'log_level',
+ 'log_format',
+ 'uri_parse',
+ 'uri_format',
+ 'PMurHash32',
+ 'PMurHash32_Process',
+ 'PMurHash32_Result',
+ 'crc32_calc',
+ 'mp_encode_double',
+ 'mp_encode_float',
+ 'mp_encode_decimal',
+ 'mp_decode_double',
+ 'mp_decode_float',
+ 'mp_decode_extl',
+ 'mp_sizeof_decimal',
+ 'decimal_unpack',
+
+ 'log_type',
+ 'say_set_log_level',
+ 'say_logrotate',
+ 'say_set_log_format',
+ 'tarantool_uptime',
+ 'tarantool_exit',
+ 'log_pid',
+ 'space_by_id',
+ 'space_run_triggers',
+ 'space_bsize',
+ 'box_schema_version',
+
+ 'crypto_EVP_MD_CTX_new',
+ 'crypto_EVP_MD_CTX_free',
+ 'crypto_HMAC_CTX_new',
+ 'crypto_HMAC_CTX_free',
+ 'crypto_stream_new',
+ 'crypto_stream_begin',
+ 'crypto_stream_append',
+ 'crypto_stream_commit',
+ 'crypto_stream_delete',
+
+ -- Module API
+
+ '_say',
+ 'swim_cfg',
+ 'swim_quit',
+ 'fiber_new',
+ 'fiber_cancel',
+ 'coio_wait',
+ 'coio_close',
+ 'coio_call',
+ 'coio_getaddrinfo',
+ 'luaT_call',
+ 'box_txn',
+ 'box_select',
+ 'clock_realtime',
+ 'string_strip_helper',
+
+ -- Lua / LuaJIT
+
+ 'lua_newstate',
+ 'lua_close',
+ 'luaL_loadstring',
+ 'luaJIT_profile_start',
+ 'luaJIT_profile_stop',
+ 'luaJIT_profile_dumpstack',
+
+ 'ERR_error_string',
+ 'ERR_get_error',
+
+ 'EVP_get_digestbyname',
+ 'EVP_get_cipherbyname',
+ 'EVP_CIPHER_CTX_new',
+ 'EVP_CIPHER_CTX_free',
+ 'EVP_CIPHER_block_size',
+ 'HMAC_Init_ex',
+ 'HMAC_Update',
+ 'HMAC_Final',
+
+ 'ZSTD_compress',
+ 'ZSTD_decompress',
+ 'ZSTD_free',
+ 'ZSTD_malloc',
+ 'ZSTD_versionString',
+}
+
+test:plan(#check_symbols)
+for _, sym in ipairs(check_symbols) do
+ check_symbol(sym)
+end
+
+os.exit(test:check() and 0 or 1)
diff --git a/static-build/test/static-build/traceback.test.lua b/static-build/test/static-build/traceback.test.lua
new file mode 100755
index 000000000..32e1ed8b5
--- /dev/null
+++ b/static-build/test/static-build/traceback.test.lua
@@ -0,0 +1,15 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local fiber = require('fiber')
+
+local test = tap.test('traceback')
+test:plan(1)
+
+local info = fiber.info()[fiber.id()]
+test:ok(info.backtrace ~= nil, 'fiber.info() has backtrace')
+for _, l in pairs(info.backtrace or {}) do
+ test:diag('%s: %s', next(l))
+end
+
+os.exit(test:check() and 0 or 1)
--
2.26.2
More information about the Tarantool-patches
mailing list