Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v2] build: refactor static build process
@ 2020-08-25 14:01 HustonMmmavr
  2020-08-25 14:51 ` Alexander Turenko
  2020-09-04 17:41 ` Igor Munkin
  0 siblings, 2 replies; 6+ messages in thread
From: HustonMmmavr @ 2020-08-25 14:01 UTC (permalink / raw)
  To: tarantool-patches, yaroslav.dynnikov, alexander.turenko, imun

From: Yaroslav Dynnikov <yaroslav.dynnikov@gmail.com>

Refactored static build process to use static-build/CMakeLists.txt
instead of Dockerfile.staticbuild (this allows to support static
build on macOS). Following third-party dependencies for static build
are installed via cmake `ExternalProject_Add`:
  - OpenSSL
  - Zlib
  - Ncurses
  - Readline
  - Unwind
  - ICU

* Added support static build for macOS
* Prevented linking tarantool binary with system libcurses (which is
  entire copy of libncurses) by setting flag `CURSES_NEED_NCURSES`
  to TRUE at file cmake/FindReadline.cmake. (This flag defined at
  FindCurses.cmake module and it's a workaround for linking with
  correct library)
* Fixed `CONFIGURE_COMMAND` while building bundled libcurl for staic
  build at file cmake/BuildLibCURL.cmake:
    - disable building shared libcurl libraries (by setting
      `--disable-shared` option)
    - disable hiding libcurl symbols (by setting
      `--disable-symbol-hiding` option)
    - prevent linking libcurl with system libz by settign
      `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
* Removed Dockerfile.staticbuild
* Added new gitlab.ci jobs to test new style static build:
  - static_build_cmake_linux
  - static_build_cmake_osx_15
* Removed static_docker_build gitlab.ci job

Closes #5095
---
 .gitlab-ci.yml                                |  11 +-
 .travis.mk                                    |  52 +++-
 Dockerfile.staticbuild                        |  98 -------
 cmake/BuildLibCURL.cmake                      |  18 +-
 cmake/FindReadline.cmake                      |  10 +
 cmake/compiler.cmake                          |  24 +-
 cmake/os.cmake                                |   5 +-
 static-build/.gitignore                       |   4 +
 static-build/CMakeLists.txt                   | 240 ++++++++++++++++++
 static-build/README.md                        |  58 +++++
 static-build/test/CheckDependencies.cmake     |  43 ++++
 static-build/test/static-build/box.lua        |   3 +
 .../test/static-build/exports.test.lua        | 148 +++++++++++
 static-build/test/static-build/suite.ini      |   5 +
 .../test/static-build/traceback.test.lua      |  15 ++
 static-build/test/test-run.py                 |   1 +
 16 files changed, 608 insertions(+), 127 deletions(-)
 delete mode 100644 Dockerfile.staticbuild
 create mode 100644 static-build/.gitignore
 create mode 100644 static-build/CMakeLists.txt
 create mode 100644 static-build/README.md
 create mode 100644 static-build/test/CheckDependencies.cmake
 create mode 100755 static-build/test/static-build/box.lua
 create mode 100755 static-build/test/static-build/exports.test.lua
 create mode 100644 static-build/test/static-build/suite.ini
 create mode 100755 static-build/test/static-build/traceback.test.lua
 create mode 120000 static-build/test/test-run.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 0ead08711..c9aef3dc7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -534,9 +534,14 @@ static_build:
   script:
     - ${GITLAB_MAKE} test_static_build
 
-static_docker_build:
+static_build_cmake_linux:
+  <<: *docker_test_definition
+  script:
+    - ${GITLAB_MAKE} test_static_build_cmake_linux
+
+static_build_cmake_osx_15:
   stage: test
   tags:
-    - deploy_test
+    - osx_15
   script:
-    - ${GITLAB_MAKE} test_static_docker_build
+    - ${GITLAB_MAKE} test_static_build_cmake_osx
diff --git a/.travis.mk b/.travis.mk
index efc05cf05..482672429 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -148,8 +148,12 @@ deps_debian_static:
 test_static_build: deps_debian_static
 	CMAKE_EXTRA_PARAMS=-DBUILD_STATIC=ON make -f .travis.mk test_debian_no_deps
 
-test_static_docker_build:
-	docker build --no-cache --network=host --build-arg RUN_TESTS=ON -f Dockerfile.staticbuild .
+# New static build
+# builddir used in this target - is a default build path from cmake ExternalProject_Add()
+test_static_build_cmake_linux:
+	cd static-build && cmake . && make -j && ctest -V
+	cd test && /usr/bin/python test-run.py --force \
+		--builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build $(TEST_RUN_EXTRA_PARAMS)
 
 # ###################
 # Static Analysis
@@ -193,15 +197,16 @@ build_osx:
 	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
 	make -j
 
-test_osx_no_deps: build_osx
-	# Limits: Increase the maximum number of open file descriptors on macOS:
-	#   Travis-ci needs the "ulimit -n <value>" call
-	#   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
-	# Also gitlib-ci needs the password to change the limits, while
-	# travis-ci runs under root user. Limit setup must be in the same
-	# call as tests runs call.
-	# Tests: Temporary excluded replication/ suite with some tests
-	#        from other suites by issues #4357 and #4370
+
+# Limits: Increase the maximum number of open file descriptors on macOS:
+#   Travis-ci needs the "ulimit -n <value>" call
+#   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
+# Also gitlib-ci needs the password to change the limits, while
+# travis-ci runs under root user. Limit setup must be in the same
+# call as tests runs call.
+# Tests: Temporary excluded replication/ suite with some tests
+#        from other suites by issues #4357 and #4370
+INIT_TEST_ENV_OSX=\
 	sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
 		launchctl limit maxfiles || : ; \
 		ulimit -n ${MAX_FILES} || : ; \
@@ -210,11 +215,32 @@ test_osx_no_deps: build_osx
 		launchctl limit maxproc || : ; \
 		ulimit -u ${MAX_PROC} || : ; \
 		ulimit -u ; \
-		rm -rf /tmp/tnt ; \
-		cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
+		rm -rf /tmp/tnt
+
+test_osx_no_deps: build_osx
+	${INIT_TEST_ENV_OSX}; \
+	cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS)
 
 test_osx: deps_osx test_osx_no_deps
 
+# Static macOS build
+
+STATIC_OSX_PKGS=autoconf automake libtool cmake file://$${PWD}/tools/brew_taps/tntpython2.rb
+base_deps_osx:
+	brew update || echo | /usr/bin/ruby -e \
+		"$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+	brew install --force ${STATIC_OSX_PKGS} || brew upgrade ${STATIC_OSX_PKGS}
+	pip install --force-reinstall -r test-run/requirements.txt
+
+# builddir used in this target - is a default build path from cmake ExternalProject_Add()
+test_static_build_cmake_osx: base_deps_osx
+	cd static-build && cmake . && make -j && ctest -V
+	${INIT_TEST_ENV_OSX}; \
+	cd test && ./test-run.py --vardir /tmp/tnt \
+		--builddir ${PWD}/static-build/tarantool-prefix/src/tarantool-build \
+		--force $(TEST_RUN_EXTRA_PARAMS)
+
+
 ###########
 # FreeBSD #
 ###########
diff --git a/Dockerfile.staticbuild b/Dockerfile.staticbuild
deleted file mode 100644
index f67f46f5e..000000000
--- a/Dockerfile.staticbuild
+++ /dev/null
@@ -1,98 +0,0 @@
-FROM centos:7
-
-RUN yum install -y epel-release
-RUN yum install -y yum install https://centos7.iuscommunity.org/ius-release.rpm
-
-RUN set -x \
-    && yum -y install \
-        libstdc++ \
-        libstdc++-static \
-        readline \
-        openssl \
-        lz4 \
-        binutils \
-        ncurses \
-        libgomp \
-        lua \
-        curl \
-        tar \
-        zip \
-        unzip \
-        libunwind \
-        zlib \
-    && yum -y install \
-        perl \
-        gcc-c++ \
-        cmake \
-        lz4-devel \
-        binutils-devel \
-        lua-devel \
-        make \
-        git \
-        autoconf \
-        automake \
-        libtool \
-        wget
-
-RUN yum -y install ncurses-static readline-static zlib-static pcre-static glibc-static
-
-RUN yum -y install python-devel python-pip
-
-RUN set -x && \
-    cd / && \
-    curl -O -L https://www.openssl.org/source/openssl-1.1.1f.tar.gz && \
-    tar -xvf openssl-1.1.1f.tar.gz && \
-    cd openssl-1.1.1f && \
-    ./config --libdir=lib && \
-    make -j && make install
-
-RUN set -x && \
-    cd / && \
-    curl -O -L https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz && \
-    tar -xvf icu4c-62_1-src.tgz && \
-    cd icu/source && \
-    ./configure --with-data-packaging=static --enable-static --enable-shared && \
-    make -j && make install
-
-RUN set -x && \
-    cd / && \
-    curl -O -L http://download.savannah.nongnu.org/releases/libunwind/libunwind-1.3-rc1.tar.gz && \
-    tar -xvf libunwind-1.3-rc1.tar.gz && \
-    cd libunwind-1.3-rc1 && \
-    ./configure --enable-static --enable-shared && \
-    make -j && make install
-
-COPY . /tarantool
-
-WORKDIR /tarantool
-
-RUN set -x && \
-    git submodule init && \
-    git submodule update
-
-# Cleanup for 'build' directory added, because it purges all artefacts
-# produced for curl build, including the old configuration in build/curl
-RUN set -x && \
-    find . -name 'CMakeFiles' -type d -exec rm -rf {} + && \
-    find . -name 'CMakeCache.txt' -type f -delete && \
-    rm -rf build test/small test/luajit-tap
-
-RUN pip install -r /tarantool/test-run/requirements.txt
-
-RUN set -x && \
-    cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-         -DENABLE_DIST:BOOL=ON \
-         -DBUILD_STATIC=ON \
-         -DOPENSSL_USE_STATIC_LIBS=ON \
-         -DOPENSSL_ROOT_DIR=/usr/local \
-         . && \
-    make -j && make install
-
-ARG RUN_TESTS
-RUN if [ -n "${RUN_TESTS}" ]; then \
-        set -x && \
-        cd test && \
-        /usr/bin/python test-run.py --force; \
-    fi
-
-ENTRYPOINT ["/bin/bash"]
diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
index 5f8b15a63..365c14284 100644
--- a/cmake/BuildLibCURL.cmake
+++ b/cmake/BuildLibCURL.cmake
@@ -4,15 +4,20 @@ macro(curl_build)
     set(LIBCURL_BINARY_DIR ${PROJECT_BINARY_DIR}/build/curl/work)
     set(LIBCURL_INSTALL_DIR ${PROJECT_BINARY_DIR}/build/curl/dest)
 
+    message(STATUS "Looking for zlib")
+    find_path(ZLIB_INCLUDE_DIR zlib.h)
+    message(STATUS "Looking for zlib.h - ${ZLIB_INCLUDE_DIR}")
     if (BUILD_STATIC)
-        set(LIBZ_LIB_NAME libz.a)
+        find_library(LIBZ_LIBRARY NAMES libz.a)
     else()
-        set(LIBZ_LIB_NAME z)
+        find_library(LIBZ_LIBRARY NAMES z)
     endif()
-    find_library(LIBZ_LIBRARY NAMES ${LIBZ_LIB_NAME})
-    if ("${LIBZ_LIBRARY}" STREQUAL "LIBZ_LIBRARY-NOTFOUND")
+    message(STATUS "Looking for libz - ${LIBZ_LIBRARY}")
+
+    if (NOT ZLIB_INCLUDE_DIR OR NOT LIBZ_LIBRARY)
         message(FATAL_ERROR "Unable to find zlib")
     endif()
+    get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} DIRECTORY)
 
     # Use the same OpenSSL library for libcurl as is used for
     # tarantool itself.
@@ -88,9 +93,10 @@ macro(curl_build)
 
                 --prefix <INSTALL_DIR>
                 --enable-static
-                --enable-shared
+                --disable-shared
+                --disable-symbol-hiding
 
-                --with-zlib
+                --with-zlib=${FOUND_ZLIB_ROOT_DIR}
                 ${LIBCURL_OPENSSL_OPT}
                 --with-ca-fallback
 
diff --git a/cmake/FindReadline.cmake b/cmake/FindReadline.cmake
index c48bdcb3e..afe480679 100644
--- a/cmake/FindReadline.cmake
+++ b/cmake/FindReadline.cmake
@@ -14,7 +14,17 @@ if(BUILD_STATIC)
     if (NOT CURSES_INFO_LIBRARY)
         set(CURSES_INFO_LIBRARY "")
     endif()
+
+    # From Modules/FindCurses.cmake:
+    # Set ``CURSES_NEED_NCURSES`` to ``TRUE`` before the
+    # ``find_package(Curses)`` call if NCurses functionality is required.
+    # This flag is set for linking with required library (installed
+    # via static-build/CMakeLists.txt). If this variable won't be set
+    # then tarantool binary links with system library curses which is an
+    # entire copy of ncurses
+    set(CURSES_NEED_NCURSES TRUE)
 endif()
+
 find_package(Curses)
 if(NOT CURSES_FOUND)
     find_package(Termcap)
diff --git a/cmake/compiler.cmake b/cmake/compiler.cmake
index 5a1141ebd..db2ae6227 100644
--- a/cmake/compiler.cmake
+++ b/cmake/compiler.cmake
@@ -33,10 +33,6 @@ if (CMAKE_COMPILER_IS_GNUCC)
            Your GCC version is ${CMAKE_CXX_COMPILER_VERSION}, please update
                    ")
        endif()
-else()
-     if (BUILD_STATIC)
-           message(FATAL_ERROR "Static build is supported for GCC only")
-     endif()
 endif()
 
 #
@@ -120,10 +116,23 @@ set (CMAKE_CXX_FLAGS_RELWITHDEBINFO
 
 unset(CC_DEBUG_OPT)
 
+message(STATUS "Looking for libunwind.h")
+find_path(UNWIND_INCLUDE_DIR libunwind.h)
+message(STATUS "Looking for libunwind.h - ${UNWIND_INCLUDE_DIR}")
+
+if (UNWIND_INCLUDE_DIR)
+    include_directories(${UNWIND_INCLUDE_DIR})
+endif()
+
+set(CMAKE_REQUIRED_INCLUDES ${UNWIND_INCLUDE_DIR})
 check_include_file(libunwind.h HAVE_LIBUNWIND_H)
-if(BUILD_STATIC)
+set(CMAKE_REQUIRED_INCLUDES "")
+
+if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
     set(UNWIND_LIB_NAME libunwind.a)
 else()
+    # libunwind can't be compiled on macOS.
+    # But there exists libunwind.dylib as a part of MacOSSDK
     set(UNWIND_LIB_NAME unwind)
 endif()
 find_library(UNWIND_LIBRARY PATH_SUFFIXES system NAMES ${UNWIND_LIB_NAME})
@@ -185,7 +194,10 @@ if (ENABLE_BACKTRACE)
     find_package_message(UNWIND_LIBRARIES "Found unwind" "${UNWIND_LIBRARIES}")
 endif()
 
-if(BUILD_STATIC)
+# On macOS there is no '-static-libstdc++' flag and it's use will
+# raise following error:
+# error: argument unused during compilation: '-static-libstdc++'
+if(BUILD_STATIC AND NOT TARGET_OS_DARWIN)
     # Static linking for c++ routines
     add_compile_flags("C;CXX" "-static-libstdc++")
 endif()
diff --git a/cmake/os.cmake b/cmake/os.cmake
index 905be61df..276a79b42 100644
--- a/cmake/os.cmake
+++ b/cmake/os.cmake
@@ -107,7 +107,10 @@ elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Darwin")
 
     # Latest versions of Homebrew wont 'link --force' for libraries, that were
     # preinstalled in system. So we'll use this dirty hack
-    find_program(HOMEBREW_EXECUTABLE brew)
+
+    if (NOT BUILD_STATIC)
+        find_program(HOMEBREW_EXECUTABLE brew)
+    endif()
     if(EXISTS ${HOMEBREW_EXECUTABLE})
         execute_process(COMMAND ${HOMEBREW_EXECUTABLE} --prefix
                         OUTPUT_VARIABLE HOMEBREW_PREFIX
diff --git a/static-build/.gitignore b/static-build/.gitignore
new file mode 100644
index 000000000..c8028a870
--- /dev/null
+++ b/static-build/.gitignore
@@ -0,0 +1,4 @@
+*-prefix
+/Makefile
+/test/var
+/build
diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
new file mode 100644
index 000000000..d90a642e6
--- /dev/null
+++ b/static-build/CMakeLists.txt
@@ -0,0 +1,240 @@
+cmake_minimum_required(VERSION 2.8)
+
+project(tarantool-env NONE)
+
+include(ExternalProject)
+set(OPENSSL_VERSION 1.1.1f)
+set(ZLIB_VERSION 1.2.11)
+set(NCURSES_VERSION 6.2)
+set(READLINE_VERSION 8.0)
+set(UNWIND_VERSION 1.3-rc1)
+
+if (APPLE)
+    find_program(C_COMPILER clang)
+    find_program(CXX_COMPILER clang++)
+else()
+    find_program(C_COMPILER gcc)
+    find_program(CXX_COMPILER g++)
+endif()
+set(CMAKE_C_COMPILER ${C_COMPILER})
+set(CMAKE_CXX_COMPILER ${CXX_COMPILER})
+
+# Install all libraries required by tarantool at current build dir
+
+#
+# OpenSSL
+#
+ExternalProject_Add(openssl
+    URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
+    CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+        CXX=${CMAKE_CXX_COMPILER}
+        <SOURCE_DIR>/config
+        --prefix=<INSTALL_DIR>
+        --libdir=lib
+        no-shared
+    INSTALL_COMMAND ${CMAKE_MAKE_PROGRAM} install_sw
+)
+
+#
+# ICU
+#
+ExternalProject_Add(icu
+    URL https://github.com/unicode-org/icu/releases/download/release-62-1/icu4c-62_1-src.tgz
+    CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+        CXX=${CMAKE_CXX_COMPILER}
+        <SOURCE_DIR>/source/configure
+        --with-data-packaging=static
+        --prefix=<INSTALL_DIR>
+        --disable-shared
+        --enable-static
+)
+
+#
+# ZLIB
+#
+ExternalProject_Add(zlib
+    URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
+    CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+        <SOURCE_DIR>/configure
+        --prefix=<INSTALL_DIR>
+        --static
+    TEST_COMMAND ${CMAKE_MAKE_PROGRAM} check
+)
+
+#
+# Ncurses
+#
+ExternalProject_Add(ncurses
+    URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz
+    CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+        CXX=${CMAKE_CXX_COMPILER}
+        <SOURCE_DIR>/configure
+        --prefix=<INSTALL_DIR>
+)
+
+#
+# ReadLine
+#
+ExternalProject_Add(readline
+    URL https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz
+    CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+        <SOURCE_DIR>/configure
+        --prefix=<INSTALL_DIR>
+        --disable-shared
+    # STEP_TARGETS download
+)
+
+#
+# ICONV
+#
+if (APPLE)
+    ExternalProject_Add(iconv
+        URL https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
+        CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+            <SOURCE_DIR>/configure
+            --prefix=<INSTALL_DIR>
+            --disable-shared
+            --enable-static
+            --with-gnu-ld
+        STEP_TARGETS download
+    )
+else()
+    # In linux iconv is embedded into glibc
+    # So we find system header and copy it locally
+    find_path(ICONV_INCLUDE_DIR iconv.h)
+    if(NOT ICONV_INCLUDE_DIR)
+        message(FATAL_ERROR "iconv include header not found")
+    endif()
+
+    set(ICONV_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix")
+
+    add_custom_command(
+        OUTPUT "${ICONV_INSTALL_PREFIX}/include/iconv.h"
+        COMMAND ${CMAKE_COMMAND} -E make_directory
+            "${ICONV_INSTALL_PREFIX}/include"
+        COMMAND ${CMAKE_COMMAND} -E copy
+            "${ICONV_INCLUDE_DIR}/iconv.h"
+            "${ICONV_INSTALL_PREFIX}/include/iconv.h"
+    )
+    add_custom_target(iconv
+        DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/iconv-prefix/include/iconv.h"
+    )
+    # This is a hack for further getting install directory of library
+    # by ExternalProject_Get_Property
+    set_target_properties(iconv
+        PROPERTIES _EP_INSTALL_DIR ${ICONV_INSTALL_PREFIX}
+    )
+endif()
+
+#
+# Unwind
+#
+if (APPLE)
+    # On macOS libunwind is a part of MacOSX.sdk
+    # So we need to find library and header and
+    # copy it locally
+    find_path(UNWIND_INCLUDE_DIR libunwind.h)
+    find_library(UNWIND_LIBRARY libunwind.dylib
+        PATH_SUFFIXES system
+    )
+
+    set(UNWIND_INSTALL_PREFIX "${CMAKE_CURRENT_BINARY_DIR}/unwind-prefix")
+
+    set(UNWIND_DEPENDENCIES)
+
+    if (UNWIND_INCLUDE_DIR AND UNWIND_LIBRARY)
+        add_custom_command(
+            OUTPUT "${UNWIND_INSTALL_PREFIX}/include/unwind.h"
+            OUTPUT "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
+            COMMAND ${CMAKE_COMMAND} -E make_directory
+                "${UNWIND_INSTALL_PREFIX}/include"
+            COMMAND ${CMAKE_COMMAND} -E copy
+                "${UNWIND_INCLUDE_DIR}/libunwind.h"
+                "${UNWIND_INCLUDE_DIR}/unwind.h"
+                "${UNWIND_INSTALL_PREFIX}/include/"
+        )
+        add_custom_command(
+            OUTPUT "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
+            COMMAND ${CMAKE_COMMAND} -E make_directory
+                "${UNWIND_INSTALL_PREFIX}/lib"
+            COMMAND ${CMAKE_COMMAND} -E copy
+                "${UNWIND_LIBRARY}"
+                "${UNWIND_INSTALL_PREFIX}/lib/"
+        )
+        set(UNWIND_DEPENDENCIES
+            ${UNWIND_DEPENDENCIES}
+            "${UNWIND_INSTALL_PREFIX}/lib/libunwind.dylib"
+            "${UNWIND_INSTALL_PREFIX}/include/libunwind.h"
+        )
+    else()
+        message(STATUS "Unwind not found")
+    endif()
+
+    add_custom_target(unwind DEPENDS ${UNWIND_DEPENDENCIES})
+    # This is a hack for further getting install directory of library
+    # by ExternalProject_Get_Property
+    set_target_properties(unwind
+        PROPERTIES _EP_INSTALL_DIR ${UNWIND_INSTALL_PREFIX}
+    )
+else()
+    ExternalProject_Add(unwind
+        URL https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz
+        CONFIGURE_COMMAND CC=${CMAKE_C_COMPILER}
+            CXX=${CMAKE_CXX_COMPILER}
+            <SOURCE_DIR>/configure
+            --prefix=<INSTALL_DIR>
+            --disable-shared
+            --enable-static
+            --disable-minidebuginfo # to prevent linking with liblzma
+        STEP_TARGETS download
+    )
+endif()
+
+# Get install directories of builded libraries for building
+# tarantool with custon CMAKE_PREFIX_PATH
+foreach(PROJ openssl icu zlib ncurses readline iconv unwind)
+    ExternalProject_Get_Property(${PROJ} install_dir)
+    set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH}:${install_dir})
+    set(TARANTOOL_DEPENDS ${PROJ} ${TARANTOOL_DEPENDS})
+    message(STATUS "Add external project ${PROJ} in ${install_dir}")
+endforeach()
+
+ExternalProject_Add(tarantool
+    DEPENDS ${TARANTOOL_DEPENDS}
+    SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/..
+    LIST_SEPARATOR :
+    CMAKE_ARGS
+        # Override LOCALSTATEDIR to avoid cmake "special" cases:
+        # https://cmake.org/cmake/help/v3.4/module/GNUInstallDirs.html#special-cases
+        -DCMAKE_INSTALL_LOCALSTATEDIR=<INSTALL_DIR>/var
+        -DCMAKE_INSTALL_PREFIX=<INSTALL_DIR>
+        -DCMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}
+        -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=FALSE
+        -DOPENSSL_USE_STATIC_LIBS=TRUE
+        -DBUILD_STATIC=TRUE
+        -DENABLE_DIST=TRUE
+        -DENABLE_BACKTRACE=TRUE
+        -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+        -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+        ${CMAKE_TARANTOOL_ARGS}
+    STEP_TARGETS build
+    BUILD_COMMAND $(MAKE)
+)
+
+enable_testing()
+ExternalProject_Get_Property(tarantool binary_dir)
+SET(TARANTOOL_BINARY_DIR ${binary_dir})
+
+add_test(
+    NAME check-dependencies
+    COMMAND ${CMAKE_COMMAND}
+        -D FILE=${TARANTOOL_BINARY_DIR}/src/tarantool
+        -P CheckDependencies.cmake
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
+)
+
+add_test(
+    NAME test-run-static
+    COMMAND ./test-run.py --builddir ${TARANTOOL_BINARY_DIR}
+    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/test
+)
diff --git a/static-build/README.md b/static-build/README.md
new file mode 100644
index 000000000..0019e963f
--- /dev/null
+++ b/static-build/README.md
@@ -0,0 +1,58 @@
+# Tarantool static build tooling
+
+These files help to prepare environment for building Tarantool
+statically. And builds it.
+
+## Prerequisites
+
+CentOS:
+
+```bash
+yum install -y \
+    git perl gcc cmake make gcc-c++ libstdc++-static autoconf automake libtool \
+    python-msgpack python-yaml python-argparse python-six python-gevent
+```
+
+MacOS:
+
+Before you start please install default Xcode Tools by Apple:
+
+```
+sudo xcode-select --install
+sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
+```
+
+Install brew using command from
+[Homebrew repository instructions](https://github.com/Homebrew/inst)
+
+After that run next script:
+
+```bash
+  brew install autoconf automake libtool cmake file://$${PWD}/tools/brew_taps/tntpython2.rbs
+  pip install --force-reinstall -r test-run/requirements.txt
+```
+
+### Usage
+
+```bash
+cmake .
+make -j
+ctest -V
+```
+
+## Customize your build
+
+If you want to customise build, you need to set `CMAKE_TARANTOOL_ARGS` variable
+
+### Usage
+
+There is three types of `CMAKE_BUILD_TYPE`:
+* Debug - default
+* Release
+* RelWithDebInfo
+
+And you want to build tarantool with RelWithDebInfo:
+
+```bash
+cmake -DCMAKE_TARANTOOL_ARGS="-DCMAKE_BUILD_TYPE=RelWithDebInfo" .
+```
diff --git a/static-build/test/CheckDependencies.cmake b/static-build/test/CheckDependencies.cmake
new file mode 100644
index 000000000..49e91e7fa
--- /dev/null
+++ b/static-build/test/CheckDependencies.cmake
@@ -0,0 +1,43 @@
+## This is a cmake-based test, it checks that tarantool static binary
+# has no dependencies except allowed ones.
+
+include(GetPrerequisites)
+if(NOT FILE)
+    message(FATAL_ERROR "Usage: "
+        "${CMAKE_COMMAND} -DFILE=<FILENAME> -P CheckDependencies.cmake")
+elseif(NOT EXISTS ${FILE})
+    message(FATAL_ERROR "${FILE}: No such file")
+endif()
+
+get_prerequisites(${FILE} DEPENDENCIES 0 0 "" "")
+
+if (APPLE)
+    set(ALLOWLIST
+        libSystem
+        CoreFoundation
+        libc++
+    )
+elseif(UNIX)
+    set(ALLOWLIST
+        libdl
+        librt
+        libc
+        libm
+        libgcc_s
+        libpthread
+    )
+else()
+    message(FATAL_ERROR "Unknown platform")
+endif()
+
+foreach(DEPENDENCY_FILE ${DEPENDENCIES})
+    message("Dependency: ${DEPENDENCY_FILE}")
+endforeach()
+
+foreach(DEPENDENCY_FILE ${DEPENDENCIES})
+    get_filename_component(libname ${DEPENDENCY_FILE} NAME_WE)
+    list (FIND ALLOWLIST ${libname} _index)
+    if (_index EQUAL -1)
+        message(FATAL_ERROR "Blocklisted dependency: ${DEPENDENCY_FILE}")
+    endif()
+endforeach()
diff --git a/static-build/test/static-build/box.lua b/static-build/test/static-build/box.lua
new file mode 100755
index 000000000..bad8a9055
--- /dev/null
+++ b/static-build/test/static-build/box.lua
@@ -0,0 +1,3 @@
+#!/usr/bin/env tarantool
+
+require('console').listen(os.getenv('ADMIN'))
diff --git a/static-build/test/static-build/exports.test.lua b/static-build/test/static-build/exports.test.lua
new file mode 100755
index 000000000..63dc163a9
--- /dev/null
+++ b/static-build/test/static-build/exports.test.lua
@@ -0,0 +1,148 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local ffi = require('ffi')
+ffi.cdef([[
+    void *dlsym(void *handle, const char *symbol);
+]])
+
+local test = tap.test('exports')
+
+
+local RTLD_DEFAULT
+-- See `man 3 dlsym`:
+-- RTLD_DEFAULT
+--   Find  the  first occurrence of the desired symbol using the default
+--   shared object search order.  The search will include global symbols
+--   in the executable and its dependencies, as well as symbols in shared
+--   objects that were dynamically loaded with the RTLD_GLOBAL flag.
+if jit.os == "OSX" then
+    RTLD_DEFAULT = ffi.cast("void *", -2LL)
+else
+    RTLD_DEFAULT = ffi.cast("void *", 0LL)
+end
+
+local function check_symbol(sym)
+    test:ok(ffi.C.dlsym(RTLD_DEFAULT, sym) ~= nil, ('Symbol %q found'):format(sym))
+end
+
+local check_symbols = {
+    -- FFI
+
+    'guava',
+    'base64_decode',
+    'base64_encode',
+    'SHA1internal',
+    'random_bytes',
+    'fiber_time',
+    'ibuf_create',
+    'ibuf_destroy',
+    'port_destroy',
+    'csv_create',
+    'csv_destroy',
+    'title_get',
+    'title_update',
+    'tnt_iconv',
+    'tnt_iconv_open',
+    'tnt_iconv_close',
+    'exception_get_int',
+    'exception_get_string',
+
+    'tarantool_lua_ibuf',
+    'uuid_nil',
+    'tt_uuid_create',
+    'tt_uuid_str',
+    'tt_uuid_is_equal',
+    'tt_uuid_is_nil',
+    'tt_uuid_bswap',
+    'tt_uuid_from_string',
+    'log_level',
+    'log_format',
+    'uri_parse',
+    'uri_format',
+    'PMurHash32',
+    'PMurHash32_Process',
+    'PMurHash32_Result',
+    'crc32_calc',
+    'mp_encode_double',
+    'mp_encode_float',
+    'mp_encode_decimal',
+    'mp_decode_double',
+    'mp_decode_float',
+    'mp_decode_extl',
+    'mp_sizeof_decimal',
+    'decimal_unpack',
+
+    'log_type',
+    'say_set_log_level',
+    'say_logrotate',
+    'say_set_log_format',
+    'tarantool_uptime',
+    'tarantool_exit',
+    'log_pid',
+    'space_by_id',
+    'space_run_triggers',
+    'space_bsize',
+    'box_schema_version',
+
+    'crypto_EVP_MD_CTX_new',
+    'crypto_EVP_MD_CTX_free',
+    'crypto_HMAC_CTX_new',
+    'crypto_HMAC_CTX_free',
+    'crypto_stream_new',
+    'crypto_stream_begin',
+    'crypto_stream_append',
+    'crypto_stream_commit',
+    'crypto_stream_delete',
+
+    -- Module API
+
+    '_say',
+    'swim_cfg',
+    'swim_quit',
+    'fiber_new',
+    'fiber_cancel',
+    'coio_wait',
+    'coio_close',
+    'coio_call',
+    'coio_getaddrinfo',
+    'luaT_call',
+    'box_txn',
+    'box_select',
+    'clock_realtime',
+    'string_strip_helper',
+
+    -- Lua / LuaJIT
+
+    'lua_newstate',
+    'lua_close',
+    'luaL_loadstring',
+    'luaJIT_profile_start',
+    'luaJIT_profile_stop',
+    'luaJIT_profile_dumpstack',
+
+    'ERR_error_string',
+    'ERR_get_error',
+
+    'EVP_get_digestbyname',
+    'EVP_get_cipherbyname',
+    'EVP_CIPHER_CTX_new',
+    'EVP_CIPHER_CTX_free',
+    'EVP_CIPHER_block_size',
+    'HMAC_Init_ex',
+    'HMAC_Update',
+    'HMAC_Final',
+
+    'ZSTD_compress',
+    'ZSTD_decompress',
+    'ZSTD_free',
+    'ZSTD_malloc',
+    'ZSTD_versionString',
+}
+
+test:plan(#check_symbols)
+for _, sym in ipairs(check_symbols) do
+    check_symbol(sym)
+end
+
+os.exit(test:check() and 0 or 1)
diff --git a/static-build/test/static-build/suite.ini b/static-build/test/static-build/suite.ini
new file mode 100644
index 000000000..92e349466
--- /dev/null
+++ b/static-build/test/static-build/suite.ini
@@ -0,0 +1,5 @@
+[default]
+core = app
+description = Static build tests
+script = box.lua
+is_parallel = True
diff --git a/static-build/test/static-build/traceback.test.lua b/static-build/test/static-build/traceback.test.lua
new file mode 100755
index 000000000..32e1ed8b5
--- /dev/null
+++ b/static-build/test/static-build/traceback.test.lua
@@ -0,0 +1,15 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local fiber = require('fiber')
+
+local test = tap.test('traceback')
+test:plan(1)
+
+local info = fiber.info()[fiber.id()]
+test:ok(info.backtrace ~= nil, 'fiber.info() has backtrace')
+for _, l in pairs(info.backtrace or {}) do
+    test:diag('%s: %s', next(l))
+end
+
+os.exit(test:check() and 0 or 1)
diff --git a/static-build/test/test-run.py b/static-build/test/test-run.py
new file mode 120000
index 000000000..02187cdc5
--- /dev/null
+++ b/static-build/test/test-run.py
@@ -0,0 +1 @@
+../../test-run/test-run.py
\ No newline at end of file
-- 
2.26.2

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-09 14:49 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-25 14:01 [Tarantool-patches] [PATCH v2] build: refactor static build process HustonMmmavr
2020-08-25 14:51 ` Alexander Turenko
2020-08-27 21:43   ` Alexandr Barulev
2020-08-31 17:37     ` Alexandr Barulev
2020-09-04 17:41 ` Igor Munkin
2020-09-09 14:49   ` Alexandr Barulev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox