* [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake
@ 2020-10-14 8:27 Alexander V. Tikhonov
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
` (5 more replies)
0 siblings, 6 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-14 8:27 UTC (permalink / raw)
To: Kirill Yukhin, Igor Munkin; +Cc: tarantool-patches
Blocked check if the current cmake of the older than
3.2...3.16 version to be able to build curl at the old OS:
- CentOS 6
- CentOS 7
- Debian 8
- Ubuntu 14.04
Used in 'third_party/curl' repository branch:
curl-7_71_1-tarantool
Needed for #4968
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4968
third_party/curl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/third_party/curl b/third_party/curl
index 5a1fc8d33..12af024bc 160000
--- a/third_party/curl
+++ b/third_party/curl
@@ -1 +1 @@
-Subproject commit 5a1fc8d33808d7b22f57bdf9403cda7ff07b0670
+Subproject commit 12af024bc85606b14ffc415413a7e86e6bbee7eb
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
@ 2020-10-14 8:29 ` Alexander V. Tikhonov
2020-10-15 9:18 ` Igor Munkin
2021-03-12 8:55 ` Kirill Yukhin via Tarantool-patches
2020-10-14 8:30 ` [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory Alexander V. Tikhonov
` (4 subsequent siblings)
5 siblings, 2 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-14 8:29 UTC (permalink / raw)
To: Kirill Yukhin, Igor Munkin; +Cc: tarantool-patches
Running out-of-source build got the issue:
/source/src/box/memtx_engine.c:48:23: fatal error: bootstrap.h: No such file or directory
To fix it added path to the header in cmake file.
Part of #4968
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4968
src/box/CMakeLists.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 8b2e704cf..6e25401a1 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -77,6 +77,7 @@ set_property(DIRECTORY PROPERTY ADDITIONAL_MAKE_CLEAN_FILES ${lua_sources})
include_directories(${ZSTD_INCLUDE_DIRS})
include_directories(${CMAKE_BINARY_DIR}/src/box/sql)
+include_directories(${CMAKE_BINARY_DIR}/src/box)
add_library(vclock STATIC vclock.c)
target_link_libraries(vclock core bit)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
@ 2020-10-14 8:30 ` Alexander V. Tikhonov
2020-10-15 12:24 ` Igor Munkin
2020-10-14 8:35 ` [Tarantool-patches] [PATCH v1 5/5] gitlab-ci: add out-of-source build Alexander V. Tikhonov
` (3 subsequent siblings)
5 siblings, 1 reply; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-14 8:30 UTC (permalink / raw)
To: Kirill Yukhin, Igor Munkin; +Cc: tarantool-patches
Running out-of-source build found that destination path was set to
source directory. Changed it to binary path.
Part of #4968
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4968
cmake/utils.cmake | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/cmake/utils.cmake b/cmake/utils.cmake
index 3ab2d3ff2..eaec821b3 100644
--- a/cmake/utils.cmake
+++ b/cmake/utils.cmake
@@ -70,9 +70,9 @@ endfunction()
function(bin_source varname srcfile dstfile)
set(var ${${varname}})
- set(${varname} ${var} ${dstfile} PARENT_SCOPE)
set (srcfile "${CMAKE_CURRENT_SOURCE_DIR}/${srcfile}")
- set (dstfile "${CMAKE_CURRENT_SOURCE_DIR}/${dstfile}")
+ set (dstfile "${CMAKE_CURRENT_BINARY_DIR}/${dstfile}")
+ set(${varname} ${var} "${dstfile}" PARENT_SCOPE)
set (tmpfile "${dstfile}.tmp")
get_filename_component(module ${dstfile} NAME_WE)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH v1 5/5] gitlab-ci: add out-of-source build
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
2020-10-14 8:30 ` [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory Alexander V. Tikhonov
@ 2020-10-14 8:35 ` Alexander V. Tikhonov
2020-10-15 12:30 ` [Tarantool-patches] [PATCH v2 " Igor Munkin
2020-10-14 9:25 ` [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build Alexander V. Tikhonov
` (2 subsequent siblings)
5 siblings, 1 reply; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-14 8:35 UTC (permalink / raw)
To: Kirill Yukhin, Igor Munkin; +Cc: tarantool-patches
Implemented out-of-source build at cmake files. Added out of source
build make targets and added test job to gitlab-ci.
Closes #4874
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4874
V2: removed changes not for gitlab-ci.
.gitlab-ci.yml | 7 +++++++
.travis.mk | 26 +++++++++++++++++++++++++-
2 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d40dc74e3..d5dc52f82 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -174,6 +174,13 @@ luacheck:
# Tests
+out_of_source:
+ stage: test
+ tags:
+ - deploy_test
+ script:
+ - ${GITLAB_MAKE} test_oos_build
+
release:
<<: *docker_test_definition
script:
diff --git a/.travis.mk b/.travis.mk
index d797472e2..5730e1097 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -7,7 +7,9 @@ DOCKER_IMAGE_TARANTOOL="registry.gitlab.com/tarantool/tarantool/testing/debian-s
TEST_RUN_EXTRA_PARAMS?=
MAX_FILES?=65534
MAX_PROC?=2500
-OOS_SRC_PATH="/source"
+OOS_SRC_PATH?="/source"
+OOS_BUILD_PATH?="/rw_bins"
+OOS_BUILD_RULE?=test_oos_no_deps
BIN_DIR=/usr/local/bin
OSX_VARDIR?=/tmp/tnt
@@ -207,6 +209,28 @@ test_debian_install_luacheck:
test_debian_luacheck: test_debian_install_luacheck configure_debian
make luacheck
+# Out-Of-Source build
+
+build_oos:
+ mkdir ${OOS_BUILD_PATH} 2>/dev/null || : ; \
+ cd ${OOS_BUILD_PATH} && \
+ cmake ${OOS_SRC_PATH} ${CMAKE_EXTRA_PARAMS} && \
+ make -j
+
+test_oos_no_deps: build_oos
+ cd ${OOS_BUILD_PATH}/test && \
+ ${OOS_SRC_PATH}/test/test-run.py \
+ --builddir ${OOS_BUILD_PATH} \
+ --vardir ${OOS_BUILD_PATH}/test/var --force
+
+test_oos: deps_debian test_oos_no_deps
+
+test_oos_build:
+ docker run --network=host -w ${OOS_SRC_PATH} \
+ --mount type=bind,source="${PWD}",target=${OOS_SRC_PATH},readonly,bind-propagation=rslave \
+ -i ${DOCKER_IMAGE_TARANTOOL} \
+ make -f .travis.mk ${OOS_BUILD_RULE}
+
#######
# OSX #
#######
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
` (2 preceding siblings ...)
2020-10-14 8:35 ` [Tarantool-patches] [PATCH v1 5/5] gitlab-ci: add out-of-source build Alexander V. Tikhonov
@ 2020-10-14 9:25 ` Alexander V. Tikhonov
2020-10-15 14:17 ` Igor Munkin
2020-10-15 12:23 ` [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Igor Munkin
2020-10-16 6:55 ` Kirill Yukhin
5 siblings, 1 reply; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-14 9:25 UTC (permalink / raw)
To: Kirill Yukhin, Igor Munkin; +Cc: tarantool-patches
Initially tried to change autoconf tools in Curl build to cmake and
found the following build issue on:
CentOS 6,7
Debian 8
Ubuntu 14.04
Issue found:
CMake Error at CMakeLists.txt:41 (cmake_minimum_required):
CMake 3.0 or higher is required. You are running version 2.8.12.2
To fix the issue check is removed of the version from curl sources
in Tarantool's third party '<root Tarantool sources>/third_party/curl':
12af024bc85606b14ffc415413a7e86e6bbee7eb ('Enable curl build with old cmake')
After this fix completely changed autoconf to cmake in curl build.
Autoconf part was completely removed and code cleaned up for cmake.
For curl cmake build all autoconf options were ported to cmake
configuration call, check the accurate list of the change in [1].
Also the following issues resolved:
1. Found that CURL cmake configuration file:
third_party/curl/lib/CMakeLists.txt
has installation part for built libcurl.a library:
install(TARGETS ${LIB_NAME}
EXPORT ${TARGETS_EXPORT_NAME}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
where it changes CMAKE_INSTALL_LIBDIR to appropriate name with
suffix of the architecture, like:
lib
lib64
x86_64
Found that find_library routine from the file:
cmake/FindLibCURL.cmake
returns only 'lib' value and it breaks the building of the depends
binaries. To avoid of it the CMAKE_INSTALL_LIBDIR option was set to
cmake call:
-DCMAKE_INSTALL_LIBDIR=lib
2. Found issue with building on CentOS 6:
Linking C executable curl
build/ares/dest/lib/libcares.a(ares__timeval.c.o): In function `ares__tvnow':
ares__timeval.c:(.text+0x15): undefined reference to `clock_gettime'
collect2: error: ld returned 1 exit status
It was fixed with added "-lrt" flag to CMAKE_C_FLAGS and
CMAKE_CXX_FLAGS build flags, when cmake version is lower
than 3.0 and RT library had needed function.
3. Found issues with building Tarantool statically on Debian 9 and
its package build on CentOS 8.
Building statically got the issues with openssl linking, like [2]:
static-build/openssl-prefix/lib/libcrypto.a(threads_pthread.o): In function `CRYPTO_THREAD_lock_new':
threads_pthread.c:(.text+0x45): undefined reference to `pthread_rwlock_init'
It happened because in openssl radically changed how threads-related
things were handled before 1.1.0 version. It required the application
to provide us with the locking functionality in form of callbacks.
Since 1.1.0, these matters are entirely internal, so libcrypto
requires the presence of the platform specific thread implementation
of our choosing, which is pthread on everything.
After '-pthread' added to C compile flags package build on CentOS 8
failed with the issue [3]:
/build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:101:4: error: #error "SIZEOF_CURL_OFF_T not defined"
# error "SIZEOF_CURL_OFF_T not defined"
^~~~~
/build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c: In function 'curlx_uztoso':
/build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:192:40: error: 'CURL_MASK_SCOFFT' undeclared (first use in this function); did you mean 'CURL_MASK_SINT'?
return (curl_off_t)(uznum & (size_t) CURL_MASK_SCOFFT);
^~~~~~~~~~~~~~~~
CURL_MASK_SINT
To avoid of the issue decided to use '-pthread' flag only for openssl
when it had not this flag in openssl compilation:
# check '-pthread' in openssl compile options
execute_process(COMMAND openssl version -f
OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
# add pthread library for openssl static library linking
if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
endif()
4. Found issue with static build using CentOS 7, where SSL cmake rule
failed. Building the image got the issue:
[ 1%] Performing configure step for 'bundled-libcurl-project'
CMake Warning at CMakeLists.txt:50 (message):
the curl cmake build system is poorly maintained. Be aware
-- curl version=[7.66.0-DEV]
-- Found c-ares: /tarantool/build/ares/dest/lib/libcares.a
Found *nroff option: -- -man
CMake Error at /usr/share/cmake/Modules/FindOpenSSL.cmake:278 (list):
list GET given empty list
Call Stack (most recent call first):
CMakeLists.txt:347 (find_package)
Root cause of the issue that Dockerfile uses globaly
installed openSSL with:
cmake ... -DOPENSSL_ROOT_DIR=/usr/local ...
Its cmake build file:
/usr/share/cmake/Modules/FindOpenSSL.cmake
fails on parsing the SSL version:
it has:
REGEX "^#define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*")
but it should to use:
REGEX "^#[\t ]*define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*")
Anyway we want to use the same OpenSSL library for libcurl, as is used
for Tarantool itself. So the path to it set for cURL build:
list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_MODULE_PATH=${PROJECT_SOURCE_DIR}/cmake")
5. Found that on Gentoo need to switch off use of libSSH2:
list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
to avoid of linking issues, like:
ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
this issue exists in curl issues [4].
Closes #4968
Closes #5019
Closes #5396
[1] - https://github.com/tarantool/tarantool/issues/4968#issue-617183031
[2] - https://gitlab.com/tarantool/tarantool/-/jobs/779176133#L6021
[3] - https://gitlab.com/tarantool/tarantool/-/jobs/778309145#L3060
[4] - https://github.com/curl/curl/issues/1146
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4968
V4: Changes:
- Disabled libSSH2.
- Added disable symbols hiding flag.
cmake/BuildLibCURL.cmake | 250 +++++++++++++++++++--------------------
1 file changed, 121 insertions(+), 129 deletions(-)
diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
index 86fec39e9..0c3b2fda4 100644
--- a/cmake/BuildLibCURL.cmake
+++ b/cmake/BuildLibCURL.cmake
@@ -3,6 +3,7 @@ macro(curl_build)
set(LIBCURL_SOURCE_DIR ${PROJECT_SOURCE_DIR}/third_party/curl)
set(LIBCURL_BINARY_DIR ${PROJECT_BINARY_DIR}/build/curl/work)
set(LIBCURL_INSTALL_DIR ${PROJECT_BINARY_DIR}/build/curl/dest)
+ set(LIBCURL_CMAKE_FLAGS "")
message(STATUS "Looking for zlib")
find_path(ZLIB_INCLUDE_DIR zlib.h)
@@ -19,40 +20,126 @@ macro(curl_build)
message(FATAL_ERROR "Unable to find zlib")
endif()
get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} DIRECTORY)
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DZLIB_ROOT=${FOUND_ZLIB_ROOT_DIR}")
+
+ # check '-pthread' in openssl compile options
+ execute_process(COMMAND openssl version -f
+ OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
+ # add pthread library for openssl static library linking
+ if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
+ endif()
+
+ # add librt for clock_gettime function definition
+ if(${CMAKE_MAJOR_VERSION} VERSION_LESS "3")
+ CHECK_LIBRARY_EXISTS (rt clock_gettime "" HAVE_LIBRT)
+ if (HAVE_LIBRT)
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-lrt")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_CXX_FLAGS=-lrt")
+ endif()
+ endif()
+
+ # switch on the static build
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_STATICLIB=ON")
+
+ # switch off the shared build
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_SHARED_LIBS=OFF")
- # Use the same OpenSSL library for libcurl as is used for
- # tarantool itself.
+ # curl uses visibility hiding mode for its symbols by default and
+ # Tarantool tests can't use it. To fix it symbols hiding disabled
+ # for gcc and clang.
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_HIDDEN_SYMBOLS=OFF")
+
+ # let's disable testing for curl to save build time
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_TESTING=OFF")
+
+ # Setup use of openssl, use the same OpenSSL library
+ # for libcurl as is used for tarantool itself.
get_filename_component(FOUND_OPENSSL_ROOT_DIR ${OPENSSL_INCLUDE_DIR} DIRECTORY)
- set(LIBCURL_OPENSSL_OPT "--with-ssl=${FOUND_OPENSSL_ROOT_DIR}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_OPENSSL=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DOPENSSL_ROOT_DIR=${FOUND_OPENSSL_ROOT_DIR}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_MODULE_PATH=${PROJECT_SOURCE_DIR}/cmake")
- # Use either c-ares bundled with tarantool or
- # libcurl-default threaded resolver.
+ # Setup ARES and its library path, use either c-ares bundled
+ # with tarantool or libcurl-default threaded resolver.
if(BUNDLED_LIBCURL_USE_ARES)
- set(ASYN_DNS_USED "ares")
- set(ASYN_DNS_UNUSED "threaded-resolver")
- set(ASYN_DNS_PATH "=${ARES_INSTALL_DIR}")
+ set(ENABLE_ARES "ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_FIND_ROOT_PATH=${ARES_INSTALL_DIR}")
else()
- set(ASYN_DNS_USED "threaded-resolver")
- set(ASYN_DNS_UNUSED "ares")
- set(ASYN_DNS_PATH "")
- endif()
-
- set(ENABLED_DNS_OPT "--enable-${ASYN_DNS_USED}${ASYN_DNS_PATH}")
- set(DISABLED_DNS_OPT "--disable-${ASYN_DSN_UNUSED}")
-
- # Pass -isysroot=<SDK_PATH> option on Mac OS to a preprocessor
- # and a C compiler to find header files installed with an SDK.
- #
- # The idea here is to don't pass all compiler/linker options
- # that is set for tarantool, but only a subset that is
- # necessary for choosen toolchain, and let curl's configure
- # script set options that are appropriate for libcurl.
- set(LIBCURL_CPPFLAGS "")
- set(LIBCURL_CFLAGS "")
- if (TARGET_OS_DARWIN AND NOT "${CMAKE_OSX_SYSROOT}" STREQUAL "")
- set(LIBCURL_CPPFLAGS "${LIBCURL_CPPFLAGS} ${CMAKE_C_SYSROOT_FLAG} ${CMAKE_OSX_SYSROOT}")
- set(LIBCURL_CFLAGS "${LIBCURL_CFLAGS} ${CMAKE_C_SYSROOT_FLAG} ${CMAKE_OSX_SYSROOT}")
+ set(ENABLE_ARES "OFF")
endif()
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DENABLE_ARES=${ENABLE_ARES}")
+
+ # Found that on Gentoo need to switch off use of libSSH2 to avoid of issues, like:
+ # ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
+
+ # Switch off the group of protocols with special flag HTTP_ONLY:
+ # ftp, file, ldap, ldaps, rtsp, dict, telnet, tftp, pop3, imap, smtp
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DHTTP_ONLY=ON")
+
+ # additionaly disable some more protocols
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_SMB=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_GOPHER=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_CRYPTO_AUTH=ON")
+
+ # switch on ca-fallback feature
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_CA_FALLBACK=ON")
+
+ # Even though we set the external project's install dir
+ # below, we still need to pass the corresponding install
+ # prefix via cmake arguments.
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_INSTALL_PREFIX=${LIBCURL_INSTALL_DIR}")
+
+ # The default values for the options below are not always
+ # "./lib", "./bin" and "./include", while curl expects them
+ # to be.
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_INSTALL_LIBDIR=lib")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_INSTALL_INCLUDEDIR=include")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_INSTALL_BINDIR=bin")
+
+ # Pass the same toolchain as is used to build tarantool itself,
+ # because they can be incompatible.
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_LINKER=${CMAKE_LINKER}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_AR=${CMAKE_AR}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_RANLIB=${CMAKE_RANLIB}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_NM=${CMAKE_NM}")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_STRIP=${CMAKE_STRIP}")
+
+ # Need to set values explicitly everything that is default, because
+ # we don't know how defaults will be changed in a future and we don't
+ # want to verify them each time we'll bump new libcurl version.
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_BROTLI=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_GNUTLS=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_MBEDTLS=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_WOLFSSL=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_NSS=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_CA_BUNDLE=none")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_CA_PATH=none")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_LIBRTMP=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DHAVE_LIBIDN2=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_NGHTTP2=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_NGTCP2=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_NGHTTP3=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DUSE_QUICHE=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_HTTP=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_PROXY=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DENABLE_IPV6=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_COOKIES=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DENABLE_UNIX_SOCKETS=ON")
+ # should be already set by "-DHTTP_ONLY=ON" above
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_FTP=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_FILE=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_LDAP=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_LDAPS=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_RTSP=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_DICT=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_TELNET=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_TFTP=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_POP3=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_IMAP=ON")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_SMTP=ON")
include(ExternalProject)
ExternalProject_Add(
@@ -62,101 +149,11 @@ macro(curl_build)
DOWNLOAD_DIR ${LIBCURL_BINARY_DIR}
TMP_DIR ${LIBCURL_BINARY_DIR}/tmp
STAMP_DIR ${LIBCURL_BINARY_DIR}/stamp
- BINARY_DIR ${LIBCURL_BINARY_DIR}
+ BINARY_DIR ${LIBCURL_BINARY_DIR}/curl
CONFIGURE_COMMAND
- cd <SOURCE_DIR> && ./buildconf &&
- cd <BINARY_DIR> && <SOURCE_DIR>/configure
- # Pass the same toolchain as is used to build
- # tarantool itself, because they can be
- # incompatible.
- CC=${CMAKE_C_COMPILER}
- LD=${CMAKE_LINKER}
- AR=${CMAKE_AR}
- RANLIB=${CMAKE_RANLIB}
- NM=${CMAKE_NM}
- STRIP=${CMAKE_STRIP}
-
- # Pass -isysroot=<SDK_PATH> option on Mac OS, see
- # above.
- # Note: Passing of CPPFLAGS / CFLAGS explicitly
- # discards using of corresponsing environment
- # variables.
- CPPFLAGS=${LIBCURL_CPPFLAGS}
- CFLAGS=${LIBCURL_CFLAGS}
-
- # Pass empty LDFLAGS to discard using of
- # corresponding environment variable.
- # It is possible that a linker flag assumes that
- # some compilation flag is set. We don't pass
- # CFLAGS from environment, so we should not do it
- # for LDFLAGS too.
- LDFLAGS=
-
- --prefix <INSTALL_DIR>
- --enable-static
- --disable-shared
- --disable-symbol-hiding
-
- --with-zlib=${FOUND_ZLIB_ROOT_DIR}
- ${LIBCURL_OPENSSL_OPT}
- --with-ca-fallback
-
- --without-brotli
- --without-gnutls
- --without-mbedtls
- --without-cyassl
- --without-wolfssl
- --without-mesalink
- --without-nss
- --without-ca-bundle
- --without-ca-path
- --without-libpsl
- --without-libmetalink
- --without-librtmp
- --without-winidn
- --without-libidn2
- --without-nghttp2
- --without-ngtcp2
- --without-nghttp3
- --without-quiche
- --without-zsh-functions-dir
- --without-fish-functions-dir
-
- ${ENABLED_DNS_OPT}
- --enable-http
- --enable-proxy
- --enable-ipv6
- --enable-unix-sockets
- --enable-cookies
- --enable-http-auth
- --enable-mime
- --enable-dateparse
-
- ${DISABLED_DNS_OPT}
- --disable-ftp
- --disable-file
- --disable-ldap
- --disable-ldaps
- --disable-rtsp
- --disable-dict
- --disable-telnet
- --disable-tftp
- --disable-pop3
- --disable-imap
- --disable-smb
- --disable-smtp
- --disable-gopher
- --disable-manual
- --disable-sspi
- --disable-crypto-auth
- --disable-ntlm-wb
- --disable-tls-srp
- --disable-doh
- --disable-netrc
- --disable-progress-meter
- --disable-dnsshuffle
- --disable-alt-svc
- BUILD_COMMAND cd <BINARY_DIR> && $(MAKE)
+ cd <BINARY_DIR> && cmake <SOURCE_DIR>
+ ${LIBCURL_CMAKE_FLAGS}
+ BUILD_COMMAND cd <BINARY_DIR> && $(MAKE) -j
INSTALL_COMMAND cd <BINARY_DIR> && $(MAKE) install)
add_library(bundled-libcurl STATIC IMPORTED GLOBAL)
@@ -168,6 +165,7 @@ macro(curl_build)
endif()
add_dependencies(bundled-libcurl bundled-libcurl-project)
+ # setup CURL_INCLUDE_DIRS & CURL_LIBRARIES for global use
set(CURL_INCLUDE_DIRS ${LIBCURL_INSTALL_DIR}/include)
set(CURL_LIBRARIES bundled-libcurl ${LIBZ_LIBRARY})
if (BUNDLED_LIBCURL_USE_ARES)
@@ -177,13 +175,7 @@ macro(curl_build)
set(CURL_LIBRARIES ${CURL_LIBRARIES} rt)
endif()
- unset(ASYN_DNS_USED)
- unset(ASYN_DNS_UNUSED)
- unset(ASYN_DNS_PATH)
- unset(ENABLED_DNS_OPT)
- unset(DISABLED_DNS_OPT)
- unset(LIBCURL_CPPFLAGS)
- unset(LIBCURL_CFLAGS)
+ unset(FOUND_ZLIB_ROOT_DIR)
unset(FOUND_OPENSSL_ROOT_DIR)
unset(LIBCURL_INSTALL_DIR)
unset(LIBCURL_BINARY_DIR)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
@ 2020-10-15 9:18 ` Igor Munkin
2021-03-12 8:55 ` Kirill Yukhin via Tarantool-patches
1 sibling, 0 replies; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 9:18 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks for the patch! The patch LGTM per se, but this is just a partial
fix: there is no need to split this one and the following[1] into
separate patches; just move these changes out from gitlab-ci ones.
Furthermore, the commit message doesn't relate to the root problem at
all. This error occurs when bootstrap.h is generated outside the source
directory (as a result of the next patch, right?). So I propose the
following:
* join this patch with the following[1], since their separation breaks
out of source build
* write the original purpose for these changes
Here is my version, feel free to adjust it on your own:
| build: generate bootstrap.h in CMAKE_BINARY_DIR
|
| Prior to these changes bootstrap.h was generated right in the source
| directory even for out of source build. Firstly such approach doesn't
| respect the idea of building outside the source files. Furthermore
| this leads to build failures when the source directory is located on
| read-only file system.
|
| As a result of the patch bootstrap.h is generated within the build
| tree and include directories are adjusted the corresponding way.
On 14.10.20, Alexander V. Tikhonov wrote:
> Running out-of-source build got the issue:
>
> /source/src/box/memtx_engine.c:48:23: fatal error: bootstrap.h: No such file or directory
>
> To fix it added path to the header in cmake file.
>
> Part of #4968
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
> Issue: https://github.com/tarantool/tarantool/issues/4968
>
> src/box/CMakeLists.txt | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
> index 8b2e704cf..6e25401a1 100644
> --- a/src/box/CMakeLists.txt
> +++ b/src/box/CMakeLists.txt
> @@ -77,6 +77,7 @@ set_property(DIRECTORY PROPERTY ADDITIONAL_MAKE_CLEAN_FILES ${lua_sources})
>
> include_directories(${ZSTD_INCLUDE_DIRS})
> include_directories(${CMAKE_BINARY_DIR}/src/box/sql)
> +include_directories(${CMAKE_BINARY_DIR}/src/box)
>
> add_library(vclock STATIC vclock.c)
> target_link_libraries(vclock core bit)
> --
> 2.25.1
>
[1]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-October/020159.html
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
` (3 preceding siblings ...)
2020-10-14 9:25 ` [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build Alexander V. Tikhonov
@ 2020-10-15 12:23 ` Igor Munkin
2020-10-16 6:55 ` Kirill Yukhin
5 siblings, 0 replies; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 12:23 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks for the patch! It would be better to either explicitly drop this
line or change it to the valid one (e.g. specify 2.8 version as the
least one, since it's provided by ancient distros), but I don't insist.
This remark is minor, your fix is temporary and just works, so I don't
want to block the patch further. LGTM.
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory
2020-10-14 8:30 ` [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory Alexander V. Tikhonov
@ 2020-10-15 12:24 ` Igor Munkin
0 siblings, 0 replies; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 12:24 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks for the patch! Consider the related comments here[1].
Otherwise, the changes LGTM.
[1]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-October/020236.html
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v2 5/5] gitlab-ci: add out-of-source build
2020-10-14 8:35 ` [Tarantool-patches] [PATCH v1 5/5] gitlab-ci: add out-of-source build Alexander V. Tikhonov
@ 2020-10-15 12:30 ` Igor Munkin
0 siblings, 0 replies; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 12:30 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks for the patch! LGTM, except a couple of nits below.
On 14.10.20, Alexander V. Tikhonov wrote:
> Implemented out-of-source build at cmake files. Added out of source
Typo: The first sentence doesn't relate to the patch.
> build make targets and added test job to gitlab-ci.
>
> Closes #4874
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
> Issue: https://github.com/tarantool/tarantool/issues/4874
>
> V2: removed changes not for gitlab-ci.
>
> .gitlab-ci.yml | 7 +++++++
> .travis.mk | 26 +++++++++++++++++++++++++-
> 2 files changed, 32 insertions(+), 1 deletion(-)
>
<snipped>
> diff --git a/.travis.mk b/.travis.mk
> index d797472e2..5730e1097 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -7,7 +7,9 @@ DOCKER_IMAGE_TARANTOOL="registry.gitlab.com/tarantool/tarantool/testing/debian-s
> TEST_RUN_EXTRA_PARAMS?=
> MAX_FILES?=65534
> MAX_PROC?=2500
> -OOS_SRC_PATH="/source"
> +OOS_SRC_PATH?="/source"
> +OOS_BUILD_PATH?="/rw_bins"
Minor: All these quotes are excess[1], but break nothing.
> +OOS_BUILD_RULE?=test_oos_no_deps
> BIN_DIR=/usr/local/bin
> OSX_VARDIR?=/tmp/tnt
>
<snipped>
> --
> 2.25.1
>
[1]: https://gitlab.com/tarantool/tarantool/-/jobs/790306507#L3757
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build
2020-10-14 9:25 ` [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build Alexander V. Tikhonov
@ 2020-10-15 14:17 ` Igor Munkin
2020-10-15 19:00 ` Alexander V. Tikhonov
0 siblings, 1 reply; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 14:17 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks for the patch! There is one question and several nits below,
please consider them.
On 14.10.20, Alexander V. Tikhonov wrote:
> Initially tried to change autoconf tools in Curl build to cmake and
> found the following build issue on:
>
> CentOS 6,7
Minor (again): IMHO, it's better to split these lines as you did in the
commit you're referring below.
> Debian 8
> Ubuntu 14.04
>
> Issue found:
>
<snipped>
>
> 3. Found issues with building Tarantool statically on Debian 9 and
> its package build on CentOS 8.
>
> Building statically got the issues with openssl linking, like [2]:
>
> static-build/openssl-prefix/lib/libcrypto.a(threads_pthread.o): In function `CRYPTO_THREAD_lock_new':
> threads_pthread.c:(.text+0x45): undefined reference to `pthread_rwlock_init'
>
> It happened because in openssl radically changed how threads-related
> things were handled before 1.1.0 version. It required the application
> to provide us with the locking functionality in form of callbacks.
> Since 1.1.0, these matters are entirely internal, so libcrypto
> requires the presence of the platform specific thread implementation
> of our choosing, which is pthread on everything.
>
> After '-pthread' added to C compile flags package build on CentOS 8
> failed with the issue [3]:
>
> /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:101:4: error: #error "SIZEOF_CURL_OFF_T not defined"
> # error "SIZEOF_CURL_OFF_T not defined"
> ^~~~~
> /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c: In function 'curlx_uztoso':
> /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:192:40: error: 'CURL_MASK_SCOFFT' undeclared (first use in this function); did you mean 'CURL_MASK_SINT'?
> return (curl_off_t)(uznum & (size_t) CURL_MASK_SCOFFT);
> ^~~~~~~~~~~~~~~~
> CURL_MASK_SINT
>
> To avoid of the issue decided to use '-pthread' flag only for openssl
> when it had not this flag in openssl compilation:
>
> # check '-pthread' in openssl compile options
> execute_process(COMMAND openssl version -f
> OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
> # add pthread library for openssl static library linking
> if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
> list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
> endif()
Minor: The wording is nice and quite clear, feel free to drop the code
snipped above.
>
<snipped>
>
> 5. Found that on Gentoo need to switch off use of libSSH2:
Minor: This doesn't relate to Gentoo per se. As I mentioned here[1], the
issue occurred as a result of moving curl build from autotools to CMake:
CMAKE_USE_LIBSSH2 flag is enabled by default, but --with-libssh2 is not.
>
> list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
>
> to avoid of linking issues, like:
>
> ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
>
> this issue exists in curl issues [4].
>
> Closes #4968
> Closes #5019
> Closes #5396
>
> [1] - https://github.com/tarantool/tarantool/issues/4968#issue-617183031
> [2] - https://gitlab.com/tarantool/tarantool/-/jobs/779176133#L6021
> [3] - https://gitlab.com/tarantool/tarantool/-/jobs/778309145#L3060
> [4] - https://github.com/curl/curl/issues/1146
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
> Issue: https://github.com/tarantool/tarantool/issues/4968
>
> V4: Changes:
> - Disabled libSSH2.
> - Added disable symbols hiding flag.
>
> cmake/BuildLibCURL.cmake | 250 +++++++++++++++++++--------------------
> 1 file changed, 121 insertions(+), 129 deletions(-)
>
> diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
> index 86fec39e9..0c3b2fda4 100644
> --- a/cmake/BuildLibCURL.cmake
> +++ b/cmake/BuildLibCURL.cmake
<snipped>
> @@ -19,40 +20,126 @@ macro(curl_build)
> message(FATAL_ERROR "Unable to find zlib")
> endif()
> get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} DIRECTORY)
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DZLIB_ROOT=${FOUND_ZLIB_ROOT_DIR}")
> +
> + # check '-pthread' in openssl compile options
Typo: s/check/Check/.
Typo: s/options/options./.
> + execute_process(COMMAND openssl version -f
> + OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
> + # add pthread library for openssl static library linking
Typo: s/add/Add/.
Typo: s/linking/linking./.
> + if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
> + endif()
> +
> + # add librt for clock_gettime function definition
Typo: s/add/Add/.
Typo: s/definition/definition./.
> + if(${CMAKE_MAJOR_VERSION} VERSION_LESS "3")
> + CHECK_LIBRARY_EXISTS (rt clock_gettime "" HAVE_LIBRT)
> + if (HAVE_LIBRT)
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-lrt")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_CXX_FLAGS=-lrt")
> + endif()
> + endif()
> +
> + # switch on the static build
Typo: s/switch/Switch/.
Typo: s/build/build./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_STATICLIB=ON")
> +
> + # switch off the shared build
Typo: s/switch/Switch/.
Typo: s/build/build./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_SHARED_LIBS=OFF")
<snipped>
> + # let's disable testing for curl to save build time
Typo: s/let/Let/.
Typo: s/time/time./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_TESTING=OFF")
<snipped>
> + # Found that on Gentoo need to switch off use of libSSH2 to avoid of issues, like:
> + # ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
Minor: This doesn't relate to Gentoo per se. As I mentioned here[1], the
issue occurred as a result of moving curl build from autotools to CMake:
CMAKE_USE_LIBSSH2 flag is enabled by default, but --with-libssh2 is not.
Anyway, I see no additions related to this inconsistency. Do other CMake
defaults match the corresponding autotools values?
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
> +
> + # Switch off the group of protocols with special flag HTTP_ONLY:
> + # ftp, file, ldap, ldaps, rtsp, dict, telnet, tftp, pop3, imap, smtp
Typo: s/smtp/smtp./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DHTTP_ONLY=ON")
> +
> + # additionaly disable some more protocols
Typo: s/additionaly/Additionally/.
Typo: s/protocols/protocols./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_SMB=ON")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_GOPHER=ON")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_CRYPTO_AUTH=ON")
> +
> + # switch on ca-fallback feature
Typo: s/switch/Switch/.
Typo: s/feature/feature./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_CA_FALLBACK=ON")
<snipped>
> + # should be already set by "-DHTTP_ONLY=ON" above
Typo: s/should/Should/.
Typo: s/above/above./.
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_FTP=ON")
<snipped>
> + BUILD_COMMAND cd <BINARY_DIR> && $(MAKE) -j
Side note: This change is reverted on the branch, as discussed offline.
> INSTALL_COMMAND cd <BINARY_DIR> && $(MAKE) install)
<snipped>
> @@ -168,6 +165,7 @@ macro(curl_build)
> endif()
> add_dependencies(bundled-libcurl bundled-libcurl-project)
>
> + # setup CURL_INCLUDE_DIRS & CURL_LIBRARIES for global use
Typo: s/setup/Setup/.
Typo: s/use/use./.
> set(CURL_INCLUDE_DIRS ${LIBCURL_INSTALL_DIR}/include)
> set(CURL_LIBRARIES bundled-libcurl ${LIBZ_LIBRARY})
> if (BUNDLED_LIBCURL_USE_ARES)
<snipped>
> --
> 2.25.1
>
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build
2020-10-15 14:17 ` Igor Munkin
@ 2020-10-15 19:00 ` Alexander V. Tikhonov
2020-10-15 20:43 ` Igor Munkin
0 siblings, 1 reply; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-10-15 19:00 UTC (permalink / raw)
To: Igor Munkin; +Cc: tarantool-patches
Hi Igor, I've made all corrections as you suggested and checked all
the new options set in cmake. Founf 3 new options which I've blocked
in curl build:
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DPICKY_COMPILER=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_CURL_EXE=OFF")
+ list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_ZLIB=OFF")
On Thu, Oct 15, 2020 at 05:17:13PM +0300, Igor Munkin wrote:
> Sasha,
>
> Thanks for the patch! There is one question and several nits below,
> please consider them.
>
> On 14.10.20, Alexander V. Tikhonov wrote:
> > Initially tried to change autoconf tools in Curl build to cmake and
> > found the following build issue on:
> >
> > CentOS 6,7
>
> Minor (again): IMHO, it's better to split these lines as you did in the
> commit you're referring below.
>
> > Debian 8
> > Ubuntu 14.04
> >
> > Issue found:
> >
>
> <snipped>
>
> >
> > 3. Found issues with building Tarantool statically on Debian 9 and
> > its package build on CentOS 8.
> >
> > Building statically got the issues with openssl linking, like [2]:
> >
> > static-build/openssl-prefix/lib/libcrypto.a(threads_pthread.o): In function `CRYPTO_THREAD_lock_new':
> > threads_pthread.c:(.text+0x45): undefined reference to `pthread_rwlock_init'
> >
> > It happened because in openssl radically changed how threads-related
> > things were handled before 1.1.0 version. It required the application
> > to provide us with the locking functionality in form of callbacks.
> > Since 1.1.0, these matters are entirely internal, so libcrypto
> > requires the presence of the platform specific thread implementation
> > of our choosing, which is pthread on everything.
> >
> > After '-pthread' added to C compile flags package build on CentOS 8
> > failed with the issue [3]:
> >
> > /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:101:4: error: #error "SIZEOF_CURL_OFF_T not defined"
> > # error "SIZEOF_CURL_OFF_T not defined"
> > ^~~~~
> > /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c: In function 'curlx_uztoso':
> > /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:192:40: error: 'CURL_MASK_SCOFFT' undeclared (first use in this function); did you mean 'CURL_MASK_SINT'?
> > return (curl_off_t)(uznum & (size_t) CURL_MASK_SCOFFT);
> > ^~~~~~~~~~~~~~~~
> > CURL_MASK_SINT
> >
> > To avoid of the issue decided to use '-pthread' flag only for openssl
> > when it had not this flag in openssl compilation:
> >
> > # check '-pthread' in openssl compile options
> > execute_process(COMMAND openssl version -f
> > OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
> > # add pthread library for openssl static library linking
> > if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
> > list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
> > endif()
>
> Minor: The wording is nice and quite clear, feel free to drop the code
> snipped above.
>
> >
>
> <snipped>
>
> >
> > 5. Found that on Gentoo need to switch off use of libSSH2:
>
> Minor: This doesn't relate to Gentoo per se. As I mentioned here[1], the
> issue occurred as a result of moving curl build from autotools to CMake:
> CMAKE_USE_LIBSSH2 flag is enabled by default, but --with-libssh2 is not.
>
> >
> > list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
> >
> > to avoid of linking issues, like:
> >
> > ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
> >
> > this issue exists in curl issues [4].
> >
> > Closes #4968
> > Closes #5019
> > Closes #5396
> >
> > [1] - https://github.com/tarantool/tarantool/issues/4968#issue-617183031
> > [2] - https://gitlab.com/tarantool/tarantool/-/jobs/779176133#L6021
> > [3] - https://gitlab.com/tarantool/tarantool/-/jobs/778309145#L3060
> > [4] - https://github.com/curl/curl/issues/1146
> > ---
> >
> > Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4874-out-of-source-build-full-ci
> > Issue: https://github.com/tarantool/tarantool/issues/4968
> >
> > V4: Changes:
> > - Disabled libSSH2.
> > - Added disable symbols hiding flag.
> >
> > cmake/BuildLibCURL.cmake | 250 +++++++++++++++++++--------------------
> > 1 file changed, 121 insertions(+), 129 deletions(-)
> >
> > diff --git a/cmake/BuildLibCURL.cmake b/cmake/BuildLibCURL.cmake
> > index 86fec39e9..0c3b2fda4 100644
> > --- a/cmake/BuildLibCURL.cmake
> > +++ b/cmake/BuildLibCURL.cmake
>
> <snipped>
>
> > @@ -19,40 +20,126 @@ macro(curl_build)
> > message(FATAL_ERROR "Unable to find zlib")
> > endif()
> > get_filename_component(FOUND_ZLIB_ROOT_DIR ${ZLIB_INCLUDE_DIR} DIRECTORY)
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DZLIB_ROOT=${FOUND_ZLIB_ROOT_DIR}")
> > +
> > + # check '-pthread' in openssl compile options
>
> Typo: s/check/Check/.
> Typo: s/options/options./.
>
> > + execute_process(COMMAND openssl version -f
> > + OUTPUT_VARIABLE OPENSSL_COMPILE_OPTIONS)
> > + # add pthread library for openssl static library linking
>
> Typo: s/add/Add/.
> Typo: s/linking/linking./.
>
> > + if(NOT OPENSSL_COMPILE_OPTIONS MATCHES ".* -pthread .*")
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-pthread")
> > + endif()
> > +
> > + # add librt for clock_gettime function definition
>
> Typo: s/add/Add/.
> Typo: s/definition/definition./.
>
> > + if(${CMAKE_MAJOR_VERSION} VERSION_LESS "3")
> > + CHECK_LIBRARY_EXISTS (rt clock_gettime "" HAVE_LIBRT)
> > + if (HAVE_LIBRT)
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_C_FLAGS=-lrt")
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_CXX_FLAGS=-lrt")
> > + endif()
> > + endif()
> > +
> > + # switch on the static build
>
> Typo: s/switch/Switch/.
> Typo: s/build/build./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_STATICLIB=ON")
> > +
> > + # switch off the shared build
>
> Typo: s/switch/Switch/.
> Typo: s/build/build./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_SHARED_LIBS=OFF")
>
> <snipped>
>
> > + # let's disable testing for curl to save build time
>
> Typo: s/let/Let/.
> Typo: s/time/time./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_TESTING=OFF")
>
> <snipped>
>
> > + # Found that on Gentoo need to switch off use of libSSH2 to avoid of issues, like:
> > + # ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
>
> Minor: This doesn't relate to Gentoo per se. As I mentioned here[1], the
> issue occurred as a result of moving curl build from autotools to CMake:
> CMAKE_USE_LIBSSH2 flag is enabled by default, but --with-libssh2 is not.
>
> Anyway, I see no additions related to this inconsistency. Do other CMake
> defaults match the corresponding autotools values?
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
> > +
> > + # Switch off the group of protocols with special flag HTTP_ONLY:
> > + # ftp, file, ldap, ldaps, rtsp, dict, telnet, tftp, pop3, imap, smtp
>
> Typo: s/smtp/smtp./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DHTTP_ONLY=ON")
> > +
> > + # additionaly disable some more protocols
>
> Typo: s/additionaly/Additionally/.
> Typo: s/protocols/protocols./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_SMB=ON")
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_GOPHER=ON")
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_CRYPTO_AUTH=ON")
> > +
> > + # switch on ca-fallback feature
>
> Typo: s/switch/Switch/.
> Typo: s/feature/feature./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_CA_FALLBACK=ON")
>
> <snipped>
>
> > + # should be already set by "-DHTTP_ONLY=ON" above
>
> Typo: s/should/Should/.
> Typo: s/above/above./.
>
> > + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_DISABLE_FTP=ON")
>
> <snipped>
>
> > + BUILD_COMMAND cd <BINARY_DIR> && $(MAKE) -j
>
> Side note: This change is reverted on the branch, as discussed offline.
>
> > INSTALL_COMMAND cd <BINARY_DIR> && $(MAKE) install)
>
> <snipped>
>
> > @@ -168,6 +165,7 @@ macro(curl_build)
> > endif()
> > add_dependencies(bundled-libcurl bundled-libcurl-project)
> >
> > + # setup CURL_INCLUDE_DIRS & CURL_LIBRARIES for global use
>
> Typo: s/setup/Setup/.
> Typo: s/use/use./.
>
> > set(CURL_INCLUDE_DIRS ${LIBCURL_INSTALL_DIR}/include)
> > set(CURL_LIBRARIES bundled-libcurl ${LIBZ_LIBRARY})
> > if (BUNDLED_LIBCURL_USE_ARES)
>
> <snipped>
>
> > --
> > 2.25.1
> >
>
> --
> Best regards,
> IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build
2020-10-15 19:00 ` Alexander V. Tikhonov
@ 2020-10-15 20:43 ` Igor Munkin
0 siblings, 0 replies; 14+ messages in thread
From: Igor Munkin @ 2020-10-15 20:43 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Sasha,
Thanks, I've checked the new version on the branch. LGTM.
On 15.10.20, Alexander V. Tikhonov wrote:
> Hi Igor, I've made all corrections as you suggested and checked all
> the new options set in cmake. Founf 3 new options which I've blocked
> in curl build:
>
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DPICKY_COMPILER=OFF")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_CURL_EXE=OFF")
> + list(APPEND LIBCURL_CMAKE_FLAGS "-DCURL_ZLIB=OFF")
Side note: Discussed this change offline, and there is no the last line
on the branch.
>
<snipped>
--
Best regards,
IM
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
` (4 preceding siblings ...)
2020-10-15 12:23 ` [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Igor Munkin
@ 2020-10-16 6:55 ` Kirill Yukhin
5 siblings, 0 replies; 14+ messages in thread
From: Kirill Yukhin @ 2020-10-16 6:55 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Hello,
On 14 окт 11:27, Alexander V. Tikhonov wrote:
> Blocked check if the current cmake of the older than
> 3.2...3.16 version to be able to build curl at the old OS:
>
> - CentOS 6
> - CentOS 7
> - Debian 8
> - Ubuntu 14.04
>
> Used in 'third_party/curl' repository branch:
>
> curl-7_71_1-tarantool
I've checked your patchset into master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
2020-10-15 9:18 ` Igor Munkin
@ 2021-03-12 8:55 ` Kirill Yukhin via Tarantool-patches
1 sibling, 0 replies; 14+ messages in thread
From: Kirill Yukhin via Tarantool-patches @ 2021-03-12 8:55 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
Hello,
On 14 окт 11:29, Alexander V. Tikhonov wrote:
> Running out-of-source build got the issue:
>
> /source/src/box/memtx_engine.c:48:23: fatal error: bootstrap.h: No such file or directory
>
> To fix it added path to the header in cmake file.
>
> Part of #4968
I've cherry-picked your patch into 1.10.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-03-12 8:55 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-14 8:27 [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Alexander V. Tikhonov
2020-10-14 8:29 ` [Tarantool-patches] [PATCH v1 2/5] build: set include path to bootstrap.h Alexander V. Tikhonov
2020-10-15 9:18 ` Igor Munkin
2021-03-12 8:55 ` Kirill Yukhin via Tarantool-patches
2020-10-14 8:30 ` [Tarantool-patches] [PATCH v1 3/5] build: correct destination build directory Alexander V. Tikhonov
2020-10-15 12:24 ` Igor Munkin
2020-10-14 8:35 ` [Tarantool-patches] [PATCH v1 5/5] gitlab-ci: add out-of-source build Alexander V. Tikhonov
2020-10-15 12:30 ` [Tarantool-patches] [PATCH v2 " Igor Munkin
2020-10-14 9:25 ` [Tarantool-patches] [PATCH v4 4/5] build: enable cmake in curl build Alexander V. Tikhonov
2020-10-15 14:17 ` Igor Munkin
2020-10-15 19:00 ` Alexander V. Tikhonov
2020-10-15 20:43 ` Igor Munkin
2020-10-15 12:23 ` [Tarantool-patches] [PATCH v1 1/5] curl: enable curl build with old cmake Igor Munkin
2020-10-16 6:55 ` Kirill Yukhin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox