[Tarantool-patches] [PATCH v1] static-build: use hashes of downloaded files

[Tarantool-patches] [PATCH v1] static-build: use hashes of downloaded files

[Sergey Bronnikov via Tarantool-patches]

From: Sergey Bronnikov <sergeyb@tarantool.org>

Using hashes in ExternalProject_Add() [1] allows to avoid extra
downloads on rebuilds.

1. https://cmake.org/cmake/help/latest/module/ExternalProject.html

Closes #5761
---
Gitlab CI: https://gitlab.com/tarantool/tarantool/-/pipelines/247666614
Branch: ligurio/gh-5761-hashes
Issue: https://github.com/tarantool/tarantool/issues/5761

 static-build/CMakeLists.txt | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
index 9a2f85052..39cb321d0 100644
--- a/static-build/CMakeLists.txt
+++ b/static-build/CMakeLists.txt
@@ -40,6 +40,7 @@ endif()
 #
 ExternalProject_Add(openssl
     URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
+    URL_MD5 3f486f2f4435ef14b81814dbbc7b48bb
     CONFIGURE_COMMAND <SOURCE_DIR>/config
         CC=${CMAKE_C_COMPILER}
         CXX=${CMAKE_CXX_COMPILER}
@@ -84,6 +85,7 @@ ExternalProject_Add(icu
 #
 ExternalProject_Add(zlib
     URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
+    URL_MD5 1c9f62f0778697a09d36121ead88e08e
     CONFIGURE_COMMAND env
         CC=${CMAKE_C_COMPILER}
         CFLAGS=${DEPENDENCY_CFLAGS}
@@ -99,6 +101,7 @@ ExternalProject_Add(zlib
 #
 ExternalProject_Add(ncurses
     URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz
+    URL_MD5 e812da327b1c2214ac1aed440ea3ae8d
     CONFIGURE_COMMAND <SOURCE_DIR>/configure
         CC=${CMAKE_C_COMPILER}
         CXX=${CMAKE_CXX_COMPILER}
@@ -132,6 +135,7 @@ ExternalProject_Add(ncurses
 #
 ExternalProject_Add(readline
     URL https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz
+    URL_MD5 7e6c1f16aee3244a69aba6e438295ca3
     CONFIGURE_COMMAND <SOURCE_DIR>/configure
         CC=${CMAKE_C_COMPILER}
         CFLAGS=${DEPENDENCY_CFLAGS}
@@ -241,6 +245,7 @@ if (APPLE)
 else()
     ExternalProject_Add(unwind
         URL https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz
+        URL_MD5 f09b670de5db6430a3de666e6aed60e3
         CONFIGURE_COMMAND <SOURCE_DIR>/configure
             CC=${CMAKE_C_COMPILER}
             CXX=${CMAKE_CXX_COMPILER}
-- 
2.25.1

Re: [Tarantool-patches] [PATCH v1] static-build: use hashes of downloaded files

[Alexander Turenko via Tarantool-patches]

On Wed, Jan 27, 2021 at 07:37:01PM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
> 
> Using hashes in ExternalProject_Add() [1] allows to avoid extra
> downloads on rebuilds.
> 
> 1. https://cmake.org/cmake/help/latest/module/ExternalProject.html
> 
> Closes #5761

I don't know this machinery much, but it looks worthful and I don't see
any downsides.

LGTM.

> diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
> index 9a2f85052..39cb321d0 100644
> --- a/static-build/CMakeLists.txt
> +++ b/static-build/CMakeLists.txt
> @@ -40,6 +40,7 @@ endif()
>  #
>  ExternalProject_Add(openssl
>      URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
> +    URL_MD5 3f486f2f4435ef14b81814dbbc7b48bb

Nit: I would place certain hash values near to the versions block to
don't spread dependent information and descrease probability to forget
to update a checksum together with a version.

Re: [Tarantool-patches] [PATCH v1] static-build: use hashes of downloaded files

[Sergey Bronnikov via Tarantool-patches]

Thanks for review!

On 27.01.2021 21:26, Alexander Turenko wrote:
> On Wed, Jan 27, 2021 at 07:37:01PM +0300, sergeyb@tarantool.org wrote:
>> From: Sergey Bronnikov <sergeyb@tarantool.org>
>>
>> Using hashes in ExternalProject_Add() [1] allows to avoid extra
>> downloads on rebuilds.
>>
>> 1. https://cmake.org/cmake/help/latest/module/ExternalProject.html
>>
>> Closes #5761
> I don't know this machinery much, but it looks worthful and I don't see
> any downsides.
>
> LGTM.
>
>> diff --git a/static-build/CMakeLists.txt b/static-build/CMakeLists.txt
>> index 9a2f85052..39cb321d0 100644
>> --- a/static-build/CMakeLists.txt
>> +++ b/static-build/CMakeLists.txt
>> @@ -40,6 +40,7 @@ endif()
>>   #
>>   ExternalProject_Add(openssl
>>       URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
>> +    URL_MD5 3f486f2f4435ef14b81814dbbc7b48bb
> Nit: I would place certain hash values near to the versions block to
> don't spread dependent information and descrease probability to forget
> to update a checksum together with a version.

Changing version without updating hashes is not possible.

Although, I'm agree placing hashes near version would be more convenient.

@@ -9,10 +9,15 @@ project(tarantool-static C CXX)

  include(ExternalProject)
  set(OPENSSL_VERSION 1.1.1f)
+set(OPENSSL_HASH 3f486f2f4435ef14b81814dbbc7b48bb)
  set(ZLIB_VERSION 1.2.11)
+set(ZLIB_HASH 1c9f62f0778697a09d36121ead88e08e)
  set(NCURSES_VERSION 6.2)
+set(NCURSES_HASH e812da327b1c2214ac1aed440ea3ae8d)
  set(READLINE_VERSION 8.0)
+set(READLINE_HASH 7e6c1f16aee3244a69aba6e438295ca3)
  set(UNWIND_VERSION 1.3-rc1)
+set(UNWIND_HASH f09b670de5db6430a3de666e6aed60e3)

  # Pass -isysroot=<SDK_PATH> option on Mac OS to a preprocessor and a C
  # compiler to find header files installed with an SDK.
@@ -40,7 +45,7 @@ endif()
  #
  ExternalProject_Add(openssl
      URL https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
-    URL_MD5 3f486f2f4435ef14b81814dbbc7b48bb
+    URL_MD5 ${OPENSSL_HASH}
      CONFIGURE_COMMAND <SOURCE_DIR>/config
          CC=${CMAKE_C_COMPILER}
          CXX=${CMAKE_CXX_COMPILER}
@@ -85,7 +90,7 @@ ExternalProject_Add(icu
  #
  ExternalProject_Add(zlib
      URL https://zlib.net/zlib-${ZLIB_VERSION}.tar.gz
-    URL_MD5 1c9f62f0778697a09d36121ead88e08e
+    URL_MD5 ${ZLIB_HASH}
      CONFIGURE_COMMAND env
          CC=${CMAKE_C_COMPILER}
          CFLAGS=${DEPENDENCY_CFLAGS}
@@ -101,7 +106,7 @@ ExternalProject_Add(zlib
  #
  ExternalProject_Add(ncurses
      URL https://ftp.gnu.org/gnu/ncurses/ncurses-${NCURSES_VERSION}.tar.gz
-    URL_MD5 e812da327b1c2214ac1aed440ea3ae8d
+    URL_MD5 ${NCURSES_HASH}
      CONFIGURE_COMMAND <SOURCE_DIR>/configure
          CC=${CMAKE_C_COMPILER}
          CXX=${CMAKE_CXX_COMPILER}
@@ -135,7 +140,7 @@ ExternalProject_Add(ncurses
  #
  ExternalProject_Add(readline
      URL 
https://ftp.gnu.org/gnu/readline/readline-${READLINE_VERSION}.tar.gz
-    URL_MD5 7e6c1f16aee3244a69aba6e438295ca3
+    URL_MD5 ${READLINE_HASH}
      CONFIGURE_COMMAND <SOURCE_DIR>/configure
          CC=${CMAKE_C_COMPILER}
          CFLAGS=${DEPENDENCY_CFLAGS}
@@ -245,7 +250,7 @@ if (APPLE)
  else()
      ExternalProject_Add(unwind
          URL 
https://download.savannah.nongnu.org/releases/libunwind/libunwind-${UNWIND_VERSION}.tar.gz
-        URL_MD5 f09b670de5db6430a3de666e6aed60e3
+        URL_MD5 ${UNWIND_HASH}
          CONFIGURE_COMMAND <SOURCE_DIR>/configure
              CC=${CMAKE_C_COMPILER}
              CXX=${CMAKE_CXX_COMPILER}

Re: [Tarantool-patches] [PATCH v1] static-build: use hashes of downloaded files

[Kirill Yukhin via Tarantool-patches]

Hello,

On 27 янв 19:37, Sergey Bronnikov via Tarantool-patches wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
> 
> Using hashes in ExternalProject_Add() [1] allows to avoid extra
> downloads on rebuilds.
> 
> 1. https://cmake.org/cmake/help/latest/module/ExternalProject.html
> 
> Closes #5761

I've checked your patch into 2.6, 2.7 and master.

--
Regards, Kirill Yukhin