[Tarantool-patches] [PATCH v2 1/1] gitlab-ci: disable at OSX curl SSL cert check

[Tarantool-patches] [PATCH v2 1/1] gitlab-ci: disable at OSX curl SSL cert check

[Alexander V. Tikhonov]

Formula tntpython2.rb consist of the packages which download target host
does not have valid SSL certificate, disabled curl SSL check for it.

Close #4883
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4883

 .travis.mk | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/.travis.mk b/.travis.mk
index 001752844..68cc12c0b 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -154,22 +154,33 @@ test_static_docker_build:
 # OSX #
 #######
 
-# since Python 2 is EOL it's latest commit from tapped local formula is used
 OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool \
-	cmake file://$${PWD}/tools/brew_taps/tntpython2.rb
+	cmake
 
 deps_osx:
-	# install brew using command from Homebrew repository instructions:
+	# Install brew using command from Homebrew repository instructions:
 	#   https://github.com/Homebrew/install
 	# NOTE: 'echo' command below is required since brew installation
 	# script obliges the one to enter a newline for confirming the
 	# installation via Ruby script.
 	brew update || echo | /usr/bin/ruby -e \
 		"$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-	# try to install the packages either upgrade it to avoid of fails
-	# if the package already exists with the previous version
+	# Try to install the packages either upgrade it to avoid of fails
+	# if the package already exists with the previous version.
 	brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
-	pip install --force-reinstall -r test-run/requirements.txt
+	# Since Python 2 is EOL, it's latest commit from tapped local formula is
+	# used. Some packages from tntpython2.rb formula use external download
+	# hosts which do not have valid SSL certificate. To resolve it the SSL
+	# certificates check need to be disabled during formula installation.
+	echo insecure >>$${HOME}/.curlrc
+	brew install --force file://$${PWD}/tools/brew_taps/tntpython2.rb || :
+	sed '$$d' $${HOME}/.curlrc >$${HOME}/.curlrc.new && \
+		mv $${HOME}/.curlrc.new $${HOME}/.curlrc
+	python2 -V
+	pip install --trusted-host files.pythonhosted.org \
+		--upgrade pip setuptools
+	pip install --trusted-host files.pythonhosted.org \
+		--force-reinstall -r test-run/requirements.txt
 
 build_osx:
 	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 1/1] gitlab-ci: disable at OSX curl SSL cert check

[Sergey Bronnikov]

Hi,

Thanks for the patch. See 2 comments inline.

On 17:53 Mon 13 Apr , Alexander V. Tikhonov wrote:
> Formula tntpython2.rb consist of the packages which download target host
> does not have valid SSL certificate, disabled curl SSL check for it.

1. From patch it is unclear what host has an invalid SSL certificate.
Could you elaborate?

> Close #4883
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina-full-ci
> Issue: https://github.com/tarantool/tarantool/issues/4883
> 
>  .travis.mk | 23 +++++++++++++++++------
>  1 file changed, 17 insertions(+), 6 deletions(-)
> 
> diff --git a/.travis.mk b/.travis.mk
> index 001752844..68cc12c0b 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -154,22 +154,33 @@ test_static_docker_build:
>  # OSX #
>  #######
>  
> -# since Python 2 is EOL it's latest commit from tapped local formula is used
>  OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool \
> -	cmake file://$${PWD}/tools/brew_taps/tntpython2.rb
> +	cmake
>  
>  deps_osx:
> -	# install brew using command from Homebrew repository instructions:
> +	# Install brew using command from Homebrew repository instructions:
>  	#   https://github.com/Homebrew/install
>  	# NOTE: 'echo' command below is required since brew installation
>  	# script obliges the one to enter a newline for confirming the
>  	# installation via Ruby script.
>  	brew update || echo | /usr/bin/ruby -e \
>  		"$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
> -	# try to install the packages either upgrade it to avoid of fails
> -	# if the package already exists with the previous version
> +	# Try to install the packages either upgrade it to avoid of fails
> +	# if the package already exists with the previous version.
>  	brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
> -	pip install --force-reinstall -r test-run/requirements.txt
> +	# Since Python 2 is EOL, it's latest commit from tapped local formula is
> +	# used. Some packages from tntpython2.rb formula use external download
> +	# hosts which do not have valid SSL certificate. To resolve it the SSL
> +	# certificates check need to be disabled during formula installation.
> +	echo insecure >>$${HOME}/.curlrc
> +	brew install --force file://$${PWD}/tools/brew_taps/tntpython2.rb || :
> +	sed '$$d' $${HOME}/.curlrc >$${HOME}/.curlrc.new && \
> +		mv $${HOME}/.curlrc.new $${HOME}/.curlrc

2. Failed to understand this magick with .curlrc. Could you explain?

> +	python2 -V
> +	pip install --trusted-host files.pythonhosted.org \
> +		--upgrade pip setuptools
> +	pip install --trusted-host files.pythonhosted.org \
> +		--force-reinstall -r test-run/requirements.txt
>  
>  build_osx:
>  	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
> -- 
> 2.17.1
> 

-- 
sergeyb@

Re: [Tarantool-patches] [PATCH v2 1/1] gitlab-ci: disable at OSX curl SSL cert check

[Alexander Tikhonov]

[-- Attachment #1: Type: text/plain, Size: 3828 bytes --]


Hi Sergey, thanks for the review, please check my comments below.

  
>Вторник, 14 апреля 2020, 11:07 +03:00 от Sergey Bronnikov <sergeyb@tarantool.org>:
> 
>Hi,
>
>Thanks for the patch. See 2 comments inline.
>
>On 17:53 Mon 13 Apr , Alexander V. Tikhonov wrote:
>> Formula tntpython2.rb consist of the packages which download target host
>> does not have valid SSL certificate, disabled curl SSL check for it.
>
>1. From patch it is unclear what host has an invalid SSL certificate.
>Could you elaborate?
The root cause of the issue is not that SSL certificate wrong, but local host may have wrong local setup which produce the issue with SSL certificate checking, like date my be not synchronized either some other issues, which we don’t want to fix. That is why the regular  https://files.pythonhosted.org  host may have SSL certificate which will block the installation process even it is fine, but the target host may not approve it. So the better way to avoid our hosts from it is to block just for this formula SSL certificate even we know the host name and prove it.
>
>> Close #4883
>> ---
>>
>> Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina-full-ci
>> Issue:  https://github.com/tarantool/tarantool/issues/4883
>>
>> .travis.mk | 23 +++++++++++++++++------
>> 1 file changed, 17 insertions(+), 6 deletions(-)
>>
>> diff --git a/.travis.mk b/.travis.mk
>> index 001752844..68cc12c0b 100644
>> --- a/.travis.mk
>> +++ b/.travis.mk
>> @@ -154,22 +154,33 @@ test_static_docker_build:
>> # OSX #
>> #######
>>
>> -# since Python 2 is EOL it's latest commit from tapped local formula is used
>> OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool \
>> - cmake file://${PWD}/tools/brew_taps/tntpython2.rb
>> + cmake
>>
>> deps_osx:
>> - # install brew using command from Homebrew repository instructions:
>> + # Install brew using command from Homebrew repository instructions:
>> #  https://github.com/Homebrew/install
>> # NOTE: 'echo' command below is required since brew installation
>> # script obliges the one to enter a newline for confirming the
>> # installation via Ruby script.
>> brew update || echo | /usr/bin/ruby -e \
>> "$(curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install )"
>> - # try to install the packages either upgrade it to avoid of fails
>> - # if the package already exists with the previous version
>> + # Try to install the packages either upgrade it to avoid of fails
>> + # if the package already exists with the previous version.
>> brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>> - pip install --force-reinstall -r test-run/requirements.txt
>> + # Since Python 2 is EOL, it's latest commit from tapped local formula is
>> + # used. Some packages from tntpython2.rb formula use external download
>> + # hosts which do not have valid SSL certificate. To resolve it the SSL
>> + # certificates check need to be disabled during formula installation.
>> + echo insecure >>${HOME}/.curlrc
>> + brew install --force file://${PWD}/tools/brew_taps/tntpython2.rb || :
>> + sed '$d' ${HOME}/.curlrc >${HOME}/.curlrc.new && \
>> + mv ${HOME}/.curlrc.new ${HOME}/.curlrc
>
>2. Failed to understand this magick with .curlrc. Could you explain?
Actually seems that I forgot to optimize the temporary code - fixed it to «sed … -i .curlrc». By the way sed here removes last line from the file.
>
>> + python2 -V
>> + pip install --trusted-host files.pythonhosted.org \
>> + --upgrade pip setuptools
>> + pip install --trusted-host files.pythonhosted.org \
>> + --force-reinstall -r test-run/requirements.txt
>>
>> build_osx:
>> cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>> --
>> 2.17.1
>>
>
>--
>sergeyb@ 
 
 
--
Alexander Tikhonov
 

[-- Attachment #2: Type: text/html, Size: 5395 bytes --]