[Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Alexander V. Tikhonov]

Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
due to decided to have only 2 last major releases, for now it is
10.14 and 10.15 OSX versions. Also changed the commit job for branches
from 10.14 to 10.15 OSX version.

Close #4873
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina-full-ci
Issue: https://github.com/tarantool/tarantool/issues/4873

 .gitlab-ci.yml | 26 +++++++++++---------------
 1 file changed, 11 insertions(+), 15 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fd265aa60..09feda7f3 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -139,32 +139,28 @@ release_asan_clang8:
   script:
     - ${GITLAB_MAKE} test_asan_debian_no_deps
 
-osx_13_release:
+osx_14_release:
   <<: *release_only_definition
-  <<: *vbox_definition
+  stage: test
   tags:
-    - vms_osx_13
-  variables:
-    VMS_NAME: 'osx_13'
-    VMS_USER: 'tarantool'
-    VMS_PORT: '2212'
+    - osx_14
   script:
-    - ${GITLAB_MAKE} vms_test_osx
+    - ${GITLAB_MAKE} test_osx
 
-osx_14_release:
-  <<: *docker_test_definition
+osx_15_release:
+  stage: test
   tags:
-    - osx_14
+    - osx_15
   script:
     - ${GITLAB_MAKE} test_osx
 
-osx_14_release_lto:
+osx_15_release_lto:
   <<: *release_only_definition
-  <<: *docker_test_definition
+  stage: test
   tags:
-    - osx_14
+    - osx_15
   variables:
-    EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
+    EXTRA_ENV: 'export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;'
   script:
     - ${GITLAB_MAKE} test_osx
 
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Sergey Bronnikov]

On 15:23 Thu 30 Apr , Alexander V. Tikhonov wrote:
> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
> due to decided to have only 2 last major releases, for now it is
> 10.14 and 10.15 OSX versions. Also changed the commit job for branches
> from 10.14 to 10.15 OSX version.
> 
> Close #4873
> ---

LGTM

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Oleg Piskunov]

[-- Attachment #1: Type: text/plain, Size: 1767 bytes --]


LGTM.
  
>Четверг, 30 апреля 2020, 15:23 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
> 
>Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
>due to decided to have only 2 last major releases, for now it is
>10.14 and 10.15 OSX versions. Also changed the commit job for branches
>from 10.14 to 10.15 OSX version.
>
>Close #4873
>---
>
>Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina-full-ci
>Issue:  https://github.com/tarantool/tarantool/issues/4873
>
> .gitlab-ci.yml | 26 +++++++++++---------------
> 1 file changed, 11 insertions(+), 15 deletions(-)
>
>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>index fd265aa60..09feda7f3 100644
>--- a/.gitlab-ci.yml
>+++ b/.gitlab-ci.yml
>@@ -139,32 +139,28 @@ release_asan_clang8:
>   script:
>     - ${GITLAB_MAKE} test_asan_debian_no_deps
> 
>-osx_13_release:
>+osx_14_release:
>   <<: *release_only_definition
>- <<: *vbox_definition
>+ stage: test
>   tags:
>- - vms_osx_13
>- variables:
>- VMS_NAME: 'osx_13'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2212'
>+ - osx_14
>   script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
> 
>-osx_14_release:
>- <<: *docker_test_definition
>+osx_15_release:
>+ stage: test
>   tags:
>- - osx_14
>+ - osx_15
>   script:
>     - ${GITLAB_MAKE} test_osx
> 
>-osx_14_release_lto:
>+osx_15_release_lto:
>   <<: *release_only_definition
>- <<: *docker_test_definition
>+ stage: test
>   tags:
>- - osx_14
>+ - osx_15
>   variables:
>- EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>+ EXTRA_ENV: 'export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;'
>   script:
>     - ${GITLAB_MAKE} test_osx
> 
>--
>2.17.1
>  
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 2750 bytes --]

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Kirill Yukhin]

Hello,

On 30 апр 15:23, Alexander V. Tikhonov wrote:
> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
> due to decided to have only 2 last major releases, for now it is
> 10.14 and 10.15 OSX versions. Also changed the commit job for branches
> from 10.14 to 10.15 OSX version.
> 
> Close #4873

I've checked your patch into 1.10, 2.3, 2.4 and master.

--
Regards, Kirill Yukhin

[Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Alexander V. Tikhonov]

Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
due to decided to have only 2 last major releases, for now it is
10.14 and 10.15 OSX versions.

Also added homebrew installation routine as it was suggested in its
instructions. Added path in environment to reach the pip binary that
was additionally installed based on python2 version in addition to
pip3 which is the default in Catalina OSX 10.15. Added installation
of the cmake and tool. Added upgrade of the OSX packages to avoid of
fails on already existed packages, but of the previous versions.
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina

 .gitlab-ci.yml |  8 ++++----
 .travis.mk     | 30 +++++++++++++++++++++++++-----
 2 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3af5a3c8a..d3cb959d0 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -90,15 +90,15 @@ release_asan_clang8:
   script:
     - ${GITLAB_MAKE} test_asan_debian_no_deps
 
-osx_13_release:
+osx_15_release:
   <<: *release_only_definition
   <<: *vbox_definition
   tags:
-    - vms_osx_13
+    - vms_test
   variables:
-    VMS_NAME: 'osx_13'
+    VMS_NAME: 'osx_15'
     VMS_USER: 'tarantool'
-    VMS_PORT: '2212'
+    VMS_PORT: '2242'
   script:
     - ${GITLAB_MAKE} vms_test_osx
 
diff --git a/.travis.mk b/.travis.mk
index 42969ff56..c8cb273fa 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -127,17 +127,37 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
 # OSX #
 #######
 
+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake
+
 deps_osx:
-	brew update
-	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
+	# install brew using command from Homebrew repository instructions:
+	#   https://github.com/Homebrew/install
+	export PATH=${PATH}:/usr/local/bin ; \
+		brew update || echo | /usr/bin/ruby -e \
+			"`curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install`"
+	# try to install the packages either upgrade it to avoid of fails
+	# if the package allready exists but of the previous version
+	export PATH=${PATH}:/usr/local/bin ; \
+		brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
+	# OSX 10.15 Catalina started to use Xcode 11 which has 'Deprications'
+	# part in Release Notes:
+	#   Use of Python 2.7 isn’t recommended. This version is included in
+	#   macOS for compatibility with legacy software. Future versions of
+	#   macOS won’t include Python 2.7. Instead, it’s recommended that
+	#   you run python3 in Terminal. (51097165)
 	python2 -V || brew install python2 --force
 	curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
 	python get-pip.py --user
-	pip install --user --force-reinstall -r test-run/requirements.txt
+	# To make pip tools available, the PATH environment must include the
+	# path to its binaries
+	export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; \
+		pip install --user --force-reinstall -r test-run/requirements.txt
 
 build_osx:
-	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
-	make -j
+	export PATH=${PATH}:/usr/local/bin ; \
+		cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
+			-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} && \
+		make -j
 
 test_osx_no_deps: build_osx
 	# Limits: Increase the maximum number of open file descriptors on macOS:
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Igor Munkin]

Sasha,

Thanks for the patch, it totally LGTM. I left a couple of minor comments
below, feel free to ignore them.

On 09.12.19, Alexander V. Tikhonov wrote:
> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
> due to decided to have only 2 last major releases, for now it is
> 10.14 and 10.15 OSX versions.
> 
> Also added homebrew installation routine as it was suggested in its
> instructions. Added path in environment to reach the pip binary that
> was additionally installed based on python2 version in addition to
> pip3 which is the default in Catalina OSX 10.15. Added installation
> of the cmake and tool. Added upgrade of the OSX packages to avoid of
> fails on already existed packages, but of the previous versions.
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina
> 
>  .gitlab-ci.yml |  8 ++++----
>  .travis.mk     | 30 +++++++++++++++++++++++++-----
>  2 files changed, 29 insertions(+), 9 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 3af5a3c8a..d3cb959d0 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -90,15 +90,15 @@ release_asan_clang8:
>    script:
>      - ${GITLAB_MAKE} test_asan_debian_no_deps
>  
> -osx_13_release:
> +osx_15_release:
>    <<: *release_only_definition
>    <<: *vbox_definition
>    tags:
> -    - vms_osx_13
> +    - vms_test
>    variables:
> -    VMS_NAME: 'osx_13'
> +    VMS_NAME: 'osx_15'
>      VMS_USER: 'tarantool'
> -    VMS_PORT: '2212'
> +    VMS_PORT: '2242'
>    script:
>      - ${GITLAB_MAKE} vms_test_osx
>  
> diff --git a/.travis.mk b/.travis.mk
> index 42969ff56..c8cb273fa 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -127,17 +127,37 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>  # OSX #
>  #######
>  
> +OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake
> +
>  deps_osx:
> -	brew update
> -	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
> +	# install brew using command from Homebrew repository instructions:
> +	#   https://github.com/Homebrew/install

Minor: As discussed offline, please consider to add a comment regarding
the echo command below. I guess something like one below will be a great
one:
| XXX Fragile: echo command below is required since brew installation
| script obligues the one to enter a newline for confirming the
| installation via Ruby script.

Minor: Please sub `...` to $(...) if possible.

> +	export PATH=${PATH}:/usr/local/bin ; \
> +		brew update || echo | /usr/bin/ruby -e \
> +			"`curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install`"
> +	# try to install the packages either upgrade it to avoid of fails
> +	# if the package allready exists but of the previous version
> +	export PATH=${PATH}:/usr/local/bin ; \
> +		brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
> +	# OSX 10.15 Catalina started to use Xcode 11 which has 'Deprications'
> +	# part in Release Notes:
> +	#   Use of Python 2.7 isn’t recommended. This version is included in
> +	#   macOS for compatibility with legacy software. Future versions of
> +	#   macOS won’t include Python 2.7. Instead, it’s recommended that
> +	#   you run python3 in Terminal. (51097165)
>  	python2 -V || brew install python2 --force
>  	curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
>  	python get-pip.py --user
> -	pip install --user --force-reinstall -r test-run/requirements.txt
> +	# To make pip tools available, the PATH environment must include the
> +	# path to its binaries
> +	export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; \
> +		pip install --user --force-reinstall -r test-run/requirements.txt
>  
>  build_osx:
> -	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
> -	make -j
> +	export PATH=${PATH}:/usr/local/bin ; \
> +		cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
> +			-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} && \
> +		make -j
>  
>  test_osx_no_deps: build_osx
>  	# Limits: Increase the maximum number of open file descriptors on macOS:
> -- 
> 2.17.1
> 

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Alexander Tikhonov]

[-- Attachment #1: Type: text/plain, Size: 4521 bytes --]

Igor, thanks for your review, I've made all the changes as you suggested.


>Понедельник, 16 декабря 2019, 16:25 +03:00 от Igor Munkin <imun@tarantool.org>:
>
>Sasha,
>
>Thanks for the patch, it totally LGTM. I left a couple of minor comments
>below, feel free to ignore them.
>
>On 09.12.19, Alexander V. Tikhonov wrote:
>> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
>> due to decided to have only 2 last major releases, for now it is
>> 10.14 and 10.15 OSX versions.
>> 
>> Also added homebrew installation routine as it was suggested in its
>> instructions. Added path in environment to reach the pip binary that
>> was additionally installed based on python2 version in addition to
>> pip3 which is the default in Catalina OSX 10.15. Added installation
>> of the cmake and tool. Added upgrade of the OSX packages to avoid of
>> fails on already existed packages, but of the previous versions.
>> ---
>> 
>> Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina
>> 
>>  .gitlab-ci.yml |  8 ++++----
>>  .travis.mk     | 30 +++++++++++++++++++++++++-----
>>  2 files changed, 29 insertions(+), 9 deletions(-)
>> 
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 3af5a3c8a..d3cb959d0 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -90,15 +90,15 @@ release_asan_clang8:
>>    script:
>>      - ${GITLAB_MAKE} test_asan_debian_no_deps
>> 
>> -osx_13_release:
>> +osx_15_release:
>>    <<: *release_only_definition
>>    <<: *vbox_definition
>>    tags:
>> -    - vms_osx_13
>> +    - vms_test
>>    variables:
>> -    VMS_NAME: 'osx_13'
>> +    VMS_NAME: 'osx_15'
>>      VMS_USER: 'tarantool'
>> -    VMS_PORT: '2212'
>> +    VMS_PORT: '2242'
>>    script:
>>      - ${GITLAB_MAKE} vms_test_osx
>> 
>> diff --git a/.travis.mk b/.travis.mk
>> index 42969ff56..c8cb273fa 100644
>> --- a/.travis.mk
>> +++ b/.travis.mk
>> @@ -127,17 +127,37 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>>  # OSX #
>>  #######
>> 
>> +OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake
>> +
>>  deps_osx:
>> -	brew update
>> -	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>> +	# install brew using command from Homebrew repository instructions:
>> +	#  https://github.com/Homebrew/install
>
>Minor: As discussed offline, please consider to add a comment regarding
>the echo command below. I guess something like one below will be a great
>one:
>| XXX Fragile: echo command below is required since brew installation
>| script obliges the one to enter a newline for confirming the
>| installation via Ruby script.
Added.
>
>Minor: Please sub `...` to $(...) if possible.
Changed.
>
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		brew update || echo | /usr/bin/ruby -e \
>> +			"`curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install` "
>> +	# try to install the packages either upgrade it to avoid of fails
>> +	# if the package allready exists but of the previous version
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>> +	# OSX 10.15 Catalina started to use Xcode 11 which has 'Deprications'
>> +	# part in Release Notes:
>> +	#   Use of Python 2.7 isn’t recommended. This version is included in
>> +	#   macOS for compatibility with legacy software. Future versions of
>> +	#   macOS won’t include Python 2.7. Instead, it’s recommended that
>> +	#   you run python3 in Terminal. (51097165)
>>  	python2 -V || brew install python2 --force
>>  	curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>>  	python get-pip.py --user
>> -	pip install --user --force-reinstall -r test-run/requirements.txt
>> +	# To make pip tools available, the PATH environment must include the
>> +	# path to its binaries
>> +	export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; \
>> +		pip install --user --force-reinstall -r test-run/requirements.txt
>> 
>>  build_osx:
>> -	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>> -	make -j
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
>> +			-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} && \
>> +		make -j
>> 
>>  test_osx_no_deps: build_osx
>>  	# Limits: Increase the maximum number of open file descriptors on macOS:
>> -- 
>> 2.17.1
>> 
>
>-- 
>Best regards,
>IM


-- 
Alexander Tikhonov

[-- Attachment #2: Type: text/html, Size: 6233 bytes --]

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Alexander Turenko]

To be honest, I didn't get most of changes here. So below I asked for
explanation.

WBR, Alexander Turenko.

On Mon, Dec 09, 2019 at 02:45:54PM +0300, Alexander V. Tikhonov wrote:
> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
> due to decided to have only 2 last major releases, for now it is
> 10.14 and 10.15 OSX versions.
> 
> Also added homebrew installation routine as it was suggested in its
> instructions. Added path in environment to reach the pip binary that
> was additionally installed based on python2 version in addition to
> pip3 which is the default in Catalina OSX 10.15. Added installation
> of the cmake and tool. Added upgrade of the OSX packages to avoid of
> fails on already existed packages, but of the previous versions.
> ---
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina
> 
>  .gitlab-ci.yml |  8 ++++----
>  .travis.mk     | 30 +++++++++++++++++++++++++-----
>  2 files changed, 29 insertions(+), 9 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 3af5a3c8a..d3cb959d0 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -90,15 +90,15 @@ release_asan_clang8:
>    script:
>      - ${GITLAB_MAKE} test_asan_debian_no_deps
>  
> -osx_13_release:
> +osx_15_release:
>    <<: *release_only_definition
>    <<: *vbox_definition
>    tags:
> -    - vms_osx_13
> +    - vms_test

Is it separate change? Is is about load balancing between our hosts?

If so, let's move this change into its own commit.

>    variables:
> -    VMS_NAME: 'osx_13'
> +    VMS_NAME: 'osx_15'
>      VMS_USER: 'tarantool'
> -    VMS_PORT: '2212'
> +    VMS_PORT: '2242'
>    script:
>      - ${GITLAB_MAKE} vms_test_osx
>  
> diff --git a/.travis.mk b/.travis.mk
> index 42969ff56..c8cb273fa 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -127,17 +127,37 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>  # OSX #
>  #######
>  
> +OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake
> +
>  deps_osx:
> -	brew update
> -	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
> +	# install brew using command from Homebrew repository instructions:
> +	#   https://github.com/Homebrew/install
> +	export PATH=${PATH}:/usr/local/bin ; \
> +		brew update || echo | /usr/bin/ruby -e \
> +			"`curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install`"

The only motivation I found is that it is suggested in homebrew
instructions. What did not work before this change? Is it actually
related to 1.0.13 -> 10.15 update?

> +	# try to install the packages either upgrade it to avoid of fails
> +	# if the package allready exists but of the previous version

Typo: allready.

> +	export PATH=${PATH}:/usr/local/bin ; \
> +		brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}

I see the following warnings on 10.14 in Travis-CI on current master:

 | brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
 | Warning: openssl@1.1 1.1.1d is already installed, it's just not linked
 | You can use `brew link openssl@1.1` to link this version.

Is it about them? So, it forces brew to install last packages versions?
It seems this is not related to 10.13 -> 10.15 update (correct me if I'm
wrong) and deserves its own commit.

> +	# OSX 10.15 Catalina started to use Xcode 11 which has 'Deprications'
> +	# part in Release Notes:
> +	#   Use of Python 2.7 isn’t recommended. This version is included in
> +	#   macOS for compatibility with legacy software. Future versions of
> +	#   macOS won’t include Python 2.7. Instead, it’s recommended that
> +	#   you run python3 in Terminal. (51097165)
>  	python2 -V || brew install python2 --force
>  	curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
>  	python get-pip.py --user

Is it python3 on Mac OS 10.15? If so, I didn't get what are we doing
here. Maybe it didn't matter before, but now we should explicitly call
python2 here?

> -	pip install --user --force-reinstall -r test-run/requirements.txt
> +	# To make pip tools available, the PATH environment must include the
> +	# path to its binaries
> +	export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; \
> +		pip install --user --force-reinstall -r test-run/requirements.txt

We just install pip and it is not with default PATH? This looks strange.
Can you verify that this extra path is necessary?

>  
>  build_osx:
> -	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
> -	make -j
> +	export PATH=${PATH}:/usr/local/bin ; \
> +		cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
> +			-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} && \
> +		make -j

Didn't get this change: what is now in /usr/local/bin that was not there
before?

>  
>  test_osx_no_deps: build_osx
>  	# Limits: Increase the maximum number of open file descriptors on macOS:
> -- 
> 2.17.1
> 

Re: [Tarantool-patches] [PATCH v1] gitlab-ci: add Catalina OSX 10.15

[Alexander Tikhonov]

[-- Attachment #1: Type: text/plain, Size: 9252 bytes --]

Alexander, thank you for the review, I've made the changes as you suggested.


>Вторник, 17 декабря 2019, 18:46 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:
>
>To be honest, I didn't get most of changes here. So below I asked for
>explanation.
>
>WBR, Alexander Turenko.
>
>On Mon, Dec 09, 2019 at 02:45:54PM +0300, Alexander V. Tikhonov wrote:
>> Added Catalina OSX 10.15 to gitlab-ci testing and removed OSX 10.13,
>> due to decided to have only 2 last major releases, for now it is
>> 10.14 and 10.15 OSX versions.
>> 
>> Also added homebrew installation routine as it was suggested in its
>> instructions. Added path in environment to reach the pip binary that
>> was additionally installed based on python2 version in addition to
>> pip3 which is the default in Catalina OSX 10.15. Added installation
>> of the cmake and tool. Added upgrade of the OSX packages to avoid of
>> fails on already existed packages, but of the previous versions.
>> ---
>> 
>> Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_15_catalina
>> 
>>  .gitlab-ci.yml |  8 ++++----
>>  .travis.mk     | 30 +++++++++++++++++++++++++-----
>>  2 files changed, 29 insertions(+), 9 deletions(-)
>> 
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 3af5a3c8a..d3cb959d0 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -90,15 +90,15 @@ release_asan_clang8:
>>    script:
>>      - ${GITLAB_MAKE} test_asan_debian_no_deps
>> 
>> -osx_13_release:
>> +osx_15_release:
>>    <<: *release_only_definition
>>    <<: *vbox_definition
>>    tags:
>> -    - vms_osx_13
>> +    - vms_test
>
>Is it separate change? Is is about load balancing between our hosts?
>
>If so, let's move this change into its own commit.
Actually no, this is the new tag for the gitlab-ci runners, because in the current
version we have different tags for each of the OSX/FreeBSD versions, so to
have the new 'osx_15' version it was needed to create the new tag and I've
created it with the name 'vms_test'. The only question which can be here is why
it was not named as 'vms_osx_15' - the answer is simple, it is just the new name
which will be changed in the nearest time to the common 'vms_test' for all the
gitlab runners jobs, in this way we'll have only the following steps
- create tag 'vms_test'
while naming as 'vms_osx_15' will have extra steps like:
- create tag 'vms_osx_15'
- create tag 'vms_test'
- move job from tag 'vms_osx_15' to 'vms_test'
- remove tag 'vms_osx_15'
So I don't see any obstacles just to name the tag as 'vms_test' from the very start,
anyway the name is new.
>
>
>>    variables:
>> -    VMS_NAME: 'osx_13'
>> +    VMS_NAME: 'osx_15'
>>      VMS_USER: 'tarantool'
>> -    VMS_PORT: '2212'
>> +    VMS_PORT: '2242'
>>    script:
>>      - ${GITLAB_MAKE} vms_test_osx
>> 
>> diff --git a/.travis.mk b/.travis.mk
>> index 42969ff56..c8cb273fa 100644
>> --- a/.travis.mk
>> +++ b/.travis.mk
>> @@ -127,17 +127,37 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>>  # OSX #
>>  #######
>> 
>> +OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake
>> +
>>  deps_osx:
>> -	brew update
>> -	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>> +	# install brew using command from Homebrew repository instructions:
>> +	#  https://github.com/Homebrew/install
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		brew update || echo | /usr/bin/ruby -e \
>> +			"`curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install` "
>
>The only motivation I found is that it is suggested in homebrew
>instructions. What did not work before this change? Is it actually
>related to 1.0.13 -> 10.15 update? 
Additional tool homebrew is not installed at macOS by default, also it uses the
'/usr/local' path (check  https://en.wikipedia.org/wiki/Homebrew_(package_management_software) )
which is not in the PATH environment by default. Right, the fix can be separated to the
standalone patch for the other macOS version, but the current part of code doesn't
depend to any of macOS version, that is why I suggested to add this change to the
current commit, anyway I'll move it to the separate commit, as suggested.
>
>
>> +	# try to install the packages either upgrade it to avoid of fails
>> +	# if the package allready exists but of the previous version
>
>Typo: allready. 
Corrected.
>
>
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>
>I see the following warnings on 10.14 in Travis-CI on current master:
>
> | brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
> | Warning:  openssl@1.1 1.1.1d is already installed, it's just not linked
> | You can use `brew link openssl@1.1` to link this version.
>
>Is it about them? So, it forces brew to install last packages versions?
>It seems this is not related to 10.13 -> 10.15 update (correct me if I'm
>wrong) and deserves its own commit. 
No, the problem is that if the package already exists with the previous version
then 'brew install' fails with message about it and suggests to upgrade it. To avoid of
its issue we can try to install all needed packages and try to upgrade it on failed
installation process, if the upgrade command fails too than it means the some other
fail occurred. Ok, let's move it to separate commit.
>
>
>> +	# OSX 10.15 Catalina started to use Xcode 11 which has 'Deprications'
>> +	# part in Release Notes:
>> +	#   Use of Python 2.7 isn’t recommended. This version is included in
>> +	#   macOS for compatibility with legacy software. Future versions of
>> +	#   macOS won’t include Python 2.7. Instead, it’s recommended that
>> +	#   you run python3 in Terminal. (51097165)
>>  	python2 -V || brew install python2 --force
>>  	curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>>  	python get-pip.py --user
>
>Is it python3 on Mac OS 10.15? If so, I didn't get what are we doing
>here. Maybe it didn't matter before, but now we should explicitly call
>python2 here?
Here we check that python2 is installed, by default macOS 10.15 use python3
instead of python2 as previous MacOS 10.14 version used. After the manual
installation of python2 the link 'python' points to the python2 binary and we don't
need to call python2 explicitly, check:
tarantool@Tarantools-iMac tarantool % ls -al `which python`
/usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
>
>
>> -	pip install --user --force-reinstall -r test-run/requirements.txt
>> +	# To make pip tools available, the PATH environment must include the
>> +	# path to its binaries
>> +	export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; \
>> +		pip install --user --force-reinstall -r test-run/requirements.txt
>
>We just install pip and it is not with default PATH? This looks strange.
>Can you verify that this extra path is necessary?
Sure, please check the following:
tarantool@Tarantools-iMac tarantool % curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py

tarantool@Tarantools-iMac tarantool % python get-pip.py --user
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting pip
Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.3.1
Uninstalling pip-19.3.1:
Successfully uninstalled pip-19.3.1
WARNING: The scripts pip, pip2 and pip2.7 are installed in '/Users/tarantool/Library/Python/2.7/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-19.3.1

tarantool@Tarantools-iMac tarantool % which pip
pip not found

tarantool@Tarantools-iMac tarantool % export PATH=${PATH}:/Users/tarantool/Library/Python/2.7/bin ; which pip
/Users/tarantool/Library/Python/2.7/bin/pip
>
>> 
>>  build_osx:
>> -	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>> -	make -j
>> +	export PATH=${PATH}:/usr/local/bin ; \
>> +		cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
>> +			-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS} && \
>> +		make -j
>
>Didn't get this change: what is now in /usr/local/bin that was not there
>before?
The following example shows that 'make' that was installed by default is in
the default path '/usr/bin':

tarantool@Tarantools-iMac tarantool % which make
/usr/bin/make
while 'cmake' additionally installed by the homebrew exists only at the additional
path '/usr/local/bin':
tarantool@Tarantools-iMac tarantool % which cmake
/usr/local/bin/cmake
>
>> 
>>  test_osx_no_deps: build_osx
>>  	# Limits: Increase the maximum number of open file descriptors on macOS:
>> -- 
>> 2.17.1
>> 


-- 
Alexander Tikhonov

[-- Attachment #2: Type: text/html, Size: 12560 bytes --]