Re: [Tarantool-patches] [PATCH v3] gitlab-ci: implement OSX 10.14 testing on mac mini

["Alexander Tikhonov" <avtikhon@tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 6048 bytes --]

Oleg, thanks for your review, I've made the changes and commented the review.


>Пятница, 28 февраля 2020, 22:23 +03:00 от Oleg Piskunov <o.piskunov@tarantool.org>:
>
>Sasha,
>I guess it doesn’t work correctly around python and pip installation.
>Here is your latest log from the gitlab-ci run ( https://gitlab.com/tarantool/tarantool/-/jobs/452400831 ):
>brew install --force openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake python || brew upgrade openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake python
>...
>Warning: python 3.7.6_1 is already installed and up-to-date
>To reinstall 3.7.6_1, run `brew reinstall python`
>As you can see brew install updating python3 instead of python2.7
>The reason that it still working correctly is that python2.7 preinstalled before into different path which you are using.
>Need to investigate and fix. 
Found that python 2 was removed just a few days ago at all from the brew formulas. To make
it workable found the solution to use the latest Python 2 commit.
>>Четверг, 27 февраля 2020, 16:23 +03:00 от Alexander V. Tikhonov < avtikhon@tarantool.org >:
>>Added mac mini hosts into testing and set to it all
>>OSX 10.14 test jobs with new osx_14 gitlab-runner tag.
>>Also made additional setup:
>> - added PATHs to python2 and pip
>> - set 'var' directory for test-run tool to the shorter
>>   path name to avoid of issues with long names.
>>---
>>
>>Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
>>
>> .gitlab-ci.yml | 20 ++++++++------------
>> .gitlab.mk | 2 +-
>> .travis.mk | 32 +++++++++++++++++++++++++-------
>> 3 files changed, 34 insertions(+), 20 deletions(-)
>>
>>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>index 5177a9903..7414dd578 100644
>>--- a/.gitlab-ci.yml
>>+++ b/.gitlab-ci.yml
>>@@ -152,28 +152,24 @@ osx_13_release:
>>     - ${GITLAB_MAKE} vms_test_osx
>>
>> osx_14_release:
>>- <<: *vbox_definition
>>+ <<: *docker_test_definition
>>   tags:
>>- - vms_osx_14
>>+ - osx_14
>>   variables:
>>- VMS_NAME: 'osx_14'
>>- VMS_USER: 'tarantool'
>>- VMS_PORT: '2222'
>>+ PYTHON2: "python"
>>   script:
>>- - ${GITLAB_MAKE} vms_test_osx
>>+ - ${GITLAB_MAKE} test_osx
>>
>> osx_14_release_lto:
>>   <<: *release_only_definition
>>- <<: *vbox_definition
>>+ <<: *docker_test_definition
>>   tags:
>>- - vms_osx_14
>>+ - osx_14
>>   variables:
>>     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>>- VMS_NAME: 'osx_14'
>>- VMS_USER: 'tarantool'
>>- VMS_PORT: '2222'
>>+ PYTHON2: "python"
>>   script:
>>- - ${GITLAB_MAKE} vms_test_osx
>>+ - ${GITLAB_MAKE} test_osx
>>
>> freebsd_12_release:
>>   <<: *vbox_definition
>>diff --git a/.gitlab.mk b/.gitlab.mk
>>index b39c5c651..d804d3cf6 100644
>>--- a/.gitlab.mk
>>+++ b/.gitlab.mk
>>@@ -14,7 +14,7 @@ git_submodule_update:
>>  git submodule update --recursive --init
>>
>> # Pass *_no_deps goals to .travis.mk.
>>-test_%_no_deps: git_submodule_update
>>+test_%: git_submodule_update
>>  ${TRAVIS_MAKE} $@
>>
>> # #######################################################
>>diff --git a/.travis.mk b/.travis.mk
>>index 42969ff56..9db49a6b7 100644
>>--- a/.travis.mk
>>+++ b/.travis.mk
>>@@ -5,6 +5,7 @@
>> DOCKER_IMAGE?=packpack/packpack:debian-stretch
>> TEST_RUN_EXTRA_PARAMS?=
>> MAX_FILES?=65534
>>+MAX_PROC?=2500
>>
>> all: package
>>
>>@@ -127,13 +128,24 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>> # OSX #
>> #######
>>
>>+# Python2 is EOF and we need to be sure that it is in use instead of Python3
>>+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake ${PYTHON2}
>>+
>> deps_osx:
>>- brew update
>>- brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>>- python2 -V || brew install python2 --force
>>+ # install brew using command from Homebrew repository instructions:
>>+ #  https://github.com/Homebrew/install
>>+ # NOTE: 'echo' command below is required since brew installation
>>+ # script obliges the one to enter a newline for confirming the
>>+ # installation via Ruby script.
>>+ brew update || echo | /usr/bin/ruby -e \
>>+ "$(curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install )"
>>+ # try to install the packages either upgrade it to avoid of fails
>>+ # if the package already exists with the previous version
>>+ brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>>  curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>>- python get-pip.py --user
>>- pip install --user --force-reinstall -r test-run/requirements.txt
>>+ python get-pip.py
>>+ export PATH=/Users/${USER}/Library/Python/2.7/bin:${PATH} ; \
>>+ pip install --force-reinstall -r test-run/requirements.txt
>>
>> build_osx:
>>  cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>>@@ -148,11 +160,17 @@ test_osx_no_deps: build_osx
>>  # call as tests runs call.
>>  # Tests: Temporary excluded replication/ suite with some tests
>>  # from other suites by issues #4357 and #4370
>>- echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>>+ sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>>  launchctl limit maxfiles || : ; \
>>  ulimit -n ${MAX_FILES} || : ; \
>>  ulimit -n ; \
>>- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
>>+ sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
>>+ launchctl limit maxproc || : ; \
>>+ ulimit -u ${MAX_PROC} || : ; \
>>+ ulimit -u ; \
>>+ export PATH=/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH} ; \
>>+ rm -rf /tmp/tnt ; \
>>+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
>>  app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
>>  replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
>>
>>--
>>2.17.1
>--
>Oleg Piskunov

-- 
Alexander Tikhonov

[-- Attachment #2: Type: text/html, Size: 11206 bytes --]