Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini
@ 2020-02-27 10:30 Alexander V. Tikhonov
  2020-02-27 14:26 ` Oleg Piskunov
  0 siblings, 1 reply; 5+ messages in thread
From: Alexander V. Tikhonov @ 2020-02-27 10:30 UTC (permalink / raw)
  To: Oleg Piskunov; +Cc: tarantool-patches

Added mac mini hosts into testing and set to it all
OSX 10.14 test jobs with new osx_14 gitlab-runner tag.
Also made additional setup:
 - added PATHs to python2 and pip
 - set 'var' directory for test-run tool to the shorter
   path name to avoid of issues with long names.
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci

 .gitlab-ci.yml | 20 ++++++++------------
 .gitlab.mk     |  2 +-
 .travis.mk     | 45 ++++++++++++++++++++++++++++++++-------------
 3 files changed, 41 insertions(+), 26 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5177a9903..7414dd578 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -152,28 +152,24 @@ osx_13_release:
     - ${GITLAB_MAKE} vms_test_osx
 
 osx_14_release:
-  <<: *vbox_definition
+  <<: *docker_test_definition
   tags:
-    - vms_osx_14
+    - osx_14
   variables:
-    VMS_NAME: 'osx_14'
-    VMS_USER: 'tarantool'
-    VMS_PORT: '2222'
+    PYTHON2: "python"
   script:
-    - ${GITLAB_MAKE} vms_test_osx
+    - ${GITLAB_MAKE} test_osx
 
 osx_14_release_lto:
   <<: *release_only_definition
-  <<: *vbox_definition
+  <<: *docker_test_definition
   tags:
-    - vms_osx_14
+    - osx_14
   variables:
     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
-    VMS_NAME: 'osx_14'
-    VMS_USER: 'tarantool'
-    VMS_PORT: '2222'
+    PYTHON2: "python"
   script:
-    - ${GITLAB_MAKE} vms_test_osx
+    - ${GITLAB_MAKE} test_osx
 
 freebsd_12_release:
   <<: *vbox_definition
diff --git a/.gitlab.mk b/.gitlab.mk
index b39c5c651..d804d3cf6 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -14,7 +14,7 @@ git_submodule_update:
 		git submodule update --recursive --init
 
 # Pass *_no_deps goals to .travis.mk.
-test_%_no_deps: git_submodule_update
+test_%: git_submodule_update
 	${TRAVIS_MAKE} $@
 
 # #######################################################
diff --git a/.travis.mk b/.travis.mk
index 42969ff56..90ad0b412 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -5,6 +5,7 @@
 DOCKER_IMAGE?=packpack/packpack:debian-stretch
 TEST_RUN_EXTRA_PARAMS?=
 MAX_FILES?=65534
+MAX_PROC?=4096
 
 all: package
 
@@ -127,16 +128,28 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
 # OSX #
 #######
 
+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake ${PYTHON2}
+
 deps_osx:
-	brew update
-	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
-	python2 -V || brew install python2 --force
-	curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
-	python get-pip.py --user
-	pip install --user --force-reinstall -r test-run/requirements.txt
+	# install brew using command from Homebrew repository instructions:
+	#   https://github.com/Homebrew/install
+	# NOTE: 'echo' command below is required since brew installation
+	# script obliges the one to enter a newline for confirming the
+	# installation via Ruby script.
+	brew update || echo | /usr/bin/ruby -e \
+		"$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+	# try to install the packages either upgrade it to avoid of fails
+	# if the package already exists with the previous version
+	brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
+	curl --silent --show-error --retry 5 \
+		https://bootstrap.pypa.io/get-pip.py >get-pip.py
+	python get-pip.py
+	export PATH=/Users/${USER}/Library/Python/2.7/bin:${PATH} ; \
+		pip install --force-reinstall -r test-run/requirements.txt
 
 build_osx:
-	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
+	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
+		-DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
 	make -j
 
 test_osx_no_deps: build_osx
@@ -148,13 +161,19 @@ test_osx_no_deps: build_osx
 	# call as tests runs call.
 	# Tests: Temporary excluded replication/ suite with some tests
 	#        from other suites by issues #4357 and #4370
-	echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
+	sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
 		launchctl limit maxfiles || : ; \
-		ulimit -n ${MAX_FILES} || : ; \
-		ulimit -n ; \
-		cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
-			app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
-			replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
+		ulimit -n ${MAX_FILES} || : ulimit -n ; \
+		sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
+		launchctl limit maxproc || : ; \
+		ulimit -u ${MAX_PROC} || : ulimit -u ; \
+		export PATH=/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH} ; \
+		rm -rf /tmp/tnt ; \
+		cd test && ./test-run.py --vardir /tmp/tnt \
+			--force $(TEST_RUN_EXTRA_PARAMS) \
+			app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ \
+			long_run-py/ luajit-tap/ replication-py/ small/ sql/ \
+			sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
 
 test_osx: deps_osx test_osx_no_deps
 
-- 
2.17.1

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini
  2020-02-27 10:30 [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini Alexander V. Tikhonov
@ 2020-02-27 14:26 ` Oleg Piskunov
  0 siblings, 0 replies; 5+ messages in thread
From: Oleg Piskunov @ 2020-02-27 14:26 UTC (permalink / raw)
  To: Alexander V. Tikhonov; +Cc: tarantool-patches

[-- Attachment #1: Type: text/plain, Size: 5702 bytes --]


Alexander, below my comments:
  
>Четверг, 27 февраля 2020, 13:30 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
> 
>Added mac mini hosts into testing and set to it all
>OSX 10.14 test jobs with new osx_14 gitlab-runner tag.
>Also made additional setup:
> - added PATHs to python2 and pip
> - set 'var' directory for test-run tool to the shorter
>   path name to avoid of issues with long names.
>---
>
>Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
>
> .gitlab-ci.yml | 20 ++++++++------------
> .gitlab.mk | 2 +-
> .travis.mk | 45 ++++++++++++++++++++++++++++++++-------------
> 3 files changed, 41 insertions(+), 26 deletions(-)
>
>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>index 5177a9903..7414dd578 100644
>--- a/.gitlab-ci.yml
>+++ b/.gitlab-ci.yml
>@@ -152,28 +152,24 @@ osx_13_release:
>     - ${GITLAB_MAKE} vms_test_osx
> 
> osx_14_release:
>- <<: *vbox_definition
>+ <<: *docker_test_definition
>   tags:
>- - vms_osx_14
>+ - osx_14
>   variables:
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
>+ PYTHON2: "python"
Please add comment with explanation about bypassing situation with python2 EOL.
>   script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
> 
> osx_14_release_lto:
>   <<: *release_only_definition
>- <<: *vbox_definition
>+ <<: *docker_test_definition
>   tags:
>- - vms_osx_14
>+ - osx_14
>   variables:
>     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
>+ PYTHON2: "python"
>   script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
> 
> freebsd_12_release:
>   <<: *vbox_definition
>diff --git a/.gitlab.mk b/.gitlab.mk
>index b39c5c651..d804d3cf6 100644
>--- a/.gitlab.mk
>+++ b/.gitlab.mk
>@@ -14,7 +14,7 @@ git_submodule_update:
>  git submodule update --recursive --init
> 
> # Pass *_no_deps goals to .travis.mk.
>-test_%_no_deps: git_submodule_update
>+test_%: git_submodule_update
>  ${TRAVIS_MAKE} $@
> 
> # #######################################################
>diff --git a/.travis.mk b/.travis.mk
>index 42969ff56..90ad0b412 100644
>--- a/.travis.mk
>+++ b/.travis.mk
>@@ -5,6 +5,7 @@
> DOCKER_IMAGE?=packpack/packpack:debian-stretch
> TEST_RUN_EXTRA_PARAMS?=
> MAX_FILES?=65534
>+MAX_PROC?=4096
Please use 2500 instead 4096. This is hard coded in OSX kernel to 2500.
> 
> all: package
> 
>@@ -127,16 +128,28 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
> # OSX #
> #######
> 
>+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake ${PYTHON2}
>+
> deps_osx:
>- brew update
>- brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>- python2 -V || brew install python2 --force
>- curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>- python get-pip.py --user
>- pip install --user --force-reinstall -r test-run/requirements.txt
>+ # install brew using command from Homebrew repository instructions:
>+ #  https://github.com/Homebrew/install
>+ # NOTE: 'echo' command below is required since brew installation
>+ # script obliges the one to enter a newline for confirming the
>+ # installation via Ruby script.
>+ brew update || echo | /usr/bin/ruby -e \
>+ "$(curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install )"
>+ # try to install the packages either upgrade it to avoid of fails
>+ # if the package already exists with the previous version
>+ brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>+ curl --silent --show-error --retry 5 \
>+  https://bootstrap.pypa.io/get-pip.py >get-pip.py
Please do not use strings refactoring
>+ python get-pip.py
>+ export PATH=/Users/${USER}/Library/Python/2.7/bin:${PATH} ; \
>+ pip install --force-reinstall -r test-run/requirements.txt
> 
> build_osx:
>- cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>+ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \
>+ -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
Please do not use strings refactoring
>  make -j
> 
> test_osx_no_deps: build_osx
>@@ -148,13 +161,19 @@ test_osx_no_deps: build_osx
>  # call as tests runs call.
>  # Tests: Temporary excluded replication/ suite with some tests
>  # from other suites by issues #4357 and #4370
>- echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>+ sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>  launchctl limit maxfiles || : ; \
>- ulimit -n ${MAX_FILES} || : ; \
>- ulimit -n ; \
>- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
>- app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
>- replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
>+ ulimit -n ${MAX_FILES} || : ulimit -n ; \
Please do not use strings refactoring
>+ sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
>+ launchctl limit maxproc || : ; \
>+ ulimit -u ${MAX_PROC} || : ulimit -u ; \
Please use the same format as above (one comand per line)
>+ export PATH=/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH} ; \
>+ rm -rf /tmp/tnt ; \
>+ cd test && ./test-run.py --vardir /tmp/tnt \
>+ --force $(TEST_RUN_EXTRA_PARAMS) \
Please join two lines above
>+ app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ \
>+ long_run-py/ luajit-tap/ replication-py/ small/ sql/ \
>+ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
Please do not use strings refactoring
> 
> test_osx: deps_osx test_osx_no_deps
> 
>--
>2.17.1
>  
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 8732 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini
  2020-03-06 18:35 ` Oleg Piskunov
@ 2020-03-10  6:25   ` Alexander Tikhonov
  0 siblings, 0 replies; 5+ messages in thread
From: Alexander Tikhonov @ 2020-03-10  6:25 UTC (permalink / raw)
  To: Oleg Piskunov, Kirill Yukhin; +Cc: tarantool-patches

[-- Attachment #1: Type: text/plain, Size: 18597 bytes --]


Oleg, thanks for the review, I've made the changes as you suggested.

Kirill, please proceed with commit process and push the patch to release branches. 
>Пятница,  6 марта 2020, 21:35 +03:00 от Oleg Piskunov < o.piskunov@tarantool.org >:
>
>LGTM.
>Minor: Sasha, could you please change 1 comment below.
>  
>>Пятница, 6 марта 2020, 15:59 +03:00 от Alexander V. Tikhonov < avtikhon@tarantool.org >:
>> 
>>Added new gitlab-ci test jobs for OSX 10.14 testings.
>>
>>Set in use Python2 locally tapped version with its
>>latest commit in it, since Python 2 is EOL.
>>
>>Set 'var' directory for test-run tool to the shorter
>>path name to avoid of issues with long names.
>>---
>>
>>Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
>>
>> .gitlab-ci.yml | 19 +--
>> .gitlab.mk | 2 +-
>> .travis.mk | 31 +++-
>> tools/brew_taps/tntpython2.rb | 306 ++++++++++++++++++++++++++++++++++
>> 4 files changed, 337 insertions(+), 21 deletions(-)
>> create mode 100644 tools/brew_taps/tntpython2.rb
>>
>>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>index 5177a9903..47e4af5a2 100644
>>--- a/.gitlab-ci.yml
>>+++ b/.gitlab-ci.yml
>>@@ -152,28 +152,21 @@ osx_13_release:
>>     - ${GITLAB_MAKE} vms_test_osx
>> 
>> osx_14_release:
>>- <<: *vbox_definition
>>+ <<: *docker_test_definition
>>   tags:
>>- - vms_osx_14
>>- variables:
>>- VMS_NAME: 'osx_14'
>>- VMS_USER: 'tarantool'
>>- VMS_PORT: '2222'
>>+ - osx_14
>>   script:
>>- - ${GITLAB_MAKE} vms_test_osx
>>+ - ${GITLAB_MAKE} test_osx
>> 
>> osx_14_release_lto:
>>   <<: *release_only_definition
>>- <<: *vbox_definition
>>+ <<: *docker_test_definition
>>   tags:
>>- - vms_osx_14
>>+ - osx_14
>>   variables:
>>     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>>- VMS_NAME: 'osx_14'
>>- VMS_USER: 'tarantool'
>>- VMS_PORT: '2222'
>>   script:
>>- - ${GITLAB_MAKE} vms_test_osx
>>+ - ${GITLAB_MAKE} test_osx
>> 
>> freebsd_12_release:
>>   <<: *vbox_definition
>>diff --git a/.gitlab.mk b/.gitlab.mk
>>index b39c5c651..d804d3cf6 100644
>>--- a/.gitlab.mk
>>+++ b/.gitlab.mk
>>@@ -14,7 +14,7 @@ git_submodule_update:
>>  git submodule update --recursive --init
>> 
>> # Pass *_no_deps goals to .travis.mk.
>>-test_%_no_deps: git_submodule_update
>>+test_%: git_submodule_update
>>  ${TRAVIS_MAKE} $@
>> 
>> # #######################################################
>>diff --git a/.travis.mk b/.travis.mk
>>index 42969ff56..a619ee95a 100644
>>--- a/.travis.mk
>>+++ b/.travis.mk
>>@@ -5,6 +5,7 @@
>> DOCKER_IMAGE?=packpack/packpack:debian-stretch
>> TEST_RUN_EXTRA_PARAMS?=
>> MAX_FILES?=65534
>>+MAX_PROC?=2500
>> 
>> all: package
>> 
>>@@ -127,13 +128,24 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
>> # OSX #
>> #######
>> 
>>+# Python2 is EOF and we need to be sure that it is in use instead of Python3
># since Python 2 is EOL it's tapped version from local formula with its latest commit will be used
>>+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake \
>>+ file://${PWD}/tools/brew_taps/tntpython2.rb
>>+
>> deps_osx:
>>- brew update
>>- brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>>- python2 -V || brew install python2 --force
>>+ # install brew using command from Homebrew repository instructions:
>>+ #  https://github.com/Homebrew/install
>>+ # NOTE: 'echo' command below is required since brew installation
>>+ # script obliges the one to enter a newline for confirming the
>>+ # installation via Ruby script.
>>+ brew update || echo | /usr/bin/ruby -e \
>>+ "$(curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install )"
>>+ # try to install the packages either upgrade it to avoid of fails
>>+ # if the package already exists with the previous version
>>+ brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>>  curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>>- python get-pip.py --user
>>- pip install --user --force-reinstall -r test-run/requirements.txt
>>+ python get-pip.py
>>+ pip install --force-reinstall -r test-run/requirements.txt
>> 
>> build_osx:
>>  cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>>@@ -148,11 +160,16 @@ test_osx_no_deps: build_osx
>>  # call as tests runs call.
>>  # Tests: Temporary excluded replication/ suite with some tests
>>  # from other suites by issues #4357 and #4370
>>- echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>>+ sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>>  launchctl limit maxfiles || : ; \
>>  ulimit -n ${MAX_FILES} || : ; \
>>  ulimit -n ; \
>>- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
>>+ sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
>>+ launchctl limit maxproc || : ; \
>>+ ulimit -u ${MAX_PROC} || : ; \
>>+ ulimit -u ; \
>>+ rm -rf /tmp/tnt ; \
>>+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
>>  app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
>>  replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
>> 
>>diff --git a/tools/brew_taps/tntpython2.rb b/tools/brew_taps/tntpython2.rb
>>new file mode 100644
>>index 000000000..59f374030
>>--- /dev/null
>>+++ b/tools/brew_taps/tntpython2.rb
>>@@ -0,0 +1,306 @@
>>+class Tntpython2 < Formula
>>+ desc "Interpreted, interactive, object-oriented programming language"
>>+ homepage " https://www.python.org/ "
>>+ url " https://www.python.org/ftp/python/2.7.17/Python-2.7.17.tar.xz "
>>+ sha256 "4d43f033cdbd0aa7b7023c81b0e986fd11e653b5248dac9144d508f11812ba41"
>>+ revision 1
>>+ head " https://github.com/python/cpython.git ", :branch => "2.7"
>>+
>>+ bottle do
>>+ sha256 "accfaa922708f00afb69ab230199f96e6ecdddd248a1eca586ce1e5e5cfd732b" => :catalina
>>+ sha256 "54d3351d6be8268b2f5017894dcc8e083811dfa3812bdb9f79f989873b9a4542" => :mojave
>>+ sha256 "cfd5c6eeac37065d19f527bb0798a9caf1928bab3340cd545224861a3c82f219" => :high_sierra
>>+ end
>>+
>>+ # setuptools remembers the build flags python is built with and uses them to
>>+ # build packages later. Xcode-only systems need different flags.
>>+ pour_bottle? do
>>+ reason <<~EOS
>>+ The bottle needs the Apple Command Line Tools to be installed.
>>+ You can install them, if desired, with:
>>+ xcode-select --install
>>+ EOS
>>+ satisfy { MacOS::CLT.installed? }
>>+ end
>>+
>>+ depends_on "pkg-config" => :build
>>+ depends_on "gdbm"
>>+ depends_on " openssl@1.1 "
>>+ depends_on "readline"
>>+ depends_on "sqlite"
>>+
>>+ resource "setuptools" do
>>+ url " https://files.pythonhosted.org/packages/f4/d5/a6c19dcbcbc267aca376558797f036d9bcdff344c9f785fe7d0fe9a5f2a7/setuptools-41.4.0.zip "
>>+ sha256 "7eae782ccf36b790c21bde7d86a4f303a441cd77036b25c559a602cf5186ce4d"
>>+ end
>>+
>>+ resource "pip" do
>>+ url " https://files.pythonhosted.org/packages/ce/ea/9b445176a65ae4ba22dce1d93e4b5fe182f953df71a145f557cffaffc1bf/pip-19.3.1.tar.gz "
>>+ sha256 "21207d76c1031e517668898a6b46a9fb1501c7a4710ef5dfd6a40ad9e6757ea7"
>>+ end
>>+
>>+ resource "wheel" do
>>+ url " https://files.pythonhosted.org/packages/59/b0/11710a598e1e148fb7cbf9220fd2a0b82c98e94efbdecb299cb25e7f0b39/wheel-0.33.6.tar.gz "
>>+ sha256 "10c9da68765315ed98850f8e048347c3eb06dd81822dc2ab1d4fde9dc9702646"
>>+ end
>>+
>>+ def lib_cellar
>>+ prefix/"Frameworks/Python.framework/Versions/2.7/lib/python2.7"
>>+ end
>>+
>>+ def site_packages_cellar
>>+ lib_cellar/"site-packages"
>>+ end
>>+
>>+ # The HOMEBREW_PREFIX location of site-packages.
>>+ def site_packages
>>+ HOMEBREW_PREFIX/"lib/python2.7/site-packages"
>>+ end
>>+
>>+ def install
>>+ # Unset these so that installing pip and setuptools puts them where we want
>>+ # and not into some other Python the user has installed.
>>+ ENV["PYTHONHOME"] = nil
>>+ ENV["PYTHONPATH"] = nil
>>+
>>+ args = %W[
>>+ --prefix=#{prefix}
>>+ --enable-ipv6
>>+ --datarootdir=#{share}
>>+ --datadir=#{share}
>>+ --enable-framework=#{frameworks}
>>+ --without-ensurepip
>>+ ]
>>+
>>+ # See upstream bug report from 22 Jan 2018 "Significant performance problems
>>+ # with Python 2.7 built with clang 3.x or 4.x"
>>+ #  https://bugs.python.org/issue32616
>>+ #  https://github.com/Homebrew/homebrew-core/issues/22743
>>+ if DevelopmentTools.clang_build_version >= 802 &&
>>+ DevelopmentTools.clang_build_version < 902
>>+ args << "--without-computed-gotos"
>>+ end
>>+
>>+ args << "--without-gcc" if ENV.compiler == :clang
>>+
>>+ cflags = []
>>+ ldflags = []
>>+ cppflags = []
>>+
>>+ if MacOS.sdk_path_if_needed
>>+ # Help Python's build system (setuptools/pip) to build things on SDK-based systems
>>+ # The setup.py looks at "-isysroot" to get the sysroot (and not at --sysroot)
>>+ cflags << "-isysroot #{MacOS.sdk_path}" << "-I#{MacOS.sdk_path}/usr/include"
>>+ ldflags << "-isysroot #{MacOS.sdk_path}"
>>+ # For the Xlib.h, Python needs this header dir with the system Tk
>>+ # Yep, this needs the absolute path where zlib needed a path relative
>>+ # to the SDK.
>>+ cflags << "-I#{MacOS.sdk_path}/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers"
>>+ end
>>+
>>+ # Avoid linking to libgcc  https://code.activestate.com/lists/python-dev/112195/
>>+ args << "MACOSX_DEPLOYMENT_TARGET=#{MacOS.version}"
>>+
>>+ # We want our readline and openssl! This is just to outsmart the detection code,
>>+ # superenv handles that cc finds includes/libs!
>>+ inreplace "setup.py" do |s|
>>+ s.gsub! "do_readline = self.compiler.find_library_file(lib_dirs, 'readline')",
>>+ "do_readline = '#{Formula["readline"].opt_lib}/libhistory.dylib'"
>>+ s.gsub! "/usr/local/ssl", Formula[" openssl@1.1 "].opt_prefix
>>+ end
>>+
>>+ inreplace "setup.py" do |s|
>>+ s.gsub! "sqlite_setup_debug = False", "sqlite_setup_debug = True"
>>+ s.gsub! "for d_ in inc_dirs + sqlite_inc_paths:",
>>+ "for d_ in ['#{Formula["sqlite"].opt_include}']:"
>>+
>>+ # Allow sqlite3 module to load extensions:
>>+ #  https://docs.python.org/library/sqlite3.html#f1
>>+ s.gsub! 'sqlite_defines.append(("SQLITE_OMIT_LOAD_EXTENSION", "1"))', ""
>>+ end
>>+
>>+ # Allow python modules to use ctypes.find_library to find homebrew's stuff
>>+ # even if homebrew is not a /usr/local/lib. Try this with:
>>+ # `brew install enchant && pip install pyenchant`
>>+ inreplace "./Lib/ctypes/macholib/dyld.py" do |f|
>>+ f.gsub! "DEFAULT_LIBRARY_FALLBACK = [", "DEFAULT_LIBRARY_FALLBACK = [ '#{HOMEBREW_PREFIX}/lib',"
>>+ f.gsub! "DEFAULT_FRAMEWORK_FALLBACK = [", "DEFAULT_FRAMEWORK_FALLBACK = [ '#{HOMEBREW_PREFIX}/Frameworks',"
>>+ end
>>+
>>+ args << "CFLAGS=#{cflags.join(" ")}" unless cflags.empty?
>>+ args << "LDFLAGS=#{ldflags.join(" ")}" unless ldflags.empty?
>>+ args << "CPPFLAGS=#{cppflags.join(" ")}" unless cppflags.empty?
>>+
>>+ system "./configure", *args
>>+ system "make"
>>+
>>+ ENV.deparallelize do
>>+ # Tell Python not to install into /Applications
>>+ system "make", "install", "PYTHONAPPSDIR=#{prefix}"
>>+ system "make", "frameworkinstallextras", "PYTHONAPPSDIR=#{pkgshare}"
>>+ end
>>+
>>+ # Fixes setting Python build flags for certain software
>>+ # See:  https://github.com/Homebrew/homebrew/pull/20182
>>+ #  https://bugs.python.org/issue3588
>>+ inreplace lib_cellar/"config/Makefile" do |s|
>>+ s.change_make_var! "LINKFORSHARED",
>>+ "-u _PyMac_Error $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)"
>>+ end
>>+
>>+ # Prevent third-party packages from building against fragile Cellar paths
>>+ inreplace [lib_cellar/"_sysconfigdata.py",
>>+ lib_cellar/"config/Makefile",
>>+ frameworks/"Python.framework/Versions/Current/lib/pkgconfig/python-2.7.pc"],
>>+ prefix, opt_prefix
>>+
>>+ # Symlink the pkgconfig files into HOMEBREW_PREFIX so they're accessible.
>>+ (lib/"pkgconfig").install_symlink Dir[frameworks/"Python.framework/Versions/Current/lib/pkgconfig/*"]
>>+
>>+ # Remove 2to3 because Python 3 also installs it
>>+ rm bin/"2to3"
>>+
>>+ # A fix, because python and python@2 both want to install Python.framework
>>+ # and therefore we can't link both into HOMEBREW_PREFIX/Frameworks
>>+ #  https://github.com/Homebrew/homebrew/issues/15943
>>+ ["Headers", "Python", "Resources"].each { |f| rm(prefix/"Frameworks/Python.framework/#{f}") }
>>+ rm prefix/"Frameworks/Python.framework/Versions/Current"
>>+
>>+ # Remove the site-packages that Python created in its Cellar.
>>+ site_packages_cellar.rmtree
>>+
>>+ (libexec/"setuptools").install resource("setuptools")
>>+ (libexec/"pip").install resource("pip")
>>+ (libexec/"wheel").install resource("wheel")
>>+ end
>>+
>>+ def post_install
>>+ # Avoid conflicts with lingering unversioned files from Python 3
>>+ rm_f %W[
>>+ #{HOMEBREW_PREFIX}/bin/easy_install
>>+ #{HOMEBREW_PREFIX}/bin/pip
>>+ #{HOMEBREW_PREFIX}/bin/wheel
>>+ ]
>>+
>>+ # Fix up the site-packages so that user-installed Python software survives
>>+ # minor updates, such as going from 2.7.0 to 2.7.1:
>>+
>>+ # Create a site-packages in HOMEBREW_PREFIX/lib/python2.7/site-packages
>>+ site_packages.mkpath
>>+
>>+ # Symlink the prefix site-packages into the cellar.
>>+ site_packages_cellar.unlink if site_packages_cellar.exist?
>>+ site_packages_cellar.parent.install_symlink site_packages
>>+
>>+ # Write our sitecustomize.py
>>+ rm_rf Dir["#{site_packages}/sitecustomize.py[co]"]
>>+ (site_packages/"sitecustomize.py").atomic_write(sitecustomize)
>>+
>>+ # Remove old setuptools installations that may still fly around and be
>>+ # listed in the easy_install.pth. This can break setuptools build with
>>+ # zipimport.ZipImportError: bad local file header
>>+ # setuptools-0.9.5-py3.3.egg
>>+ rm_rf Dir["#{site_packages}/setuptools*"]
>>+ rm_rf Dir["#{site_packages}/distribute*"]
>>+ rm_rf Dir["#{site_packages}/pip[-_.][0-9]*", "#{site_packages}/pip"]
>>+
>>+ setup_args = ["-s", "setup.py", "--no-user-cfg", "install", "--force",
>>+ "--verbose",
>>+ "--single-version-externally-managed",
>>+ "--record=installed.txt",
>>+ "--install-scripts=#{bin}",
>>+ "--install-lib=#{site_packages}"]
>>+
>>+ (libexec/"setuptools").cd { system "#{bin}/python", *setup_args }
>>+ (libexec/"pip").cd { system "#{bin}/python", *setup_args }
>>+ (libexec/"wheel").cd { system "#{bin}/python", *setup_args }
>>+
>>+ # When building from source, these symlinks will not exist, since
>>+ # post_install happens after linking.
>>+ %w[pip pip2 pip2.7 easy_install easy_install-2.7 wheel].each do |e|
>>+ (HOMEBREW_PREFIX/"bin").install_symlink bin/e
>>+ end
>>+
>>+ # Help distutils find brewed stuff when building extensions
>>+ include_dirs = [HOMEBREW_PREFIX/"include", Formula[" openssl@1.1 "].opt_include,
>>+ Formula["sqlite"].opt_include]
>>+ library_dirs = [HOMEBREW_PREFIX/"lib", Formula[" openssl@1.1 "].opt_lib,
>>+ Formula["sqlite"].opt_lib]
>>+
>>+ cfg = lib_cellar/"distutils/distutils.cfg"
>>+ cfg.atomic_write <<~EOS
>>+ [install]
>>+ prefix=#{HOMEBREW_PREFIX}
>>+ [build_ext]
>>+ include_dirs=#{include_dirs.join ":"}
>>+ library_dirs=#{library_dirs.join ":"}
>>+ EOS
>>+ end
>>+
>>+ def sitecustomize
>>+ <<~EOS
>>+ # This file is created by Homebrew and is executed on each python startup.
>>+ # Don't print from here, or else python command line scripts may fail!
>>+ # < https://docs.brew.sh/Homebrew-and-Python >
>>+ import re
>>+ import os
>>+ import sys
>>+ if sys.version_info[0] != 2:
>>+ # This can only happen if the user has set the PYTHONPATH for 3.x and run Python 2.x or vice versa.
>>+ # Every Python looks at the PYTHONPATH variable and we can't fix it here in sitecustomize.py,
>>+ # because the PYTHONPATH is evaluated after the sitecustomize.py. Many modules (e.g. PyQt4) are
>>+ # built only for a specific version of Python and will fail with cryptic error messages.
>>+ # In the end this means: Don't set the PYTHONPATH permanently if you use different Python versions.
>>+ exit('Your PYTHONPATH points to a site-packages dir for Python 2.x but you are running Python ' +
>>+ str(sys.version_info[0]) + '.x!\\n PYTHONPATH is currently: "' + str(os.environ['PYTHONPATH']) + '"\\n' +
>>+ ' You should `unset PYTHONPATH` to fix this.')
>>+ # Only do this for a brewed python:
>>+ if os.path.realpath(sys.executable).startswith('#{rack}'):
>>+ # Shuffle /Library site-packages to the end of sys.path and reject
>>+ # paths in /System pre-emptively (#14712)
>>+ library_site = '/Library/Python/2.7/site-packages'
>>+ library_packages = [p for p in sys.path if p.startswith(library_site)]
>>+ sys.path = [p for p in sys.path if not p.startswith(library_site) and
>>+ not p.startswith('/System')]
>>+ # .pth files have already been processed so don't use addsitedir
>>+ sys.path.extend(library_packages)
>>+ # the Cellar site-packages is a symlink to the HOMEBREW_PREFIX
>>+ # site_packages; prefer the shorter paths
>>+ long_prefix = re.compile(r'#{rack}/[0-9\._abrc]+/Frameworks/Python\.framework/Versions/2\.7/lib/python2\.7/site-packages')
>>+ sys.path = [long_prefix.sub('#{site_packages}', p) for p in sys.path]
>>+ # LINKFORSHARED (and python-config --ldflags) return the
>>+ # full path to the lib (yes, "Python" is actually the lib, not a
>>+ # dir) so that third-party software does not need to add the
>>+ # -F/#{HOMEBREW_PREFIX}/Frameworks switch.
>>+ try:
>>+ from _sysconfigdata import build_time_vars
>>+ build_time_vars['LINKFORSHARED'] = '-u _PyMac_Error #{opt_prefix}/Frameworks/Python.framework/Versions/2.7/Python'
>>+ except:
>>+ pass # remember: don't print here. Better to fail silently.
>>+ # Set the sys.executable to use the opt_prefix
>>+ sys.executable = '#{opt_bin}/python2.7'
>>+ EOS
>>+ end
>>+
>>+ def caveats; <<~EOS
>>+ Pip and setuptools have been installed. To update them
>>+ pip install --upgrade pip setuptools
>>+ You can install Python packages with
>>+ pip install <package>
>>+ They will install into the site-package directory
>>+ #{site_packages}
>>+ See:  https://docs.brew.sh/Homebrew-and-Python
>>+ EOS
>>+ end
>>+
>>+ test do
>>+ # Check if sqlite is ok, because we build with --enable-loadable-sqlite-extensions
>>+ # and it can occur that building sqlite silently fails if OSX's sqlite is used.
>>+ system "#{bin}/python", "-c", "import sqlite3"
>>+ # Check if some other modules import. Then the linked libs are working.
>>+ system "#{bin}/python", "-c", "import Tkinter; root = Tkinter.Tk()"
>>+ system "#{bin}/python", "-c", "import gdbm"
>>+ system "#{bin}/python", "-c", "import zlib"
>>+ system bin/"pip", "list", "--format=columns"
>>+ end
>>+end
>>--
>>2.17.1
>>  
> 
> 
>--
>Oleg Piskunov
> 

-- 
Alexander Tikhonov

[-- Attachment #2: Type: text/html, Size: 23650 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini
  2020-03-06 12:59 Alexander V. Tikhonov
@ 2020-03-06 18:35 ` Oleg Piskunov
  2020-03-10  6:25   ` Alexander Tikhonov
  0 siblings, 1 reply; 5+ messages in thread
From: Oleg Piskunov @ 2020-03-06 18:35 UTC (permalink / raw)
  To: Alexander V. Tikhonov; +Cc: tarantool-patches

[-- Attachment #1: Type: text/plain, Size: 17858 bytes --]


LGTM.
Minor: Sasha, could you please change 1 comment below.
  
>Пятница, 6 марта 2020, 15:59 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
> 
>Added new gitlab-ci test jobs for OSX 10.14 testings.
>
>Set in use Python2 locally tapped version with its
>latest commit in it, since Python 2 is EOL.
>
>Set 'var' directory for test-run tool to the shorter
>path name to avoid of issues with long names.
>---
>
>Github:  https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
>
> .gitlab-ci.yml | 19 +--
> .gitlab.mk | 2 +-
> .travis.mk | 31 +++-
> tools/brew_taps/tntpython2.rb | 306 ++++++++++++++++++++++++++++++++++
> 4 files changed, 337 insertions(+), 21 deletions(-)
> create mode 100644 tools/brew_taps/tntpython2.rb
>
>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>index 5177a9903..47e4af5a2 100644
>--- a/.gitlab-ci.yml
>+++ b/.gitlab-ci.yml
>@@ -152,28 +152,21 @@ osx_13_release:
>     - ${GITLAB_MAKE} vms_test_osx
> 
> osx_14_release:
>- <<: *vbox_definition
>+ <<: *docker_test_definition
>   tags:
>- - vms_osx_14
>- variables:
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
>+ - osx_14
>   script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
> 
> osx_14_release_lto:
>   <<: *release_only_definition
>- <<: *vbox_definition
>+ <<: *docker_test_definition
>   tags:
>- - vms_osx_14
>+ - osx_14
>   variables:
>     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
>   script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
> 
> freebsd_12_release:
>   <<: *vbox_definition
>diff --git a/.gitlab.mk b/.gitlab.mk
>index b39c5c651..d804d3cf6 100644
>--- a/.gitlab.mk
>+++ b/.gitlab.mk
>@@ -14,7 +14,7 @@ git_submodule_update:
>  git submodule update --recursive --init
> 
> # Pass *_no_deps goals to .travis.mk.
>-test_%_no_deps: git_submodule_update
>+test_%: git_submodule_update
>  ${TRAVIS_MAKE} $@
> 
> # #######################################################
>diff --git a/.travis.mk b/.travis.mk
>index 42969ff56..a619ee95a 100644
>--- a/.travis.mk
>+++ b/.travis.mk
>@@ -5,6 +5,7 @@
> DOCKER_IMAGE?=packpack/packpack:debian-stretch
> TEST_RUN_EXTRA_PARAMS?=
> MAX_FILES?=65534
>+MAX_PROC?=2500
> 
> all: package
> 
>@@ -127,13 +128,24 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
> # OSX #
> #######
> 
>+# Python2 is EOF and we need to be sure that it is in use instead of Python3
# since Python 2 is EOL it's tapped version from local formula with its latest commit will be used
>+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake \
>+ file://${PWD}/tools/brew_taps/tntpython2.rb
>+
> deps_osx:
>- brew update
>- brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
>- python2 -V || brew install python2 --force
>+ # install brew using command from Homebrew repository instructions:
>+ #  https://github.com/Homebrew/install
>+ # NOTE: 'echo' command below is required since brew installation
>+ # script obliges the one to enter a newline for confirming the
>+ # installation via Ruby script.
>+ brew update || echo | /usr/bin/ruby -e \
>+ "$(curl -fsSL  https://raw.githubusercontent.com/Homebrew/install/master/install )"
>+ # try to install the packages either upgrade it to avoid of fails
>+ # if the package already exists with the previous version
>+ brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
>  curl --silent --show-error --retry 5  https://bootstrap.pypa.io/get-pip.py >get-pip.py
>- python get-pip.py --user
>- pip install --user --force-reinstall -r test-run/requirements.txt
>+ python get-pip.py
>+ pip install --force-reinstall -r test-run/requirements.txt
> 
> build_osx:
>  cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
>@@ -148,11 +160,16 @@ test_osx_no_deps: build_osx
>  # call as tests runs call.
>  # Tests: Temporary excluded replication/ suite with some tests
>  # from other suites by issues #4357 and #4370
>- echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>+ sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
>  launchctl limit maxfiles || : ; \
>  ulimit -n ${MAX_FILES} || : ; \
>  ulimit -n ; \
>- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
>+ sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
>+ launchctl limit maxproc || : ; \
>+ ulimit -u ${MAX_PROC} || : ; \
>+ ulimit -u ; \
>+ rm -rf /tmp/tnt ; \
>+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
>  app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
>  replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
> 
>diff --git a/tools/brew_taps/tntpython2.rb b/tools/brew_taps/tntpython2.rb
>new file mode 100644
>index 000000000..59f374030
>--- /dev/null
>+++ b/tools/brew_taps/tntpython2.rb
>@@ -0,0 +1,306 @@
>+class Tntpython2 < Formula
>+ desc "Interpreted, interactive, object-oriented programming language"
>+ homepage " https://www.python.org/ "
>+ url " https://www.python.org/ftp/python/2.7.17/Python-2.7.17.tar.xz "
>+ sha256 "4d43f033cdbd0aa7b7023c81b0e986fd11e653b5248dac9144d508f11812ba41"
>+ revision 1
>+ head " https://github.com/python/cpython.git ", :branch => "2.7"
>+
>+ bottle do
>+ sha256 "accfaa922708f00afb69ab230199f96e6ecdddd248a1eca586ce1e5e5cfd732b" => :catalina
>+ sha256 "54d3351d6be8268b2f5017894dcc8e083811dfa3812bdb9f79f989873b9a4542" => :mojave
>+ sha256 "cfd5c6eeac37065d19f527bb0798a9caf1928bab3340cd545224861a3c82f219" => :high_sierra
>+ end
>+
>+ # setuptools remembers the build flags python is built with and uses them to
>+ # build packages later. Xcode-only systems need different flags.
>+ pour_bottle? do
>+ reason <<~EOS
>+ The bottle needs the Apple Command Line Tools to be installed.
>+ You can install them, if desired, with:
>+ xcode-select --install
>+ EOS
>+ satisfy { MacOS::CLT.installed? }
>+ end
>+
>+ depends_on "pkg-config" => :build
>+ depends_on "gdbm"
>+ depends_on " openssl@1.1 "
>+ depends_on "readline"
>+ depends_on "sqlite"
>+
>+ resource "setuptools" do
>+ url " https://files.pythonhosted.org/packages/f4/d5/a6c19dcbcbc267aca376558797f036d9bcdff344c9f785fe7d0fe9a5f2a7/setuptools-41.4.0.zip "
>+ sha256 "7eae782ccf36b790c21bde7d86a4f303a441cd77036b25c559a602cf5186ce4d"
>+ end
>+
>+ resource "pip" do
>+ url " https://files.pythonhosted.org/packages/ce/ea/9b445176a65ae4ba22dce1d93e4b5fe182f953df71a145f557cffaffc1bf/pip-19.3.1.tar.gz "
>+ sha256 "21207d76c1031e517668898a6b46a9fb1501c7a4710ef5dfd6a40ad9e6757ea7"
>+ end
>+
>+ resource "wheel" do
>+ url " https://files.pythonhosted.org/packages/59/b0/11710a598e1e148fb7cbf9220fd2a0b82c98e94efbdecb299cb25e7f0b39/wheel-0.33.6.tar.gz "
>+ sha256 "10c9da68765315ed98850f8e048347c3eb06dd81822dc2ab1d4fde9dc9702646"
>+ end
>+
>+ def lib_cellar
>+ prefix/"Frameworks/Python.framework/Versions/2.7/lib/python2.7"
>+ end
>+
>+ def site_packages_cellar
>+ lib_cellar/"site-packages"
>+ end
>+
>+ # The HOMEBREW_PREFIX location of site-packages.
>+ def site_packages
>+ HOMEBREW_PREFIX/"lib/python2.7/site-packages"
>+ end
>+
>+ def install
>+ # Unset these so that installing pip and setuptools puts them where we want
>+ # and not into some other Python the user has installed.
>+ ENV["PYTHONHOME"] = nil
>+ ENV["PYTHONPATH"] = nil
>+
>+ args = %W[
>+ --prefix=#{prefix}
>+ --enable-ipv6
>+ --datarootdir=#{share}
>+ --datadir=#{share}
>+ --enable-framework=#{frameworks}
>+ --without-ensurepip
>+ ]
>+
>+ # See upstream bug report from 22 Jan 2018 "Significant performance problems
>+ # with Python 2.7 built with clang 3.x or 4.x"
>+ #  https://bugs.python.org/issue32616
>+ #  https://github.com/Homebrew/homebrew-core/issues/22743
>+ if DevelopmentTools.clang_build_version >= 802 &&
>+ DevelopmentTools.clang_build_version < 902
>+ args << "--without-computed-gotos"
>+ end
>+
>+ args << "--without-gcc" if ENV.compiler == :clang
>+
>+ cflags = []
>+ ldflags = []
>+ cppflags = []
>+
>+ if MacOS.sdk_path_if_needed
>+ # Help Python's build system (setuptools/pip) to build things on SDK-based systems
>+ # The setup.py looks at "-isysroot" to get the sysroot (and not at --sysroot)
>+ cflags << "-isysroot #{MacOS.sdk_path}" << "-I#{MacOS.sdk_path}/usr/include"
>+ ldflags << "-isysroot #{MacOS.sdk_path}"
>+ # For the Xlib.h, Python needs this header dir with the system Tk
>+ # Yep, this needs the absolute path where zlib needed a path relative
>+ # to the SDK.
>+ cflags << "-I#{MacOS.sdk_path}/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers"
>+ end
>+
>+ # Avoid linking to libgcc  https://code.activestate.com/lists/python-dev/112195/
>+ args << "MACOSX_DEPLOYMENT_TARGET=#{MacOS.version}"
>+
>+ # We want our readline and openssl! This is just to outsmart the detection code,
>+ # superenv handles that cc finds includes/libs!
>+ inreplace "setup.py" do |s|
>+ s.gsub! "do_readline = self.compiler.find_library_file(lib_dirs, 'readline')",
>+ "do_readline = '#{Formula["readline"].opt_lib}/libhistory.dylib'"
>+ s.gsub! "/usr/local/ssl", Formula[" openssl@1.1 "].opt_prefix
>+ end
>+
>+ inreplace "setup.py" do |s|
>+ s.gsub! "sqlite_setup_debug = False", "sqlite_setup_debug = True"
>+ s.gsub! "for d_ in inc_dirs + sqlite_inc_paths:",
>+ "for d_ in ['#{Formula["sqlite"].opt_include}']:"
>+
>+ # Allow sqlite3 module to load extensions:
>+ #  https://docs.python.org/library/sqlite3.html#f1
>+ s.gsub! 'sqlite_defines.append(("SQLITE_OMIT_LOAD_EXTENSION", "1"))', ""
>+ end
>+
>+ # Allow python modules to use ctypes.find_library to find homebrew's stuff
>+ # even if homebrew is not a /usr/local/lib. Try this with:
>+ # `brew install enchant && pip install pyenchant`
>+ inreplace "./Lib/ctypes/macholib/dyld.py" do |f|
>+ f.gsub! "DEFAULT_LIBRARY_FALLBACK = [", "DEFAULT_LIBRARY_FALLBACK = [ '#{HOMEBREW_PREFIX}/lib',"
>+ f.gsub! "DEFAULT_FRAMEWORK_FALLBACK = [", "DEFAULT_FRAMEWORK_FALLBACK = [ '#{HOMEBREW_PREFIX}/Frameworks',"
>+ end
>+
>+ args << "CFLAGS=#{cflags.join(" ")}" unless cflags.empty?
>+ args << "LDFLAGS=#{ldflags.join(" ")}" unless ldflags.empty?
>+ args << "CPPFLAGS=#{cppflags.join(" ")}" unless cppflags.empty?
>+
>+ system "./configure", *args
>+ system "make"
>+
>+ ENV.deparallelize do
>+ # Tell Python not to install into /Applications
>+ system "make", "install", "PYTHONAPPSDIR=#{prefix}"
>+ system "make", "frameworkinstallextras", "PYTHONAPPSDIR=#{pkgshare}"
>+ end
>+
>+ # Fixes setting Python build flags for certain software
>+ # See:  https://github.com/Homebrew/homebrew/pull/20182
>+ #  https://bugs.python.org/issue3588
>+ inreplace lib_cellar/"config/Makefile" do |s|
>+ s.change_make_var! "LINKFORSHARED",
>+ "-u _PyMac_Error $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)"
>+ end
>+
>+ # Prevent third-party packages from building against fragile Cellar paths
>+ inreplace [lib_cellar/"_sysconfigdata.py",
>+ lib_cellar/"config/Makefile",
>+ frameworks/"Python.framework/Versions/Current/lib/pkgconfig/python-2.7.pc"],
>+ prefix, opt_prefix
>+
>+ # Symlink the pkgconfig files into HOMEBREW_PREFIX so they're accessible.
>+ (lib/"pkgconfig").install_symlink Dir[frameworks/"Python.framework/Versions/Current/lib/pkgconfig/*"]
>+
>+ # Remove 2to3 because Python 3 also installs it
>+ rm bin/"2to3"
>+
>+ # A fix, because python and python@2 both want to install Python.framework
>+ # and therefore we can't link both into HOMEBREW_PREFIX/Frameworks
>+ #  https://github.com/Homebrew/homebrew/issues/15943
>+ ["Headers", "Python", "Resources"].each { |f| rm(prefix/"Frameworks/Python.framework/#{f}") }
>+ rm prefix/"Frameworks/Python.framework/Versions/Current"
>+
>+ # Remove the site-packages that Python created in its Cellar.
>+ site_packages_cellar.rmtree
>+
>+ (libexec/"setuptools").install resource("setuptools")
>+ (libexec/"pip").install resource("pip")
>+ (libexec/"wheel").install resource("wheel")
>+ end
>+
>+ def post_install
>+ # Avoid conflicts with lingering unversioned files from Python 3
>+ rm_f %W[
>+ #{HOMEBREW_PREFIX}/bin/easy_install
>+ #{HOMEBREW_PREFIX}/bin/pip
>+ #{HOMEBREW_PREFIX}/bin/wheel
>+ ]
>+
>+ # Fix up the site-packages so that user-installed Python software survives
>+ # minor updates, such as going from 2.7.0 to 2.7.1:
>+
>+ # Create a site-packages in HOMEBREW_PREFIX/lib/python2.7/site-packages
>+ site_packages.mkpath
>+
>+ # Symlink the prefix site-packages into the cellar.
>+ site_packages_cellar.unlink if site_packages_cellar.exist?
>+ site_packages_cellar.parent.install_symlink site_packages
>+
>+ # Write our sitecustomize.py
>+ rm_rf Dir["#{site_packages}/sitecustomize.py[co]"]
>+ (site_packages/"sitecustomize.py").atomic_write(sitecustomize)
>+
>+ # Remove old setuptools installations that may still fly around and be
>+ # listed in the easy_install.pth. This can break setuptools build with
>+ # zipimport.ZipImportError: bad local file header
>+ # setuptools-0.9.5-py3.3.egg
>+ rm_rf Dir["#{site_packages}/setuptools*"]
>+ rm_rf Dir["#{site_packages}/distribute*"]
>+ rm_rf Dir["#{site_packages}/pip[-_.][0-9]*", "#{site_packages}/pip"]
>+
>+ setup_args = ["-s", "setup.py", "--no-user-cfg", "install", "--force",
>+ "--verbose",
>+ "--single-version-externally-managed",
>+ "--record=installed.txt",
>+ "--install-scripts=#{bin}",
>+ "--install-lib=#{site_packages}"]
>+
>+ (libexec/"setuptools").cd { system "#{bin}/python", *setup_args }
>+ (libexec/"pip").cd { system "#{bin}/python", *setup_args }
>+ (libexec/"wheel").cd { system "#{bin}/python", *setup_args }
>+
>+ # When building from source, these symlinks will not exist, since
>+ # post_install happens after linking.
>+ %w[pip pip2 pip2.7 easy_install easy_install-2.7 wheel].each do |e|
>+ (HOMEBREW_PREFIX/"bin").install_symlink bin/e
>+ end
>+
>+ # Help distutils find brewed stuff when building extensions
>+ include_dirs = [HOMEBREW_PREFIX/"include", Formula[" openssl@1.1 "].opt_include,
>+ Formula["sqlite"].opt_include]
>+ library_dirs = [HOMEBREW_PREFIX/"lib", Formula[" openssl@1.1 "].opt_lib,
>+ Formula["sqlite"].opt_lib]
>+
>+ cfg = lib_cellar/"distutils/distutils.cfg"
>+ cfg.atomic_write <<~EOS
>+ [install]
>+ prefix=#{HOMEBREW_PREFIX}
>+ [build_ext]
>+ include_dirs=#{include_dirs.join ":"}
>+ library_dirs=#{library_dirs.join ":"}
>+ EOS
>+ end
>+
>+ def sitecustomize
>+ <<~EOS
>+ # This file is created by Homebrew and is executed on each python startup.
>+ # Don't print from here, or else python command line scripts may fail!
>+ # < https://docs.brew.sh/Homebrew-and-Python >
>+ import re
>+ import os
>+ import sys
>+ if sys.version_info[0] != 2:
>+ # This can only happen if the user has set the PYTHONPATH for 3.x and run Python 2.x or vice versa.
>+ # Every Python looks at the PYTHONPATH variable and we can't fix it here in sitecustomize.py,
>+ # because the PYTHONPATH is evaluated after the sitecustomize.py. Many modules (e.g. PyQt4) are
>+ # built only for a specific version of Python and will fail with cryptic error messages.
>+ # In the end this means: Don't set the PYTHONPATH permanently if you use different Python versions.
>+ exit('Your PYTHONPATH points to a site-packages dir for Python 2.x but you are running Python ' +
>+ str(sys.version_info[0]) + '.x!\\n PYTHONPATH is currently: "' + str(os.environ['PYTHONPATH']) + '"\\n' +
>+ ' You should `unset PYTHONPATH` to fix this.')
>+ # Only do this for a brewed python:
>+ if os.path.realpath(sys.executable).startswith('#{rack}'):
>+ # Shuffle /Library site-packages to the end of sys.path and reject
>+ # paths in /System pre-emptively (#14712)
>+ library_site = '/Library/Python/2.7/site-packages'
>+ library_packages = [p for p in sys.path if p.startswith(library_site)]
>+ sys.path = [p for p in sys.path if not p.startswith(library_site) and
>+ not p.startswith('/System')]
>+ # .pth files have already been processed so don't use addsitedir
>+ sys.path.extend(library_packages)
>+ # the Cellar site-packages is a symlink to the HOMEBREW_PREFIX
>+ # site_packages; prefer the shorter paths
>+ long_prefix = re.compile(r'#{rack}/[0-9\._abrc]+/Frameworks/Python\.framework/Versions/2\.7/lib/python2\.7/site-packages')
>+ sys.path = [long_prefix.sub('#{site_packages}', p) for p in sys.path]
>+ # LINKFORSHARED (and python-config --ldflags) return the
>+ # full path to the lib (yes, "Python" is actually the lib, not a
>+ # dir) so that third-party software does not need to add the
>+ # -F/#{HOMEBREW_PREFIX}/Frameworks switch.
>+ try:
>+ from _sysconfigdata import build_time_vars
>+ build_time_vars['LINKFORSHARED'] = '-u _PyMac_Error #{opt_prefix}/Frameworks/Python.framework/Versions/2.7/Python'
>+ except:
>+ pass # remember: don't print here. Better to fail silently.
>+ # Set the sys.executable to use the opt_prefix
>+ sys.executable = '#{opt_bin}/python2.7'
>+ EOS
>+ end
>+
>+ def caveats; <<~EOS
>+ Pip and setuptools have been installed. To update them
>+ pip install --upgrade pip setuptools
>+ You can install Python packages with
>+ pip install <package>
>+ They will install into the site-package directory
>+ #{site_packages}
>+ See:  https://docs.brew.sh/Homebrew-and-Python
>+ EOS
>+ end
>+
>+ test do
>+ # Check if sqlite is ok, because we build with --enable-loadable-sqlite-extensions
>+ # and it can occur that building sqlite silently fails if OSX's sqlite is used.
>+ system "#{bin}/python", "-c", "import sqlite3"
>+ # Check if some other modules import. Then the linked libs are working.
>+ system "#{bin}/python", "-c", "import Tkinter; root = Tkinter.Tk()"
>+ system "#{bin}/python", "-c", "import gdbm"
>+ system "#{bin}/python", "-c", "import zlib"
>+ system bin/"pip", "list", "--format=columns"
>+ end
>+end
>--
>2.17.1
>  
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 21701 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini
@ 2020-03-06 12:59 Alexander V. Tikhonov
  2020-03-06 18:35 ` Oleg Piskunov
  0 siblings, 1 reply; 5+ messages in thread
From: Alexander V. Tikhonov @ 2020-03-06 12:59 UTC (permalink / raw)
  To: Oleg Piskunov; +Cc: tarantool-patches

Added new gitlab-ci test jobs for OSX 10.14 testings.

Set in use Python2 locally tapped version with its
latest commit in it, since Python 2 is EOL.

Set 'var' directory for test-run tool to the shorter
path name to avoid of issues with long names.
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci

 .gitlab-ci.yml                |  19 +--
 .gitlab.mk                    |   2 +-
 .travis.mk                    |  31 +++-
 tools/brew_taps/tntpython2.rb | 306 ++++++++++++++++++++++++++++++++++
 4 files changed, 337 insertions(+), 21 deletions(-)
 create mode 100644 tools/brew_taps/tntpython2.rb

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5177a9903..47e4af5a2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -152,28 +152,21 @@ osx_13_release:
     - ${GITLAB_MAKE} vms_test_osx
 
 osx_14_release:
-  <<: *vbox_definition
+  <<: *docker_test_definition
   tags:
-    - vms_osx_14
-  variables:
-    VMS_NAME: 'osx_14'
-    VMS_USER: 'tarantool'
-    VMS_PORT: '2222'
+    - osx_14
   script:
-    - ${GITLAB_MAKE} vms_test_osx
+    - ${GITLAB_MAKE} test_osx
 
 osx_14_release_lto:
   <<: *release_only_definition
-  <<: *vbox_definition
+  <<: *docker_test_definition
   tags:
-    - vms_osx_14
+    - osx_14
   variables:
     EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
-    VMS_NAME: 'osx_14'
-    VMS_USER: 'tarantool'
-    VMS_PORT: '2222'
   script:
-    - ${GITLAB_MAKE} vms_test_osx
+    - ${GITLAB_MAKE} test_osx
 
 freebsd_12_release:
   <<: *vbox_definition
diff --git a/.gitlab.mk b/.gitlab.mk
index b39c5c651..d804d3cf6 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -14,7 +14,7 @@ git_submodule_update:
 		git submodule update --recursive --init
 
 # Pass *_no_deps goals to .travis.mk.
-test_%_no_deps: git_submodule_update
+test_%: git_submodule_update
 	${TRAVIS_MAKE} $@
 
 # #######################################################
diff --git a/.travis.mk b/.travis.mk
index 42969ff56..a619ee95a 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -5,6 +5,7 @@
 DOCKER_IMAGE?=packpack/packpack:debian-stretch
 TEST_RUN_EXTRA_PARAMS?=
 MAX_FILES?=65534
+MAX_PROC?=2500
 
 all: package
 
@@ -127,13 +128,24 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
 # OSX #
 #######
 
+# Python2 is EOF and we need to be sure that it is in use instead of Python3
+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake \
+	 file://$${PWD}/tools/brew_taps/tntpython2.rb
+
 deps_osx:
-	brew update
-	brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
-	python2 -V || brew install python2 --force
+	# install brew using command from Homebrew repository instructions:
+	#   https://github.com/Homebrew/install
+	# NOTE: 'echo' command below is required since brew installation
+	# script obliges the one to enter a newline for confirming the
+	# installation via Ruby script.
+	brew update || echo | /usr/bin/ruby -e \
+		"$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+	# try to install the packages either upgrade it to avoid of fails
+	# if the package already exists with the previous version
+	brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
 	curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
-	python get-pip.py --user
-	pip install --user --force-reinstall -r test-run/requirements.txt
+	python get-pip.py
+	pip install --force-reinstall -r test-run/requirements.txt
 
 build_osx:
 	cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
@@ -148,11 +160,16 @@ test_osx_no_deps: build_osx
 	# call as tests runs call.
 	# Tests: Temporary excluded replication/ suite with some tests
 	#        from other suites by issues #4357 and #4370
-	echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
+	sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
 		launchctl limit maxfiles || : ; \
 		ulimit -n ${MAX_FILES} || : ; \
 		ulimit -n ; \
-		cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+		sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
+		launchctl limit maxproc || : ; \
+		ulimit -u ${MAX_PROC} || : ; \
+		ulimit -u ; \
+		rm -rf /tmp/tnt ; \
+		cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
 			app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
 			replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
 
diff --git a/tools/brew_taps/tntpython2.rb b/tools/brew_taps/tntpython2.rb
new file mode 100644
index 000000000..59f374030
--- /dev/null
+++ b/tools/brew_taps/tntpython2.rb
@@ -0,0 +1,306 @@
+class Tntpython2 < Formula
+  desc "Interpreted, interactive, object-oriented programming language"
+  homepage "https://www.python.org/"
+  url "https://www.python.org/ftp/python/2.7.17/Python-2.7.17.tar.xz"
+  sha256 "4d43f033cdbd0aa7b7023c81b0e986fd11e653b5248dac9144d508f11812ba41"
+  revision 1
+  head "https://github.com/python/cpython.git", :branch => "2.7"
+
+  bottle do
+    sha256 "accfaa922708f00afb69ab230199f96e6ecdddd248a1eca586ce1e5e5cfd732b" => :catalina
+    sha256 "54d3351d6be8268b2f5017894dcc8e083811dfa3812bdb9f79f989873b9a4542" => :mojave
+    sha256 "cfd5c6eeac37065d19f527bb0798a9caf1928bab3340cd545224861a3c82f219" => :high_sierra
+  end
+
+  # setuptools remembers the build flags python is built with and uses them to
+  # build packages later. Xcode-only systems need different flags.
+  pour_bottle? do
+    reason <<~EOS
+      The bottle needs the Apple Command Line Tools to be installed.
+        You can install them, if desired, with:
+          xcode-select --install
+    EOS
+    satisfy { MacOS::CLT.installed? }
+  end
+
+  depends_on "pkg-config" => :build
+  depends_on "gdbm"
+  depends_on "openssl@1.1"
+  depends_on "readline"
+  depends_on "sqlite"
+
+  resource "setuptools" do
+    url "https://files.pythonhosted.org/packages/f4/d5/a6c19dcbcbc267aca376558797f036d9bcdff344c9f785fe7d0fe9a5f2a7/setuptools-41.4.0.zip"
+    sha256 "7eae782ccf36b790c21bde7d86a4f303a441cd77036b25c559a602cf5186ce4d"
+  end
+
+  resource "pip" do
+    url "https://files.pythonhosted.org/packages/ce/ea/9b445176a65ae4ba22dce1d93e4b5fe182f953df71a145f557cffaffc1bf/pip-19.3.1.tar.gz"
+    sha256 "21207d76c1031e517668898a6b46a9fb1501c7a4710ef5dfd6a40ad9e6757ea7"
+  end
+
+  resource "wheel" do
+    url "https://files.pythonhosted.org/packages/59/b0/11710a598e1e148fb7cbf9220fd2a0b82c98e94efbdecb299cb25e7f0b39/wheel-0.33.6.tar.gz"
+    sha256 "10c9da68765315ed98850f8e048347c3eb06dd81822dc2ab1d4fde9dc9702646"
+  end
+
+  def lib_cellar
+    prefix/"Frameworks/Python.framework/Versions/2.7/lib/python2.7"
+  end
+
+  def site_packages_cellar
+    lib_cellar/"site-packages"
+  end
+
+  # The HOMEBREW_PREFIX location of site-packages.
+  def site_packages
+    HOMEBREW_PREFIX/"lib/python2.7/site-packages"
+  end
+
+  def install
+    # Unset these so that installing pip and setuptools puts them where we want
+    # and not into some other Python the user has installed.
+    ENV["PYTHONHOME"] = nil
+    ENV["PYTHONPATH"] = nil
+
+    args = %W[
+      --prefix=#{prefix}
+      --enable-ipv6
+      --datarootdir=#{share}
+      --datadir=#{share}
+      --enable-framework=#{frameworks}
+      --without-ensurepip
+    ]
+
+    # See upstream bug report from 22 Jan 2018 "Significant performance problems
+    # with Python 2.7 built with clang 3.x or 4.x"
+    # https://bugs.python.org/issue32616
+    # https://github.com/Homebrew/homebrew-core/issues/22743
+    if DevelopmentTools.clang_build_version >= 802 &&
+       DevelopmentTools.clang_build_version < 902
+      args << "--without-computed-gotos"
+    end
+
+    args << "--without-gcc" if ENV.compiler == :clang
+
+    cflags   = []
+    ldflags  = []
+    cppflags = []
+
+    if MacOS.sdk_path_if_needed
+      # Help Python's build system (setuptools/pip) to build things on SDK-based systems
+      # The setup.py looks at "-isysroot" to get the sysroot (and not at --sysroot)
+      cflags  << "-isysroot #{MacOS.sdk_path}" << "-I#{MacOS.sdk_path}/usr/include"
+      ldflags << "-isysroot #{MacOS.sdk_path}"
+      # For the Xlib.h, Python needs this header dir with the system Tk
+      # Yep, this needs the absolute path where zlib needed a path relative
+      # to the SDK.
+      cflags  << "-I#{MacOS.sdk_path}/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers"
+    end
+
+    # Avoid linking to libgcc https://code.activestate.com/lists/python-dev/112195/
+    args << "MACOSX_DEPLOYMENT_TARGET=#{MacOS.version}"
+
+    # We want our readline and openssl! This is just to outsmart the detection code,
+    # superenv handles that cc finds includes/libs!
+    inreplace "setup.py" do |s|
+      s.gsub! "do_readline = self.compiler.find_library_file(lib_dirs, 'readline')",
+              "do_readline = '#{Formula["readline"].opt_lib}/libhistory.dylib'"
+      s.gsub! "/usr/local/ssl", Formula["openssl@1.1"].opt_prefix
+    end
+
+    inreplace "setup.py" do |s|
+      s.gsub! "sqlite_setup_debug = False", "sqlite_setup_debug = True"
+      s.gsub! "for d_ in inc_dirs + sqlite_inc_paths:",
+              "for d_ in ['#{Formula["sqlite"].opt_include}']:"
+
+      # Allow sqlite3 module to load extensions:
+      # https://docs.python.org/library/sqlite3.html#f1
+      s.gsub! 'sqlite_defines.append(("SQLITE_OMIT_LOAD_EXTENSION", "1"))', ""
+    end
+
+    # Allow python modules to use ctypes.find_library to find homebrew's stuff
+    # even if homebrew is not a /usr/local/lib. Try this with:
+    # `brew install enchant && pip install pyenchant`
+    inreplace "./Lib/ctypes/macholib/dyld.py" do |f|
+      f.gsub! "DEFAULT_LIBRARY_FALLBACK = [", "DEFAULT_LIBRARY_FALLBACK = [ '#{HOMEBREW_PREFIX}/lib',"
+      f.gsub! "DEFAULT_FRAMEWORK_FALLBACK = [", "DEFAULT_FRAMEWORK_FALLBACK = [ '#{HOMEBREW_PREFIX}/Frameworks',"
+    end
+
+    args << "CFLAGS=#{cflags.join(" ")}" unless cflags.empty?
+    args << "LDFLAGS=#{ldflags.join(" ")}" unless ldflags.empty?
+    args << "CPPFLAGS=#{cppflags.join(" ")}" unless cppflags.empty?
+
+    system "./configure", *args
+    system "make"
+
+    ENV.deparallelize do
+      # Tell Python not to install into /Applications
+      system "make", "install", "PYTHONAPPSDIR=#{prefix}"
+      system "make", "frameworkinstallextras", "PYTHONAPPSDIR=#{pkgshare}"
+    end
+
+    # Fixes setting Python build flags for certain software
+    # See: https://github.com/Homebrew/homebrew/pull/20182
+    # https://bugs.python.org/issue3588
+    inreplace lib_cellar/"config/Makefile" do |s|
+      s.change_make_var! "LINKFORSHARED",
+        "-u _PyMac_Error $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)"
+    end
+
+    # Prevent third-party packages from building against fragile Cellar paths
+    inreplace [lib_cellar/"_sysconfigdata.py",
+               lib_cellar/"config/Makefile",
+               frameworks/"Python.framework/Versions/Current/lib/pkgconfig/python-2.7.pc"],
+              prefix, opt_prefix
+
+    # Symlink the pkgconfig files into HOMEBREW_PREFIX so they're accessible.
+    (lib/"pkgconfig").install_symlink Dir[frameworks/"Python.framework/Versions/Current/lib/pkgconfig/*"]
+
+    # Remove 2to3 because Python 3 also installs it
+    rm bin/"2to3"
+
+    # A fix, because python and python@2 both want to install Python.framework
+    # and therefore we can't link both into HOMEBREW_PREFIX/Frameworks
+    # https://github.com/Homebrew/homebrew/issues/15943
+    ["Headers", "Python", "Resources"].each { |f| rm(prefix/"Frameworks/Python.framework/#{f}") }
+    rm prefix/"Frameworks/Python.framework/Versions/Current"
+
+    # Remove the site-packages that Python created in its Cellar.
+    site_packages_cellar.rmtree
+
+    (libexec/"setuptools").install resource("setuptools")
+    (libexec/"pip").install resource("pip")
+    (libexec/"wheel").install resource("wheel")
+  end
+
+  def post_install
+    # Avoid conflicts with lingering unversioned files from Python 3
+    rm_f %W[
+      #{HOMEBREW_PREFIX}/bin/easy_install
+      #{HOMEBREW_PREFIX}/bin/pip
+      #{HOMEBREW_PREFIX}/bin/wheel
+    ]
+
+    # Fix up the site-packages so that user-installed Python software survives
+    # minor updates, such as going from 2.7.0 to 2.7.1:
+
+    # Create a site-packages in HOMEBREW_PREFIX/lib/python2.7/site-packages
+    site_packages.mkpath
+
+    # Symlink the prefix site-packages into the cellar.
+    site_packages_cellar.unlink if site_packages_cellar.exist?
+    site_packages_cellar.parent.install_symlink site_packages
+
+    # Write our sitecustomize.py
+    rm_rf Dir["#{site_packages}/sitecustomize.py[co]"]
+    (site_packages/"sitecustomize.py").atomic_write(sitecustomize)
+
+    # Remove old setuptools installations that may still fly around and be
+    # listed in the easy_install.pth. This can break setuptools build with
+    # zipimport.ZipImportError: bad local file header
+    # setuptools-0.9.5-py3.3.egg
+    rm_rf Dir["#{site_packages}/setuptools*"]
+    rm_rf Dir["#{site_packages}/distribute*"]
+    rm_rf Dir["#{site_packages}/pip[-_.][0-9]*", "#{site_packages}/pip"]
+
+    setup_args = ["-s", "setup.py", "--no-user-cfg", "install", "--force",
+                  "--verbose",
+                  "--single-version-externally-managed",
+                  "--record=installed.txt",
+                  "--install-scripts=#{bin}",
+                  "--install-lib=#{site_packages}"]
+
+    (libexec/"setuptools").cd { system "#{bin}/python", *setup_args }
+    (libexec/"pip").cd { system "#{bin}/python", *setup_args }
+    (libexec/"wheel").cd { system "#{bin}/python", *setup_args }
+
+    # When building from source, these symlinks will not exist, since
+    # post_install happens after linking.
+    %w[pip pip2 pip2.7 easy_install easy_install-2.7 wheel].each do |e|
+      (HOMEBREW_PREFIX/"bin").install_symlink bin/e
+    end
+
+    # Help distutils find brewed stuff when building extensions
+    include_dirs = [HOMEBREW_PREFIX/"include", Formula["openssl@1.1"].opt_include,
+                    Formula["sqlite"].opt_include]
+    library_dirs = [HOMEBREW_PREFIX/"lib", Formula["openssl@1.1"].opt_lib,
+                    Formula["sqlite"].opt_lib]
+
+    cfg = lib_cellar/"distutils/distutils.cfg"
+    cfg.atomic_write <<~EOS
+      [install]
+      prefix=#{HOMEBREW_PREFIX}
+      [build_ext]
+      include_dirs=#{include_dirs.join ":"}
+      library_dirs=#{library_dirs.join ":"}
+    EOS
+  end
+
+  def sitecustomize
+    <<~EOS
+      # This file is created by Homebrew and is executed on each python startup.
+      # Don't print from here, or else python command line scripts may fail!
+      # <https://docs.brew.sh/Homebrew-and-Python>
+      import re
+      import os
+      import sys
+      if sys.version_info[0] != 2:
+          # This can only happen if the user has set the PYTHONPATH for 3.x and run Python 2.x or vice versa.
+          # Every Python looks at the PYTHONPATH variable and we can't fix it here in sitecustomize.py,
+          # because the PYTHONPATH is evaluated after the sitecustomize.py. Many modules (e.g. PyQt4) are
+          # built only for a specific version of Python and will fail with cryptic error messages.
+          # In the end this means: Don't set the PYTHONPATH permanently if you use different Python versions.
+          exit('Your PYTHONPATH points to a site-packages dir for Python 2.x but you are running Python ' +
+               str(sys.version_info[0]) + '.x!\\n     PYTHONPATH is currently: "' + str(os.environ['PYTHONPATH']) + '"\\n' +
+               '     You should `unset PYTHONPATH` to fix this.')
+      # Only do this for a brewed python:
+      if os.path.realpath(sys.executable).startswith('#{rack}'):
+          # Shuffle /Library site-packages to the end of sys.path and reject
+          # paths in /System pre-emptively (#14712)
+          library_site = '/Library/Python/2.7/site-packages'
+          library_packages = [p for p in sys.path if p.startswith(library_site)]
+          sys.path = [p for p in sys.path if not p.startswith(library_site) and
+                                             not p.startswith('/System')]
+          # .pth files have already been processed so don't use addsitedir
+          sys.path.extend(library_packages)
+          # the Cellar site-packages is a symlink to the HOMEBREW_PREFIX
+          # site_packages; prefer the shorter paths
+          long_prefix = re.compile(r'#{rack}/[0-9\._abrc]+/Frameworks/Python\.framework/Versions/2\.7/lib/python2\.7/site-packages')
+          sys.path = [long_prefix.sub('#{site_packages}', p) for p in sys.path]
+          # LINKFORSHARED (and python-config --ldflags) return the
+          # full path to the lib (yes, "Python" is actually the lib, not a
+          # dir) so that third-party software does not need to add the
+          # -F/#{HOMEBREW_PREFIX}/Frameworks switch.
+          try:
+              from _sysconfigdata import build_time_vars
+              build_time_vars['LINKFORSHARED'] = '-u _PyMac_Error #{opt_prefix}/Frameworks/Python.framework/Versions/2.7/Python'
+          except:
+              pass  # remember: don't print here. Better to fail silently.
+          # Set the sys.executable to use the opt_prefix
+          sys.executable = '#{opt_bin}/python2.7'
+    EOS
+  end
+
+  def caveats; <<~EOS
+    Pip and setuptools have been installed. To update them
+      pip install --upgrade pip setuptools
+    You can install Python packages with
+      pip install <package>
+    They will install into the site-package directory
+      #{site_packages}
+    See: https://docs.brew.sh/Homebrew-and-Python
+  EOS
+  end
+
+  test do
+    # Check if sqlite is ok, because we build with --enable-loadable-sqlite-extensions
+    # and it can occur that building sqlite silently fails if OSX's sqlite is used.
+    system "#{bin}/python", "-c", "import sqlite3"
+    # Check if some other modules import. Then the linked libs are working.
+    system "#{bin}/python", "-c", "import Tkinter; root = Tkinter.Tk()"
+    system "#{bin}/python", "-c", "import gdbm"
+    system "#{bin}/python", "-c", "import zlib"
+    system bin/"pip", "list", "--format=columns"
+  end
+end
-- 
2.17.1

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-03-10  6:25 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-27 10:30 [Tarantool-patches] [PATCH v2] gitlab-ci: implement OSX 10.14 testing on mac mini Alexander V. Tikhonov
2020-02-27 14:26 ` Oleg Piskunov
2020-03-06 12:59 Alexander V. Tikhonov
2020-03-06 18:35 ` Oleg Piskunov
2020-03-10  6:25   ` Alexander Tikhonov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox