* [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
@ 2020-03-27 7:29 Alexander V. Tikhonov
2020-03-31 5:34 ` Oleg Piskunov
2020-04-02 7:43 ` Kirill Yukhin
0 siblings, 2 replies; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-03-27 7:29 UTC (permalink / raw)
To: Oleg Piskunov, Sergey Bronnikov; +Cc: tarantool-patches
Switched OSX 10.14 test jobs to MAC mini hosts.
Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
used to pass all appropriate target calls to Travis make rule target.
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
.gitlab-ci.yml | 19 ++++++-------------
.gitlab.mk | 2 +-
2 files changed, 7 insertions(+), 14 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5177a9903..47e4af5a2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -152,28 +152,21 @@ osx_13_release:
- ${GITLAB_MAKE} vms_test_osx
osx_14_release:
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
- variables:
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
+ - osx_14
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
osx_14_release_lto:
<<: *release_only_definition
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
+ - osx_14
variables:
EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
freebsd_12_release:
<<: *vbox_definition
diff --git a/.gitlab.mk b/.gitlab.mk
index b39c5c651..d804d3cf6 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -14,7 +14,7 @@ git_submodule_update:
git submodule update --recursive --init
# Pass *_no_deps goals to .travis.mk.
-test_%_no_deps: git_submodule_update
+test_%: git_submodule_update
${TRAVIS_MAKE} $@
# #######################################################
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
2020-03-27 7:29 [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini Alexander V. Tikhonov
@ 2020-03-31 5:34 ` Oleg Piskunov
2020-03-31 9:10 ` Sergey Bronnikov
2020-04-02 7:43 ` Kirill Yukhin
1 sibling, 1 reply; 6+ messages in thread
From: Oleg Piskunov @ 2020-03-31 5:34 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
[-- Attachment #1: Type: text/plain, Size: 1887 bytes --]
LGTM
>Пятница, 27 марта 2020, 10:29 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
>
>Switched OSX 10.14 test jobs to MAC mini hosts.
>Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
>used to pass all appropriate target calls to Travis make rule target.
>---
>
>Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
>
> .gitlab-ci.yml | 19 ++++++-------------
> .gitlab.mk | 2 +-
> 2 files changed, 7 insertions(+), 14 deletions(-)
>
>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>index 5177a9903..47e4af5a2 100644
>--- a/.gitlab-ci.yml
>+++ b/.gitlab-ci.yml
>@@ -152,28 +152,21 @@ osx_13_release:
> - ${GITLAB_MAKE} vms_test_osx
>
> osx_14_release:
>- <<: *vbox_definition
>+ <<: *docker_test_definition
> tags:
>- - vms_osx_14
>- variables:
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
>+ - osx_14
> script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
>
> osx_14_release_lto:
> <<: *release_only_definition
>- <<: *vbox_definition
>+ <<: *docker_test_definition
> tags:
>- - vms_osx_14
>+ - osx_14
> variables:
> EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
>- VMS_NAME: 'osx_14'
>- VMS_USER: 'tarantool'
>- VMS_PORT: '2222'
> script:
>- - ${GITLAB_MAKE} vms_test_osx
>+ - ${GITLAB_MAKE} test_osx
>
> freebsd_12_release:
> <<: *vbox_definition
>diff --git a/.gitlab.mk b/.gitlab.mk
>index b39c5c651..d804d3cf6 100644
>--- a/.gitlab.mk
>+++ b/.gitlab.mk
>@@ -14,7 +14,7 @@ git_submodule_update:
> git submodule update --recursive --init
>
> # Pass *_no_deps goals to .travis.mk.
>-test_%_no_deps: git_submodule_update
>+test_%: git_submodule_update
> ${TRAVIS_MAKE} $@
>
> # #######################################################
>--
>2.17.1
>
--
Oleg Piskunov
[-- Attachment #2: Type: text/html, Size: 2779 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
2020-03-31 5:34 ` Oleg Piskunov
@ 2020-03-31 9:10 ` Sergey Bronnikov
0 siblings, 0 replies; 6+ messages in thread
From: Sergey Bronnikov @ 2020-03-31 9:10 UTC (permalink / raw)
To: Oleg Piskunov; +Cc: tarantool-patches
LGTM
On 08:34 Tue 31 Mar , Oleg Piskunov wrote:
>
> LGTM
>
>
> >Пятница, 27 марта 2020, 10:29 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
> >
> >Switched OSX 10.14 test jobs to MAC mini hosts.
> >Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
> >used to pass all appropriate target calls to Travis make rule target.
> >---
> >
> >Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
> >
> > .gitlab-ci.yml | 19 ++++++-------------
> > .gitlab.mk | 2 +-
> > 2 files changed, 7 insertions(+), 14 deletions(-)
> >
> >diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> >index 5177a9903..47e4af5a2 100644
> >--- a/.gitlab-ci.yml
> >+++ b/.gitlab-ci.yml
> >@@ -152,28 +152,21 @@ osx_13_release:
> > - ${GITLAB_MAKE} vms_test_osx
> >
> > osx_14_release:
> >- <<: *vbox_definition
> >+ <<: *docker_test_definition
> > tags:
> >- - vms_osx_14
> >- variables:
> >- VMS_NAME: 'osx_14'
> >- VMS_USER: 'tarantool'
> >- VMS_PORT: '2222'
> >+ - osx_14
> > script:
> >- - ${GITLAB_MAKE} vms_test_osx
> >+ - ${GITLAB_MAKE} test_osx
> >
> > osx_14_release_lto:
> > <<: *release_only_definition
> >- <<: *vbox_definition
> >+ <<: *docker_test_definition
> > tags:
> >- - vms_osx_14
> >+ - osx_14
> > variables:
> > EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
> >- VMS_NAME: 'osx_14'
> >- VMS_USER: 'tarantool'
> >- VMS_PORT: '2222'
> > script:
> >- - ${GITLAB_MAKE} vms_test_osx
> >+ - ${GITLAB_MAKE} test_osx
> >
> > freebsd_12_release:
> > <<: *vbox_definition
> >diff --git a/.gitlab.mk b/.gitlab.mk
> >index b39c5c651..d804d3cf6 100644
> >--- a/.gitlab.mk
> >+++ b/.gitlab.mk
> >@@ -14,7 +14,7 @@ git_submodule_update:
> > git submodule update --recursive --init
> >
> > # Pass *_no_deps goals to .travis.mk.
> >-test_%_no_deps: git_submodule_update
> >+test_%: git_submodule_update
> > ${TRAVIS_MAKE} $@
> >
> > # #######################################################
> >--
> >2.17.1
> >
>
>
> --
> Oleg Piskunov
>
--
sergeyb@
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
2020-03-27 7:29 [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini Alexander V. Tikhonov
2020-03-31 5:34 ` Oleg Piskunov
@ 2020-04-02 7:43 ` Kirill Yukhin
1 sibling, 0 replies; 6+ messages in thread
From: Kirill Yukhin @ 2020-04-02 7:43 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: Oleg Piskunov, tarantool-patches
Hello,
On 27 мар 10:29, Alexander V. Tikhonov wrote:
> Switched OSX 10.14 test jobs to MAC mini hosts.
> Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is
> used to pass all appropriate target calls to Travis make rule target.
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
I've checked your patch into 1.10, 2.2, 2.3 and master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
@ 2020-03-06 12:45 Alexander V. Tikhonov
0 siblings, 0 replies; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-03-06 12:45 UTC (permalink / raw)
To: Oleg Piskunov; +Cc: tarantool-patches
Added new gitlab-ci test jobs for OSX 10.14 testings.
Set in use Python2 tapped version with its latest commit in it,
since Python 2 is EOL.
Set 'var' directory for test-run tool to the shorter path name
to avoid of issues with long names.
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
.gitlab-ci.yml | 19 +--
.gitlab.mk | 2 +-
.travis.mk | 34 +++-
tools/brew_taps/tntpython2.rb | 306 ++++++++++++++++++++++++++++++++++
4 files changed, 340 insertions(+), 21 deletions(-)
create mode 100644 tools/brew_taps/tntpython2.rb
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5177a9903..47e4af5a2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -152,28 +152,21 @@ osx_13_release:
- ${GITLAB_MAKE} vms_test_osx
osx_14_release:
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
- variables:
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
+ - osx_14
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
osx_14_release_lto:
<<: *release_only_definition
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
+ - osx_14
variables:
EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
freebsd_12_release:
<<: *vbox_definition
diff --git a/.gitlab.mk b/.gitlab.mk
index b39c5c651..d804d3cf6 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -14,7 +14,7 @@ git_submodule_update:
git submodule update --recursive --init
# Pass *_no_deps goals to .travis.mk.
-test_%_no_deps: git_submodule_update
+test_%: git_submodule_update
${TRAVIS_MAKE} $@
# #######################################################
diff --git a/.travis.mk b/.travis.mk
index 42969ff56..fcd24fa67 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -5,6 +5,7 @@
DOCKER_IMAGE?=packpack/packpack:debian-stretch
TEST_RUN_EXTRA_PARAMS?=
MAX_FILES?=65534
+MAX_PROC?=2500
all: package
@@ -127,13 +128,27 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
# OSX #
#######
+# Python2 is EOF and we need to be sure that it is in use instead of Python3
+OSX_PKGS=openssl readline curl icu4c libiconv zlib autoconf automake libtool cmake \
+ file://$${PWD}/tools/brew_taps/tntpython2.rb
+
deps_osx:
- brew update
- brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
- python2 -V || brew install python2 --force
+ # since Python 2 is EOL its tapped version
+ # with its latest commit will be used:
+ brew tap avtikhon/tnt_taps https://github.com/avtikhon/tnt_taps.git
+ # install brew using command from Homebrew repository instructions:
+ # https://github.com/Homebrew/install
+ # NOTE: 'echo' command below is required since brew installation
+ # script obliges the one to enter a newline for confirming the
+ # installation via Ruby script.
+ brew update || echo | /usr/bin/ruby -e \
+ "$$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+ # try to install the packages either upgrade it to avoid of fails
+ # if the package already exists with the previous version
+ brew install --force ${OSX_PKGS} || brew upgrade ${OSX_PKGS}
curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
- python get-pip.py --user
- pip install --user --force-reinstall -r test-run/requirements.txt
+ python get-pip.py
+ pip install --force-reinstall -r test-run/requirements.txt
build_osx:
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
@@ -148,11 +163,16 @@ test_osx_no_deps: build_osx
# call as tests runs call.
# Tests: Temporary excluded replication/ suite with some tests
# from other suites by issues #4357 and #4370
- echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
+ sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
launchctl limit maxfiles || : ; \
ulimit -n ${MAX_FILES} || : ; \
ulimit -n ; \
- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+ sudo -S launchctl limit maxproc ${MAX_PROC} || : ; \
+ launchctl limit maxproc || : ; \
+ ulimit -u ${MAX_PROC} || : ; \
+ ulimit -u ; \
+ rm -rf /tmp/tnt ; \
+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
diff --git a/tools/brew_taps/tntpython2.rb b/tools/brew_taps/tntpython2.rb
new file mode 100644
index 000000000..59f374030
--- /dev/null
+++ b/tools/brew_taps/tntpython2.rb
@@ -0,0 +1,306 @@
+class Tntpython2 < Formula
+ desc "Interpreted, interactive, object-oriented programming language"
+ homepage "https://www.python.org/"
+ url "https://www.python.org/ftp/python/2.7.17/Python-2.7.17.tar.xz"
+ sha256 "4d43f033cdbd0aa7b7023c81b0e986fd11e653b5248dac9144d508f11812ba41"
+ revision 1
+ head "https://github.com/python/cpython.git", :branch => "2.7"
+
+ bottle do
+ sha256 "accfaa922708f00afb69ab230199f96e6ecdddd248a1eca586ce1e5e5cfd732b" => :catalina
+ sha256 "54d3351d6be8268b2f5017894dcc8e083811dfa3812bdb9f79f989873b9a4542" => :mojave
+ sha256 "cfd5c6eeac37065d19f527bb0798a9caf1928bab3340cd545224861a3c82f219" => :high_sierra
+ end
+
+ # setuptools remembers the build flags python is built with and uses them to
+ # build packages later. Xcode-only systems need different flags.
+ pour_bottle? do
+ reason <<~EOS
+ The bottle needs the Apple Command Line Tools to be installed.
+ You can install them, if desired, with:
+ xcode-select --install
+ EOS
+ satisfy { MacOS::CLT.installed? }
+ end
+
+ depends_on "pkg-config" => :build
+ depends_on "gdbm"
+ depends_on "openssl@1.1"
+ depends_on "readline"
+ depends_on "sqlite"
+
+ resource "setuptools" do
+ url "https://files.pythonhosted.org/packages/f4/d5/a6c19dcbcbc267aca376558797f036d9bcdff344c9f785fe7d0fe9a5f2a7/setuptools-41.4.0.zip"
+ sha256 "7eae782ccf36b790c21bde7d86a4f303a441cd77036b25c559a602cf5186ce4d"
+ end
+
+ resource "pip" do
+ url "https://files.pythonhosted.org/packages/ce/ea/9b445176a65ae4ba22dce1d93e4b5fe182f953df71a145f557cffaffc1bf/pip-19.3.1.tar.gz"
+ sha256 "21207d76c1031e517668898a6b46a9fb1501c7a4710ef5dfd6a40ad9e6757ea7"
+ end
+
+ resource "wheel" do
+ url "https://files.pythonhosted.org/packages/59/b0/11710a598e1e148fb7cbf9220fd2a0b82c98e94efbdecb299cb25e7f0b39/wheel-0.33.6.tar.gz"
+ sha256 "10c9da68765315ed98850f8e048347c3eb06dd81822dc2ab1d4fde9dc9702646"
+ end
+
+ def lib_cellar
+ prefix/"Frameworks/Python.framework/Versions/2.7/lib/python2.7"
+ end
+
+ def site_packages_cellar
+ lib_cellar/"site-packages"
+ end
+
+ # The HOMEBREW_PREFIX location of site-packages.
+ def site_packages
+ HOMEBREW_PREFIX/"lib/python2.7/site-packages"
+ end
+
+ def install
+ # Unset these so that installing pip and setuptools puts them where we want
+ # and not into some other Python the user has installed.
+ ENV["PYTHONHOME"] = nil
+ ENV["PYTHONPATH"] = nil
+
+ args = %W[
+ --prefix=#{prefix}
+ --enable-ipv6
+ --datarootdir=#{share}
+ --datadir=#{share}
+ --enable-framework=#{frameworks}
+ --without-ensurepip
+ ]
+
+ # See upstream bug report from 22 Jan 2018 "Significant performance problems
+ # with Python 2.7 built with clang 3.x or 4.x"
+ # https://bugs.python.org/issue32616
+ # https://github.com/Homebrew/homebrew-core/issues/22743
+ if DevelopmentTools.clang_build_version >= 802 &&
+ DevelopmentTools.clang_build_version < 902
+ args << "--without-computed-gotos"
+ end
+
+ args << "--without-gcc" if ENV.compiler == :clang
+
+ cflags = []
+ ldflags = []
+ cppflags = []
+
+ if MacOS.sdk_path_if_needed
+ # Help Python's build system (setuptools/pip) to build things on SDK-based systems
+ # The setup.py looks at "-isysroot" to get the sysroot (and not at --sysroot)
+ cflags << "-isysroot #{MacOS.sdk_path}" << "-I#{MacOS.sdk_path}/usr/include"
+ ldflags << "-isysroot #{MacOS.sdk_path}"
+ # For the Xlib.h, Python needs this header dir with the system Tk
+ # Yep, this needs the absolute path where zlib needed a path relative
+ # to the SDK.
+ cflags << "-I#{MacOS.sdk_path}/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers"
+ end
+
+ # Avoid linking to libgcc https://code.activestate.com/lists/python-dev/112195/
+ args << "MACOSX_DEPLOYMENT_TARGET=#{MacOS.version}"
+
+ # We want our readline and openssl! This is just to outsmart the detection code,
+ # superenv handles that cc finds includes/libs!
+ inreplace "setup.py" do |s|
+ s.gsub! "do_readline = self.compiler.find_library_file(lib_dirs, 'readline')",
+ "do_readline = '#{Formula["readline"].opt_lib}/libhistory.dylib'"
+ s.gsub! "/usr/local/ssl", Formula["openssl@1.1"].opt_prefix
+ end
+
+ inreplace "setup.py" do |s|
+ s.gsub! "sqlite_setup_debug = False", "sqlite_setup_debug = True"
+ s.gsub! "for d_ in inc_dirs + sqlite_inc_paths:",
+ "for d_ in ['#{Formula["sqlite"].opt_include}']:"
+
+ # Allow sqlite3 module to load extensions:
+ # https://docs.python.org/library/sqlite3.html#f1
+ s.gsub! 'sqlite_defines.append(("SQLITE_OMIT_LOAD_EXTENSION", "1"))', ""
+ end
+
+ # Allow python modules to use ctypes.find_library to find homebrew's stuff
+ # even if homebrew is not a /usr/local/lib. Try this with:
+ # `brew install enchant && pip install pyenchant`
+ inreplace "./Lib/ctypes/macholib/dyld.py" do |f|
+ f.gsub! "DEFAULT_LIBRARY_FALLBACK = [", "DEFAULT_LIBRARY_FALLBACK = [ '#{HOMEBREW_PREFIX}/lib',"
+ f.gsub! "DEFAULT_FRAMEWORK_FALLBACK = [", "DEFAULT_FRAMEWORK_FALLBACK = [ '#{HOMEBREW_PREFIX}/Frameworks',"
+ end
+
+ args << "CFLAGS=#{cflags.join(" ")}" unless cflags.empty?
+ args << "LDFLAGS=#{ldflags.join(" ")}" unless ldflags.empty?
+ args << "CPPFLAGS=#{cppflags.join(" ")}" unless cppflags.empty?
+
+ system "./configure", *args
+ system "make"
+
+ ENV.deparallelize do
+ # Tell Python not to install into /Applications
+ system "make", "install", "PYTHONAPPSDIR=#{prefix}"
+ system "make", "frameworkinstallextras", "PYTHONAPPSDIR=#{pkgshare}"
+ end
+
+ # Fixes setting Python build flags for certain software
+ # See: https://github.com/Homebrew/homebrew/pull/20182
+ # https://bugs.python.org/issue3588
+ inreplace lib_cellar/"config/Makefile" do |s|
+ s.change_make_var! "LINKFORSHARED",
+ "-u _PyMac_Error $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)"
+ end
+
+ # Prevent third-party packages from building against fragile Cellar paths
+ inreplace [lib_cellar/"_sysconfigdata.py",
+ lib_cellar/"config/Makefile",
+ frameworks/"Python.framework/Versions/Current/lib/pkgconfig/python-2.7.pc"],
+ prefix, opt_prefix
+
+ # Symlink the pkgconfig files into HOMEBREW_PREFIX so they're accessible.
+ (lib/"pkgconfig").install_symlink Dir[frameworks/"Python.framework/Versions/Current/lib/pkgconfig/*"]
+
+ # Remove 2to3 because Python 3 also installs it
+ rm bin/"2to3"
+
+ # A fix, because python and python@2 both want to install Python.framework
+ # and therefore we can't link both into HOMEBREW_PREFIX/Frameworks
+ # https://github.com/Homebrew/homebrew/issues/15943
+ ["Headers", "Python", "Resources"].each { |f| rm(prefix/"Frameworks/Python.framework/#{f}") }
+ rm prefix/"Frameworks/Python.framework/Versions/Current"
+
+ # Remove the site-packages that Python created in its Cellar.
+ site_packages_cellar.rmtree
+
+ (libexec/"setuptools").install resource("setuptools")
+ (libexec/"pip").install resource("pip")
+ (libexec/"wheel").install resource("wheel")
+ end
+
+ def post_install
+ # Avoid conflicts with lingering unversioned files from Python 3
+ rm_f %W[
+ #{HOMEBREW_PREFIX}/bin/easy_install
+ #{HOMEBREW_PREFIX}/bin/pip
+ #{HOMEBREW_PREFIX}/bin/wheel
+ ]
+
+ # Fix up the site-packages so that user-installed Python software survives
+ # minor updates, such as going from 2.7.0 to 2.7.1:
+
+ # Create a site-packages in HOMEBREW_PREFIX/lib/python2.7/site-packages
+ site_packages.mkpath
+
+ # Symlink the prefix site-packages into the cellar.
+ site_packages_cellar.unlink if site_packages_cellar.exist?
+ site_packages_cellar.parent.install_symlink site_packages
+
+ # Write our sitecustomize.py
+ rm_rf Dir["#{site_packages}/sitecustomize.py[co]"]
+ (site_packages/"sitecustomize.py").atomic_write(sitecustomize)
+
+ # Remove old setuptools installations that may still fly around and be
+ # listed in the easy_install.pth. This can break setuptools build with
+ # zipimport.ZipImportError: bad local file header
+ # setuptools-0.9.5-py3.3.egg
+ rm_rf Dir["#{site_packages}/setuptools*"]
+ rm_rf Dir["#{site_packages}/distribute*"]
+ rm_rf Dir["#{site_packages}/pip[-_.][0-9]*", "#{site_packages}/pip"]
+
+ setup_args = ["-s", "setup.py", "--no-user-cfg", "install", "--force",
+ "--verbose",
+ "--single-version-externally-managed",
+ "--record=installed.txt",
+ "--install-scripts=#{bin}",
+ "--install-lib=#{site_packages}"]
+
+ (libexec/"setuptools").cd { system "#{bin}/python", *setup_args }
+ (libexec/"pip").cd { system "#{bin}/python", *setup_args }
+ (libexec/"wheel").cd { system "#{bin}/python", *setup_args }
+
+ # When building from source, these symlinks will not exist, since
+ # post_install happens after linking.
+ %w[pip pip2 pip2.7 easy_install easy_install-2.7 wheel].each do |e|
+ (HOMEBREW_PREFIX/"bin").install_symlink bin/e
+ end
+
+ # Help distutils find brewed stuff when building extensions
+ include_dirs = [HOMEBREW_PREFIX/"include", Formula["openssl@1.1"].opt_include,
+ Formula["sqlite"].opt_include]
+ library_dirs = [HOMEBREW_PREFIX/"lib", Formula["openssl@1.1"].opt_lib,
+ Formula["sqlite"].opt_lib]
+
+ cfg = lib_cellar/"distutils/distutils.cfg"
+ cfg.atomic_write <<~EOS
+ [install]
+ prefix=#{HOMEBREW_PREFIX}
+ [build_ext]
+ include_dirs=#{include_dirs.join ":"}
+ library_dirs=#{library_dirs.join ":"}
+ EOS
+ end
+
+ def sitecustomize
+ <<~EOS
+ # This file is created by Homebrew and is executed on each python startup.
+ # Don't print from here, or else python command line scripts may fail!
+ # <https://docs.brew.sh/Homebrew-and-Python>
+ import re
+ import os
+ import sys
+ if sys.version_info[0] != 2:
+ # This can only happen if the user has set the PYTHONPATH for 3.x and run Python 2.x or vice versa.
+ # Every Python looks at the PYTHONPATH variable and we can't fix it here in sitecustomize.py,
+ # because the PYTHONPATH is evaluated after the sitecustomize.py. Many modules (e.g. PyQt4) are
+ # built only for a specific version of Python and will fail with cryptic error messages.
+ # In the end this means: Don't set the PYTHONPATH permanently if you use different Python versions.
+ exit('Your PYTHONPATH points to a site-packages dir for Python 2.x but you are running Python ' +
+ str(sys.version_info[0]) + '.x!\\n PYTHONPATH is currently: "' + str(os.environ['PYTHONPATH']) + '"\\n' +
+ ' You should `unset PYTHONPATH` to fix this.')
+ # Only do this for a brewed python:
+ if os.path.realpath(sys.executable).startswith('#{rack}'):
+ # Shuffle /Library site-packages to the end of sys.path and reject
+ # paths in /System pre-emptively (#14712)
+ library_site = '/Library/Python/2.7/site-packages'
+ library_packages = [p for p in sys.path if p.startswith(library_site)]
+ sys.path = [p for p in sys.path if not p.startswith(library_site) and
+ not p.startswith('/System')]
+ # .pth files have already been processed so don't use addsitedir
+ sys.path.extend(library_packages)
+ # the Cellar site-packages is a symlink to the HOMEBREW_PREFIX
+ # site_packages; prefer the shorter paths
+ long_prefix = re.compile(r'#{rack}/[0-9\._abrc]+/Frameworks/Python\.framework/Versions/2\.7/lib/python2\.7/site-packages')
+ sys.path = [long_prefix.sub('#{site_packages}', p) for p in sys.path]
+ # LINKFORSHARED (and python-config --ldflags) return the
+ # full path to the lib (yes, "Python" is actually the lib, not a
+ # dir) so that third-party software does not need to add the
+ # -F/#{HOMEBREW_PREFIX}/Frameworks switch.
+ try:
+ from _sysconfigdata import build_time_vars
+ build_time_vars['LINKFORSHARED'] = '-u _PyMac_Error #{opt_prefix}/Frameworks/Python.framework/Versions/2.7/Python'
+ except:
+ pass # remember: don't print here. Better to fail silently.
+ # Set the sys.executable to use the opt_prefix
+ sys.executable = '#{opt_bin}/python2.7'
+ EOS
+ end
+
+ def caveats; <<~EOS
+ Pip and setuptools have been installed. To update them
+ pip install --upgrade pip setuptools
+ You can install Python packages with
+ pip install <package>
+ They will install into the site-package directory
+ #{site_packages}
+ See: https://docs.brew.sh/Homebrew-and-Python
+ EOS
+ end
+
+ test do
+ # Check if sqlite is ok, because we build with --enable-loadable-sqlite-extensions
+ # and it can occur that building sqlite silently fails if OSX's sqlite is used.
+ system "#{bin}/python", "-c", "import sqlite3"
+ # Check if some other modules import. Then the linked libs are working.
+ system "#{bin}/python", "-c", "import Tkinter; root = Tkinter.Tk()"
+ system "#{bin}/python", "-c", "import gdbm"
+ system "#{bin}/python", "-c", "import zlib"
+ system bin/"pip", "list", "--format=columns"
+ end
+end
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini
@ 2020-02-27 7:47 Alexander V. Tikhonov
0 siblings, 0 replies; 6+ messages in thread
From: Alexander V. Tikhonov @ 2020-02-27 7:47 UTC (permalink / raw)
To: Oleg Piskunov; +Cc: tarantool-patches
Added mac mini hosts into testing and set to it all
OSX 10.14 test jobs with new osx_14 gitlab-runner tag.
Also made additional setup:
- added PATHs to python2 and pip
- set 'var' directory for test-run tool to the shorter
path name to avoid of issues with long names.
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_on_mini14-full-ci
.gitlab-ci.yml | 19 ++++++-------------
.gitlab.mk | 2 +-
.travis.mk | 14 +++++++++-----
3 files changed, 16 insertions(+), 19 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5177a9903..47e4af5a2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -152,28 +152,21 @@ osx_13_release:
- ${GITLAB_MAKE} vms_test_osx
osx_14_release:
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
- variables:
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
+ - osx_14
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
osx_14_release_lto:
<<: *release_only_definition
- <<: *vbox_definition
+ <<: *docker_test_definition
tags:
- - vms_osx_14
+ - osx_14
variables:
EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"
- VMS_NAME: 'osx_14'
- VMS_USER: 'tarantool'
- VMS_PORT: '2222'
script:
- - ${GITLAB_MAKE} vms_test_osx
+ - ${GITLAB_MAKE} test_osx
freebsd_12_release:
<<: *vbox_definition
diff --git a/.gitlab.mk b/.gitlab.mk
index b39c5c651..d804d3cf6 100644
--- a/.gitlab.mk
+++ b/.gitlab.mk
@@ -14,7 +14,7 @@ git_submodule_update:
git submodule update --recursive --init
# Pass *_no_deps goals to .travis.mk.
-test_%_no_deps: git_submodule_update
+test_%: git_submodule_update
${TRAVIS_MAKE} $@
# #######################################################
diff --git a/.travis.mk b/.travis.mk
index 42969ff56..fca86b569 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -130,10 +130,12 @@ test_asan_debian: deps_debian deps_buster_clang_8 test_asan_debian_no_deps
deps_osx:
brew update
brew install openssl readline curl icu4c libiconv zlib autoconf automake libtool --force
- python2 -V || brew install python2 --force
- curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
- python get-pip.py --user
- pip install --user --force-reinstall -r test-run/requirements.txt
+ export PATH=/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH} ; \
+ export PATH=${PATH}:/Users/${USER}/Library/Python/2.7/bin ; \
+ python2 -V || brew install python2 --force ; \
+ curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py ; \
+ python get-pip.py --user ; \
+ pip install --user --force-reinstall -r test-run/requirements.txt
build_osx:
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
@@ -152,7 +154,9 @@ test_osx_no_deps: build_osx
launchctl limit maxfiles || : ; \
ulimit -n ${MAX_FILES} || : ; \
ulimit -n ; \
- cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+ export PATH=/System/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH} ; \
+ rm -rf /tmp/tnt ; \
+ cd test && ./test-run.py --vardir /tmp/tnt --force $(TEST_RUN_EXTRA_PARAMS) \
app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-04-02 7:43 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-27 7:29 [Tarantool-patches] [PATCH v1] gitlab-ci: implement OSX 10.14 testing on mac mini Alexander V. Tikhonov
2020-03-31 5:34 ` Oleg Piskunov
2020-03-31 9:10 ` Sergey Bronnikov
2020-04-02 7:43 ` Kirill Yukhin
-- strict thread matches above, loose matches on Subject: below --
2020-03-06 12:45 Alexander V. Tikhonov
2020-02-27 7:47 Alexander V. Tikhonov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox