[Tarantool-patches] [PATCH v1 0/3] test: set complete OSX testing on 1.10

[Tarantool-patches] [PATCH v1 0/3] test: set complete OSX testing on 1.10

[Alexander V. Tikhonov]

Cherry pick from master branch OSX testing

Cherry picked 2 commits from tarantool master branch:
- "travis-ci: fix OSX max files limits"
  (cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
- "test: need to run all test suites on OSX"
  (cherry picked from commit 73319ae8bfcdabda01277d085150ad08c21b7c6d)

and 1 commit from third party small:
- "Test quota hangs on OSX"
  (cherry picked from commit 50cb7874380b286f3b34c82299cf5eb88b0d0059)

To test OSX completely as in master branch.

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_1.10

Alexander V. Tikhonov (3):
  Bump libsmall version
  test: need to run all test suites on OSX
  travis-ci: fix OSX max files limits

 .travis.mk                        | 23 ++++++++++++++++-------
 src/lib/small                     |  2 +-
 test/engine/replica_join.skipcond |  7 +++++++
 3 files changed, 24 insertions(+), 8 deletions(-)
 create mode 100644 test/engine/replica_join.skipcond

-- 
2.17.1

[Tarantool-patches] [PATCH v1 1/3] Bump libsmall version

[Alexander V. Tikhonov]

---
 src/lib/small | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/lib/small b/src/lib/small
index cdf7d4a8e..086f51f0c 160000
--- a/src/lib/small
+++ b/src/lib/small
@@ -1 +1 @@
-Subproject commit cdf7d4a8e24ae465298d0ddacaf1aaed1085281e
+Subproject commit 086f51f0cc7cb3bf3f7879257bf9df296b5995f4
-- 
2.17.1

[Tarantool-patches] [PATCH v1 2/3] test: need to run all test suites on OSX

[Alexander V. Tikhonov]

Enabled complete testing for OSX, temporary skipped the
failed suite replication/ and tests engine/replica_join
with issue #4370 and small/quota from repository
https://github.com/tarantool/small.git with issue #4357
to enable it back. Also fixed the OSX ulimit setup
different for VBOX and Travis-ci images.

Closes #4358

(cherry picked from commit 73319ae8bfcdabda01277d085150ad08c21b7c6d)
---
 .travis.mk                        | 22 ++++++++++++++++------
 test/engine/replica_join.skipcond |  7 +++++++
 2 files changed, 23 insertions(+), 6 deletions(-)
 create mode 100644 test/engine/replica_join.skipcond

diff --git a/.travis.mk b/.travis.mk
index bfb3016da..e5112ef30 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -4,6 +4,7 @@
 
 DOCKER_IMAGE?=packpack/packpack:debian-stretch
 TEST_RUN_EXTRA_PARAMS?=
+MAX_FILES?=65534
 
 all: package
 
@@ -115,12 +116,21 @@ build_osx:
 
 test_osx_no_deps: build_osx
 	# Increase the maximum number of open file descriptors on macOS
-	sudo sysctl -w kern.maxfiles=20480 || :
-	sudo sysctl -w kern.maxfilesperproc=20480 || :
-	sudo launchctl limit maxfiles 20480 || :
-	ulimit -S -n 20480 || :
-	ulimit -n
-	cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) unit/ app/ app-tap/ box/ box-tap/
+	echo tarantool | sudo -S echo "Initialize sudo for the next commands..." :
+	sudo sysctl -w kern.maxfiles=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfiles=${MAX_FILES} || :
+	sysctl -w kern.maxfiles || :
+	sudo sysctl -w kern.maxfilesperproc=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfilesperproc=${MAX_FILES} || :
+	sysctl -w kern.maxfilesperproc || :
+	sudo launchctl limit maxfiles ${MAX_FILES} || echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || :
+	launchctl limit maxfiles || :
+	sudo ulimit -S -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -S -n ${MAX_FILES} || :
+	ulimit -S -n || :
+	sudo ulimit -H -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -H -n ${MAX_FILES} || :
+	ulimit -H -n
+	# temporary excluded replication/ suite with some tests from other suites by issues #4357 and #4370
+	cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+		app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
+		replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
 
 test_osx: deps_osx test_osx_no_deps
 
diff --git a/test/engine/replica_join.skipcond b/test/engine/replica_join.skipcond
new file mode 100644
index 000000000..56f938312
--- /dev/null
+++ b/test/engine/replica_join.skipcond
@@ -0,0 +1,7 @@
+import platform
+
+# Disabled on OSX due to fail #4370.
+if platform.system() == 'Darwin':
+    self.skip = 1
+
+# vim: set ft=python:
-- 
2.17.1

[Tarantool-patches] [PATCH v1 3/3] travis-ci: fix OSX max files limits

[Alexander V. Tikhonov]

Increased the maximum number of open file descriptors on macOS.

Travis-ci: needed the "ulimit -n <value>" call, because found
that the tests fail without it even with launchctl limit tool set.

Gitlab-ci: needed the "launchctl limit maxfiles <value>" call,
because under gitlib-ci it is needed the password to change the
limits and we call the sudo tool which separates the local set
of the environment.

Closes #4373

(cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
---
 .travis.mk | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/.travis.mk b/.travis.mk
index e5112ef30..98600bae2 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -115,22 +115,21 @@ build_osx:
 	make -j
 
 test_osx_no_deps: build_osx
-	# Increase the maximum number of open file descriptors on macOS
-	echo tarantool | sudo -S echo "Initialize sudo for the next commands..." :
-	sudo sysctl -w kern.maxfiles=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfiles=${MAX_FILES} || :
-	sysctl -w kern.maxfiles || :
-	sudo sysctl -w kern.maxfilesperproc=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfilesperproc=${MAX_FILES} || :
-	sysctl -w kern.maxfilesperproc || :
-	sudo launchctl limit maxfiles ${MAX_FILES} || echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || :
-	launchctl limit maxfiles || :
-	sudo ulimit -S -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -S -n ${MAX_FILES} || :
-	ulimit -S -n || :
-	sudo ulimit -H -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -H -n ${MAX_FILES} || :
-	ulimit -H -n
-	# temporary excluded replication/ suite with some tests from other suites by issues #4357 and #4370
-	cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
-		app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
-		replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
+	# Limits: Increase the maximum number of open file descriptors on macOS:
+	#   Travis-ci needs the "ulimit -n <value>" call
+	#   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
+	# Also gitlib-ci needs the password to change the limits, while
+	# travis-ci runs under root user. Limit setup must be in the same
+	# call as tests runs call.
+	# Tests: Temporary excluded replication/ suite with some tests
+	#        from other suites by issues #4357 and #4370
+	echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
+		launchctl limit maxfiles || : ; \
+		ulimit -n ${MAX_FILES} || : ; \
+		ulimit -n ; \
+		cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+			app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
+			replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
 
 test_osx: deps_osx test_osx_no_deps
 
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v1 0/3] test: set complete OSX testing on 1.10

[Igor Munkin]

Sasha,

Thanks for the patchset, all three patches LGTM. Feel free to proceed.

On 06.11.19, Alexander V. Tikhonov wrote:
> Cherry pick from master branch OSX testing
> 
> Cherry picked 2 commits from tarantool master branch:
> - "travis-ci: fix OSX max files limits"
>   (cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
> - "test: need to run all test suites on OSX"
>   (cherry picked from commit 73319ae8bfcdabda01277d085150ad08c21b7c6d)
> 
> and 1 commit from third party small:
> - "Test quota hangs on OSX"
>   (cherry picked from commit 50cb7874380b286f3b34c82299cf5eb88b0d0059)
> 
> To test OSX completely as in master branch.
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_1.10
> 
> Alexander V. Tikhonov (3):
>   Bump libsmall version
>   test: need to run all test suites on OSX
>   travis-ci: fix OSX max files limits
> 
>  .travis.mk                        | 23 ++++++++++++++++-------
>  src/lib/small                     |  2 +-
>  test/engine/replica_join.skipcond |  7 +++++++
>  3 files changed, 24 insertions(+), 8 deletions(-)
>  create mode 100644 test/engine/replica_join.skipcond
> 
> -- 
> 2.17.1
> 

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH v1 0/3] test: set complete OSX testing on 1.10

[Kirill Yukhin]

Hello,

On 06 ноя 15:57, Alexander V. Tikhonov wrote:
> Cherry pick from master branch OSX testing
> 
> Cherry picked 2 commits from tarantool master branch:
> - "travis-ci: fix OSX max files limits"
>   (cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
> - "test: need to run all test suites on OSX"
>   (cherry picked from commit 73319ae8bfcdabda01277d085150ad08c21b7c6d)
> 
> and 1 commit from third party small:
> - "Test quota hangs on OSX"
>   (cherry picked from commit 50cb7874380b286f3b34c82299cf5eb88b0d0059)
> 
> To test OSX completely as in master branch.
> 
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_1.10

I've checked your patch set into 1.10.

--
Regards, Kirill Yukhin

[Tarantool-patches] [PATCH v1 0/3] test: set complete OSX testing on 1.10

[Alexander V. Tikhonov]

Cherry pick from master branch OSX testing

Cherry picked 2 commits from tarantool master branch:
- "travis-ci: fix OSX max files limits"
  (cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
- "test: need to run all test suites on OSX"
  (cherry picked from commit 73319ae8bfcdabda01277d085150ad08c21b7c6d)

and 1 commit from third party small:
- "Test quota hangs on OSX"
  (cherry picked from commit 50cb7874380b286f3b34c82299cf5eb88b0d0059)

To test OSX completely as in master branch.

Github: https://github.com/tarantool/tarantool/tree/avtikhon/osx_1.10

Alexander V. Tikhonov (3):
  Bump libsmall version
  test: need to run all test suites on OSX
  travis-ci: fix OSX max files limits

 .travis.mk                        | 23 ++++++++++++++++-------
 src/lib/small                     |  2 +-
 test/engine/replica_join.skipcond |  7 +++++++
 3 files changed, 24 insertions(+), 8 deletions(-)
 create mode 100644 test/engine/replica_join.skipcond

-- 
2.17.1

[Tarantool-patches] [PATCH v1 3/3] travis-ci: fix OSX max files limits

[Alexander V. Tikhonov]

Increased the maximum number of open file descriptors on macOS.

Travis-ci: needed the "ulimit -n <value>" call, because found
that the tests fail without it even with launchctl limit tool set.

Gitlab-ci: needed the "launchctl limit maxfiles <value>" call,
because under gitlib-ci it is needed the password to change the
limits and we call the sudo tool which separates the local set
of the environment.

Closes #4373

(cherry picked from commit 027fa1a0699dc3b72d2f613bc32ba098c84938cf)
---
 .travis.mk | 31 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/.travis.mk b/.travis.mk
index e5112ef30..98600bae2 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -115,22 +115,21 @@ build_osx:
 	make -j
 
 test_osx_no_deps: build_osx
-	# Increase the maximum number of open file descriptors on macOS
-	echo tarantool | sudo -S echo "Initialize sudo for the next commands..." :
-	sudo sysctl -w kern.maxfiles=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfiles=${MAX_FILES} || :
-	sysctl -w kern.maxfiles || :
-	sudo sysctl -w kern.maxfilesperproc=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfilesperproc=${MAX_FILES} || :
-	sysctl -w kern.maxfilesperproc || :
-	sudo launchctl limit maxfiles ${MAX_FILES} || echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || :
-	launchctl limit maxfiles || :
-	sudo ulimit -S -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -S -n ${MAX_FILES} || :
-	ulimit -S -n || :
-	sudo ulimit -H -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -H -n ${MAX_FILES} || :
-	ulimit -H -n
-	# temporary excluded replication/ suite with some tests from other suites by issues #4357 and #4370
-	cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
-		app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
-		replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
+	# Limits: Increase the maximum number of open file descriptors on macOS:
+	#   Travis-ci needs the "ulimit -n <value>" call
+	#   Gitlab-ci needs the "launchctl limit maxfiles <value>" call
+	# Also gitlib-ci needs the password to change the limits, while
+	# travis-ci runs under root user. Limit setup must be in the same
+	# call as tests runs call.
+	# Tests: Temporary excluded replication/ suite with some tests
+	#        from other suites by issues #4357 and #4370
+	echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || : ; \
+		launchctl limit maxfiles || : ; \
+		ulimit -n ${MAX_FILES} || : ; \
+		ulimit -n ; \
+		cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
+			app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
+			replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
 
 test_osx: deps_osx test_osx_no_deps
 
-- 
2.17.1