* [tarantool-patches] Re: [PATCH v1] travis-ci: fix OSX max files limits
[not found] <34290551efba4c2287e6ffdaea5caf6aea7539b5.1563868186.git.avtikhon@tarantool.org>
@ 2019-07-24 9:37 ` Alexander Turenko
[not found] ` <1563995566.727319504@f118.i.mail.ru>
0 siblings, 1 reply; 2+ messages in thread
From: Alexander Turenko @ 2019-07-24 9:37 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
To be honest, I know nothing about launchctl and rlimits on Mac OS. So I
cannot give 'ok' or 'not ok' here. Please, consider some common comments
below.
WBR, Alexander Turenko.
On Tue, Jul 23, 2019 at 10:51:09AM +0300, Alexander V. Tikhonov wrote:
> Increased the maximum number of open file descriptors on macOS:
> Travis-ci needed the "ulimit -n <value>" call
> Gitlab-ci needed the "launchctl limit maxfiles <value>" call
> Also gitlib-ci needed the password to change the limits, while
> travis-ci run under root user. Limit setup set in the same
> call as tests runs call.
Are you know what is a reason of the difference between travis-ci and
gitlab-ci? I don't insist, but if it is known, it worth to mention it.
> Also found that the travis-ci global environment values schema
> had the mistake in definition - fixed with added "global:" tag
> and added the missed travis-ci environment value to the docker
> call.
I would extract it into separate commit.
>
> Closes #4373
> ---
>
> Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4373-travis-osx-files
> Issue: https://github.com/tarantool/tarantool/issues/4373
>
> .travis.mk | 30 ++++++++++++++----------------
> .travis.yml | 5 +++--
> 2 files changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/.travis.mk b/.travis.mk
> index 7d349bc21..f6c5ecbd0 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -34,6 +34,7 @@ docker_%:
> -e TRAVIS_JOB_ID=${TRAVIS_JOB_ID} \
> -e CMAKE_EXTRA_PARAMS=${CMAKE_EXTRA_PARAMS} \
> -e APT_EXTRA_FLAGS="${APT_EXTRA_FLAGS}" \
> + -e TEST_RUN_EXTRA_PARAMS="${TEST_RUN_EXTRA_PARAMS}" \
> -e CC=${CC} \
> -e CXX=${CXX} \
> ${DOCKER_IMAGE} \
> @@ -134,22 +135,19 @@ build_osx:
> make -j
>
> test_osx_no_deps: build_osx
> - # Increase the maximum number of open file descriptors on macOS
> - echo tarantool | sudo -S echo "Initialize sudo for the next commands..." :
> - sudo sysctl -w kern.maxfiles=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfiles=${MAX_FILES} || :
> - sysctl -w kern.maxfiles || :
> - sudo sysctl -w kern.maxfilesperproc=${MAX_FILES} || echo tarantool | sudo -S sysctl -w kern.maxfilesperproc=${MAX_FILES} || :
> - sysctl -w kern.maxfilesperproc || :
> - sudo launchctl limit maxfiles ${MAX_FILES} || echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || :
> - launchctl limit maxfiles || :
> - sudo ulimit -S -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -S -n ${MAX_FILES} || :
> - ulimit -S -n || :
> - sudo ulimit -H -n ${MAX_FILES} || echo tarantool | sudo -S ulimit -H -n ${MAX_FILES} || :
> - ulimit -H -n
> - # temporary excluded replication/ suite with some tests from other suites by issues #4357 and #4370
> - cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
> - app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
> - replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
> + # Limits: Increase the maximum number of open file descriptors on macOS:
> + # Travis-ci needs the "ulimit -n <value>" call
> + # Gitlab-ci needs the "launchctl limit maxfiles <value>" call
> + # Also gitlib-ci needs the password to change the limits, while
> + # travis-ci runs under root user. Limit setup must be in the same
> + # call as tests runs call.
> + # Tests: Temporary excluded replication/ suite with some tests from other suites by issues #4357 and #4370
> + sudo launchctl limit maxfiles ${MAX_FILES} || echo tarantool | sudo -S launchctl limit maxfiles ${MAX_FILES} || true ; \
> + launchctl limit maxfiles || true ; \
> + ulimit -n ${MAX_FILES} || echo tarantool | sudo -S ulimit ${MAX_FILES} || true ; ulimit -n ; \
> + cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) \
> + app/ app-tap/ box/ box-py/ box-tap/ engine/ engine_long/ long_run-py/ luajit-tap/ \
> + replication-py/ small/ sql/ sql-tap/ swim/ unit/ vinyl/ wal_off/ xlog/ xlog-py/
rlimit (it is the thing that ulimit command get/set) is a process
property (which is inherited by its childs). When you do `sudo ulimit -n
${MAX_FILES}` it runs ulimit under childs process and it does not alter
parent's resource limits. ulimit is the bash builtin, not an external
command: it changes resource limits for processes that bash starts.
Also I dislike `echo password | sudo <...>` construction, because root
password in our images is more our internal infrastructure thing.
>
> test_osx: deps_osx test_osx_no_deps
>
> diff --git a/.travis.yml b/.travis.yml
> index 61c887296..21a69d15a 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -17,8 +17,9 @@ cache:
> git:
> depth: 100500
>
> -env: >
> - TEST_RUN_EXTRA_PARAMS="-j 1"
> +env:
> + global:
> + - TEST_RUN_EXTRA_PARAMS="-j 1"
>
> jobs:
> include:
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 2+ messages in thread