* [tarantool-patches] [PATCH v3 0/2] Implement Gitlab-ci testing process
@ 2019-06-19 6:47 Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 1/2] Temporary disabled on_shutdown.test.lua test Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 2/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
0 siblings, 2 replies; 3+ messages in thread
From: Alexander V. Tikhonov @ 2019-06-19 6:47 UTC (permalink / raw)
To: Alexander Turenko; +Cc: Alexander V. Tikhonov, tarantool-patches
Implement Gitlab-ci testing process
Implemented Gitlba-ci testing process additionaly to travis-ci
that is currently uses. New testing process was added to make
able to control the high load of the testing processes to avoid
of flaky fails on timouts, disks layouts and memory swapping.
Also the ability to use needed number of hosts for testing jobs
and control jobs per each host, currently jobs set to hosts:
sh2 - runs tests which need to be run in line where internal
testing suites may run its tests in parallel
sh3 - ready to run performance testings
sh4 - runs tests OSX and FreeBSD under Virtual Box tool
sh5 - runs jobs in parallel: bootstrap jobs which build docker
images to speedup testing and build deploy jobs
Fixes #4156
Alexander V. Tikhonov (2):
Temporary disabled on_shutdown.test.lua test
Implement Gitlab-ci testing process
.gitlab-ci.yml | 324 +++++++++++++++++++++++++++++++++++++++++++++
.gitlab.mk | 42 ++++++
.travis.mk | 107 +++++++++++----
.travis.yml | 3 +
test/box/suite.ini | 2 +-
5 files changed, 450 insertions(+), 28 deletions(-)
create mode 100644 .gitlab-ci.yml
create mode 100644 .gitlab.mk
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [tarantool-patches] [PATCH v3 1/2] Temporary disabled on_shutdown.test.lua test
2019-06-19 6:47 [tarantool-patches] [PATCH v3 0/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
@ 2019-06-19 6:47 ` Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 2/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
1 sibling, 0 replies; 3+ messages in thread
From: Alexander V. Tikhonov @ 2019-06-19 6:47 UTC (permalink / raw)
To: Alexander Turenko; +Cc: Alexander V. Tikhonov, tarantool-patches
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4156-gitlab-ci-testing
Issue: https://github.com/tarantool/tarantool/issues/4156
test/box/suite.ini | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/box/suite.ini b/test/box/suite.ini
index 9197e3c0f..beb925faa 100644
--- a/test/box/suite.ini
+++ b/test/box/suite.ini
@@ -2,7 +2,7 @@
core = tarantool
description = Database tests
script = box.lua
-disabled = rtree_errinj.test.lua tuple_bench.test.lua
+disabled = rtree_errinj.test.lua tuple_bench.test.lua on_shutdown.test.lua
release_disabled = errinj.test.lua errinj_index.test.lua rtree_errinj.test.lua upsert_errinj.test.lua iproto_stress.test.lua
lua_libs = lua/fifo.lua lua/utils.lua lua/bitset.lua lua/index_random_test.lua lua/push.lua lua/identifier.lua
use_unix_sockets = True
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [tarantool-patches] [PATCH v3 2/2] Implement Gitlab-ci testing process
2019-06-19 6:47 [tarantool-patches] [PATCH v3 0/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 1/2] Temporary disabled on_shutdown.test.lua test Alexander V. Tikhonov
@ 2019-06-19 6:47 ` Alexander V. Tikhonov
1 sibling, 0 replies; 3+ messages in thread
From: Alexander V. Tikhonov @ 2019-06-19 6:47 UTC (permalink / raw)
To: Alexander Turenko; +Cc: Alexander V. Tikhonov, tarantool-patches
Implemented Gitlba-ci testing process additionaly to travis-ci
that is currently uses. New testing process was added to make
able to control the high load of the testing processes to avoid
of flaky fails on timouts, disks layouts and memory swapping.
Also the ability to use needed number of hosts for testing jobs
and control jobs per each host, currently jobs set to hosts:
sh2 - runs tests which need to be run in line where internal
testing suites may run its tests in parallel
sh3 - ready to run performance testings
sh4 - runs tests OSX and FreeBSD under Virtual Box tool
sh5 - runs jobs in parallel: bootstrap jobs which build docker
images to speedup testing and build deploy jobs
Fixes #4156
---
Github: https://github.com/tarantool/tarantool/tree/avtikhon/gh-4156-gitlab-ci-testing
Issue: https://github.com/tarantool/tarantool/issues/4156
.gitlab-ci.yml | 324 +++++++++++++++++++++++++++++++++++++++++++++++++
.gitlab.mk | 42 +++++++
.travis.mk | 107 +++++++++++-----
.travis.yml | 3 +
4 files changed, 449 insertions(+), 27 deletions(-)
create mode 100644 .gitlab-ci.yml
create mode 100644 .gitlab.mk
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 000000000..35809719a
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,324 @@
+stages:
+ - bootstrap
+ - test
+ - deploy
+
+variables:
+ DEPS_VERSION: "master"
+ IMAGE_TEST: "${CI_REGISTRY}/${CI_PROJECT_PATH}/debian-stretch:${DEPS_VERSION}"
+ IMAGE_TEST_LTO: "${CI_REGISTRY}/${CI_PROJECT_PATH}/debian-buster:${DEPS_VERSION}"
+ DENABLE_LTO_ON: "-DENABLE_LTO=ON"
+ GITLAB_MAKE: "make -f .gitlab.mk"
+
+# Bootstraps
+
+debian-9:
+ stage: bootstrap
+ tags:
+ - bootstrap
+ variables:
+ IMAGE: ${IMAGE_TEST}
+ DOCKERFILE: "FROM packpack/packpack:debian-stretch\\nCOPY .travis.mk .\\nRUN make -f .travis.mk deps_debian"
+ script:
+ - ${GITLAB_MAKE} docker_bootstrap
+
+debian-10_lto:
+ only:
+ refs:
+ - master
+ - /.*/
+ stage: bootstrap
+ tags:
+ - bootstrap
+ variables:
+ IMAGE: ${IMAGE_TEST_LTO}
+ DOCKERFILE: "FROM packpack/packpack:debian-buster\\nCOPY .travis.mk .\\nRUN make -f .travis.mk deps_buster_clang_8"
+ script:
+ - ${GITLAB_MAKE} docker_bootstrap
+
+# Tests
+
+release:
+ image: ${IMAGE_TEST}
+ stage: test
+ tags:
+ - docker_test
+ script:
+ - ${GITLAB_MAKE} test_debian_no_deps
+
+debug:
+ image: ${IMAGE_TEST}
+ stage: test
+ tags:
+ - docker_test
+ script:
+ - ${GITLAB_MAKE} test_coverage_debian_no_deps
+
+release_clang:
+ image: ${IMAGE_TEST}
+ stage: test
+ tags:
+ - docker_test
+ variables:
+ CC: clang
+ CXX: clang++
+ script:
+ - ${GITLAB_MAKE} test_debian_no_deps
+
+release_lto:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: ${IMAGE_TEST_LTO}
+ stage: test
+ tags:
+ - docker_test
+ variables:
+ CMAKE_EXTRA_PARAMS: ${DENABLE_LTO_ON}
+ script:
+ - ${GITLAB_MAKE} test_debian_no_deps
+
+release_lto_clang8:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: ${IMAGE_TEST_LTO}
+ stage: test
+ tags:
+ - docker_test
+ variables:
+ CC: clang-8
+ CXX: clang++-8
+ CMAKE_EXTRA_PARAMS: ${DENABLE_LTO_ON}
+ script:
+ - ${GITLAB_MAKE} test_debian_no_deps
+
+osx_13_release:
+ only:
+ refs:
+ - master
+ - /.*/
+ stage: test
+ tags:
+ - vms_sierra
+ variables:
+ VMS_NAME: 'Sierra'
+ VMS_USER: 'tarantool'
+ VMS_PORT: '2212'
+ before_script:
+ - ${GITLAB_MAKE} vms_start
+ script:
+ - ${GITLAB_MAKE} vms_test_osx
+ after_script:
+ - ${GITLAB_MAKE} vms_shutdown
+
+osx_14_release:
+ stage: test
+ tags:
+ - vms_mojave
+ variables:
+ VMS_NAME: 'Mojave'
+ VMS_USER: 'tarantool'
+ VMS_PORT: '2222'
+ before_script:
+ - ${GITLAB_MAKE} vms_start
+ script:
+ - ${GITLAB_MAKE} vms_test_osx
+ after_script:
+ - ${GITLAB_MAKE} vms_shutdown
+
+osx_14_release_lto:
+ only:
+ refs:
+ - master
+ - /.*/
+ stage: test
+ tags:
+ - vms_mojave
+ variables:
+ EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=${DENABLE_LTO_ON} ;"
+ VMS_NAME: 'Mojave'
+ VMS_USER: 'tarantool'
+ VMS_PORT: '2222'
+ before_script:
+ - ${GITLAB_MAKE} vms_start
+ script:
+ - ${GITLAB_MAKE} vms_test_osx
+ after_script:
+ - ${GITLAB_MAKE} vms_shutdown
+
+freebsd_12_release:
+ stage: test
+ tags:
+ - vms_freebsd
+ variables:
+ VMS_NAME: 'Freebsd'
+ VMS_USER: 'vagrant'
+ VMS_PORT: '2232'
+ TRAVIS_MAKE: 'gmake -f .travis.mk'
+ before_script:
+ - ${GITLAB_MAKE} vms_start
+ script:
+ - ${GITLAB_MAKE} vms_test_freebsd
+ after_script:
+ - ${GITLAB_MAKE} vms_shutdown
+
+# Packs
+
+centos_6:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:el-6
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+centos_7:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:el-7
+ stage: deploy
+ tags:
+ - deploy_test
+ script:
+ - ${GITLAB_MAKE} package
+
+fedora_28:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:fedora-28
+ stage: deploy
+ tags:
+ - deploy_test
+ script:
+ - ${GITLAB_MAKE} package
+
+fedora_29:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:fedora-29
+ stage: deploy
+ tags:
+ - deploy_test
+ script:
+ - ${GITLAB_MAKE} package
+
+fedora_30:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:fedora-30
+ stage: deploy
+ tags:
+ - deploy_test
+ script:
+ - ${GITLAB_MAKE} package
+
+ubuntu_14_04:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:ubuntu-trusty
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+ubuntu_16_04:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:ubuntu-xenial
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+ubuntu_18_04:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:ubuntu-bionic
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+ubuntu_18_10:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:ubuntu-cosmic
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+ubuntu_19_04:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:ubuntu-disco
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+debian_8:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:debian-jessie
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+debian_9:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:debian-stretch
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
+
+debian_10:
+ only:
+ refs:
+ - master
+ - /.*/
+ image: packpack/packpack:debian-buster
+ stage: deploy
+ tags:
+ - deploy
+ script:
+ - ${GITLAB_MAKE} package
diff --git a/.gitlab.mk b/.gitlab.mk
new file mode 100644
index 000000000..f3e0cfad3
--- /dev/null
+++ b/.gitlab.mk
@@ -0,0 +1,42 @@
+GITLAB_MAKE?=make -f .gitlab.mk
+TRAVIS_MAKE?=make -f .travis.mk
+
+# docker images bootstrap
+docker_bootstrap:
+ docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} \
+ ${CI_REGISTRY}
+ echo -e "$(DOCKERFILE)" | docker build --network=host -t ${IMAGE} -f- .
+ docker push ${IMAGE}
+
+# build sources with different flags and test
+git_submodule_update:
+ git submodule update --force --recursive --init 2>/dev/null || \
+ git submodule update --recursive --init
+
+test_%_no_deps: git_submodule_update
+ ${TRAVIS_MAKE} $@
+
+# build sources with different flags and test under VBox virtual machines
+vms_start:
+ VBoxManage controlvm ${VMS_NAME} poweroff || true
+ VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME}
+ VBoxManage startvm ${VMS_NAME} --type headless
+
+vms_test_%:
+ ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \
+ '${EXTRA_ENV} \
+ export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin ; \
+ cd tarantool && \
+ git fetch -p && \
+ git checkout --force ${CI_BUILD_REF} && \
+ ${GITLAB_MAKE} git_submodule_update && \
+ ${TRAVIS_MAKE} $(subst vms_,,$@)'"
+
+vms_shutdown:
+ VBoxManage controlvm ${VMS_NAME} poweroff
+
+# build packs and test
+package: git_submodule_update
+ git clone https://github.com/packpack/packpack.git packpack
+ # to avoid of issue with too long names build path is short
+ make -f packpack/pack/Makefile -C . BUILDDIR=/builds/pack -j
diff --git a/.travis.mk b/.travis.mk
index 6d0c42207..dc3bdb5ab 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -3,6 +3,7 @@
#
DOCKER_IMAGE?=packpack/packpack:debian-stretch
+TEST_RUN_EXTRA_PARAMS?=
all: package
@@ -13,8 +14,8 @@ package:
test: test_$(TRAVIS_OS_NAME)
# Redirect some targets via docker
-test_linux: docker_test_ubuntu
-coverage: docker_coverage_ubuntu
+test_linux: docker_test_debian
+coverage: docker_coverage_debian
docker_%:
mkdir -p ~/.cache/ccache
@@ -33,43 +34,42 @@ docker_%:
${DOCKER_IMAGE} \
make -f .travis.mk $(subst docker_,,$@)
-deps_ubuntu:
- sudo apt-get update && apt-get install -y -f \
+#########
+# Linux #
+#########
+
+deps_debian:
+ apt-get -y update && apt-get install -y -f \
build-essential cmake coreutils sed \
libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
libcurl4-openssl-dev libunwind-dev libicu-dev \
python python-pip python-setuptools python-dev \
python-msgpack python-yaml python-argparse python-six python-gevent \
- lcov ruby clang llvm llvm-dev
+ lcov ruby clang llvm-dev
+
+deps_buster_clang_8: deps_debian
+ echo "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" > /etc/apt/sources.list.d/clang_8.list
+ echo "deb-src http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" >> /etc/apt/sources.list.d/clang_8.list
+ wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -
+ apt-get -y update
+ apt-get -y install clang-8 llvm-8-dev
-test_ubuntu: deps_ubuntu
+build_debian:
cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
- make -j8
- cd test && /usr/bin/python test-run.py --force -j 1
+ make -j
-deps_osx:
- brew update
- brew install openssl readline curl icu4c --force
- python2 -V || brew install python2 --force
- curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python
- pip install -r test-run/requirements.txt
+test_debian_no_deps: build_debian
+ cd test && /usr/bin/python test-run.py --force $(TEST_RUN_EXTRA_PARAMS)
-test_osx: deps_osx
- cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
- # Increase the maximum number of open file descriptors on macOS
- sudo sysctl -w kern.maxfiles=20480 || :
- sudo sysctl -w kern.maxfilesperproc=20480 || :
- sudo launchctl limit maxfiles 20480 || :
- ulimit -S -n 20480 || :
- ulimit -n
- make -j8
- cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/
+test_debian: deps_debian test_debian_no_deps
-coverage_ubuntu: deps_ubuntu
+build_coverage_debian:
cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON
- make -j8
+ make -j
+
+test_coverage_debian_no_deps: build_coverage_debian
# Enable --long tests for coverage
- cd test && /usr/bin/python test-run.py --force -j 1 --long
+ cd test && /usr/bin/python test-run.py --force $(TEST_RUN_EXTRA_PARAMS) --long
lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp
lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \
--output-file coverage.info
@@ -81,6 +81,59 @@ coverage_ubuntu: deps_ubuntu
coveralls-lcov --service-name travis-ci --service-job-id $(TRAVIS_JOB_ID) --repo-token $(COVERALLS_TOKEN) coverage.info; \
fi;
+coverage_debian: deps_debian test_coverage_debian_no_deps
+
+#######
+# OSX #
+#######
+
+deps_osx:
+ brew update
+ brew install openssl readline curl icu4c --force
+ python2 -V || brew install python2 --force
+ curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py >get-pip.py
+ python get-pip.py --user
+ pip install --user --ignore-installed -r test-run/requirements.txt
+
+build_osx:
+ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
+ make -j
+
+test_osx_no_deps: build_osx
+ # Increase the maximum number of open file descriptors on macOS
+ sudo sysctl -w kern.maxfiles=20480 || :
+ sudo sysctl -w kern.maxfilesperproc=20480 || :
+ sudo launchctl limit maxfiles 20480 || :
+ ulimit -S -n 20480 || :
+ ulimit -n
+ cd test && ./test-run.py --force $(TEST_RUN_EXTRA_PARAMS) unit/ app/ app-tap/ box/ box-tap/
+
+test_osx: deps_osx test_osx_no_deps
+
+###########
+# FreeBSD #
+###########
+
+deps_freebsd:
+ sudo pkg install -y git cmake gmake gcc coreutils \
+ readline ncurses libyaml openssl curl libunwind icu \
+ python27 py27-pip py27-setuptools py27-daemon \
+ py27-yaml py27-argparse py27-six py27-gevent \
+ gdb bash
+
+build_freebsd:
+ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
+ gmake -j
+
+test_freebsd_no_deps: build_freebsd
+ cd test && /usr/bin/python test-run.py --force $(TEST_RUN_EXTRA_PARAMS) || true
+
+test_freebsd: deps_freebsd test_freebsd_no_deps
+
+####################
+# Sources tarballs #
+####################
+
source:
git clone https://github.com/packpack/packpack.git packpack
TARBALL_COMPRESSOR=gz packpack/packpack tarball
diff --git a/.travis.yml b/.travis.yml
index e94d02ef5..4b39dbd56 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -17,6 +17,9 @@ cache:
git:
depth: 100500
+env: >
+ TEST_RUN_EXTRA_PARAMS="-j 1"
+
jobs:
include:
# Testing targets (just run tests on Debian Stretch or OS X).
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-06-19 6:47 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-19 6:47 [tarantool-patches] [PATCH v3 0/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 1/2] Temporary disabled on_shutdown.test.lua test Alexander V. Tikhonov
2019-06-19 6:47 ` [tarantool-patches] [PATCH v3 2/2] Implement Gitlab-ci testing process Alexander V. Tikhonov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox