<HTML><BODY><div>LGTM<br> <blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;">Четверг, 13 февраля 2020, 19:01 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:<br> <div id=""><div class="js-helper js-readmsg-msg"><style type="text/css"></style><div><div id="style_15816096900182422458_BODY">Enabled Tarantool performance testing on Gitlab-CI<br>for release/master branches and "*-perf" named branches.<br>For this purpose 'perf' and 'cleanup' stages were added<br>into Gitlab-CI pipeline.<br><br>Performance testing support next benchmarks:<br>- cbench<br>- linkbench<br>- nosqlbench (hash and tree Tarantool run modes)<br>- sysbench<br>- tpcc<br>- ycsb (hash and tree Tarantool run modes)<br><br>Benchmarks use scripts from repository:<br> <a href="http://gitlab.com/tarantool/bench-run" target="_blank">http://gitlab.com/tarantool/bench-run</a><br><br>Perfomance testing uses docker images, built<br>with docker files from bench-run repository:<br>- perf/ubuntu-bionic:perf_master<br> parent image with benchmarks only<br>- perf_tmp/ubuntu-bionic:perf_<commit_SHA><br> child images used for testing Tarantool sources<br>---<br><br>Changes v3:<br>- updated commit message<br>- merged all make targets for perf testing into single<br>- set to use variables in gitlab-ci for running perf testings<br><br>Changes v2:<br>- moved performance variables from global setup<br> to performance template<br>- updated commit message<br>- added more comments to sources<br><br> .gitlab-ci.yml | 109 +++++++++++++++++++++++++++++++++++++++++++++++++<br> .gitlab.mk | 38 +++++++++++++++++<br> 2 files changed, 147 insertions(+)<br><br>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml<br>index 727b7c94e..758db900f 100644<br>--- a/.gitlab-ci.yml<br>+++ b/.gitlab-ci.yml<br>@@ -1,5 +1,7 @@<br> stages:<br> - test<br>+ - perf<br>+ - cleanup<br> <br> variables:<br> GITLAB_MAKE: "make -f .gitlab.mk"<br>@@ -26,6 +28,14 @@ variables:<br> - merge_requests<br> - /^.*-full-ci$/<br> <br>+.perf_only_template: &perf_only_definition<br>+ only:<br>+ - master<br>+ - /^.*-perf$/<br>+ variables: &perf_vars_definition<br>+ IMAGE_PERF: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf/ubuntu-bionic:perf_master"<br>+ IMAGE_PERF_BUILT: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf_tmp/ubuntu-bionic:perf_${CI_COMMIT_SHORT_SHA}"<br>+<br> .docker_test_template: &docker_test_definition<br> image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-stretch:latest"<br> stage: test<br>@@ -77,6 +87,15 @@ variables:<br> after_script:<br> - ${GITLAB_MAKE} vms_shutdown<br> <br>+.perf_docker_test_template: &perf_docker_test_definition<br>+ <<: *perf_only_definition<br>+ image: ${IMAGE_PERF_BUILT}<br>+ stage: perf<br>+ tags:<br>+ - docker_perf<br>+ script:<br>+ - ${GITLAB_MAKE} perf_run<br>+<br> # Tests<br> <br> release:<br>@@ -168,6 +187,96 @@ freebsd_12_release:<br> script:<br> - ${GITLAB_MAKE} vms_test_freebsd<br> <br>+# ####<br>+# Perf<br>+# ####<br>+<br>+# Pre-testing part<br>+<br>+perf_bootstrap:<br>+ <<: *perf_only_definition<br>+ stage: test<br>+ tags:<br>+ - perf<br>+ script:<br>+ - ${GITLAB_MAKE} docker_perf_bootstrap<br>+<br>+# Testing part<br>+<br>+perf_sysbench:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'sysbench'<br>+<br>+perf_tpcc:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'tpcc'<br>+<br>+perf_ycsb_hash:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'ycsb'<br>+ ARG: 'hash'<br>+<br>+perf_ycsb_tree:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'ycsb'<br>+ ARG: 'tree'<br>+<br>+perf_nosqlbench_hash:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'nosqlbench'<br>+ ARG: 'hash'<br>+<br>+perf_nosqlbench_tree:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'nosqlbench'<br>+ ARG: 'tree'<br>+<br>+perf_cbench:<br>+ <<: *perf_docker_test_definition<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'cbench'<br>+<br>+# TODO: need to tune the docker memory size<br>+#perf_linkbench:<br>+# <<: *perf_docker_test_definition<br>+# tags:<br>+# - docker_perf_hdd<br>+# variables:<br>+# <<: *perf_vars_definition<br>+# BENCH: 'linkbench'<br>+<br>+perf_linkbench_ssd:<br>+ <<: *perf_docker_test_definition<br>+ tags:<br>+ - docker_perf_ssd<br>+ variables:<br>+ <<: *perf_vars_definition<br>+ BENCH: 'linkbench'<br>+<br>+# Post-testing part<br>+<br>+remove_images:<br>+ <<: *perf_only_definition<br>+ stage: cleanup<br>+ when: always<br>+ tags:<br>+ - perf<br>+ script:<br>+ - ${GITLAB_MAKE} docker_perf_tmp_image_remove<br>+<br> # Packs<br> <br> centos_6:<br>diff --git a/.gitlab.mk b/.gitlab.mk<br>index 24b18b7f2..6745e589a 100644<br>--- a/.gitlab.mk<br>+++ b/.gitlab.mk<br>@@ -41,6 +41,7 @@ test_%_no_deps: git_submodule_update<br> <br> GITLAB_REGISTRY?=registry.gitlab.com<br> DOCKER_BUILD=docker build --network=host -f - .<br>+DOCKERFILE_BUILD=docker build --network=host<br> <br> define DEBIAN_STRETCH_DOCKERFILE<br> FROM packpack/packpack:debian-stretch<br>@@ -78,6 +79,36 @@ docker_bootstrap:<br> docker push ${DEBIAN_STRETCH_IMAGE}:latest<br> docker push ${DEBIAN_BUSTER_IMAGE}:latest<br> <br>+# #########################################################<br>+# Prepare 2 images for performance testings:<br>+# - perf/ubuntu-bionic:perf_master<br>+# image with only built benchmarks w/o Tarantool sources,<br>+# image rare changed, for its fast rebuild better to save<br>+# it on performance build hosts<br>+# - perf_tmp/ubuntu-bionic:perf_<commit_SHA><br>+# images with always changed Tarantool sources and its<br>+# depends benchmarks like 'cbench', image need to be<br>+# removed after each testing to save the disk space<br>+# #########################################################<br>+docker_perf_bootstrap:<br>+ git clone <a href="https://github.com/tarantool/bench-run.git" target="_blank">https://github.com/tarantool/bench-run.git</a><br>+ docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} \<br>+ ${CI_REGISTRY}<br>+ # build all benchmarks w/o depends on Tarantool sources<br>+ ${DOCKERFILE_BUILD} --add-host $(shell hostname):127.0.0.1 \<br>+ -t ${IMAGE_PERF} -f bench-run/dockerfiles/ubuntu_benchs .<br>+ docker push ${IMAGE_PERF}<br>+ # build Tarantool and benchmarks with depends on Tarantool sources<br>+ ${DOCKERFILE_BUILD} --build-arg image_from=${IMAGE_PERF} \<br>+ -t ${IMAGE_PERF_BUILT} -f bench-run/dockerfiles/ubuntu_tnt .<br>+ docker push ${IMAGE_PERF_BUILT}<br>+<br>+# #####################################################<br>+# Remove temporary performance image from the test host<br>+# #####################################################<br>+docker_perf_tmp_image_remove:<br>+ docker rmi --force ${IMAGE_PERF_BUILT}<br>+<br> # #################################<br> # Run tests under a virtual machine<br> # #################################<br>@@ -126,3 +157,10 @@ deploy: package<br> <br> static_build:<br> docker build --network=host --build-arg RUN_TESTS="${RUN_TESTS}" -f Dockerfile.staticbuild .<br>+<br>+# ###################<br>+# Performance testing<br>+# ###################<br>+<br>+perf_run:<br>+ /opt/bench-run/${BENCH}/run.sh ${ARG}<br>--<br>2.17.1<br> </div></div></div></div></blockquote> <div> </div><div data-signature-widget="container"><div data-signature-widget="content"><div>--<br>Oleg Piskunov</div></div></div><div> </div></div></BODY></HTML>