Re: [Tarantool-patches] [PATCH v3] Implement perf testing at gitlab-ci

["Oleg Piskunov" <o.piskunov@tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 6659 bytes --]


LGTM
  
>Четверг, 13 февраля 2020, 19:01 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:
> 
>Enabled Tarantool performance testing on Gitlab-CI
>for release/master branches and "*-perf" named branches.
>For this purpose 'perf' and 'cleanup' stages were added
>into Gitlab-CI pipeline.
>
>Performance testing support next benchmarks:
>- cbench
>- linkbench
>- nosqlbench (hash and tree Tarantool run modes)
>- sysbench
>- tpcc
>- ycsb (hash and tree Tarantool run modes)
>
>Benchmarks use scripts from repository:
>   http://gitlab.com/tarantool/bench-run
>
>Perfomance testing uses docker images, built
>with docker files from bench-run repository:
>- perf/ubuntu-bionic:perf_master
>    parent image with benchmarks only
>- perf_tmp/ubuntu-bionic:perf_<commit_SHA>
>    child images used for testing Tarantool sources
>---
>
>Changes v3:
>- updated commit message
>- merged all make targets for perf testing into single
>- set to use variables in gitlab-ci for running perf testings
>
>Changes v2:
>- moved performance variables from global setup
>  to performance template
>- updated commit message
>- added more comments to sources
>
> .gitlab-ci.yml | 109 +++++++++++++++++++++++++++++++++++++++++++++++++
> .gitlab.mk | 38 +++++++++++++++++
> 2 files changed, 147 insertions(+)
>
>diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>index 727b7c94e..758db900f 100644
>--- a/.gitlab-ci.yml
>+++ b/.gitlab-ci.yml
>@@ -1,5 +1,7 @@
> stages:
>   - test
>+ - perf
>+ - cleanup
> 
> variables:
>   GITLAB_MAKE: "make -f .gitlab.mk"
>@@ -26,6 +28,14 @@ variables:
>     - merge_requests
>     - /^.*-full-ci$/
> 
>+.perf_only_template: &perf_only_definition
>+ only:
>+ - master
>+ - /^.*-perf$/
>+ variables: &perf_vars_definition
>+ IMAGE_PERF: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf/ubuntu-bionic:perf_master"
>+ IMAGE_PERF_BUILT: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf_tmp/ubuntu-bionic:perf_${CI_COMMIT_SHORT_SHA}"
>+
> .docker_test_template: &docker_test_definition
>   image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-stretch:latest"
>   stage: test
>@@ -77,6 +87,15 @@ variables:
>   after_script:
>     - ${GITLAB_MAKE} vms_shutdown
> 
>+.perf_docker_test_template: &perf_docker_test_definition
>+ <<: *perf_only_definition
>+ image: ${IMAGE_PERF_BUILT}
>+ stage: perf
>+ tags:
>+ - docker_perf
>+ script:
>+ - ${GITLAB_MAKE} perf_run
>+
> # Tests
> 
> release:
>@@ -168,6 +187,96 @@ freebsd_12_release:
>   script:
>     - ${GITLAB_MAKE} vms_test_freebsd
> 
>+# ####
>+# Perf
>+# ####
>+
>+# Pre-testing part
>+
>+perf_bootstrap:
>+ <<: *perf_only_definition
>+ stage: test
>+ tags:
>+ - perf
>+ script:
>+ - ${GITLAB_MAKE} docker_perf_bootstrap
>+
>+# Testing part
>+
>+perf_sysbench:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'sysbench'
>+
>+perf_tpcc:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'tpcc'
>+
>+perf_ycsb_hash:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'ycsb'
>+ ARG: 'hash'
>+
>+perf_ycsb_tree:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'ycsb'
>+ ARG: 'tree'
>+
>+perf_nosqlbench_hash:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'nosqlbench'
>+ ARG: 'hash'
>+
>+perf_nosqlbench_tree:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'nosqlbench'
>+ ARG: 'tree'
>+
>+perf_cbench:
>+ <<: *perf_docker_test_definition
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'cbench'
>+
>+# TODO: need to tune the docker memory size
>+#perf_linkbench:
>+# <<: *perf_docker_test_definition
>+# tags:
>+# - docker_perf_hdd
>+# variables:
>+# <<: *perf_vars_definition
>+# BENCH: 'linkbench'
>+
>+perf_linkbench_ssd:
>+ <<: *perf_docker_test_definition
>+ tags:
>+ - docker_perf_ssd
>+ variables:
>+ <<: *perf_vars_definition
>+ BENCH: 'linkbench'
>+
>+# Post-testing part
>+
>+remove_images:
>+ <<: *perf_only_definition
>+ stage: cleanup
>+ when: always
>+ tags:
>+ - perf
>+ script:
>+ - ${GITLAB_MAKE} docker_perf_tmp_image_remove
>+
> # Packs
> 
> centos_6:
>diff --git a/.gitlab.mk b/.gitlab.mk
>index 24b18b7f2..6745e589a 100644
>--- a/.gitlab.mk
>+++ b/.gitlab.mk
>@@ -41,6 +41,7 @@ test_%_no_deps: git_submodule_update
> 
> GITLAB_REGISTRY?=registry.gitlab.com
> DOCKER_BUILD=docker build --network=host -f - .
>+DOCKERFILE_BUILD=docker build --network=host
> 
> define DEBIAN_STRETCH_DOCKERFILE
> FROM packpack/packpack:debian-stretch
>@@ -78,6 +79,36 @@ docker_bootstrap:
>  docker push ${DEBIAN_STRETCH_IMAGE}:latest
>  docker push ${DEBIAN_BUSTER_IMAGE}:latest
> 
>+# #########################################################
>+# Prepare 2 images for performance testings:
>+# - perf/ubuntu-bionic:perf_master
>+# image with only built benchmarks w/o Tarantool sources,
>+# image rare changed, for its fast rebuild better to save
>+# it on performance build hosts
>+# - perf_tmp/ubuntu-bionic:perf_<commit_SHA>
>+# images with always changed Tarantool sources and its
>+# depends benchmarks like 'cbench', image need to be
>+# removed after each testing to save the disk space
>+# #########################################################
>+docker_perf_bootstrap:
>+ git clone  https://github.com/tarantool/bench-run.git
>+ docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} \
>+ ${CI_REGISTRY}
>+ # build all benchmarks w/o depends on Tarantool sources
>+ ${DOCKERFILE_BUILD} --add-host $(shell hostname):127.0.0.1 \
>+ -t ${IMAGE_PERF} -f bench-run/dockerfiles/ubuntu_benchs .
>+ docker push ${IMAGE_PERF}
>+ # build Tarantool and benchmarks with depends on Tarantool sources
>+ ${DOCKERFILE_BUILD} --build-arg image_from=${IMAGE_PERF} \
>+ -t ${IMAGE_PERF_BUILT} -f bench-run/dockerfiles/ubuntu_tnt .
>+ docker push ${IMAGE_PERF_BUILT}
>+
>+# #####################################################
>+# Remove temporary performance image from the test host
>+# #####################################################
>+docker_perf_tmp_image_remove:
>+ docker rmi --force ${IMAGE_PERF_BUILT}
>+
> # #################################
> # Run tests under a virtual machine
> # #################################
>@@ -126,3 +157,10 @@ deploy: package
> 
> static_build:
>  docker build --network=host --build-arg RUN_TESTS="${RUN_TESTS}" -f Dockerfile.staticbuild .
>+
>+# ###################
>+# Performance testing
>+# ###################
>+
>+perf_run:
>+ /opt/bench-run/${BENCH}/run.sh ${ARG}
>--
>2.17.1
>  
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 8193 bytes --]