From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 515A5469719 for ; Thu, 13 Feb 2020 19:01:31 +0300 (MSK) From: "Alexander V. Tikhonov" Date: Thu, 13 Feb 2020 19:01:29 +0300 Message-Id: Subject: [Tarantool-patches] [PATCH v3] Implement perf testing at gitlab-ci List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Oleg Piskunov Cc: tarantool-patches@dev.tarantool.org Enabled Tarantool performance testing on Gitlab-CI for release/master branches and "*-perf" named branches. For this purpose 'perf' and 'cleanup' stages were added into Gitlab-CI pipeline. Performance testing support next benchmarks: - cbench - linkbench - nosqlbench (hash and tree Tarantool run modes) - sysbench - tpcc - ycsb (hash and tree Tarantool run modes) Benchmarks use scripts from repository: http://gitlab.com/tarantool/bench-run Perfomance testing uses docker images, built with docker files from bench-run repository: - perf/ubuntu-bionic:perf_master parent image with benchmarks only - perf_tmp/ubuntu-bionic:perf_ child images used for testing Tarantool sources --- Changes v3: - updated commit message - merged all make targets for perf testing into single - set to use variables in gitlab-ci for running perf testings Changes v2: - moved performance variables from global setup to performance template - updated commit message - added more comments to sources .gitlab-ci.yml | 109 +++++++++++++++++++++++++++++++++++++++++++++++++ .gitlab.mk | 38 +++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 727b7c94e..758db900f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,7 @@ stages: - test + - perf + - cleanup variables: GITLAB_MAKE: "make -f .gitlab.mk" @@ -26,6 +28,14 @@ variables: - merge_requests - /^.*-full-ci$/ +.perf_only_template: &perf_only_definition + only: + - master + - /^.*-perf$/ + variables: &perf_vars_definition + IMAGE_PERF: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf/ubuntu-bionic:perf_master" + IMAGE_PERF_BUILT: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf_tmp/ubuntu-bionic:perf_${CI_COMMIT_SHORT_SHA}" + .docker_test_template: &docker_test_definition image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-stretch:latest" stage: test @@ -77,6 +87,15 @@ variables: after_script: - ${GITLAB_MAKE} vms_shutdown +.perf_docker_test_template: &perf_docker_test_definition + <<: *perf_only_definition + image: ${IMAGE_PERF_BUILT} + stage: perf + tags: + - docker_perf + script: + - ${GITLAB_MAKE} perf_run + # Tests release: @@ -168,6 +187,96 @@ freebsd_12_release: script: - ${GITLAB_MAKE} vms_test_freebsd +# #### +# Perf +# #### + +# Pre-testing part + +perf_bootstrap: + <<: *perf_only_definition + stage: test + tags: + - perf + script: + - ${GITLAB_MAKE} docker_perf_bootstrap + +# Testing part + +perf_sysbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'sysbench' + +perf_tpcc: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'tpcc' + +perf_ycsb_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'hash' + +perf_ycsb_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'tree' + +perf_nosqlbench_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'hash' + +perf_nosqlbench_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'tree' + +perf_cbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'cbench' + +# TODO: need to tune the docker memory size +#perf_linkbench: +# <<: *perf_docker_test_definition +# tags: +# - docker_perf_hdd +# variables: +# <<: *perf_vars_definition +# BENCH: 'linkbench' + +perf_linkbench_ssd: + <<: *perf_docker_test_definition + tags: + - docker_perf_ssd + variables: + <<: *perf_vars_definition + BENCH: 'linkbench' + +# Post-testing part + +remove_images: + <<: *perf_only_definition + stage: cleanup + when: always + tags: + - perf + script: + - ${GITLAB_MAKE} docker_perf_tmp_image_remove + # Packs centos_6: diff --git a/.gitlab.mk b/.gitlab.mk index 24b18b7f2..6745e589a 100644 --- a/.gitlab.mk +++ b/.gitlab.mk @@ -41,6 +41,7 @@ test_%_no_deps: git_submodule_update GITLAB_REGISTRY?=registry.gitlab.com DOCKER_BUILD=docker build --network=host -f - . +DOCKERFILE_BUILD=docker build --network=host define DEBIAN_STRETCH_DOCKERFILE FROM packpack/packpack:debian-stretch @@ -78,6 +79,36 @@ docker_bootstrap: docker push ${DEBIAN_STRETCH_IMAGE}:latest docker push ${DEBIAN_BUSTER_IMAGE}:latest +# ######################################################### +# Prepare 2 images for performance testings: +# - perf/ubuntu-bionic:perf_master +# image with only built benchmarks w/o Tarantool sources, +# image rare changed, for its fast rebuild better to save +# it on performance build hosts +# - perf_tmp/ubuntu-bionic:perf_ +# images with always changed Tarantool sources and its +# depends benchmarks like 'cbench', image need to be +# removed after each testing to save the disk space +# ######################################################### +docker_perf_bootstrap: + git clone https://github.com/tarantool/bench-run.git + docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} \ + ${CI_REGISTRY} + # build all benchmarks w/o depends on Tarantool sources + ${DOCKERFILE_BUILD} --add-host $(shell hostname):127.0.0.1 \ + -t ${IMAGE_PERF} -f bench-run/dockerfiles/ubuntu_benchs . + docker push ${IMAGE_PERF} + # build Tarantool and benchmarks with depends on Tarantool sources + ${DOCKERFILE_BUILD} --build-arg image_from=${IMAGE_PERF} \ + -t ${IMAGE_PERF_BUILT} -f bench-run/dockerfiles/ubuntu_tnt . + docker push ${IMAGE_PERF_BUILT} + +# ##################################################### +# Remove temporary performance image from the test host +# ##################################################### +docker_perf_tmp_image_remove: + docker rmi --force ${IMAGE_PERF_BUILT} + # ################################# # Run tests under a virtual machine # ################################# @@ -126,3 +157,10 @@ deploy: package static_build: docker build --network=host --build-arg RUN_TESTS="${RUN_TESTS}" -f Dockerfile.staticbuild . + +# ################### +# Performance testing +# ################### + +perf_run: + /opt/bench-run/${BENCH}/run.sh ${ARG} -- 2.17.1