From: "Alexander V. Tikhonov" <avtikhon@tarantool.org> To: Oleg Piskunov <o.piskunov@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: [Tarantool-patches] [PATCH v3] Implement perf testing at gitlab-ci Date: Thu, 13 Feb 2020 19:01:29 +0300 [thread overview] Message-ID: <a907b7f0b690add49208a9c9d41be5f985b3069d.1581608591.git.avtikhon@tarantool.org> (raw) Enabled Tarantool performance testing on Gitlab-CI for release/master branches and "*-perf" named branches. For this purpose 'perf' and 'cleanup' stages were added into Gitlab-CI pipeline. Performance testing support next benchmarks: - cbench - linkbench - nosqlbench (hash and tree Tarantool run modes) - sysbench - tpcc - ycsb (hash and tree Tarantool run modes) Benchmarks use scripts from repository: http://gitlab.com/tarantool/bench-run Perfomance testing uses docker images, built with docker files from bench-run repository: - perf/ubuntu-bionic:perf_master parent image with benchmarks only - perf_tmp/ubuntu-bionic:perf_<commit_SHA> child images used for testing Tarantool sources --- Changes v3: - updated commit message - merged all make targets for perf testing into single - set to use variables in gitlab-ci for running perf testings Changes v2: - moved performance variables from global setup to performance template - updated commit message - added more comments to sources .gitlab-ci.yml | 109 +++++++++++++++++++++++++++++++++++++++++++++++++ .gitlab.mk | 38 +++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 727b7c94e..758db900f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,7 @@ stages: - test + - perf + - cleanup variables: GITLAB_MAKE: "make -f .gitlab.mk" @@ -26,6 +28,14 @@ variables: - merge_requests - /^.*-full-ci$/ +.perf_only_template: &perf_only_definition + only: + - master + - /^.*-perf$/ + variables: &perf_vars_definition + IMAGE_PERF: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf/ubuntu-bionic:perf_master" + IMAGE_PERF_BUILT: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf_tmp/ubuntu-bionic:perf_${CI_COMMIT_SHORT_SHA}" + .docker_test_template: &docker_test_definition image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-stretch:latest" stage: test @@ -77,6 +87,15 @@ variables: after_script: - ${GITLAB_MAKE} vms_shutdown +.perf_docker_test_template: &perf_docker_test_definition + <<: *perf_only_definition + image: ${IMAGE_PERF_BUILT} + stage: perf + tags: + - docker_perf + script: + - ${GITLAB_MAKE} perf_run + # Tests release: @@ -168,6 +187,96 @@ freebsd_12_release: script: - ${GITLAB_MAKE} vms_test_freebsd +# #### +# Perf +# #### + +# Pre-testing part + +perf_bootstrap: + <<: *perf_only_definition + stage: test + tags: + - perf + script: + - ${GITLAB_MAKE} docker_perf_bootstrap + +# Testing part + +perf_sysbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'sysbench' + +perf_tpcc: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'tpcc' + +perf_ycsb_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'hash' + +perf_ycsb_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'tree' + +perf_nosqlbench_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'hash' + +perf_nosqlbench_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'tree' + +perf_cbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'cbench' + +# TODO: need to tune the docker memory size +#perf_linkbench: +# <<: *perf_docker_test_definition +# tags: +# - docker_perf_hdd +# variables: +# <<: *perf_vars_definition +# BENCH: 'linkbench' + +perf_linkbench_ssd: + <<: *perf_docker_test_definition + tags: + - docker_perf_ssd + variables: + <<: *perf_vars_definition + BENCH: 'linkbench' + +# Post-testing part + +remove_images: + <<: *perf_only_definition + stage: cleanup + when: always + tags: + - perf + script: + - ${GITLAB_MAKE} docker_perf_tmp_image_remove + # Packs centos_6: diff --git a/.gitlab.mk b/.gitlab.mk index 24b18b7f2..6745e589a 100644 --- a/.gitlab.mk +++ b/.gitlab.mk @@ -41,6 +41,7 @@ test_%_no_deps: git_submodule_update GITLAB_REGISTRY?=registry.gitlab.com DOCKER_BUILD=docker build --network=host -f - . +DOCKERFILE_BUILD=docker build --network=host define DEBIAN_STRETCH_DOCKERFILE FROM packpack/packpack:debian-stretch @@ -78,6 +79,36 @@ docker_bootstrap: docker push ${DEBIAN_STRETCH_IMAGE}:latest docker push ${DEBIAN_BUSTER_IMAGE}:latest +# ######################################################### +# Prepare 2 images for performance testings: +# - perf/ubuntu-bionic:perf_master +# image with only built benchmarks w/o Tarantool sources, +# image rare changed, for its fast rebuild better to save +# it on performance build hosts +# - perf_tmp/ubuntu-bionic:perf_<commit_SHA> +# images with always changed Tarantool sources and its +# depends benchmarks like 'cbench', image need to be +# removed after each testing to save the disk space +# ######################################################### +docker_perf_bootstrap: + git clone https://github.com/tarantool/bench-run.git + docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} \ + ${CI_REGISTRY} + # build all benchmarks w/o depends on Tarantool sources + ${DOCKERFILE_BUILD} --add-host $(shell hostname):127.0.0.1 \ + -t ${IMAGE_PERF} -f bench-run/dockerfiles/ubuntu_benchs . + docker push ${IMAGE_PERF} + # build Tarantool and benchmarks with depends on Tarantool sources + ${DOCKERFILE_BUILD} --build-arg image_from=${IMAGE_PERF} \ + -t ${IMAGE_PERF_BUILT} -f bench-run/dockerfiles/ubuntu_tnt . + docker push ${IMAGE_PERF_BUILT} + +# ##################################################### +# Remove temporary performance image from the test host +# ##################################################### +docker_perf_tmp_image_remove: + docker rmi --force ${IMAGE_PERF_BUILT} + # ################################# # Run tests under a virtual machine # ################################# @@ -126,3 +157,10 @@ deploy: package static_build: docker build --network=host --build-arg RUN_TESTS="${RUN_TESTS}" -f Dockerfile.staticbuild . + +# ################### +# Performance testing +# ################### + +perf_run: + /opt/bench-run/${BENCH}/run.sh ${ARG} -- 2.17.1
next reply other threads:[~2020-02-13 16:01 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-02-13 16:01 Alexander V. Tikhonov [this message] 2020-02-14 7:38 ` Oleg Piskunov 2020-02-16 22:04 ` Alexander Turenko 2020-02-17 7:25 ` Alexander Tikhonov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=a907b7f0b690add49208a9c9d41be5f985b3069d.1581608591.git.avtikhon@tarantool.org \ --to=avtikhon@tarantool.org \ --cc=o.piskunov@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v3] Implement perf testing at gitlab-ci' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox