From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 4DA9B469719 for ; Mon, 17 Feb 2020 13:00:16 +0300 (MSK) From: "Alexander V. Tikhonov" Date: Mon, 17 Feb 2020 13:00:13 +0300 Message-Id: <60bfc8cbcb4e1e6489bb9a9344b2dbff787bb048.1581933288.git.avtikhon@tarantool.org> Subject: [Tarantool-patches] [PATCH v4] Implement perf testing at gitlab-ci List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Oleg Piskunov Cc: tarantool-patches@dev.tarantool.org Enabled Tarantool performance testing on Gitlab-CI for release/master branches and "*-perf" named branches. For this purpose 'perf' and 'cleanup' stages were added into Gitlab-CI pipeline. Performance testing support next benchmarks: - cbench - linkbench - nosqlbench (hash and tree Tarantool run modes) - sysbench - tpcc - ycsb (hash and tree Tarantool run modes) Benchmarks use scripts from repository: http://gitlab.com/tarantool/bench-run Perfomance testing uses docker images, built with docker files from bench-run repository: - perf/ubuntu-bionic:perf_master parent image with benchmarks only - perf_tmp/ubuntu-bionic:perf_ child images used for testing Tarantool sources --- Changes v4: - moved the rest of Docker infrastructure for perf testing into the bench-run repository - renamed calls to performance prepare/run/cleanup stages Changes v3: - updated commit message - merged all make targets for perf testing into single - set to use variables in gitlab-ci for running perf testings Changes v2: - moved performance variables from global setup to performance template - updated commit message - added more comments to sources .gitlab-ci.yml | 100 +++++++++++++++++++++++++++++++++++++++++++++++++ .gitlab.mk | 16 ++++++++ 2 files changed, 116 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 727b7c94e..1bc53d0fb 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,7 @@ stages: - test + - perf + - cleanup variables: GITLAB_MAKE: "make -f .gitlab.mk" @@ -26,6 +28,14 @@ variables: - merge_requests - /^.*-full-ci$/ +.perf_only_template: &perf_only_definition + only: + - master + - /^.*-perf$/ + variables: &perf_vars_definition + IMAGE_PERF: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf/ubuntu-bionic:perf_master" + IMAGE_PERF_BUILT: "${CI_REGISTRY}/${CI_PROJECT_PATH}/perf_tmp/ubuntu-bionic:perf_${CI_COMMIT_SHORT_SHA}" + .docker_test_template: &docker_test_definition image: "${CI_REGISTRY}/${CI_PROJECT_PATH}/testing/debian-stretch:latest" stage: test @@ -77,6 +87,15 @@ variables: after_script: - ${GITLAB_MAKE} vms_shutdown +.perf_docker_test_template: &perf_docker_test_definition + <<: *perf_only_definition + image: ${IMAGE_PERF_BUILT} + stage: perf + tags: + - docker_perf + script: + - ${GITLAB_MAKE} perf_run + # Tests release: @@ -168,6 +187,87 @@ freebsd_12_release: script: - ${GITLAB_MAKE} vms_test_freebsd +# #### +# Perf +# #### + +# Pre-testing part + +perf_bootstrap: + <<: *perf_only_definition + stage: test + tags: + - perf + script: + - ${GITLAB_MAKE} perf_prepare + +# Testing part + +perf_sysbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'sysbench' + +perf_tpcc: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'tpcc' + +perf_ycsb_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'hash' + +perf_ycsb_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'ycsb' + ARG: 'tree' + +perf_nosqlbench_hash: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'hash' + +perf_nosqlbench_tree: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'nosqlbench' + ARG: 'tree' + +perf_cbench: + <<: *perf_docker_test_definition + variables: + <<: *perf_vars_definition + BENCH: 'cbench' + +perf_linkbench_ssd: + <<: *perf_docker_test_definition + tags: + - docker_perf_ssd + variables: + <<: *perf_vars_definition + BENCH: 'linkbench' + +# Post-testing part + +remove_images: + <<: *perf_only_definition + stage: cleanup + when: always + tags: + - perf + script: + - ${GITLAB_MAKE} perf_cleanup + # Packs centos_6: diff --git a/.gitlab.mk b/.gitlab.mk index 24b18b7f2..bc37ed7aa 100644 --- a/.gitlab.mk +++ b/.gitlab.mk @@ -78,6 +78,15 @@ docker_bootstrap: docker push ${DEBIAN_STRETCH_IMAGE}:latest docker push ${DEBIAN_BUSTER_IMAGE}:latest +# Build images for performance testing +perf_prepare: + git clone https://github.com/tarantool/bench-run.git + make -f bench-run/targets.mk prepare + +# Remove temporary performance image from the test host +perf_cleanup: + make -f bench-run/targets.mk cleanup + # ################################# # Run tests under a virtual machine # ################################# @@ -126,3 +135,10 @@ deploy: package static_build: docker build --network=host --build-arg RUN_TESTS="${RUN_TESTS}" -f Dockerfile.staticbuild . + +# ################### +# Performance testing +# ################### + +perf_run: + /opt/bench-run/benchs/${BENCH}/run.sh ${ARG} -- 2.17.1