[Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Олег Пискунов]

[-- Attachment #1: Type: text/plain, Size: 2891 bytes --]


Gitlab-ci pipeline modified in order to keep performance results into gitlab-ci artifacts.
 
Closes #4920
 
Github: https://github.com/tarantool/tarantool/tree/opiskunov/gh-4920-perf-artifacts
Issue: https://github.com/tarantool/tarantool/issues/4920
---
 .gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fd265aa..4b05fa5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -201,12 +201,22 @@ perf_sysbench:
   variables:
     <<: *perf_vars_definition
     BENCH: 'sysbench'
+  artifacts:
+    when: always
+    paths:
+      - Sysbench_result.txt
+      - Sysbench_t_version.txt
 
 perf_tpcc:
   <<: *perf_docker_test_definition
   variables:
     <<: *perf_vars_definition
     BENCH: 'tpcc'
+  artifacts:
+    when: always
+    paths:
+      - tpc.c_result.txt
+      - tpc.c_t_version.txt
 
 perf_ycsb_hash:
   <<: *perf_docker_test_definition
@@ -214,6 +224,11 @@ perf_ycsb_hash:
     <<: *perf_vars_definition
     BENCH: 'ycsb'
     ARG: 'hash'
+  artifacts:
+    when: always
+    paths:
+      - ycsb.hash_result.txt
+      - ycsb.hash_t_version.txt
 
 perf_ycsb_tree:
   <<: *perf_docker_test_definition
@@ -221,6 +236,11 @@ perf_ycsb_tree:
     <<: *perf_vars_definition
     BENCH: 'ycsb'
     ARG: 'tree'
+  artifacts:
+    when: always
+    paths:
+      - ycsb.tree_result.txt
+      - ycsb.tree_t_version.txt
 
 perf_nosqlbench_hash:
   <<: *perf_docker_test_definition
@@ -228,6 +248,11 @@ perf_nosqlbench_hash:
     <<: *perf_vars_definition
     BENCH: 'nosqlbench'
     ARG: 'hash'
+  artifacts:
+    when: always
+    paths:
+      - nosqlbench.hash_result.txt
+      - nosqlbench.hash_t_version.txt
 
 perf_nosqlbench_tree:
   <<: *perf_docker_test_definition
@@ -235,12 +260,25 @@ perf_nosqlbench_tree:
     <<: *perf_vars_definition
     BENCH: 'nosqlbench'
     ARG: 'tree'
+  artifacts:
+    when: always
+    paths:
+      - nosqlbench.tree_result.txt
+      - nosqlbench.tree_t_version.txt
 
 perf_cbench:
   <<: *perf_docker_test_definition
   variables:
     <<: *perf_vars_definition
     BENCH: 'cbench'
+  artifacts:
+    when: always
+    paths:
+      - cbench-memtx-tree_result.txt
+      - cbench-memtx-hash_result.txt
+      - cbench-vinyl-fsync_result.txt
+      - cbench-vinyl-write_result.txt
+      - cbench_t_version.txt
 
 perf_linkbench_ssd:
   <<: *perf_docker_test_definition
@@ -249,6 +287,11 @@ perf_linkbench_ssd:
   variables:
     <<: *perf_vars_definition
     BENCH: 'linkbench'
+  artifacts:
+    when: always
+    paths:
+      - linkbench.ssd_result.txt
+      - linkbench.ssd_t_version.txt
 
 # Post-testing part
 
--
1.8.3.1
 

[-- Attachment #2: Type: text/html, Size: 4334 bytes --]

Re: [Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Sergey Bronnikov]

LGTM

On 10:53 Mon 27 Apr , Олег Пискунов wrote:
> 
> Gitlab-ci pipeline modified in order to keep performance results into gitlab-ci artifacts.
>  
> Closes #4920
>  
> Github: https://github.com/tarantool/tarantool/tree/opiskunov/gh-4920-perf-artifacts
> Issue: https://github.com/tarantool/tarantool/issues/4920
> ---
>  .gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
>  
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fd265aa..4b05fa5 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -201,12 +201,22 @@ perf_sysbench:
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'sysbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - Sysbench_result.txt
> +      - Sysbench_t_version.txt
>  
>  perf_tpcc:
>    <<: *perf_docker_test_definition
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'tpcc'
> +  artifacts:
> +    when: always
> +    paths:
> +      - tpc.c_result.txt
> +      - tpc.c_t_version.txt
>  
>  perf_ycsb_hash:
>    <<: *perf_docker_test_definition
> @@ -214,6 +224,11 @@ perf_ycsb_hash:
>      <<: *perf_vars_definition
>      BENCH: 'ycsb'
>      ARG: 'hash'
> +  artifacts:
> +    when: always
> +    paths:
> +      - ycsb.hash_result.txt
> +      - ycsb.hash_t_version.txt
>  
>  perf_ycsb_tree:
>    <<: *perf_docker_test_definition
> @@ -221,6 +236,11 @@ perf_ycsb_tree:
>      <<: *perf_vars_definition
>      BENCH: 'ycsb'
>      ARG: 'tree'
> +  artifacts:
> +    when: always
> +    paths:
> +      - ycsb.tree_result.txt
> +      - ycsb.tree_t_version.txt
>  
>  perf_nosqlbench_hash:
>    <<: *perf_docker_test_definition
> @@ -228,6 +248,11 @@ perf_nosqlbench_hash:
>      <<: *perf_vars_definition
>      BENCH: 'nosqlbench'
>      ARG: 'hash'
> +  artifacts:
> +    when: always
> +    paths:
> +      - nosqlbench.hash_result.txt
> +      - nosqlbench.hash_t_version.txt
>  
>  perf_nosqlbench_tree:
>    <<: *perf_docker_test_definition
> @@ -235,12 +260,25 @@ perf_nosqlbench_tree:
>      <<: *perf_vars_definition
>      BENCH: 'nosqlbench'
>      ARG: 'tree'
> +  artifacts:
> +    when: always
> +    paths:
> +      - nosqlbench.tree_result.txt
> +      - nosqlbench.tree_t_version.txt
>  
>  perf_cbench:
>    <<: *perf_docker_test_definition
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'cbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - cbench-memtx-tree_result.txt
> +      - cbench-memtx-hash_result.txt
> +      - cbench-vinyl-fsync_result.txt
> +      - cbench-vinyl-write_result.txt
> +      - cbench_t_version.txt
>  
>  perf_linkbench_ssd:
>    <<: *perf_docker_test_definition
> @@ -249,6 +287,11 @@ perf_linkbench_ssd:
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'linkbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - linkbench.ssd_result.txt
> +      - linkbench.ssd_t_version.txt
>  
>  # Post-testing part
>  
> --
> 1.8.3.1
>  

-- 
sergeyb@

Re: [Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Alexander V. Tikhonov]

Hi Oleg, thanks for the patch, LGTM, please proceed to Kirill to push.

On Mon, Apr 27, 2020 at 10:53:59AM +0300, Олег Пискунов wrote:
> 
> Gitlab-ci pipeline modified in order to keep performance results into gitlab-ci artifacts.
>  
> Closes #4920
>  
> Github: https://github.com/tarantool/tarantool/tree/opiskunov/gh-4920-perf-artifacts
> Issue: https://github.com/tarantool/tarantool/issues/4920
> ---
>  .gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
>  
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fd265aa..4b05fa5 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -201,12 +201,22 @@ perf_sysbench:
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'sysbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - Sysbench_result.txt
> +      - Sysbench_t_version.txt
>  
>  perf_tpcc:
>    <<: *perf_docker_test_definition
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'tpcc'
> +  artifacts:
> +    when: always
> +    paths:
> +      - tpc.c_result.txt
> +      - tpc.c_t_version.txt
>  
>  perf_ycsb_hash:
>    <<: *perf_docker_test_definition
> @@ -214,6 +224,11 @@ perf_ycsb_hash:
>      <<: *perf_vars_definition
>      BENCH: 'ycsb'
>      ARG: 'hash'
> +  artifacts:
> +    when: always
> +    paths:
> +      - ycsb.hash_result.txt
> +      - ycsb.hash_t_version.txt
>  
>  perf_ycsb_tree:
>    <<: *perf_docker_test_definition
> @@ -221,6 +236,11 @@ perf_ycsb_tree:
>      <<: *perf_vars_definition
>      BENCH: 'ycsb'
>      ARG: 'tree'
> +  artifacts:
> +    when: always
> +    paths:
> +      - ycsb.tree_result.txt
> +      - ycsb.tree_t_version.txt
>  
>  perf_nosqlbench_hash:
>    <<: *perf_docker_test_definition
> @@ -228,6 +248,11 @@ perf_nosqlbench_hash:
>      <<: *perf_vars_definition
>      BENCH: 'nosqlbench'
>      ARG: 'hash'
> +  artifacts:
> +    when: always
> +    paths:
> +      - nosqlbench.hash_result.txt
> +      - nosqlbench.hash_t_version.txt
>  
>  perf_nosqlbench_tree:
>    <<: *perf_docker_test_definition
> @@ -235,12 +260,25 @@ perf_nosqlbench_tree:
>      <<: *perf_vars_definition
>      BENCH: 'nosqlbench'
>      ARG: 'tree'
> +  artifacts:
> +    when: always
> +    paths:
> +      - nosqlbench.tree_result.txt
> +      - nosqlbench.tree_t_version.txt
>  
>  perf_cbench:
>    <<: *perf_docker_test_definition
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'cbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - cbench-memtx-tree_result.txt
> +      - cbench-memtx-hash_result.txt
> +      - cbench-vinyl-fsync_result.txt
> +      - cbench-vinyl-write_result.txt
> +      - cbench_t_version.txt
>  
>  perf_linkbench_ssd:
>    <<: *perf_docker_test_definition
> @@ -249,6 +287,11 @@ perf_linkbench_ssd:
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'linkbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - linkbench.ssd_result.txt
> +      - linkbench.ssd_t_version.txt
>  
>  # Post-testing part
>  
> --
> 1.8.3.1
>  

Re: [Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Alexander Turenko]

> Keep perf results as gitlab-ci artifacts

Nit: Let's add some prefix: I would use 'gitlab-ci'. Using of prefixes
is convenient and allows to traverse over commits faster.

> 
> Gitlab-ci pipeline modified in order to keep performance results into gitlab-ci artifacts.

Nit: Keep commit message body lines within 72 symbols.

>  
> Closes #4920
>  
> Github: https://github.com/tarantool/tarantool/tree/opiskunov/gh-4920-perf-artifacts
> Issue: https://github.com/tarantool/tarantool/issues/4920

Nit: Don't include issue no / branch into a commit message, only within
a email. Place them under '---' to distinguish from a commit message
itself.

> ---
>  .gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
>  
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fd265aa..4b05fa5 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -201,12 +201,22 @@ perf_sysbench:
>    variables:
>      <<: *perf_vars_definition
>      BENCH: 'sysbench'
> +  artifacts:
> +    when: always
> +    paths:
> +      - Sysbench_result.txt
> +      - Sysbench_t_version.txt

I would deploy a directory with result as an artefact or use globs
(GitLab CI supports both). gitlab-ci.yml should not lean on hardcoded
bench-run paths. Can we formalize bench-run contract for performance
measurement results? Maybe, `${benchmark}_results` directory? Or
`${benchmark}_${conf}_results` if names intersection may be troublesome.

WBR, Alexander Turenko.

Re: [Tarantool-patches] [PATCH v1] Keep perf results as gitlab-ci artifacts

[Oleg Piskunov]

[-- Attachment #1: Type: text/plain, Size: 1733 bytes --]


Alexander, thanks a lot for review!
  
>Среда, 6 мая 2020, 12:08 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:
> 
>> Keep perf results as gitlab-ci artifacts
>
>Nit: Let's add some prefix: I would use 'gitlab-ci'. Using of prefixes
>is convenient and allows to traverse over commits faster.
ok.
>>
>> Gitlab-ci pipeline modified in order to keep performance results into gitlab-ci artifacts.
>
>Nit: Keep commit message body lines within 72 symbols.
ok.
>>  
>> Closes #4920
>>  
>> Github:  https://github.com/tarantool/tarantool/tree/opiskunov/gh-4920-perf-artifacts
>> Issue:  https://github.com/tarantool/tarantool/issues/4920
>
>Nit: Don't include issue no / branch into a commit message, only within
>a email. Place them under '---' to distinguish from a commit message
>itself.
ok
>> ---
>>  .gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 43 insertions(+)
>>  
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index fd265aa..4b05fa5 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -201,12 +201,22 @@ perf_sysbench:
>>    variables:
>>      <<: *perf_vars_definition
>>      BENCH: 'sysbench'
>> +  artifacts:
>> +    when: always
>> +    paths:
>> +      - Sysbench_result.txt
>> +      - Sysbench_t_version.txt
>
>I would deploy a directory with result as an artefact or use globs
>(GitLab CI supports both). gitlab-ci.yml should not lean on hardcoded
>bench-run paths. Can we formalize bench-run contract for performance
>measurement results? Maybe, `${benchmark}_results` directory? Or
>`${benchmark}_${conf}_results` if names intersection may be troublesome.
I will use wildcards for this.
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 3222 bytes --]