From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <avtikhon@tarantool.org>
Received: from smtp55.i.mail.ru (smtp55.i.mail.ru [217.69.128.35])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by dev.tarantool.org (Postfix) with ESMTPS id DEFCB4696C3
 for <tarantool-patches@dev.tarantool.org>;
 Thu, 30 Apr 2020 13:56:47 +0300 (MSK)
Date: Thu, 30 Apr 2020 13:56:45 +0300
From: "Alexander V. Tikhonov" <avtikhon@tarantool.org>
Message-ID: <20200430105645.GA3782@hpalx>
References: <1588159426.603065637@f436.i.mail.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <1588159426.603065637@f436.i.mail.ru>
Subject: Re: [Tarantool-patches] [PATCH v2] bench-run: submit perf results
	into database
List-Id: Tarantool development patches <tarantool-patches.dev.tarantool.org>
List-Unsubscribe: <https://lists.tarantool.org/mailman/options/tarantool-patches>, 
 <mailto:tarantool-patches-request@dev.tarantool.org?subject=unsubscribe>
List-Archive: <https://lists.tarantool.org/pipermail/tarantool-patches/>
List-Post: <mailto:tarantool-patches@dev.tarantool.org>
List-Help: <mailto:tarantool-patches-request@dev.tarantool.org?subject=help>
List-Subscribe: <https://lists.tarantool.org/mailman/listinfo/tarantool-patches>, 
 <mailto:tarantool-patches-request@dev.tarantool.org?subject=subscribe>
To: Oleg Piskunov <o.piskunov@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org

Hi Oleg, thanks for the patch, LGTM, push into the master due to second
approvement.

On Wed, Apr 29, 2020 at 02:23:46PM +0300, Oleg Piskunov wrote:
>=20
>=20
> Resurrect bench.tarantool.org:
> - submit perf results into bench.tarantool.org database
> - refactoring bench running scripts
> =A0
> Fix docker files for perf testing:
> - adding python "requests" module needed for publish script.
> =A0
> Closes #4865
> =A0
> Github: https://github.com/tarantool/bench-run/tree/opiskunov/gh-4865-sub=
mit-perf-results
> Issue: https://github.com/tarantool/tarantool/issues/4865
> ---
> =A0benchs/cbench/run.sh =A0 =A0 =A0 =A0 =A0| 60 +++++++++++++++++++++++++=
++++-----------
> =A0benchs/linkbench/run.sh =A0 =A0 =A0 | 26 ++++++++++++------
> =A0benchs/nosqlbench/run.sh =A0 =A0 =A0| 46 +++++++++++++----------------=
--
> =A0benchs/publication/publish.py | 64 +++++++++++++++++++++++++++++++++++=
++++++++
> =A0benchs/sysbench/run.sh =A0 =A0 =A0 =A0| 52 ++++++++++++++-------------=
--------
> =A0benchs/tpcc/run.sh =A0 =A0 =A0 =A0 =A0 =A0| 38 ++++++++++++++---------=
--
> =A0benchs/ycsb/run.sh =A0 =A0 =A0 =A0 =A0 =A0| 42 ++++++++++++++++-------=
-----
> =A0dockerfiles/ubuntu_benchs =A0 =A0 | =A03 ++
> =A08 files changed, 216 insertions(+), 115 deletions(-)
> =A0create mode 100755 benchs/publication/publish.py
> =A0
> diff --git a/benchs/cbench/run.sh b/benchs/cbench/run.sh
> index 4d07972..77b88e7 100755
> --- a/benchs/cbench/run.sh
> +++ b/benchs/cbench/run.sh
> @@ -1,36 +1,64 @@
> =A0#!/usr/bin/env bash
> =A0
> -kill `pidof tarantool`
> -set -e
> +set -eu
> =A0set -o pipefail
> =A0
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee cbench_t_version.txt
> +numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D11"
> +cbench_opts=3D500
> =A0
> =A0killall tarantool 2>/dev/null || true
> =A0rm -rf 5* 0*
> -sync
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D9 tarantool /opt=
/cbench/cbench_runner.lua memtx 2>&1 | tee cbench_output_memtx.txt
> +$numaopts tarantool /opt/cbench/cbench_runner.lua memtx 2>&1 | tee cbenc=
h_output_memtx.txt
> =A0
> =A0killall tarantool 2>/dev/null || true
> =A0rm -rf 5* 0*
> -sync
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D10 tarantool /op=
t/cbench/cbench_runner.lua vinyl fsync 500 2>&1 | tee cbench_output_vinyl_f=
sync.txt
> +$numaopts tarantool /opt/cbench/cbench_runner.lua vinyl fsync $cbench_op=
ts 2>&1 | tee cbench_output_vinyl_fsync.txt
> =A0
> =A0killall tarantool 2>/dev/null || true
> =A0rm -rf 5* 0*
> -sync
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D11 tarantool /op=
t/cbench/cbench_runner.lua vinyl write 500 2>&1 | tee cbench_output_memtx_w=
rite.txt
> -
> -grep "^?tab" cbench_output_memtx.txt | sed "s/.*name=3D//"| sed "s/&para=
m=3D/:/"| sed "s/cb\./cb\.memtx\./"| tee -a cbench_result.txt
> -grep "^?tab" cbench_output_vinyl_fsync.txt | sed "s/.*name=3D//"| sed "s=
/&param=3D/:/"| sed "s/cb\./cb\.vinyl\.fsync\./"| tee -a cbench_result.txt
> -grep "^?tab" cbench_output_memtx_write.txt | sed "s/.*name=3D//"| sed "s=
/&param=3D/:/"| sed "s/cb\./cb\.vinyl\.write\./"| tee -a cbench_result.txt
> +$numaopts tarantool /opt/cbench/cbench_runner.lua vinyl write $cbench_op=
ts 2>&1 | tee cbench_output_vinyl_write.txt
> =A0
> -#mv tarantool-server.log cbench_tarantool_server.log
> +grep "^?tab=3Dcbench.tree" cbench_output_memtx.txt | \
> + =A0sed "s/.*name=3D//"| sed "s/&param=3D/:/"| sed "s/cb\./cb\.memtx\./"=
| \
> + =A0tee -a cbench-memtx-tree_result.txt
> +grep "^?tab=3Dcbench.hash" cbench_output_memtx.txt | \
> + =A0sed "s/.*name=3D//"| sed "s/&param=3D/:/"| sed "s/cb\./cb\.memtx\./"=
| \
> + =A0tee -a cbench-memtx-hash_result.txt
> +grep "^?tab" cbench_output_vinyl_fsync.txt | \
> + =A0sed "s/.*name=3D//"| sed "s/&param=3D/:/"| sed "s/cb\./cb\.vinyl\.fs=
ync\./"| \
> + =A0tee -a cbench-vinyl-fsync_result.txt
> +grep "^?tab" cbench_output_vinyl_write.txt | \
> + =A0sed "s/.*name=3D//"| sed "s/&param=3D/:/"| sed "s/cb\./cb\.vinyl\.wr=
ite\./"| \
> + =A0tee -a cbench-vinyl-write_result.txt
> =A0
> -echo "RESULTS:"
> -cat cbench_result.txt
> +echo ${TAR_VER} | tee cbench-memtx-tree_t_version.txt
> +echo ${TAR_VER} | tee cbench-memtx-hash_t_version.txt
> +echo ${TAR_VER} | tee cbench-vinyl-fsync_t_version.txt
> +echo ${TAR_VER} | tee cbench-vinyl-write_t_version.txt
> +echo ${TAR_VER} | tee cbench_t_version.txt
> =A0
> +echo "Tarantool TAG:"
> +cat cbench_t_version.txt
> +echo "Overall results:"
> +echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> +echo "RESULTS (cbench-memtx-tree_result.txt):"
> +cat cbench-memtx-tree_result.txt
> +echo " "
> +echo "RESULTS (cbench-memtx-hash_result.txt):"
> +cat cbench-memtx-hash_result.txt
> +echo " "
> +echo "RESULTS (cbench-vinyl-fsync_result.txt):"
> +cat cbench-vinyl-fsync_result.txt
> +echo " "
> +echo "RESULTS (cbench-vinyl-write_result.txt):"
> +cat cbench-vinyl-write_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/benchs/linkbench/run.sh b/benchs/linkbench/run.sh
> index 6738436..aac7d3c 100755
> --- a/benchs/linkbench/run.sh
> +++ b/benchs/linkbench/run.sh
> @@ -1,19 +1,16 @@
> =A0#!/usr/bin/env bash
> =A0
> -kill `pidof tarantool`
> -
> -set -e
> +set -eu
> =A0set -o pipefail
> =A0
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee linkbench_t_version.txt
> +numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> +base_dir=3D$(pwd)
> =A0
> =A0cd /opt/linkbench && mvn clean package -Dmaven.test.skip=3Dtrue
> =A0cd src/tarantool && make
> =A0cd -
> =A0
> -numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> -
> =A0# use always newly created path specialy mounted if needed
> =A0wpath=3D/builds/ws
> =A0rm -rf $wpath
> @@ -30,8 +27,21 @@ sed "s/^maxid1 =3D .*/maxid1 =3D 5000000/g" -i /opt/li=
nkbench/config/FBWorkload.prop
> =A0sed "s/^requesters =3D .*/requesters =3D 1/g" -i $cfgfile
> =A0sed "s/^requests =3D .*/requests =3D 2000000/g" -i $cfgfile
> =A0$numaopts /opt/linkbench/bin/linkbench -c $cfgfile -l 2>&1 | tee loadi=
ng.res.txt
> -sync
> +
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> =A0$numaopts /opt/linkbench/bin/linkbench -c $cfgfile -r 2>&1 | tee linkb=
ench_output.txt
> =A0
> -grep "REQUEST PHASE COMPLETED" linkbench_output.txt | sed "s/.*second =
=3D /linkbench:/" | tee -a linkbench_result.txt
> +grep "REQUEST PHASE COMPLETED" linkbench_output.txt | sed "s/.*second =
=3D /linkbench:/" | tee -a linkbench.ssd_result.txt
> +echo ${TAR_VER} | tee linkbench.ssd_t_version.txt
> +cp -f linkbench.ssd_t_version.txt $base_dir
> +cp -f linkbench.ssd_result.txt $base_dir
> +
> +echo "Tarantool TAG:"
> +cat linkbench.ssd_t_version.txt
> +echo "Overall results:"
> +echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> +cat linkbench.ssd_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/benchs/nosqlbench/run.sh b/benchs/nosqlbench/run.sh
> index c9491f5..157f372 100755
> --- a/benchs/nosqlbench/run.sh
> +++ b/benchs/nosqlbench/run.sh
> @@ -1,34 +1,22 @@
> =A0#!/usr/bin/env bash
> =A0
> +set -eu
> +set -o pipefail
> +
> =A0type=3D$1
> =A0if [ "$type" =3D=3D "" ]; then
> =A0 =A0 =A0type=3Dhash
> =A0fi
> =A0
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee nosqlbench_t_version.txt
> -
> -kill `pidof tarantool`
> -
> -#set -e
> -#
> -#set -o pipefail
> +numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> =A0
> =A0killall tarantool 2>/dev/null || true
> -rm -rf 0*.xlog 0*.snap
> -sync
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> =A0
> -#cd /opt/nosqlbench/
> -#cmake . && =A0make -j
> -#cd -
> -#
> -#tarantool -v
> -#
> -free -h
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,11 ta=
rantool `dirname $0`/tnt_${type}.lua 2>&1 &
> +$numaopts tarantool `dirname $0`/tnt_${type}.lua 2>&1 &
> =A0sed =A0"s/port 3303/port 3301/" /opt/nosqlbench/src/nosqlbench.conf -i
> -#sed =A0"s/request_count 4000000/request_count 400000/" /opt/nosqlbench/=
src/nosqlbench.conf -i
> =A0sed =A0"s/benchmark 'no_limit'/benchmark 'time_limit'/" /opt/nosqlbenc=
h/src/nosqlbench.conf -i
> =A0sed =A0"s/time_limit 10/time_limit 2000/" /opt/nosqlbench/src/nosqlben=
ch.conf -i
> =A0sed =A0"s/request_batch_count 1/request_batch_count 10/" /opt/nosqlben=
ch/src/nosqlbench.conf -i
> @@ -36,14 +24,18 @@ sed =A0"s/rps 12000/rps 20000/" /opt/nosqlbench/src/n=
osqlbench.conf -i
> =A0sleep 5
> =A0echo "Run NB"
> =A0# WARNING: don't try to save output from stderr - file will use the wh=
ole disk space !
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,11 /o=
pt/nosqlbench/src/nb /opt/nosqlbench/src/nosqlbench.conf | \
> +$numaopts =A0/opt/nosqlbench/src/nb /opt/nosqlbench/src/nosqlbench.conf =
| \
> =A0 =A0 =A0grep -v "Warmup" | grep -v "Failed to allocate" >nosqlbench_ou=
tput.txt || cat nosqlbench_output.txt
> =A0
> -#echo "Latest 1000 lines:"
> -#tail -1000 nosqlbench_output.txt
> -
> -echo "Getting results"
> -grep "TOTAL RPS STATISTICS:" nosqlbench_output.txt -A6 | awk -F "|" 'NR =
> 4 {print $2,":", $4}' > nosqlbench_result.txt
> -
> -cat nosqlbench_result.txt
> -
> +grep "TOTAL RPS STATISTICS:" nosqlbench_output.txt -A6 | \
> + =A0awk -F "|" 'NR > 4 {print $2,":", $4}' > noSQLbench.${type}_result.t=
xt
> +echo ${TAR_VER} | tee noSQLbench.${type}_t_version.txt
> +
> +echo "Tarantool TAG:"
> +cat noSQLbench.${type}_t_version.txt
> +echo "Overall results:"
> +echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> +cat noSQLbench.${type}_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/benchs/publication/publish.py b/benchs/publication/publish.py
> new file mode 100755
> index 0000000..42bc53a
> --- /dev/null
> +++ b/benchs/publication/publish.py
> @@ -0,0 +1,64 @@
> +#!/usr/bin/env python
> +import fnmatch
> +import os
> +from urllib import urlencode
> +import requests
> +
> +
> +def parse_bench(filename):
> + =A0 =A0with open(filename) as raw_data:
> + =A0 =A0 =A0 =A0return raw_data.readlines()
> +
> +
> +def get_version(filename):
> + =A0 =A0with open(filename) as raw_data:
> + =A0 =A0 =A0 =A0version =3D raw_data.readlines()[-1]
> + =A0 =A0 =A0 =A0return version.split()[0]
> +
> +
> +def push_to_microb(server, token, name, value, version, tab):
> + =A0 =A0uri =3D 'http://%s/push?%s' % (server, urlencode(dict(
> + =A0 =A0 =A0 =A0key=3Dtoken, name=3Dname, param=3Dvalue,
> + =A0 =A0 =A0 =A0v=3Dversion, unit=3D'trps', tab=3Dtab
> + =A0 =A0)))
> +
> + =A0 =A0r =3D requests.get(uri)
> + =A0 =A0if r.status_code =3D=3D 200:
> + =A0 =A0 =A0 =A0print('Export complete')
> + =A0 =A0else:
> + =A0 =A0 =A0 =A0print('Export error http: %d' % r.status_code)
> + =A0 =A0 =A0 =A0print('Export error text: %d' % r.text)
> +
> +
> +def main():
> + =A0 =A0if "MICROB_WEB_TOKEN" in os.environ and "MICROB_WEB_HOST" in os.=
environ:
> + =A0 =A0 =A0 =A0bench =3D {}
> + =A0 =A0 =A0 =A0res =3D []
> + =A0 =A0 =A0 =A0current_data =3D {}
> + =A0 =A0 =A0 =A0version =3D ''
> + =A0 =A0 =A0 =A0for file in os.listdir('.'):
> + =A0 =A0 =A0 =A0 =A0 =A0if fnmatch.fnmatch(file, '*_result.txt'):
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0values =3D parse_bench(file)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0benchmark =3D file.split('_')[0]
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0version =3D get_version('{}_t_version.tx=
t'.format(benchmark))
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0for value in values:
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0test_name =3D value.split(':')[0]
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0test_res =3D float(value.split('=
:')[1])
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0res.append(test_res)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0push_to_microb(
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0os.environ['MICROB_WEB_H=
OST'],
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0os.environ['MICROB_WEB_T=
OKEN'],
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0test_name,
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0test_res,
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0version,
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0benchmark,
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0)
> + =A0 =A0 =A0 =A0print ("VERSION - ", version)
> + =A0 =A0else:
> + =A0 =A0 =A0 =A0print("MICROB params not specified")
> +
> + =A0 =A0return 0
> +
> +
> +if __name__ =3D=3D '__main__':
> + =A0 =A0main()
> diff --git a/benchs/sysbench/run.sh b/benchs/sysbench/run.sh
> index 6a69d14..a9cf2f3 100755
> --- a/benchs/sysbench/run.sh
> +++ b/benchs/sysbench/run.sh
> @@ -1,18 +1,12 @@
> =A0#!/usr/bin/env bash
> =A0
> -runs=3D10
> -if [ "$1" !=3D "" ]; then
> - =A0 =A0runs=3D$1
> -fi
> -
> -kill `pidof tarantool`
> -
> -set -e
> +set -eu
> =A0set -o pipefail
> =A0
> +runs=3D10
> +
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee sysbench_t_version.txt
> -set -e
> +numaconf=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> =A0
> =A0ARRAY_TESTS=3D(
> =A0 =A0 =A0"oltp_read_only"
> @@ -28,27 +22,17 @@ ARRAY_TESTS=3D(
> =A0# =A0 =A0"bulk_insert"
> =A0)
> =A0
> -if [ -n "${TEST}" ]; then ARRAY_TESTS=3D("${TEST}"); fi
> -
> -
> -if [ ! -n "${WARMUP_TIME}" ]; then WARMUP_TIME=3D5; fi
> -if [ ! -n "${TIME}" ]; then TIME=3D20; fi
> -if [ ! -n "${DBMS}" ]; then DBMS=3D"tarantool"; fi
> -if [ ! -n "${THREADS}" ]; then THREADS=3D200; fi
> -
> -if [ -n "${USER}" ]; then USER=3D--${DBMS}-user=3D${USER}; fi
> -if [ -n "${PASSWORD}" ]; then PASSWORD=3D--${DBMS}-password=3D${PASSWORD=
}; fi
> -
> -numaconf=3D"--membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,1=
1"
> +WARMUP_TIME=3D5
> +TIME=3D20
> +DBMS=3D"tarantool"
> +THREADS=3D200
> =A0opts=3D"--db-driver=3D${DBMS} --threads=3D${THREADS}"
> =A0
> =A0export LD_LIBRARY_PATH=3D/usr/local/lib
> =A0
> -rm -f sysbench_result.txt
> -set -o pipefail
> =A0for test in "${ARRAY_TESTS[@]}"; do
> =A0 =A0 =A0res=3D0
> - =A0 =A0tlog=3Dsysbench_${test}_result.txt
> + =A0 =A0tlog=3Dsysbench_${test}_results.txt
> =A0 =A0 =A0rm -f $tlog
> =A0 =A0 =A0maxres=3D0
> =A0 =A0 =A0for run in `eval echo {1..$runs}` ; do
> @@ -58,10 +42,10 @@ for test in "${ARRAY_TESTS[@]}"; do
> =A0 =A0 =A0 =A0 =A0sysbench $test $opts cleanup >sysbench_output.txt
> =A0 =A0 =A0 =A0 =A0sysbench $test $opts prepare >>sysbench_output.txt
> =A0
> - =A0 =A0 =A0 =A0numactl $numaconf sysbench $test $opts \
> + =A0 =A0 =A0 =A0$numaconf sysbench $test $opts \
> =A0 =A0 =A0 =A0 =A0 =A0 =A0--time=3D${TIME} --warmup-time=3D${WARMUP_TIME=
} run >>sysbench_output.txt
> =A0
> - =A0 =A0 =A0 =A0numactl $numaconf sysbench $test $opts cleanup >>sysbenc=
h_output.txt
> + =A0 =A0 =A0 =A0$numaconf sysbench $test $opts cleanup >>sysbench_output=
.txt
> =A0
> =A0 =A0 =A0 =A0 =A0cat sysbench_output.txt | grep -e 'transactions:' | gr=
ep -oP '\(\K\S*' | tee $tlog
> =A0 =A0 =A0 =A0 =A0tres=3D`cat $tlog | sed 's#^.*:##g' | sed 's#\..*$##g'`
> @@ -69,16 +53,22 @@ for test in "${ARRAY_TESTS[@]}"; do
> =A0 =A0 =A0 =A0 =A0if [[ $tres -gt $maxres ]]; then maxres=3D$tres ; fi
> =A0 =A0 =A0done
> =A0 =A0 =A0res=3D$(($res/$runs))
> - =A0 =A0echo "${test}: $res" >>sysbench_result.txt
> + =A0 =A0echo "${test}: $res" >>Sysbench_result.txt
> =A0
> =A0 =A0 =A0echo "Subtest '$test' results:"
> =A0 =A0 =A0echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> =A0 =A0 =A0echo "Average result: $res"
> =A0 =A0 =A0echo "Maximum result: $maxres"
> - =A0 =A0printf "Diviations (AVG -> MAX): %.2f" `bc <<< "scale =3D 4; (1 =
- $res / $maxres) * 100"` ; echo %
> + =A0 =A0printf "Deviations (AVG -> MAX): %.2f" `bc <<< "scale =3D 4; (1 =
- $res / $maxres) * 100"` ; echo %
> =A0done
> =A0
> +echo ${TAR_VER} | tee Sysbench_t_version.txt
> +
> +echo "Tarantool TAG:"
> +cat Sysbench_t_version.txt
> =A0echo "Overall results:"
> =A0echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> -cat sysbench_result.txt
> -
> +cat Sysbench_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/benchs/tpcc/run.sh b/benchs/tpcc/run.sh
> index f1f60cc..fca7939 100755
> --- a/benchs/tpcc/run.sh
> +++ b/benchs/tpcc/run.sh
> @@ -1,22 +1,20 @@
> =A0#!/usr/bin/env bash
> +set -eu
> +set -o pipefail
> =A0
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee tpcc_t_version.txt
> -
> -set -e
> +numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> +base_dir=3D$(pwd)
> =A0
> -if [ ! -n "${TIME}" ]; then TIME=3D1200; fi
> -if [ ! -n "${WARMUP_TIME}" ]; then WARMUP_TIME=3D10; fi
> -
> -set -o pipefail
> +TIME=3D1200
> +WARMUP_TIME=3D10
> =A0
> =A0killall tarantool tpcc_load 2>/dev/null || true
> -rm -rf 0*.xlog 0*.snap
> -sync
> +sync && echo "sync passed" || echo "sync failed with error" $?
> =A0echo 3 > /proc/sys/vm/drop_caches
> =A0
> =A0sed 's#box.sql#box#g' -i /opt/tpcc/create_table.lua
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,11 ta=
rantool /opt/tpcc/create_table.lua &
> +$numaopts tarantool /opt/tpcc/create_table.lua &
> =A0sleep 5
> =A0
> =A0# Usage: tpcc_load -h server_host -P port -d database_name -u mysql_us=
er
> @@ -27,13 +25,21 @@ tpcc_opts=3D"-h localhost -P 3301 -d tarantool -u roo=
t -p '' -w 15"
> =A0cd /opt/tpcc
> =A0. /opt/tpcc/load.sh tarantool 15
> =A0
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,11 \
> - =A0 =A0/opt/tpcc/tpcc_start $tpcc_opts -r10 -l${TIME} -i${TIME} >tpcc_o=
utput.txt 2>/dev/null
> -
> -echo -n "tpcc:" | tee tpcc_result.txt
> -cat tpcc_output.txt | grep -e '<TpmC>' | grep -oP '\K[0-9.]*' | tee -a t=
pcc_result.txt
> +$numaopts /opt/tpcc/tpcc_start $tpcc_opts -r10 -l${TIME} -i${TIME} >tpcc=
_output.txt 2>/dev/null
> =A0
> +echo -n "tpcc:" | tee tpc.c_result.txt
> +cat tpcc_output.txt | grep -e '<TpmC>' | grep -oP '\K[0-9.]*' | tee -a t=
pc.c_result.txt
> =A0cat tpcc_output.txt
> +
> +echo ${TAR_VER} | tee tpc.c_t_version.txt
> +cp -f tpc.c_t_version.txt $base_dir
> +cp -f tpc.c_result.txt $base_dir
> +
> +echo "Tarantool TAG:"
> +cat tpc.c_t_version.txt
> =A0echo "Overall result:"
> =A0echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> -cat tpcc_result.txt
> +cat tpc.c_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/benchs/ycsb/run.sh b/benchs/ycsb/run.sh
> index 5ae1c75..e5219a1 100755
> --- a/benchs/ycsb/run.sh
> +++ b/benchs/ycsb/run.sh
> @@ -1,23 +1,18 @@
> =A0#!/usr/bin/env bash
> =A0
> +set -eu
> +set -o pipefail
> +
> =A0mode=3D$1
> -runs=3D$2
> +runs=3D1
> =A0
> =A0if [ "$mode" =3D=3D "" ]; then
> =A0 =A0 =A0mode=3Dhash
> =A0fi
> =A0
> -if [ "$runs" =3D=3D "" ]; then
> - =A0 =A0runs=3D1
> -fi
> -
> -kill `pidof tarantool`
> -
> -set -e
> -set -o pipefail
> -
> =A0TAR_VER=3D$(tarantool -v | grep -e "Tarantool" | =A0grep -oP '\s\K\S*')
> -echo ${TAR_VER} | tee ycsb_t_version.txt
> +numaopts=3D"numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,=
8,9,10,11"
> +base_dir=3D$(pwd)
> =A0
> =A0ws=3D/opt/ycsb
> =A0cd $ws
> @@ -33,7 +28,7 @@ sed 's/listen=3D.*/listen=3D3301,\n =A0 memtx_memory =
=3D 2000000000,/' -i $srvlua
> =A0sed 's/logger_nonblock.*//' -i $srvlua
> =A0sed 's/logger/log/' -i $srvlua
> =A0sed 's/read,write,execute/create,read,write,execute/' -i $srvlua
> -numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D6,7,8,9,10,11 ta=
rantool $srvlua 2>&1 &
> +$numaopts tarantool $srvlua 2>&1 &
> =A0
> =A0sleep 5
> =A0plogs=3D$ws/results
> @@ -45,12 +40,25 @@ for l in a b c d e f ; do
> =A0 =A0 =A0 =A0 =A0res=3D$plogs/run${l}_${r}
> =A0 =A0 =A0 =A0 =A0echo ---------------- ${l}: $r
> =A0 =A0 =A0 =A0 =A0echo "tarantool.port=3D3301" >> $ws/workloads/workload=
${l}
> - =A0 =A0 =A0 =A0numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D=
6,7,8,9,10,11 bin/ycsb load tarantool -s -P workloads/workload${l} >${res}.=
load 2>&1 || cat ${res}.load
> - =A0 =A0 =A0 sync
> + =A0 =A0 =A0 =A0$numaopts bin/ycsb load tarantool -s -P workloads/worklo=
ad${l} >${res}.load 2>&1 || cat ${res}.load
> + =A0 =A0 =A0 sync && echo "sync passed" || echo "sync failed with error"=
 $?
> =A0 =A0 =A0 =A0 echo 3 > /proc/sys/vm/drop_caches
> - =A0 =A0 =A0 =A0numactl --membind=3D1 --cpunodebind=3D1 --physcpubind=3D=
6,7,8,9,10,11 bin/ycsb run tarantool -s -P workloads/workload${l} >${res}.l=
og 2>&1 || cat ${res}.log
> + =A0 =A0 =A0 =A0$numaopts bin/ycsb run tarantool -s -P workloads/workloa=
d${l} >${res}.log 2>&1 || cat ${res}.log
> =A0 =A0 =A0 =A0 =A0grep Thro ${res}.log | awk '{ print "Overall result: "=
$3 }' | tee ${res}.txt
> - =A0 =A0 =A0 =A0sed "s#Overall result#$l $r#g" ${res}.txt >>${plogs}/res=
ults.txt
> + =A0 =A0 =A0 =A0sed "s#Overall result#$l $r#g" ${res}.txt >>${plogs}/ycs=
b.${mode}_result.txt
> =A0 =A0 =A0done
> =A0done
> -cat ${plogs}/results.txt
> +
> +echo ${TAR_VER} | tee ycsb.${mode}_t_version.txt
> +cp -f ${plogs}/ycsb.${mode}_result.txt .
> +cp -f ycsb.${mode}_t_version.txt $base_dir
> +cp -f ycsb.${mode}_result.txt $base_dir
> +
> +echo "Tarantool TAG:"
> +cat ycsb.${mode}_t_version.txt
> +echo "Overall results:"
> +echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D"
> +cat ycsb.${mode}_result.txt
> +echo " "
> +echo "Publish data to bench database"
> +/opt/bench-run/benchs/publication/publish.py
> diff --git a/dockerfiles/ubuntu_benchs b/dockerfiles/ubuntu_benchs
> index fd9acb4..947f082 100644
> --- a/dockerfiles/ubuntu_benchs
> +++ b/dockerfiles/ubuntu_benchs
> @@ -73,3 +73,6 @@ RUN luarocks install \
> =A0
> =A0# benchmarks runners
> =A0RUN git clone https://github.com/tarantool/bench-run.git /opt/bench-run
> +
> +# adding python dependency
> +RUN pip install requests
> --
> 1.8.3.1
> =A0