<HTML><BODY><div>Hi Oleg, thanks for the patch, I see that you just divided the<br>previous patch into this and the other one w/o changes, so LGTM.<br><br> <blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;">Среда, 22 апреля 2020, 13:18 +03:00 от Олег Пискунов <o.piskunov@corp.mail.ru>:<br> <div id=""><div class="js-helper js-readmsg-msg"><style type="text/css"></style><div><div id="style_15875506840961224297_BODY"><div class="class_1587567074"><div>Resurrect bench.tarantool.org:<br>- submit perf results into bench.tarantool.org database</div><div> </div><div>Fix docker files for perf testing:<br>- adding python "requests" module needed for publish script.</div><div> </div><div>Closes #4865</div><div> </div><div>Github: <a href="https://github.com/tarantool/bench-run/tree/opiskunov/gh-4865-submit-perf-results" target="_blank">https://github.com/tarantool/bench-run/tree/opiskunov/gh-4865-submit-perf-results</a><br>Issue: <a href="https://github.com/tarantool/tarantool/issues/4865" target="_blank">https://github.com/tarantool/tarantool/issues/4865</a><br>---<br> benchs/cbench/run.sh | 38 ++++++++++++++++++------<br> benchs/linkbench/run.sh | 13 +++++++--<br> benchs/nosqlbench/run.sh | 22 ++++++++------<br> benchs/publication/publish.py | 67 +++++++++++++++++++++++++++++++++++++++++++<br> benchs/sysbench/run.sh | 19 ++++++------<br> benchs/tpcc/run.sh | 21 ++++++++------<br> benchs/ycsb/run.sh | 16 +++++++++--<br> dockerfiles/ubuntu_benchs | 4 +--<br> 8 files changed, 158 insertions(+), 42 deletions(-)<br> create mode 100755 benchs/publication/publish.py</div><div> </div><div>diff --git a/benchs/cbench/run.sh b/benchs/cbench/run.sh<br>index 4d07972..c4867ad 100755<br>--- a/benchs/cbench/run.sh<br>+++ b/benchs/cbench/run.sh<br>@@ -1,11 +1,11 @@<br> #!/usr/bin/env bash</div><div> </div><div> kill `pidof tarantool`<br>+<br> set -e<br> set -o pipefail</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee cbench_t_version.txt</div><div> </div><div> killall tarantool 2>/dev/null || true<br> rm -rf 5* 0*<br>@@ -23,14 +23,34 @@ killall tarantool 2>/dev/null || true<br> rm -rf 5* 0*<br> sync<br> echo 3 > /proc/sys/vm/drop_caches<br>-numactl --membind=1 --cpunodebind=1 --physcpubind=11 tarantool /opt/cbench/cbench_runner.lua vinyl write 500 2>&1 | tee cbench_output_memtx_write.txt<br>+numactl --membind=1 --cpunodebind=1 --physcpubind=11 tarantool /opt/cbench/cbench_runner.lua vinyl write 500 2>&1 | tee cbench_output_vinyl_write.txt</div><div> </div><div>-grep "^?tab" cbench_output_memtx.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.memtx\./"| tee -a cbench_result.txt<br>-grep "^?tab" cbench_output_vinyl_fsync.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.vinyl\.fsync\./"| tee -a cbench_result.txt<br>-grep "^?tab" cbench_output_memtx_write.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.vinyl\.write\./"| tee -a cbench_result.txt<br>+grep "^?tab=cbench.tree" cbench_output_memtx.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.memtx\./"| tee -a cbench-memtx-tree_result.txt<br>+grep "^?tab=cbench.hash" cbench_output_memtx.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.memtx\./"| tee -a cbench-memtx-hash_result.txt<br>+grep "^?tab" cbench_output_vinyl_fsync.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.vinyl\.fsync\./"| tee -a cbench-vinyl-fsync_result.txt<br>+grep "^?tab" cbench_output_vinyl_write.txt | sed "s/.*name=//"| sed "s/¶m=/:/"| sed "s/cb\./cb\.vinyl\.write\./"| tee -a cbench-vinyl-write_result.txt</div><div> </div><div>-#mv tarantool-server.log cbench_tarantool_server.log<br>-<br>-echo "RESULTS:"<br>-cat cbench_result.txt<br>+echo ${TAR_VER} | tee cbench-memtx-tree_t_version.txt<br>+echo ${TAR_VER} | tee cbench-memtx-hash_t_version.txt<br>+echo ${TAR_VER} | tee cbench-vinyl-fsync_t_version.txt<br>+echo ${TAR_VER} | tee cbench-vinyl-write_t_version.txt<br>+echo ${TAR_VER} | tee cbench_t_version.txt</div><div> </div><div>+echo "Tarantool TAG:"<br>+cat cbench_t_version.txt<br>+echo "Overall results:"<br>+echo "================"<br>+echo "RESULTS (cbench-memtx-tree_result.txt):"<br>+cat cbench-memtx-tree_result.txt<br>+echo " "<br>+echo "RESULTS (cbench-memtx-hash_result.txt):"<br>+cat cbench-memtx-hash_result.txt<br>+echo " "<br>+echo "RESULTS (cbench-vinyl-fsync_result.txt):"<br>+cat cbench-vinyl-fsync_result.txt<br>+echo " "<br>+echo "RESULTS (cbench-vinyl-write_result.txt):"<br>+cat cbench-vinyl-write_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/benchs/linkbench/run.sh b/benchs/linkbench/run.sh<br>index 6738436..e7c77c3 100755<br>--- a/benchs/linkbench/run.sh<br>+++ b/benchs/linkbench/run.sh<br>@@ -6,7 +6,6 @@ set -e<br> set -o pipefail</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee linkbench_t_version.txt</div><div> </div><div> cd /opt/linkbench && mvn clean package -Dmaven.test.skip=true<br> cd src/tarantool && make<br>@@ -34,4 +33,14 @@ sync<br> echo 3 > /proc/sys/vm/drop_caches<br> $numaopts /opt/linkbench/bin/linkbench -c $cfgfile -r 2>&1 | tee linkbench_output.txt</div><div> </div><div>-grep "REQUEST PHASE COMPLETED" linkbench_output.txt | sed "s/.*second = /linkbench:/" | tee -a linkbench_result.txt<br>+grep "REQUEST PHASE COMPLETED" linkbench_output.txt | sed "s/.*second = /linkbench:/" | tee -a linkbench.ssd_result.txt<br>+echo ${TAR_VER} | tee linkbench.ssd_t_version.txt<br>+<br>+echo "Tarantool TAG:"<br>+cat linkbench.ssd_t_version.txt<br>+echo "Overall results:"<br>+echo "================"<br>+cat linkbench.ssd_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/benchs/nosqlbench/run.sh b/benchs/nosqlbench/run.sh<br>index c9491f5..a68248e 100755<br>--- a/benchs/nosqlbench/run.sh<br>+++ b/benchs/nosqlbench/run.sh<br>@@ -6,13 +6,11 @@ if [ "$type" == "" ]; then<br> fi</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee nosqlbench_t_version.txt</div><div> </div><div> kill `pidof tarantool`</div><div> </div><div>-#set -e<br>-#<br>-#set -o pipefail<br>+set -e<br>+set -o pipefail</div><div> </div><div> killall tarantool 2>/dev/null || true<br> rm -rf 0*.xlog 0*.snap<br>@@ -42,8 +40,14 @@ numactl --membind=1 --cpunodebind=1 --physcpubind=6,7,8,9,10,11 /opt/nosqlbench/<br> #echo "Latest 1000 lines:"</div><div> #tail -1000 nosqlbench_output.txt</div><div> </div><div>-echo "Getting results"<br>-grep "TOTAL RPS STATISTICS:" nosqlbench_output.txt -A6 | awk -F "|" 'NR > 4 {print $2,":", $4}' > nosqlbench_result.txt<br>-<br>-cat nosqlbench_result.txt<br>-<br>+grep "TOTAL RPS STATISTICS:" nosqlbench_output.txt -A6 | awk -F "|" 'NR > 4 {print $2,":", $4}' > noSQLbench.${type}_result.txt<br>+echo ${TAR_VER} | tee noSQLbench.${type}_t_version.txt<br>+<br>+echo "Tarantool TAG:"<br>+cat noSQLbench.${type}_t_version.txt<br>+echo "Overall results:"<br>+echo "================"<br>+cat noSQLbench.${type}_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/benchs/publication/publish.py b/benchs/publication/publish.py<br>new file mode 100755<br>index 0000000..d472fd7<br>--- /dev/null<br>+++ b/benchs/publication/publish.py<br>@@ -0,0 +1,67 @@<br>+#!/usr/bin/env python<br>+import fnmatch<br>+import os<br>+from urllib import urlencode<br>+import requests<br>+<br>+<br>+def parse_bench(filename):<br>+ fileHandle = open(filename)<br>+ lastline = fileHandle.readlines()<br>+ fileHandle.close()<br>+ return lastline<br>+<br>+<br>+def get_version(filename):<br>+ fileHandle = open(filename)<br>+ lastline = fileHandle.readlines()[-1]<br>+ fileHandle.close()<br>+ return lastline.split()[0]<br>+<br>+<br>+def push_to_microb(server, token, name, value, version, tab):<br>+ uri = 'http://%s/push?%s' % (server, urlencode(dict(<br>+ key=token, name=name, param=value,<br>+ v=version, unit='trps', tab=tab<br>+ )))<br>+<br>+ r = requests.get(uri)<br>+ if r.status_code == 200:<br>+ print('Export complete')<br>+ else:<br>+ print('Export error http: %d' % r.status_code)<br>+ print('Export error text: %d' % r.text)<br>+<br>+<br>+def main():<br>+ if "MICROB_WEB_TOKEN" in os.environ and "MICROB_WEB_HOST" in os.environ:<br>+ bench = {}<br>+ res = []<br>+ current_data = {}<br>+ version = ''<br>+ for file in os.listdir('.'):<br>+ if fnmatch.fnmatch(file, '*_result.txt'):<br>+ values = parse_bench(file)<br>+ benchmark = file.split('_')[0]<br>+ version = get_version('{}_t_version.txt'.format(benchmark))<br>+ for value in values:<br>+ test_name = value.split(':')[0]<br>+ test_res = float(value.split(':')[1])<br>+ res.append(test_res)<br>+ push_to_microb(<br>+ os.environ['MICROB_WEB_HOST'],<br>+ os.environ['MICROB_WEB_TOKEN'],<br>+ test_name,<br>+ test_res,<br>+ version,<br>+ benchmark,<br>+ )<br>+ print ("VERSION - ", version)<br>+ else:<br>+ print("MICROB params not specified")<br>+<br>+ return 0<br>+<br>+<br>+if __name__ == '__main__':<br>+ main()<br>diff --git a/benchs/sysbench/run.sh b/benchs/sysbench/run.sh<br>index 6a69d14..ef0af43 100755<br>--- a/benchs/sysbench/run.sh<br>+++ b/benchs/sysbench/run.sh<br>@@ -11,8 +11,6 @@ set -e<br> set -o pipefail</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee sysbench_t_version.txt<br>-set -e</div><div> </div><div> ARRAY_TESTS=(<br> "oltp_read_only"<br>@@ -44,11 +42,10 @@ opts="--db-driver=${DBMS} --threads=${THREADS}"</div><div> </div><div> export LD_LIBRARY_PATH=/usr/local/lib</div><div> </div><div>-rm -f sysbench_result.txt<br>-set -o pipefail<br>+rm -f Sysbench_result.txt<br> for test in "${ARRAY_TESTS[@]}"; do<br> res=0<br>- tlog=sysbench_${test}_result.txt<br>+ tlog=sysbench_${test}_results.txt<br> rm -f $tlog<br> maxres=0<br> for run in `eval echo {1..$runs}` ; do<br>@@ -69,7 +66,7 @@ for test in "${ARRAY_TESTS[@]}"; do<br> if [[ $tres -gt $maxres ]]; then maxres=$tres ; fi<br> done<br> res=$(($res/$runs))<br>- echo "${test}: $res" >>sysbench_result.txt<br>+ echo "${test}: $res" >>Sysbench_result.txt</div><div> echo "Subtest '$test' results:"<br> echo "==============================="<br>@@ -78,7 +75,13 @@ for test in "${ARRAY_TESTS[@]}"; do<br> printf "Diviations (AVG -> MAX): %.2f" `bc <<< "scale = 4; (1 - $res / $maxres) * 100"` ; echo %<br> done</div><div> </div><div>+echo ${TAR_VER} | tee Sysbench_t_version.txt<br>+<br>+echo "Tarantool TAG:"<br>+cat Sysbench_t_version.txt<br> echo "Overall results:"<br> echo "================"<br>-cat sysbench_result.txt<br>-<br>+cat Sysbench_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/benchs/tpcc/run.sh b/benchs/tpcc/run.sh<br>index f1f60cc..76754e5 100755<br>--- a/benchs/tpcc/run.sh<br>+++ b/benchs/tpcc/run.sh<br>@@ -1,15 +1,12 @@<br> #!/usr/bin/env bash<br>+set -e<br>+set -o pipefail</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee tpcc_t_version.txt<br>-<br>-set -e</div><div> </div><div> if [ ! -n "${TIME}" ]; then TIME=1200; fi<br> if [ ! -n "${WARMUP_TIME}" ]; then WARMUP_TIME=10; fi</div><div> </div><div>-set -o pipefail<br>-<br> killall tarantool tpcc_load 2>/dev/null || true<br> rm -rf 0*.xlog 0*.snap<br> sync<br>@@ -30,10 +27,16 @@ cd /opt/tpcc<br> numactl --membind=1 --cpunodebind=1 --physcpubind=6,7,8,9,10,11 \<br> /opt/tpcc/tpcc_start $tpcc_opts -r10 -l${TIME} -i${TIME} >tpcc_output.txt 2>/dev/null</div><div> </div><div>-echo -n "tpcc:" | tee tpcc_result.txt<br>-cat tpcc_output.txt | grep -e '<TpmC>' | grep -oP '\K[0-9.]*' | tee -a tpcc_result.txt<br>-<br>+echo -n "tpcc:" | tee tpc.c_result.txt<br>+cat tpcc_output.txt | grep -e '<TpmC>' | grep -oP '\K[0-9.]*' | tee -a tpc.c_result.txt<br> cat tpcc_output.txt<br>+echo ${TAR_VER} | tee tpc.c_t_version.txt<br>+<br>+echo "Tarantool TAG:"<br>+cat tpc.c_t_version.txt<br> echo "Overall result:"<br> echo "==============="<br>-cat tpcc_result.txt<br>+cat tpc.c_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/benchs/ycsb/run.sh b/benchs/ycsb/run.sh<br>index 5ae1c75..d2cf8af 100755<br>--- a/benchs/ycsb/run.sh<br>+++ b/benchs/ycsb/run.sh<br>@@ -17,7 +17,6 @@ set -e<br> set -o pipefail</div><div> </div><div> TAR_VER=$(tarantool -v | grep -e "Tarantool" | grep -oP '\s\K\S*')<br>-echo ${TAR_VER} | tee ycsb_t_version.txt</div><div> </div><div> ws=/opt/ycsb<br> cd $ws<br>@@ -50,7 +49,18 @@ for l in a b c d e f ; do<br> echo 3 > /proc/sys/vm/drop_caches<br> numactl --membind=1 --cpunodebind=1 --physcpubind=6,7,8,9,10,11 bin/ycsb run tarantool -s -P workloads/workload${l} >${res}.log 2>&1 || cat ${res}.log<br> grep Thro ${res}.log | awk '{ print "Overall result: "$3 }' | tee ${res}.txt<br>- sed "s#Overall result#$l $r#g" ${res}.txt >>${plogs}/results.txt<br>+ sed "s#Overall result#$l $r#g" ${res}.txt >>${plogs}/ycsb.${mode}_result.txt<br> done<br> done<br>-cat ${plogs}/results.txt<br>+<br>+echo ${TAR_VER} | tee ycsb.${mode}_t_version.txt<br>+cp -f ${plogs}/ycsb.${mode}_result.txt .<br>+<br>+echo "Tarantool TAG:"<br>+cat ycsb.${mode}_t_version.txt<br>+echo "Overall results:"<br>+echo "================"<br>+cat ycsb.${mode}_result.txt<br>+echo " "<br>+echo "Publish data to bench database"<br>+/opt/bench-run/benchs/publication/publish.py<br>diff --git a/dockerfiles/ubuntu_benchs b/dockerfiles/ubuntu_benchs<br>index fd9acb4..2c2e717 100644<br>--- a/dockerfiles/ubuntu_benchs<br>+++ b/dockerfiles/ubuntu_benchs<br>@@ -71,5 +71,5 @@ RUN luarocks install \<br> <a href="https://raw.githubusercontent.com/tarantool/gperftools/master/rockspecs/gperftools-scm-1.rockspec" target="_blank">https://raw.githubusercontent.com/tarantool/gperftools/master/rockspecs/gperftools-scm-1.rockspec</a> \<br> --local >build.log 2>&1 || ( cat build.log && false )</div><div> </div><div>-# benchmarks runners<br>-RUN git clone <a href="https://github.com/tarantool/bench-run.git" target="_blank">https://github.com/tarantool/bench-run.git</a> /opt/bench-run<br>+# adding python dependency<br>+RUN pip install requests<br>--<br>1.8.3.1</div><div> </div></div></div></div></div></div></blockquote> <div> </div><div data-signature-widget="container"><div data-signature-widget="content"><div>--<br>Alexander Tikhonov</div></div></div><div> </div></div></BODY></HTML>