<HTML><BODY><p><br>Oleg,<br><br>Please review the patch.<br><br></p><blockquote style="border-left:1px solid #0857A6; margin:10px; padding:0 0 0 10px;">
Вторник, 3 декабря 2019, 13:34 +03:00 от Alexander V. Tikhonov <avtikhon@tarantool.org>:<br>
<br>
<div id="">
<div class="js-helper js-readmsg-msg">
<style type="text/css"></style>
<div>
<div id="style_15753692491157941555_BODY">Added 'tools/run_vm.sh' script for VBOX VMs orchestrate.<br>
It starts the VM on the local host at the previously<br>
prepared configurations. It uses the template in options:<br>
osx_13_*|osx_14_*|freebsd_12_*<br>
to be able to choose free VM from the list:<br>
osx_13_[1-X]|osx_14_*[1-X]|freebsd_12_[1-X]<br>
<br>
Gitlab-ci orchestrator chooses the host, where it's job can<br>
be run, so to make it always workable without delays in<br>
waiting the free VMs by the new local orchestrator script,<br>
the limit of the overall started jobs must be not bigger<br>
than requested template can find the appropriate VMs on the<br>
host. Each of the template can use up to X pre-configured<br>
VMs, so the gitlab-runner configuration file:<br>
/etc/gitlab-runner/config.toml<br>
has common configuration option:<br>
concurrent = X<br>
and each gitlab runner has limit option:<br>
limit = X<br>
X value should be manually found by testing at each used host.<br>
---<br>
<br>
Github: <a href="https://github.com/tarantool/tarantool/tree/avtikhon/vms-orchestrate" target="_blank">https://github.com/tarantool/tarantool/tree/avtikhon/vms-orchestrate</a><br>
<br>
.gitlab-ci.yml | 30 ++-------<br>
.gitlab.mk | 15 +----<br>
tools/run_vm.sh | 176 ++++++++++++++++++++++++++++++++++++++++++++++++<br>
3 files changed, 185 insertions(+), 36 deletions(-)<br>
create mode 100755 tools/run_vm.sh<br>
<br>
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml<br>
index cf13c382e..4542f8d75 100644<br>
--- a/.gitlab-ci.yml<br>
+++ b/.gitlab-ci.yml<br>
@@ -42,10 +42,8 @@ variables:<br>
<br>
.vbox_template: &vbox_definition<br>
stage: test<br>
- before_script:<br>
- - ${GITLAB_MAKE} vms_start<br>
- after_script:<br>
- - ${GITLAB_MAKE} vms_shutdown<br>
+ tags:<br>
+ - vms_test<br>
<br>
# Tests<br>
<br>
@@ -93,47 +91,31 @@ release_asan_clang8:<br>
osx_13_release:<br>
<<: *release_only_definition<br>
<<: *vbox_definition<br>
- tags:<br>
- - vms_osx_13<br>
variables:<br>
- VMS_NAME: 'osx_13'<br>
- VMS_USER: 'tarantool'<br>
- VMS_PORT: '2212'<br>
+ VMS_NAME: 'osx_13_*'<br>
script:<br>
- ${GITLAB_MAKE} vms_test_osx<br>
<br>
osx_14_release:<br>
<<: *vbox_definition<br>
- tags:<br>
- - vms_osx_14<br>
variables:<br>
- VMS_NAME: 'osx_14'<br>
- VMS_USER: 'tarantool'<br>
- VMS_PORT: '2222'<br>
+ VMS_NAME: 'osx_14_*'<br>
script:<br>
- ${GITLAB_MAKE} vms_test_osx<br>
<br>
osx_14_release_lto:<br>
<<: *release_only_definition<br>
<<: *vbox_definition<br>
- tags:<br>
- - vms_osx_14<br>
variables:<br>
EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=-DENABLE_LTO=ON ;"<br>
- VMS_NAME: 'osx_14'<br>
- VMS_USER: 'tarantool'<br>
- VMS_PORT: '2222'<br>
+ VMS_NAME: 'osx_14_*'<br>
script:<br>
- ${GITLAB_MAKE} vms_test_osx<br>
<br>
freebsd_12_release:<br>
<<: *vbox_definition<br>
- tags:<br>
- - vms_freebsd_12<br>
variables:<br>
- VMS_NAME: 'freebsd_12'<br>
- VMS_USER: 'vagrant'<br>
- VMS_PORT: '2232'<br>
+ VMS_NAME: 'freebsd_12_*'<br>
MAKE: 'gmake'<br>
script:<br>
- ${GITLAB_MAKE} vms_test_freebsd<br>
diff --git a/.gitlab.mk b/.gitlab.mk<br>
index 48a92e518..863d5fdf7 100644<br>
--- a/.gitlab.mk<br>
+++ b/.gitlab.mk<br>
@@ -82,21 +82,12 @@ docker_bootstrap:<br>
# Run tests under a virtual machine<br>
# #################################<br>
<br>
-vms_start:<br>
- VBoxManage controlvm ${VMS_NAME} poweroff || true<br>
- VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME}<br>
- VBoxManage startvm ${VMS_NAME} --type headless<br>
-<br>
vms_test_%:<br>
- tar czf - ../tarantool | ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} tar xzf -<br>
- ssh ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \<br>
- '${EXTRA_ENV} \<br>
+ ./tools/run_vm.sh -t=${VMS_NAME} -s -a=try_restore,start,login,stop \<br>
+ -c="${EXTRA_ENV} \<br>
cd tarantool && \<br>
${GITLAB_MAKE} git_submodule_update && \<br>
- ${TRAVIS_MAKE} $(subst vms_,,$@)'"<br>
-<br>
-vms_shutdown:<br>
- VBoxManage controlvm ${VMS_NAME} poweroff<br>
+ ${TRAVIS_MAKE} $(subst vms_,,$@)"<br>
<br>
# ########################<br>
# Build RPM / Deb packages<br>
diff --git a/tools/run_vm.sh b/tools/run_vm.sh<br>
new file mode 100755<br>
index 000000000..abe409441<br>
--- /dev/null<br>
+++ b/tools/run_vm.sh<br>
@@ -0,0 +1,176 @@<br>
+#!/bin/bash<br>
+<br>
+set -e<br>
+<br>
+usage() {<br>
+ echo "Usage: $0 -t=<freebsd_12[_*]|osx_1[34][_*]> [-a=<download,install,restore,start,login,snapshot,stop>] [-c=<command to run in VM>]"<br>
+}<br>
+<br>
+NAME=<br>
+choose_vm() {<br>
+ os_dist=$1<br>
+ all_vms=`VBoxManage list vms | awk -F '"' '{print $2}' | sort | grep "^${os_dist}_[1-9]*$" || true`<br>
+ if [ "$all_vms" == "" ]; then<br>
+ echo "ERROR: the VM for the given os=$os and dist=$dist by pattern '^${os_dist}_[1-9]*\$' is not available"<br>
+ usage<br>
+ exit 1<br>
+ fi<br>
+ run_vms=`VBoxManage list runningvms | awk -F '"' '{print $2}' | sort | grep "^${os_dist}_[1-9]*$" || true`<br>
+ for vm in $all_vms ; do<br>
+ used=0<br>
+ for rvm in $run_vms ; do<br>
+ if [ "$vm" == "$rvm" ]; then<br>
+ used=1<br>
+ break<br>
+ fi<br>
+ done<br>
+<br>
+ # let's setup the lock the file which will pause the<br>
+ # next job till the current job will start the VM:<br>
+ # lockfile will hold the current VM for 10 seconds<br>
+ # to start up the VM in this delay, while next job<br>
+ # will wait for 30 seconds for the lockfile released<br>
+ # to recheck the VM, otherwise it will try to find<br>
+ # the other free VM<br>
+ lfile=/tmp/$vm<br>
+ lockfile -1 -r 30 -l 10 $lfile<br>
+ if [ "$used" == "0" ] && ! VBoxManage list runningvms | awk -F '"' '{print $2}' | grep "^${vm}$" ; then<br>
+ NAME=$vm<br>
+ echo "Found free VM: $NAME"<br>
+ break<br>
+ fi<br>
+ done<br>
+ if [ "$NAME" == "" ]; then<br>
+ echo "WARNING: all of the appropriate VMs are in use!"<br>
+ echo "VMs: $all_vms"<br>
+ fi<br>
+}<br>
+<br>
+SCP_LOCAL_SOURCES=0<br>
+for i in "$@"<br>
+do<br>
+case $i in<br>
+ -s|--scp_local_sources)<br>
+ SCP_LOCAL_SOURCES=1<br>
+ ;;<br>
+ -t=*|--template=*)<br>
+ TEMPLATE="${i#*=}"<br>
+ shift # past argument=value<br>
+ ;;<br>
+ -a=*|--actions=*)<br>
+ ACTIONS="${i#*=}"<br>
+ shift # past argument=value<br>
+ ;;<br>
+ -c=*|--command=*)<br>
+ COMMAND="${i#*=}"<br>
+ shift # past argument=value<br>
+ ;;<br>
+ -h|--help)<br>
+ usage<br>
+ exit 0<br>
+ ;;<br>
+ *)<br>
+ usage<br>
+ exit 1<br>
+ ;;<br>
+esac<br>
+done<br>
+<br>
+if [[ $TEMPLATE =~ .*_.*_\* ]]; then<br>
+ while [ 1 ]; do<br>
+ choose_vm $TEMPLATE<br>
+ if [ "$NAME" != "" ]; then<br>
+ break<br>
+ fi<br>
+ echo "INFO: wait a minute to recheck for the free VM ..."<br>
+ sleep 60<br>
+ done<br>
+else<br>
+ NAME=$TEMPLATE<br>
+fi<br>
+<br>
+if [ "$ACTIONS" == "" ]; then<br>
+ ACTIONS=restore,start,login,stop<br>
+ echo "WARNING: actions to start was not set: using default '$ACTIONS'"<br>
+ echo<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*download.* ]]; then<br>
+ echo "Images drives and configuration can be downloaded from:"<br>
+ echo " <a href="https://mcs.mail.ru/app/services/storage/bucket/objects/packages/?Prefix=testing%2Fvms%2F" target="_blank">https://mcs.mail.ru/app/services/storage/bucket/objects/packages/?Prefix=testing%2Fvms%2F</a>"<br>
+<br>
+ # download the files<br>
+ for ftype in .ovf .mf -disk001.vmdk; do<br>
+ file=${NAME}$ftype<br>
+ wget -O - "<a href="https://download.tarantool.io/testing/vms/$file" target="_blank">https://download.tarantool.io/testing/vms/$file</a>" >/tmp/$file<br>
+ done<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*install.* ]]; then<br>
+ echo "Unregister the old VM with the same name"<br>
+ VBoxManage unregistervm $NAME --delete 2>/dev/null || true<br>
+<br>
+ echo "Register the VM"<br>
+ VBoxManage import /tmp/${NAME}.ovf<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*restore.* ]]; then<br>
+ echo "Restore the virtual machine snapshot"<br>
+ if ! VBoxManage snapshot $NAME restore $NAME ; then<br>
+ if ! [[ $ACTIONS =~ .*try_restore.* ]]; then<br>
+ exit 1<br>
+ fi<br>
+ echo "WARNING: the restore action failed, but it was expected - continuing"<br>
+ fi<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*start.* ]]; then<br>
+ echo "Start the virtual machine without GUI"<br>
+ VBoxManage startvm $NAME --type headless<br>
+fi<br>
+<br>
+exit_code=0<br>
+if [[ $ACTIONS =~ .*login.* ]]; then<br>
+ if [[ $NAME =~ freebsd_.* ]] ; then<br>
+ VMUSER=vagrant<br>
+ else<br>
+ VMUSER=tarantool<br>
+ fi<br>
+<br>
+ host="${VMUSER}@127.0.0.1"<br>
+ ssh="ssh $host -p "`VBoxManage showvminfo $NAME | grep "host port = " \<br>
+ | sed 's#.* host port = ##g' | sed 's#,.*##g'`<br>
+<br>
+ echo "Try to ssh into the virtual machine"<br>
+ i=0<br>
+ while [[ $i -lt 1000 ]] || exit 1 ; do<br>
+ if $ssh "exit 0" ; then<br>
+ break<br>
+ fi<br>
+ echo "The host is not ready, let's wait a little to recheck ..."<br>
+ i=$(($i+1))<br>
+ sleep 10<br>
+ done<br>
+ if [ "$SCP_LOCAL_SOURCES" == "1" ]; then<br>
+ $ssh "cd ~ && rm -rf tarantool"<br>
+ tar czf - ../tarantool | $ssh tar xzf -<br>
+ fi<br>
+ if [ "$COMMAND" == "" ]; then<br>
+ $ssh || exit_code=$?<br>
+ else<br>
+ $ssh "$COMMAND" || exit_code=$?<br>
+ fi<br>
+ echo "The task exited with code: $exit_code"<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*snapshot.* ]]; then<br>
+ echo "Take the virtual machine snapshot"<br>
+ VBoxManage snapshot $NAME delete $NAME 2>/dev/null || true<br>
+ VBoxManage snapshot $NAME take $NAME --pause<br>
+fi<br>
+<br>
+if [[ $ACTIONS =~ .*stop.* ]]; then<br>
+ VBoxManage controlvm $NAME poweroff<br>
+fi<br>
+<br>
+exit $exit_code<br>
-- <br>
2.17.1<br>
<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<br>-- <br>Alexander Tikhonov<br></BODY></HTML>