* [Tarantool-patches] [PATCH 1/4] extra: add Terraform config files
2020-09-16 7:07 [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI sergeyb
@ 2020-09-16 7:07 ` sergeyb
2020-09-18 14:20 ` Alexander V. Tikhonov
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 2/4] cmake: add targets to run Jepsen tests sergeyb
` (3 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: sergeyb @ 2020-09-16 7:07 UTC (permalink / raw)
To: tarantool-patches, avtikhon, alexander.turenko
From: Sergey Bronnikov <sergeyb@tarantool.org>
For testing Tarantool with Jepsen we use virtual machines as they provides
better resource isolation in comparison to containers. Jepsen tests may need a
single instance or a set of instances for testing cluster. To setup virtual
machines we use Terraform [1]. Patch adds a set of configuration files for
Terraform that can create required number of virtual machines in MCS and output
IP addresses to stdout.
Terraform needs some parameters before run. They are:
- id, identificator of a test stand that should be specific for this run, id
also is a part of virtual machine name
- keypair_name, name of keypair used in a cloud, public SSH key of that key pair
will be placed to virtual machine
- instance_count, number of virtual machines in a test stand
- ssh_key, SSH private key, used to access to a virtual machine
- user_name
- password
- tenant_id
- user_domain_id
These parameters can be passed via enviroment variables with TF_VAR_ prefix
(like TF_VAR_id) or via command-line parameters.
To demonstrate full lifecycle of a test stand with Terraform one needs to
perform these commands:
terraform init extra/tf
terraform apply extra/tf
terraform output instance_names
terraform output instance_ips
terraform destroy extra/tf
1. https://www.terraform.io/
Part of #5277
---
extra/tf/main.tf | 43 ++++++++++++++++++++++++++++++++++++++
| 10 +++++++++
| 49 ++++++++++++++++++++++++++++++++++++++++++++
| 8 ++++++++
4 files changed, 110 insertions(+)
create mode 100644 extra/tf/main.tf
create mode 100644 extra/tf/outputs.tf
create mode 100644 extra/tf/provider.tf
create mode 100644 extra/tf/versions.tf
diff --git a/extra/tf/main.tf b/extra/tf/main.tf
new file mode 100644
index 000000000..0b760884e
--- /dev/null
+++ b/extra/tf/main.tf
@@ -0,0 +1,43 @@
+resource "openstack_compute_instance_v2" "instance" {
+ count = var.instance_count
+
+ name = "tarantool_testing-${count.index + 1}-${var.id}"
+
+ image_name = "Ubuntu-18.04-202003"
+
+ image_id = "3f03b393-d45b-455d-bdf5-0ac4399ecf42"
+
+ flavor_name = "Basic-1-2-20"
+
+ key_pair = var.keypair_name
+
+ config_drive = true
+
+ availability_zone = "DP1"
+
+ security_groups = [
+ "default",
+ "allow-ssh",
+ "allow-tarantool"
+ ]
+
+ network {
+ name = "ext-net"
+ }
+
+ provisioner "remote-exec" {
+ inline = [
+ "sudo hostnamectl set-hostname n${count.index + 1}",
+ "sudo apt-get update"
+ ]
+ }
+
+ connection {
+ host = self.access_ip_v4
+ type = "ssh"
+ user = "ubuntu"
+ timeout = "5m"
+ agent = true
+ private_key = file(var.ssh_key_path)
+ }
+}
--git a/extra/tf/outputs.tf b/extra/tf/outputs.tf
new file mode 100644
index 000000000..5690d6b0d
--- /dev/null
+++ b/extra/tf/outputs.tf
@@ -0,0 +1,10 @@
+output "instance_names" {
+ description = "List of names assigned to the instances."
+ value = openstack_compute_instance_v2.instance.*.name
+}
+
+output "instance_ips" {
+ description = "List of public IP addresses assigned to the instances."
+ value = openstack_compute_instance_v2.instance.*.access_ip_v4
+ sensitive = true
+}
--git a/extra/tf/provider.tf b/extra/tf/provider.tf
new file mode 100644
index 000000000..cd9488ce5
--- /dev/null
+++ b/extra/tf/provider.tf
@@ -0,0 +1,49 @@
+variable "user_name" {
+ type = string
+}
+
+variable "password" {
+ type = string
+}
+
+variable "tenant_id" {
+ type = string
+}
+
+variable "user_domain_id" {
+ type = string
+}
+
+variable "keypair_name" {
+ type = string
+}
+
+variable "ssh_key_path" {
+ type = string
+}
+
+variable "instance_count" {
+ type = number
+ default = 1
+ description = "Number of instances."
+
+ validation {
+ condition = var.instance_count > 0
+ error_message = "The instance_count value must be greater than zero."
+ }
+}
+
+variable "id" {
+ type = string
+ default = null
+}
+
+provider "openstack" {
+ user_name = var.user_name
+ password = var.password
+ tenant_id = var.tenant_id
+ user_domain_id = var.user_domain_id
+ auth_url = "https://infra.mail.ru/identity/v3/"
+ region = "RegionOne"
+ use_octavia = true
+}
--git a/extra/tf/versions.tf b/extra/tf/versions.tf
new file mode 100644
index 000000000..5af3d608b
--- /dev/null
+++ b/extra/tf/versions.tf
@@ -0,0 +1,8 @@
+terraform {
+ required_providers {
+ openstack = {
+ source = "terraform-providers/openstack"
+ }
+ }
+ required_version = ">= 0.13"
+}
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 1/4] extra: add Terraform config files
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 1/4] extra: add Terraform config files sergeyb
@ 2020-09-18 14:20 ` Alexander V. Tikhonov
0 siblings, 0 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-18 14:20 UTC (permalink / raw)
To: sergeyb; +Cc: tarantool-patches
Hi Sergey, patch has only configuration data and enough comments, so
patch LGTM.
On Wed, Sep 16, 2020 at 10:07:19AM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
>
> For testing Tarantool with Jepsen we use virtual machines as they provides
> better resource isolation in comparison to containers. Jepsen tests may need a
> single instance or a set of instances for testing cluster. To setup virtual
> machines we use Terraform [1]. Patch adds a set of configuration files for
> Terraform that can create required number of virtual machines in MCS and output
> IP addresses to stdout.
>
> Terraform needs some parameters before run. They are:
>
> - id, identificator of a test stand that should be specific for this run, id
> also is a part of virtual machine name
> - keypair_name, name of keypair used in a cloud, public SSH key of that key pair
> will be placed to virtual machine
> - instance_count, number of virtual machines in a test stand
> - ssh_key, SSH private key, used to access to a virtual machine
> - user_name
> - password
> - tenant_id
> - user_domain_id
>
> These parameters can be passed via enviroment variables with TF_VAR_ prefix
> (like TF_VAR_id) or via command-line parameters.
>
> To demonstrate full lifecycle of a test stand with Terraform one needs to
> perform these commands:
>
> terraform init extra/tf
> terraform apply extra/tf
> terraform output instance_names
> terraform output instance_ips
> terraform destroy extra/tf
>
> 1. https://www.terraform.io/
>
> Part of #5277
> ---
> extra/tf/main.tf | 43 ++++++++++++++++++++++++++++++++++++++
> extra/tf/outputs.tf | 10 +++++++++
> extra/tf/provider.tf | 49 ++++++++++++++++++++++++++++++++++++++++++++
> extra/tf/versions.tf | 8 ++++++++
> 4 files changed, 110 insertions(+)
> create mode 100644 extra/tf/main.tf
> create mode 100644 extra/tf/outputs.tf
> create mode 100644 extra/tf/provider.tf
> create mode 100644 extra/tf/versions.tf
>
> diff --git a/extra/tf/main.tf b/extra/tf/main.tf
> new file mode 100644
> index 000000000..0b760884e
> --- /dev/null
> +++ b/extra/tf/main.tf
> @@ -0,0 +1,43 @@
> +resource "openstack_compute_instance_v2" "instance" {
> + count = var.instance_count
> +
> + name = "tarantool_testing-${count.index + 1}-${var.id}"
> +
> + image_name = "Ubuntu-18.04-202003"
> +
> + image_id = "3f03b393-d45b-455d-bdf5-0ac4399ecf42"
> +
> + flavor_name = "Basic-1-2-20"
> +
> + key_pair = var.keypair_name
> +
> + config_drive = true
> +
> + availability_zone = "DP1"
> +
> + security_groups = [
> + "default",
> + "allow-ssh",
> + "allow-tarantool"
> + ]
> +
> + network {
> + name = "ext-net"
> + }
> +
> + provisioner "remote-exec" {
> + inline = [
> + "sudo hostnamectl set-hostname n${count.index + 1}",
> + "sudo apt-get update"
> + ]
> + }
> +
> + connection {
> + host = self.access_ip_v4
> + type = "ssh"
> + user = "ubuntu"
> + timeout = "5m"
> + agent = true
> + private_key = file(var.ssh_key_path)
> + }
> +}
> diff --git a/extra/tf/outputs.tf b/extra/tf/outputs.tf
> new file mode 100644
> index 000000000..5690d6b0d
> --- /dev/null
> +++ b/extra/tf/outputs.tf
> @@ -0,0 +1,10 @@
> +output "instance_names" {
> + description = "List of names assigned to the instances."
> + value = openstack_compute_instance_v2.instance.*.name
> +}
> +
> +output "instance_ips" {
> + description = "List of public IP addresses assigned to the instances."
> + value = openstack_compute_instance_v2.instance.*.access_ip_v4
> + sensitive = true
> +}
> diff --git a/extra/tf/provider.tf b/extra/tf/provider.tf
> new file mode 100644
> index 000000000..cd9488ce5
> --- /dev/null
> +++ b/extra/tf/provider.tf
> @@ -0,0 +1,49 @@
> +variable "user_name" {
> + type = string
> +}
> +
> +variable "password" {
> + type = string
> +}
> +
> +variable "tenant_id" {
> + type = string
> +}
> +
> +variable "user_domain_id" {
> + type = string
> +}
> +
> +variable "keypair_name" {
> + type = string
> +}
> +
> +variable "ssh_key_path" {
> + type = string
> +}
> +
> +variable "instance_count" {
> + type = number
> + default = 1
> + description = "Number of instances."
> +
> + validation {
> + condition = var.instance_count > 0
> + error_message = "The instance_count value must be greater than zero."
> + }
> +}
> +
> +variable "id" {
> + type = string
> + default = null
> +}
> +
> +provider "openstack" {
> + user_name = var.user_name
> + password = var.password
> + tenant_id = var.tenant_id
> + user_domain_id = var.user_domain_id
> + auth_url = "https://infra.mail.ru/identity/v3/"
> + region = "RegionOne"
> + use_octavia = true
> +}
> diff --git a/extra/tf/versions.tf b/extra/tf/versions.tf
> new file mode 100644
> index 000000000..5af3d608b
> --- /dev/null
> +++ b/extra/tf/versions.tf
> @@ -0,0 +1,8 @@
> +terraform {
> + required_providers {
> + openstack = {
> + source = "terraform-providers/openstack"
> + }
> + }
> + required_version = ">= 0.13"
> +}
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH 2/4] cmake: add targets to run Jepsen tests
2020-09-16 7:07 [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI sergeyb
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 1/4] extra: add Terraform config files sergeyb
@ 2020-09-16 7:07 ` sergeyb
2020-09-18 14:23 ` Alexander V. Tikhonov
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 3/4] tools: add script " sergeyb
` (2 subsequent siblings)
4 siblings, 1 reply; 14+ messages in thread
From: sergeyb @ 2020-09-16 7:07 UTC (permalink / raw)
To: tarantool-patches, avtikhon, alexander.turenko
From: Sergey Bronnikov <sergeyb@tarantool.org>
Added targets 'make jepsen-single' and 'make jepsen-cluster'
to run Jepsen tests on a single Tarantool instance and
cluster of Tarantool instances.
Part of #5277
---
CMakeLists.txt | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 695e80c21..e8850e320 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -17,9 +17,11 @@ include(TestBigEndian)
include(CheckFunctionExists)
include(FindOptionalPackage)
include(FindPackageMessage)
+include(ExternalProject)
find_program(ECHO echo)
find_program(CAT cat)
+find_program(BASH bash)
find_program(GIT git)
find_program(LD ld)
find_program(CTAGS ctags)
@@ -162,6 +164,31 @@ add_custom_command(TARGET luacheck
COMMENT "Perform static analysis of Lua code"
)
+ExternalProject_Add(
+ jepsen-tests
+ GIT_REPOSITORY https://github.com/tarantool/jepsen.tarantool
+ CONFIGURE_COMMAND ""
+ BUILD_COMMAND ""
+ INSTALL_COMMAND ""
+ TEST_COMMAND ""
+)
+
+#
+# Enable 'make jepsen-*' targets.
+#
+
+add_custom_target(jepsen-single DEPENDS jepsen-tests)
+add_custom_command(TARGET jepsen-single
+ COMMAND ${BASH} ${PROJECT_SOURCE_DIR}/tools/run-jepsen-tests.sh ${PROJECT_SOURCE_DIR} ${PROJECT_BINARY_DIR}
+ COMMENT "Running Jepsen tests on a single Tarantool instance"
+)
+
+add_custom_target(jepsen-cluster DEPENDS jepsen-tests)
+add_custom_command(TARGET jepsen-cluster
+ COMMAND ${BASH} ${PROJECT_SOURCE_DIR}/tools/run-jepsen-tests.sh ${PROJECT_SOURCE_DIR} ${PROJECT_BINARY_DIR} 5
+ COMMENT "Running Jepsen tests on a cluster with 5 Tarantool instances"
+)
+
#
# Get version
#
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 2/4] cmake: add targets to run Jepsen tests
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 2/4] cmake: add targets to run Jepsen tests sergeyb
@ 2020-09-18 14:23 ` Alexander V. Tikhonov
0 siblings, 0 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-18 14:23 UTC (permalink / raw)
To: sergeyb; +Cc: tarantool-patches
Hi Sergey, thanks for the patch. It has 2 make targets which will be
usable for tests reproduce. Later I hope we'll make some more
instructions on its use, but not now, so patch LGTM.
On Wed, Sep 16, 2020 at 10:07:21AM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
>
> Added targets 'make jepsen-single' and 'make jepsen-cluster'
> to run Jepsen tests on a single Tarantool instance and
> cluster of Tarantool instances.
>
> Part of #5277
> ---
> CMakeLists.txt | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
>
> diff --git a/CMakeLists.txt b/CMakeLists.txt
> index 695e80c21..e8850e320 100644
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -17,9 +17,11 @@ include(TestBigEndian)
> include(CheckFunctionExists)
> include(FindOptionalPackage)
> include(FindPackageMessage)
> +include(ExternalProject)
>
> find_program(ECHO echo)
> find_program(CAT cat)
> +find_program(BASH bash)
> find_program(GIT git)
> find_program(LD ld)
> find_program(CTAGS ctags)
> @@ -162,6 +164,31 @@ add_custom_command(TARGET luacheck
> COMMENT "Perform static analysis of Lua code"
> )
>
> +ExternalProject_Add(
> + jepsen-tests
> + GIT_REPOSITORY https://github.com/tarantool/jepsen.tarantool
> + CONFIGURE_COMMAND ""
> + BUILD_COMMAND ""
> + INSTALL_COMMAND ""
> + TEST_COMMAND ""
> +)
> +
> +#
> +# Enable 'make jepsen-*' targets.
> +#
> +
> +add_custom_target(jepsen-single DEPENDS jepsen-tests)
> +add_custom_command(TARGET jepsen-single
> + COMMAND ${BASH} ${PROJECT_SOURCE_DIR}/tools/run-jepsen-tests.sh ${PROJECT_SOURCE_DIR} ${PROJECT_BINARY_DIR}
> + COMMENT "Running Jepsen tests on a single Tarantool instance"
> +)
> +
> +add_custom_target(jepsen-cluster DEPENDS jepsen-tests)
> +add_custom_command(TARGET jepsen-cluster
> + COMMAND ${BASH} ${PROJECT_SOURCE_DIR}/tools/run-jepsen-tests.sh ${PROJECT_SOURCE_DIR} ${PROJECT_BINARY_DIR} 5
> + COMMENT "Running Jepsen tests on a cluster with 5 Tarantool instances"
> +)
> +
> #
> # Get version
> #
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH 3/4] tools: add script to run Jepsen tests
2020-09-16 7:07 [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI sergeyb
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 1/4] extra: add Terraform config files sergeyb
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 2/4] cmake: add targets to run Jepsen tests sergeyb
@ 2020-09-16 7:07 ` sergeyb
2020-09-18 14:32 ` Alexander V. Tikhonov
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI sergeyb
2020-09-18 15:03 ` [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI Kirill Yukhin
4 siblings, 1 reply; 14+ messages in thread
From: sergeyb @ 2020-09-16 7:07 UTC (permalink / raw)
To: tarantool-patches, avtikhon, alexander.turenko
From: Sergey Bronnikov <sergeyb@tarantool.org>
Main script that handle creation of set of virtual machines
using Terraform, setup for remote connection, running
Jepsen tests and teardown test environment.
Part of #5277
---
tools/run-jepsen-tests.sh | 77 +++++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)
create mode 100755 tools/run-jepsen-tests.sh
diff --git a/tools/run-jepsen-tests.sh b/tools/run-jepsen-tests.sh
new file mode 100755
index 000000000..865a8f4b0
--- /dev/null
+++ b/tools/run-jepsen-tests.sh
@@ -0,0 +1,77 @@
+#!/usr/bin/bash
+
+# Script performs setup of test environment using Terraform,
+# runs Jepsen tests and teardown test environment.
+# Script expects following environment variables:
+# TF_VAR_user_name
+# TF_VAR_password
+# TF_VAR_tenant_id
+# TF_VAR_user_domain_id
+# TF_VAR_keypair_name
+# TF_VAR_ssh_key
+# and three arguments: path to a Tarantool project source directory, path to a
+# Tarantool project binary directory and number on instances (optional).
+
+set -Eeo pipefail
+
+PROJECT_SOURCE_DIR=$1
+PROJECT_BINARY_DIR=$2
+INSTANCE_COUNT=$3
+
+TESTS_DIR="$PROJECT_BINARY_DIR/jepsen-tests-prefix/src/jepsen-tests/"
+TERRAFORM_CONFIG="$PROJECT_SOURCE_DIR/extra/tf"
+TERRAFORM_STATE="$PROJECT_BINARY_DIR/terraform.tfstate"
+SSH_KEY_FILENAME="$PROJECT_SOURCE_DIR/build/tf-cloud-init"
+NODES_FILENAME="$PROJECT_SOURCE_DIR/build/nodes"
+
+LEIN_OPTIONS="--nodes-file $NODES_FILENAME --username ubuntu --workload register"
+
+[[ -z ${PROJECT_SOURCE_DIR} ]] && (echo "Please specify path to a project source directory")
+[[ -z ${PROJECT_BINARY_DIR} ]] && (echo "Please specify path to a project binary directory")
+[[ -z ${TF_VAR_ssh_key} ]] && (echo "Please specify TF_VAR_ssh_key env var"; exit 1)
+[[ -z ${TF_VAR_keypair_name} ]] && (echo "Please specify TF_VAR_keypair_name env var"; exit 1)
+
+TERRAFORM_BIN=$(which terraform)
+LEIN_BIN=$(which lein)
+CLOJURE_BIN=$(which clojure)
+
+[[ -z ${TERRAFORM_BIN} ]] && (echo "terraform is not installed"; exit 1)
+[[ -z ${LEIN_BIN} ]] && (echo "lein is not installed"; exit 1)
+[[ -z ${CLOJURE_BIN} ]] && (echo "clojure is not installed"; exit 1)
+
+function cleanup {
+ echo "cleanup"
+ rm -f $NODES_FILENAME $SSH_KEY_FILENAME
+ [[ -e $TERRAFORM_STATE ]] && \
+ terraform destroy -state=$TERRAFORM_STATE -auto-approve $TERRAFORM_CONFIG
+}
+
+trap "{ cleanup; exit 255; }" SIGINT SIGTERM ERR
+
+echo -e "${TF_VAR_ssh_key//_/\\n}" > $SSH_KEY_FILENAME
+chmod 400 $SSH_KEY_FILENAME
+$(pgrep ssh-agent 2>&1 > /dev/null) || eval "$(ssh-agent)"
+ssh-add $SSH_KEY_FILENAME
+
+if [[ -z ${CI_JOB_ID} ]]; then
+ TF_VAR_id=$CI_JOB_ID
+else
+ RANDOM_ID=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 13; echo '')
+ TF_VAR_id=TF-$RANDOM_ID
+fi
+
+export TF_VAR_id
+export TF_VAR_ssh_key_path=$SSH_KEY_FILENAME
+[[ -z ${INSTANCE_COUNT} ]] && TF_VAR_instance_count=1 || TF_VAR_instance_count=$INSTANCE_COUNT
+export TF_VAR_instance_count
+
+[[ -e $HOME/.ssh ]] || (mkdir $HOME/.ssh && touch $HOME/.ssh/known_hosts)
+
+terraform init $TERRAFORM_CONFIG
+terraform apply -state=$TERRAFORM_STATE -auto-approve $TERRAFORM_CONFIG
+terraform providers $TERRAFORM_CONFIG
+terraform output -state=$TERRAFORM_STATE instance_names
+terraform output -state=$TERRAFORM_STATE -json instance_ips | jq --raw-output '.[]' > $NODES_FILENAME
+pushd $TESTS_DIR && lein run test $LEIN_OPTIONS
+popd
+cleanup
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 3/4] tools: add script to run Jepsen tests
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 3/4] tools: add script " sergeyb
@ 2020-09-18 14:32 ` Alexander V. Tikhonov
0 siblings, 0 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-18 14:32 UTC (permalink / raw)
To: sergeyb; +Cc: tarantool-patches
Hi Sergey, thanks a lot for the patch. IMHO the script is on bash and
its great for the initiating the new feature, because it is really easy
to fix on any needs, so patch LGTM.
On Wed, Sep 16, 2020 at 10:07:22AM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
>
> Main script that handle creation of set of virtual machines
> using Terraform, setup for remote connection, running
> Jepsen tests and teardown test environment.
>
> Part of #5277
> ---
> tools/run-jepsen-tests.sh | 77 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 77 insertions(+)
> create mode 100755 tools/run-jepsen-tests.sh
>
> diff --git a/tools/run-jepsen-tests.sh b/tools/run-jepsen-tests.sh
> new file mode 100755
> index 000000000..865a8f4b0
> --- /dev/null
> +++ b/tools/run-jepsen-tests.sh
> @@ -0,0 +1,77 @@
> +#!/usr/bin/bash
> +
> +# Script performs setup of test environment using Terraform,
> +# runs Jepsen tests and teardown test environment.
> +# Script expects following environment variables:
> +# TF_VAR_user_name
> +# TF_VAR_password
> +# TF_VAR_tenant_id
> +# TF_VAR_user_domain_id
> +# TF_VAR_keypair_name
> +# TF_VAR_ssh_key
> +# and three arguments: path to a Tarantool project source directory, path to a
> +# Tarantool project binary directory and number on instances (optional).
> +
> +set -Eeo pipefail
> +
> +PROJECT_SOURCE_DIR=$1
> +PROJECT_BINARY_DIR=$2
> +INSTANCE_COUNT=$3
> +
> +TESTS_DIR="$PROJECT_BINARY_DIR/jepsen-tests-prefix/src/jepsen-tests/"
> +TERRAFORM_CONFIG="$PROJECT_SOURCE_DIR/extra/tf"
> +TERRAFORM_STATE="$PROJECT_BINARY_DIR/terraform.tfstate"
> +SSH_KEY_FILENAME="$PROJECT_SOURCE_DIR/build/tf-cloud-init"
> +NODES_FILENAME="$PROJECT_SOURCE_DIR/build/nodes"
> +
> +LEIN_OPTIONS="--nodes-file $NODES_FILENAME --username ubuntu --workload register"
> +
> +[[ -z ${PROJECT_SOURCE_DIR} ]] && (echo "Please specify path to a project source directory")
> +[[ -z ${PROJECT_BINARY_DIR} ]] && (echo "Please specify path to a project binary directory")
> +[[ -z ${TF_VAR_ssh_key} ]] && (echo "Please specify TF_VAR_ssh_key env var"; exit 1)
> +[[ -z ${TF_VAR_keypair_name} ]] && (echo "Please specify TF_VAR_keypair_name env var"; exit 1)
> +
> +TERRAFORM_BIN=$(which terraform)
> +LEIN_BIN=$(which lein)
> +CLOJURE_BIN=$(which clojure)
> +
> +[[ -z ${TERRAFORM_BIN} ]] && (echo "terraform is not installed"; exit 1)
> +[[ -z ${LEIN_BIN} ]] && (echo "lein is not installed"; exit 1)
> +[[ -z ${CLOJURE_BIN} ]] && (echo "clojure is not installed"; exit 1)
> +
> +function cleanup {
> + echo "cleanup"
> + rm -f $NODES_FILENAME $SSH_KEY_FILENAME
> + [[ -e $TERRAFORM_STATE ]] && \
> + terraform destroy -state=$TERRAFORM_STATE -auto-approve $TERRAFORM_CONFIG
> +}
> +
> +trap "{ cleanup; exit 255; }" SIGINT SIGTERM ERR
> +
> +echo -e "${TF_VAR_ssh_key//_/\\n}" > $SSH_KEY_FILENAME
> +chmod 400 $SSH_KEY_FILENAME
> +$(pgrep ssh-agent 2>&1 > /dev/null) || eval "$(ssh-agent)"
> +ssh-add $SSH_KEY_FILENAME
> +
> +if [[ -z ${CI_JOB_ID} ]]; then
> + TF_VAR_id=$CI_JOB_ID
> +else
> + RANDOM_ID=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 13; echo '')
> + TF_VAR_id=TF-$RANDOM_ID
> +fi
> +
> +export TF_VAR_id
> +export TF_VAR_ssh_key_path=$SSH_KEY_FILENAME
> +[[ -z ${INSTANCE_COUNT} ]] && TF_VAR_instance_count=1 || TF_VAR_instance_count=$INSTANCE_COUNT
> +export TF_VAR_instance_count
> +
> +[[ -e $HOME/.ssh ]] || (mkdir $HOME/.ssh && touch $HOME/.ssh/known_hosts)
> +
> +terraform init $TERRAFORM_CONFIG
> +terraform apply -state=$TERRAFORM_STATE -auto-approve $TERRAFORM_CONFIG
> +terraform providers $TERRAFORM_CONFIG
> +terraform output -state=$TERRAFORM_STATE instance_names
> +terraform output -state=$TERRAFORM_STATE -json instance_ips | jq --raw-output '.[]' > $NODES_FILENAME
> +pushd $TESTS_DIR && lein run test $LEIN_OPTIONS
> +popd
> +cleanup
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-16 7:07 [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI sergeyb
` (2 preceding siblings ...)
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 3/4] tools: add script " sergeyb
@ 2020-09-16 7:07 ` sergeyb
2020-09-17 7:12 ` Alexander V. Tikhonov
2020-09-18 15:03 ` [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI Kirill Yukhin
4 siblings, 1 reply; 14+ messages in thread
From: sergeyb @ 2020-09-16 7:07 UTC (permalink / raw)
To: tarantool-patches, avtikhon, alexander.turenko
From: Sergey Bronnikov <sergeyb@tarantool.org>
added a new stage with a single job to run Jepsen tests.
Job is not started automatically by default, one need to
trigger it manually. Directory with test results
(logs, graphs, operations history) published to artifacts.
Closes #5277
---
.gitlab-ci.yml | 14 ++++++++++++++
.travis.mk | 34 +++++++++++++++++++++++++++++++++-
2 files changed, 47 insertions(+), 1 deletion(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 05d40b013..3afcfc245 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,6 +1,7 @@
stages:
- static_analysis
- test
+ - long_tests
- perf
- cleanup
@@ -210,6 +211,19 @@ freebsd_12_release:
- ${GITLAB_MAKE} vms_start
- ${GITLAB_MAKE} vms_test_freebsd
+jepsen:
+ <<: *docker_test_definition
+ script:
+ - ${GITLAB_MAKE} test_jepsen
+ stage: long_tests
+ when: manual
+ tags:
+ - mcs_jepsen_docker
+ artifacts:
+ paths:
+ - jepsen-tests-prefix/src/jepsen-tests/store
+ expire_in: 1 week
+
# ####
# Perf
# ####
diff --git a/.travis.mk b/.travis.mk
index efc05cf05..781b37c15 100644
--- a/.travis.mk
+++ b/.travis.mk
@@ -8,6 +8,12 @@ TEST_RUN_EXTRA_PARAMS?=
MAX_FILES?=65534
MAX_PROC?=2500
OOS_SRC_PATH="/source"
+BIN_DIR=/usr/local/bin
+
+CLOJURE_URL="https://download.clojure.org/install/linux-install-1.10.1.561.sh"
+LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein"
+TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip"
+TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME)
all: package
@@ -60,7 +66,7 @@ docker_%:
# commit, so the build requires old dependencies to be installed.
# See ce623a23416eb192ce70116fd14992e84e7ccbbe ('Enable GitLab CI
# testing') for more information.
-deps_debian:
+deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
build-essential cmake coreutils sed \
libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
@@ -69,6 +75,11 @@ deps_debian:
python-msgpack python-yaml python-argparse python-six python-gevent \
lcov ruby clang llvm llvm-dev zlib1g-dev autoconf automake libtool
+deps_debian_jepsen:
+ apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
+ openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
+ zip unzip openssh-client jq
+
deps_buster_clang_8: deps_debian
echo "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" > /etc/apt/sources.list.d/clang_8.list
echo "deb-src http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" >> /etc/apt/sources.list.d/clang_8.list
@@ -76,6 +87,19 @@ deps_buster_clang_8: deps_debian
apt-get update
apt-get install -y clang-8 llvm-8-dev
+$(BIN_DIR)/clojure: deps_debian_jepsen
+ curl $(CLOJURE_URL) | sudo bash
+
+$(BIN_DIR)/lein: deps_debian_jepsen
+ curl $(LEIN_URL) > $@
+ chmod a+x $@
+ $@ version
+
+$(BIN_DIR)/terraform: deps_debian_jepsen
+ curl -O $(TERRAFORM_URL)
+ unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
+ rm -f $(TERRAFORM_NAME)
+
# Release
configure_debian:
@@ -232,3 +256,11 @@ test_freebsd_no_deps: build_freebsd
cd test && python2.7 test-run.py --force $(TEST_RUN_EXTRA_PARAMS)
test_freebsd: deps_freebsd test_freebsd_no_deps
+
+# ###################
+# Jepsen testing
+# ###################
+
+test_jepsen: deps_debian
+ cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
+ make jepsen-single
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI sergeyb
@ 2020-09-17 7:12 ` Alexander V. Tikhonov
2020-09-17 15:25 ` Alexander Turenko
2020-09-18 12:33 ` Sergey Bronnikov
0 siblings, 2 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-17 7:12 UTC (permalink / raw)
To: sergeyb; +Cc: tarantool-patches, Alexander Turenko
Hi Sergey, thanks for the patch, please check my comments below.
On Wed, Sep 16, 2020 at 10:07:24AM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
>
> added a new stage with a single job to run Jepsen tests.
> Job is not started automatically by default, one need to
> trigger it manually. Directory with test results
> (logs, graphs, operations history) published to artifacts.
>
> Closes #5277
> ---
> .gitlab-ci.yml | 14 ++++++++++++++
> .travis.mk | 34 +++++++++++++++++++++++++++++++++-
> 2 files changed, 47 insertions(+), 1 deletion(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 05d40b013..3afcfc245 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,6 +1,7 @@
> stages:
> - static_analysis
> - test
> + - long_tests
> - perf
> - cleanup
>
May be, we may want to merge long_tests and perf stages to some single
common stage ?
> @@ -210,6 +211,19 @@ freebsd_12_release:
> - ${GITLAB_MAKE} vms_start
> - ${GITLAB_MAKE} vms_test_freebsd
>
> +jepsen:
> + <<: *docker_test_definition
> + script:
> + - ${GITLAB_MAKE} test_jepsen
> + stage: long_tests
> + when: manual
> + tags:
> + - mcs_jepsen_docker
> + artifacts:
> + paths:
> + - jepsen-tests-prefix/src/jepsen-tests/store
> + expire_in: 1 week
> +
Let's use more time for artifacts store, may be 6 months.
> # ####
> # Perf
> # ####
> diff --git a/.travis.mk b/.travis.mk
> index efc05cf05..781b37c15 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -8,6 +8,12 @@ TEST_RUN_EXTRA_PARAMS?=
> MAX_FILES?=65534
> MAX_PROC?=2500
> OOS_SRC_PATH="/source"
> +BIN_DIR=/usr/local/bin
> +
> +CLOJURE_URL="https://download.clojure.org/install/linux-install-1.10.1.561.sh"
> +LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein"
> +TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip"
> +TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME)
>
> all: package
>
> @@ -60,7 +66,7 @@ docker_%:
> # commit, so the build requires old dependencies to be installed.
> # See ce623a23416eb192ce70116fd14992e84e7ccbbe ('Enable GitLab CI
> # testing') for more information.
> -deps_debian:
> +deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
> apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> build-essential cmake coreutils sed \
> libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
I don't think we need to add jepsen's new targets to the base one, let's
move it to 'deps_debian_jepsen'.
> @@ -69,6 +75,11 @@ deps_debian:
> python-msgpack python-yaml python-argparse python-six python-gevent \
> lcov ruby clang llvm llvm-dev zlib1g-dev autoconf automake libtool
>
> +deps_debian_jepsen:
> + apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> + openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
> + zip unzip openssh-client jq
> +
> deps_buster_clang_8: deps_debian
> echo "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" > /etc/apt/sources.list.d/clang_8.list
> echo "deb-src http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" >> /etc/apt/sources.list.d/clang_8.list
> @@ -76,6 +87,19 @@ deps_buster_clang_8: deps_debian
> apt-get update
> apt-get install -y clang-8 llvm-8-dev
>
> +$(BIN_DIR)/clojure: deps_debian_jepsen
> + curl $(CLOJURE_URL) | sudo bash
> +
> +$(BIN_DIR)/lein: deps_debian_jepsen
> + curl $(LEIN_URL) > $@
> + chmod a+x $@
> + $@ version
> +
> +$(BIN_DIR)/terraform: deps_debian_jepsen
> + curl -O $(TERRAFORM_URL)
> + unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
> + rm -f $(TERRAFORM_NAME)
> +
Do we really need standalone targets that will not be used in separate?
This file is in makefile style, but not makefile in real, it has
standalone targets that provides some process meanings, like:
- prepare deps
- build
- test
> # Release
>
> configure_debian:
> @@ -232,3 +256,11 @@ test_freebsd_no_deps: build_freebsd
> cd test && python2.7 test-run.py --force $(TEST_RUN_EXTRA_PARAMS)
>
> test_freebsd: deps_freebsd test_freebsd_no_deps
> +
> +# ###################
> +# Jepsen testing
> +# ###################
> +
> +test_jepsen: deps_debian
> + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
> + make jepsen-single
Let's use deps_debian_jepsen here.
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-17 7:12 ` Alexander V. Tikhonov
@ 2020-09-17 15:25 ` Alexander Turenko
2020-09-18 12:28 ` Sergey Bronnikov
2020-09-18 12:33 ` Sergey Bronnikov
1 sibling, 1 reply; 14+ messages in thread
From: Alexander Turenko @ 2020-09-17 15:25 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches
> > +jepsen:
> > + <<: *docker_test_definition
> > + script:
> > + - ${GITLAB_MAKE} test_jepsen
> > + stage: long_tests
> > + when: manual
> > + tags:
> > + - mcs_jepsen_docker
> > + artifacts:
> > + paths:
> > + - jepsen-tests-prefix/src/jepsen-tests/store
> > + expire_in: 1 week
> > +
>
> Let's use more time for artifacts store, may be 6 months.
I agree. We sometimes return to bad results after some significant
delay.
> > -deps_debian:
> > +deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
> > apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> > build-essential cmake coreutils sed \
> > libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
>
> I don't think we need to add jepsen's new targets to the base one, let's
> move it to 'deps_debian_jepsen'.
I agree, it is a bit strange to install jepsen dependencies for all jobs
(when a separate jepsen goal exists at least).
> > +deps_debian_jepsen:
> > + apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> > + openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
> > + zip unzip openssh-client jq
> > +
> > +$(BIN_DIR)/clojure: deps_debian_jepsen
> > + curl $(CLOJURE_URL) | sudo bash
> > +
> > +$(BIN_DIR)/lein: deps_debian_jepsen
> > + curl $(LEIN_URL) > $@
> > + chmod a+x $@
> > + $@ version
> > +
> > +$(BIN_DIR)/terraform: deps_debian_jepsen
> > + curl -O $(TERRAFORM_URL)
> > + unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
> > + rm -f $(TERRAFORM_NAME)
> > +
>
> Do we really need standalone targets that will not be used in separate?
> This file is in makefile style, but not makefile in real, it has
> standalone targets that provides some process meanings, like:
> - prepare deps
> - build
> - test
In my understanding it is Makefile anyway and it is usual for a Makefile
to use a file name as a goal (so the goal will be considered as made if
the file exists, if the goal is not marked as .PHONY).
> > +
> > +# ###################
> > +# Jepsen testing
> > +# ###################
> > +
> > +test_jepsen: deps_debian
> > + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
> > + make jepsen-single
>
> Let's use deps_debian_jepsen here.
I would do it this way, yep.
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-17 15:25 ` Alexander Turenko
@ 2020-09-18 12:28 ` Sergey Bronnikov
2020-09-18 12:34 ` Alexander V. Tikhonov
0 siblings, 1 reply; 14+ messages in thread
From: Sergey Bronnikov @ 2020-09-18 12:28 UTC (permalink / raw)
To: Alexander Turenko, Alexander V. Tikhonov; +Cc: tarantool-patches
Hello!
Thanks for review! I made some changes according to your comments
and pushed them to the branch.
CI: https://gitlab.com/tarantool/tarantool/-/pipelines/191621859
On 17.09.2020 18:25, Alexander Turenko wrote:
>>> +jepsen:
>>> + <<: *docker_test_definition
>>> + script:
>>> + - ${GITLAB_MAKE} test_jepsen
>>> + stage: long_tests
>>> + when: manual
>>> + tags:
>>> + - mcs_jepsen_docker
>>> + artifacts:
>>> + paths:
>>> + - jepsen-tests-prefix/src/jepsen-tests/store
>>> + expire_in: 1 week
>>> +
>> Let's use more time for artifacts store, may be 6 months.
> I agree. We sometimes return to bad results after some significant
> delay.
increased to 6 months:
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -222,7 +222,7 @@ jepsen:
artifacts:
paths:
- jepsen-tests-prefix/src/jepsen-tests/store
- expire_in: 1 week
+ expire_in: 6 month
# ####
# Perf
>
>>> -deps_debian:
>>> +deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
>>> apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
>>> build-essential cmake coreutils sed \
>>> libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
>> I don't think we need to add jepsen's new targets to the base one, let's
>> move it to 'deps_debian_jepsen'.
> I agree, it is a bit strange to install jepsen dependencies for all jobs
> (when a separate jepsen goal exists at least).
>
Agree. Removed jepsen dependencies for deps_debian target.
>>> +deps_debian_jepsen:
>>> + apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
>>> + openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
>>> + zip unzip openssh-client jq
>>> +
>>> +$(BIN_DIR)/clojure: deps_debian_jepsen
>>> + curl $(CLOJURE_URL) | sudo bash
>>> +
>>> +$(BIN_DIR)/lein: deps_debian_jepsen
>>> + curl $(LEIN_URL) > $@
>>> + chmod a+x $@
>>> + $@ version
>>> +
>>> +$(BIN_DIR)/terraform: deps_debian_jepsen
>>> + curl -O $(TERRAFORM_URL)
>>> + unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
>>> + rm -f $(TERRAFORM_NAME)
>>> +
>> Do we really need standalone targets that will not be used in separate?
>> This file is in makefile style, but not makefile in real, it has
>> standalone targets that provides some process meanings, like:
>> - prepare deps
>> - build
>> - test
> In my understanding it is Makefile anyway and it is usual for a Makefile
> to use a file name as a goal (so the goal will be considered as made if
> the file exists, if the goal is not marked as .PHONY).
>
>>> +
>>> +# ###################
>>> +# Jepsen testing
>>> +# ###################
>>> +
>>> +test_jepsen: deps_debian
>>> + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
>>> + make jepsen-single
>> Let's use deps_debian_jepsen here.
Done.
> I would do it this way, yep.
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-18 12:28 ` Sergey Bronnikov
@ 2020-09-18 12:34 ` Alexander V. Tikhonov
0 siblings, 0 replies; 14+ messages in thread
From: Alexander V. Tikhonov @ 2020-09-18 12:34 UTC (permalink / raw)
To: Sergey Bronnikov; +Cc: tarantool-patches
Hi Sergey, thanks for the updates, patch LGTM.
On Fri, Sep 18, 2020 at 03:28:49PM +0300, Sergey Bronnikov wrote:
> Hello!
>
> Thanks for review! I made some changes according to your comments
>
> and pushed them to the branch.
>
> CI: https://gitlab.com/tarantool/tarantool/-/pipelines/191621859
>
>
> On 17.09.2020 18:25, Alexander Turenko wrote:
> > > > +jepsen:
> > > > + <<: *docker_test_definition
> > > > + script:
> > > > + - ${GITLAB_MAKE} test_jepsen
> > > > + stage: long_tests
> > > > + when: manual
> > > > + tags:
> > > > + - mcs_jepsen_docker
> > > > + artifacts:
> > > > + paths:
> > > > + - jepsen-tests-prefix/src/jepsen-tests/store
> > > > + expire_in: 1 week
> > > > +
> > > Let's use more time for artifacts store, may be 6 months.
> > I agree. We sometimes return to bad results after some significant
> > delay.
>
> increased to 6 months:
>
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -222,7 +222,7 @@ jepsen:
> artifacts:
> paths:
> - jepsen-tests-prefix/src/jepsen-tests/store
> - expire_in: 1 week
> + expire_in: 6 month
>
> # ####
> # Perf
>
> >
> > > > -deps_debian:
> > > > +deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
> > > > apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> > > > build-essential cmake coreutils sed \
> > > > libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
> > > I don't think we need to add jepsen's new targets to the base one, let's
> > > move it to 'deps_debian_jepsen'.
> > I agree, it is a bit strange to install jepsen dependencies for all jobs
> > (when a separate jepsen goal exists at least).
> >
> Agree. Removed jepsen dependencies for deps_debian target.
>
>
> > > > +deps_debian_jepsen:
> > > > + apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
> > > > + openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
> > > > + zip unzip openssh-client jq
> > > > +
> > > > +$(BIN_DIR)/clojure: deps_debian_jepsen
> > > > + curl $(CLOJURE_URL) | sudo bash
> > > > +
> > > > +$(BIN_DIR)/lein: deps_debian_jepsen
> > > > + curl $(LEIN_URL) > $@
> > > > + chmod a+x $@
> > > > + $@ version
> > > > +
> > > > +$(BIN_DIR)/terraform: deps_debian_jepsen
> > > > + curl -O $(TERRAFORM_URL)
> > > > + unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
> > > > + rm -f $(TERRAFORM_NAME)
> > > > +
> > > Do we really need standalone targets that will not be used in separate?
> > > This file is in makefile style, but not makefile in real, it has
> > > standalone targets that provides some process meanings, like:
> > > - prepare deps
> > > - build
> > > - test
> > In my understanding it is Makefile anyway and it is usual for a Makefile
> > to use a file name as a goal (so the goal will be considered as made if
> > the file exists, if the goal is not marked as .PHONY).
> >
> > > > +
> > > > +# ###################
> > > > +# Jepsen testing
> > > > +# ###################
> > > > +
> > > > +test_jepsen: deps_debian
> > > > + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
> > > > + make jepsen-single
> > > Let's use deps_debian_jepsen here.
>
> Done.
>
>
> > I would do it this way, yep.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI
2020-09-17 7:12 ` Alexander V. Tikhonov
2020-09-17 15:25 ` Alexander Turenko
@ 2020-09-18 12:33 ` Sergey Bronnikov
1 sibling, 0 replies; 14+ messages in thread
From: Sergey Bronnikov @ 2020-09-18 12:33 UTC (permalink / raw)
To: Alexander V. Tikhonov; +Cc: tarantool-patches, Alexander Turenko
Hello!
Thanks for review!
Please see my comments inline. I made some changes and pushed them to
the branch.
CI: https://gitlab.com/tarantool/tarantool/-/pipelines/191621859
On 17.09.2020 10:12, Alexander V. Tikhonov wrote:
> Hi Sergey, thanks for the patch, please check my comments below.
>
> On Wed, Sep 16, 2020 at 10:07:24AM +0300, sergeyb@tarantool.org wrote:
>> From: Sergey Bronnikov <sergeyb@tarantool.org>
>>
>> added a new stage with a single job to run Jepsen tests.
>> Job is not started automatically by default, one need to
>> trigger it manually. Directory with test results
>> (logs, graphs, operations history) published to artifacts.
>>
>> Closes #5277
>> ---
>> .gitlab-ci.yml | 14 ++++++++++++++
>> .travis.mk | 34 +++++++++++++++++++++++++++++++++-
>> 2 files changed, 47 insertions(+), 1 deletion(-)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 05d40b013..3afcfc245 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -1,6 +1,7 @@
>> stages:
>> - static_analysis
>> - test
>> + - long_tests
>> - perf
>> - cleanup
>>
> May be, we may want to merge long_tests and perf stages to some single
> common stage ?
Good idea. I believe it's a topic to discuss and we can combine stages
to a single one in scope of [1]. Right now performance jobs are not
included to a pipeline by default.
1. https://github.com/tarantool/tarantool/issues/5305
>
>> @@ -210,6 +211,19 @@ freebsd_12_release:
>> - ${GITLAB_MAKE} vms_start
>> - ${GITLAB_MAKE} vms_test_freebsd
>>
>> +jepsen:
>> + <<: *docker_test_definition
>> + script:
>> + - ${GITLAB_MAKE} test_jepsen
>> + stage: long_tests
>> + when: manual
>> + tags:
>> + - mcs_jepsen_docker
>> + artifacts:
>> + paths:
>> + - jepsen-tests-prefix/src/jepsen-tests/store
>> + expire_in: 1 week
>> +
> Let's use more time for artifacts store, may be 6 months.
Updated in a branch.
>
>> # ####
>> # Perf
>> # ####
>> diff --git a/.travis.mk b/.travis.mk
>> index efc05cf05..781b37c15 100644
>> --- a/.travis.mk
>> +++ b/.travis.mk
>> @@ -8,6 +8,12 @@ TEST_RUN_EXTRA_PARAMS?=
>> MAX_FILES?=65534
>> MAX_PROC?=2500
>> OOS_SRC_PATH="/source"
>> +BIN_DIR=/usr/local/bin
>> +
>> +CLOJURE_URL="https://download.clojure.org/install/linux-install-1.10.1.561.sh"
>> +LEIN_URL="https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein"
>> +TERRAFORM_NAME="terraform_0.13.1_linux_amd64.zip"
>> +TERRAFORM_URL="https://releases.hashicorp.com/terraform/0.13.1/"$(TERRAFORM_NAME)
>>
>> all: package
>>
>> @@ -60,7 +66,7 @@ docker_%:
>> # commit, so the build requires old dependencies to be installed.
>> # See ce623a23416eb192ce70116fd14992e84e7ccbbe ('Enable GitLab CI
>> # testing') for more information.
>> -deps_debian:
>> +deps_debian: $(BIN_DIR)/clojure $(BIN_DIR)/lein $(BIN_DIR)/terraform
>> apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
>> build-essential cmake coreutils sed \
>> libreadline-dev libncurses5-dev libyaml-dev libssl-dev \
> I don't think we need to add jepsen's new targets to the base one, let's
> move it to 'deps_debian_jepsen'.
Splitted targets.
>
>> @@ -69,6 +75,11 @@ deps_debian:
>> python-msgpack python-yaml python-argparse python-six python-gevent \
>> lcov ruby clang llvm llvm-dev zlib1g-dev autoconf automake libtool
>>
>> +deps_debian_jepsen:
>> + apt-get update ${APT_EXTRA_FLAGS} && apt-get install -y -f \
>> + openjdk-8-jre openjdk-8-jre-headless libjna-java gnuplot graphviz \
>> + zip unzip openssh-client jq
>> +
>> deps_buster_clang_8: deps_debian
>> echo "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" > /etc/apt/sources.list.d/clang_8.list
>> echo "deb-src http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" >> /etc/apt/sources.list.d/clang_8.list
>> @@ -76,6 +87,19 @@ deps_buster_clang_8: deps_debian
>> apt-get update
>> apt-get install -y clang-8 llvm-8-dev
>>
>> +$(BIN_DIR)/clojure: deps_debian_jepsen
>> + curl $(CLOJURE_URL) | sudo bash
>> +
>> +$(BIN_DIR)/lein: deps_debian_jepsen
>> + curl $(LEIN_URL) > $@
>> + chmod a+x $@
>> + $@ version
>> +
>> +$(BIN_DIR)/terraform: deps_debian_jepsen
>> + curl -O $(TERRAFORM_URL)
>> + unzip -o $(TERRAFORM_NAME) terraform -d $(dir $@)
>> + rm -f $(TERRAFORM_NAME)
>> +
> Do we really need standalone targets that will not be used in separate?
> This file is in makefile style, but not makefile in real, it has
> standalone targets that provides some process meanings, like:
> - prepare deps
> - build
> - test
We discussed orally and decided to involve third reviewer
to come to agreement regarding this point. Alexander agreed that this
set of target is ok.
>
>> # Release
>>
>> configure_debian:
>> @@ -232,3 +256,11 @@ test_freebsd_no_deps: build_freebsd
>> cd test && python2.7 test-run.py --force $(TEST_RUN_EXTRA_PARAMS)
>>
>> test_freebsd: deps_freebsd test_freebsd_no_deps
>> +
>> +# ###################
>> +# Jepsen testing
>> +# ###################
>> +
>> +test_jepsen: deps_debian
>> + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON
>> + make jepsen-single
> Let's use deps_debian_jepsen here.
Done.
>> --
>> 2.25.1
>>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI
2020-09-16 7:07 [Tarantool-patches] [PATCH 0/4] Integrate Jepsen tests to CI sergeyb
` (3 preceding siblings ...)
2020-09-16 7:07 ` [Tarantool-patches] [PATCH 4/4] ci: integrate Jepsen tests to GitLab CI sergeyb
@ 2020-09-18 15:03 ` Kirill Yukhin
4 siblings, 0 replies; 14+ messages in thread
From: Kirill Yukhin @ 2020-09-18 15:03 UTC (permalink / raw)
To: sergeyb; +Cc: tarantool-patches, alexander.turenko
Hello,
On 16 сен 10:07, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
>
> Patch series adds Terraform configs to setup test stand in a cloud
> infrastructure, script to run Jepsen tests, separate targets to
> Makefile.
>
> Issue: https://github.com/tarantool/tarantool/issues/5277
> CI: https://gitlab.com/tarantool/tarantool/-/pipelines/190225147
In future, please mention branch name in cover letter.
I've checked your patchset into master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 14+ messages in thread