[Tarantool-patches] [PATCH v1] gitlab-ci: integration testting connectors

[Sergey Bronnikov]

Hi, Oleg

Thanks for the patch! See my coments inline.

On 18:29 Thu 21 May , Oleg Piskunov wrote:
> 
> 
> - Adding new stage "integration_test" for integration
>   testing connectors and modules with Tarantool.
> - Adding testing connectors in Tarantool
>   development cycle:
>   - tarantool-python
>   - asynctnt-python
>   - tarantool-php
>   - tarantool-java
>   - tarantool-go
>   - viciious-go
>  
> Closes #4856
> ---
>  
> Github:  https://github.com/tarantool/tarantool/tree/opiskunov/gh-4856-integration-test-connectors
> Issue:  https://github.com/tarantool/tarantool/issues/4856
> 
>  .gitlab-ci.yml | 41 +++++++++++++++++++++++++++++++++++++++++
>  .travis.mk     | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 95 insertions(+), 1 deletion(-)
>  
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 256b368..3ab0103 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,5 +1,6 @@
>  stages:
>    - test
> +  - integration_test
>    - perf
>    - cleanup
>  
> @@ -183,6 +184,46 @@ freebsd_12_release:
>    script:
>      - ${GITLAB_MAKE} vms_test_freebsd
>  
> +# ###################
> +# Integration testing
> +# ###################
> +
> +release_connector_python_asynctnt_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_python_asynctnt
> +
> +release_connector_python_tarantool_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_python_tarantool
> +
> +release_connector_go_tarantool_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_go_tarantool
> +
> +release_connector_go_viciious_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_go_viciious
> +
> +release_connector_php_tarantool_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_php_tarantool
> +
> +release_connector_java_tarantool_test:
> +  <<: *docker_test_definition
> +  stage: integration_test
> +  script:
> +    - ${GITLAB_MAKE} test_connector_java_tarantool
> +
>  # ####
>  # Perf
>  # ####
> diff --git a/.travis.mk b/.travis.mk
> index 063537f..024351e 100644
> --- a/.travis.mk
> +++ b/.travis.mk
> @@ -77,7 +77,7 @@ deps_buster_clang_8: deps_debian
>  # Release
>  
>  build_debian:
> -       cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}
> +       cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON -DENABLE_DIST=ON ${CMAKE_EXTRA_PARAMS}

Why do we need ENABLE_WERROR here?

>         make -j
>  
>  test_debian_no_deps: build_debian
> @@ -87,6 +87,59 @@ test_debian: deps_debian test_debian_no_deps
>  
>  test_debian_clang8: deps_debian deps_buster_clang_8 test_debian_no_deps
>  
> +# Integration testing
> +
> +test_connector_python_asynctnt: build_debian
> +       make install

Let's create a separate target for tarantool installation and add it to
dependence. Same for targets below.

> +       apt-get install -y python3-pip python3-dev pandoc python3-setuptools
> +       git clone --depth=1  --recurse-submodules https://github.com/igorcoding/asynctnt.git asynctnt-python
> +       cd asynctnt-python && pip3 install -r requirements.txt && export PYTHON=python3 \
> +               && make && pip3 install -e . && make quicktest
> +
> +test_connector_python_tarantool: build_debian
> +       make install
> +       git clone --depth=1 https://github.com/tarantool/tarantool-python.git tarantool-python
> +       cd tarantool-python && pip install -r requirements.txt && python setup.py install \
> +               && python setup.py test
> +
> +test_connector_go_tarantool: build_debian
> +       make install
> +       wget --progress=dot:mega https://dl.google.com/go/go1.10.linux-amd64.tar.gz
> +       tar -C /usr/local -xzf go1.10.linux-amd64.tar.gz
> +       chmod -R a+rwx /usr/local/go
> +       export PATH=/usr/local/go/bin:$$PATH && export GOPATH=/usr/local/go/go-tarantool \
> +               && go get github.com/tarantool/go-tarantool \
> +               && cd /usr/local/go/go-tarantool/src/github.com/tarantool/go-tarantool \
> +               && mkdir snap xlog && (tarantool config.lua &) && go test -v .
> +
> +test_connector_go_viciious: build_debian
> +       make install
> +       wget --progress=dot:mega https://dl.google.com/go/go1.10.linux-amd64.tar.gz
> +       tar -C /usr/local -xzf go1.10.linux-amd64.tar.gz
> +       chmod -R a+rwx /usr/local/go
> +       export PATH=/usr/local/go/bin:$$PATH && export GOPATH=/usr/local/go/go-tarantool \
> +               && export GOBIN=$$GOPATH/bin && go get github.com/viciious/go-tarantool \
> +               && cd /usr/local/go/go-tarantool/src/github.com/viciious/go-tarantool \
> +               && mkdir log && export TNT_LOG_DIR=`pwd`/log && go test -v .
> +
> +test_connector_php_tarantool: build_debian
> +       make install
> +       apt-get update && apt-get install -y lsb-release
> +       wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
> +       echo "deb https://packages.sury.org/php/ $$(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
> +       apt-get update && apt-get install -y php7.4 php7.4-dev php7.4-xml
> +       curl -SsLf -o /usr/local/bin/phpunit https://phar.phpunit.de/phpunit-9.phar
> +       chmod a+x /usr/local/bin/phpunit
> +       git clone --depth=1 https://github.com/tarantool/tarantool-php.git --branch=php7-v2 tarantool-php
> +       cd tarantool-php && phpize && ./configure \
> +               && make && make install && /usr/bin/python test-run.py
> +
> +test_connector_java_tarantool: build_debian
> +       make install
> +       apt-get update && apt-get install -y openjdk-8-jre openjdk-8-jdk
> +       git clone --depth=1 https://github.com/tarantool/tarantool-java.git tarantool-java
> +       cd tarantool-java && ./mvnw clean test && ./mvnw clean verify

The same question as for vshard - why should we test tarantool against
unreleased versions of connectors?

> +
>  # Debug with coverage
>  
>  build_coverage_debian:
> --
> 1.8.3.1
>  

-- 
sergeyb@