[PATCH] test: update test-run

[PATCH] test: update test-run

[Alexander Turenko]

* Implement SQL driver (#4123).
* Support line carrying with backslash.
* Support new result file format (for new tests).

See the link below for a full description of the new features.

https://github.com/tarantool/test-run/commit/a04b5b096c607172ce4fc86a84e3531c9f3a7304

Fixes #4123.
---

https://github.com/tarantool/tarantool/issues/4123

 test-run | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test-run b/test-run
index 5c71ff670..a04b5b096 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit 5c71ff6707a96e9e06869b0336a153ae45456d77
+Subproject commit a04b5b096c607172ce4fc86a84e3531c9f3a7304
-- 
2.21.0

Re: [PATCH] test: update test-run

[Alexander Turenko]

Pushed the test-run update to master, 2.1 and 1.10.

I rewrote one SQL test as *.test.sql file for example (it is on a
branch; I doubt I should send it for review / check-in):
https://github.com/tarantool/tarantool/commit/3ffd899bb322758d2c34007674a76d0e72e747ba

WBR, Alexander Turenko.

On Mon, Jul 01, 2019 at 01:50:45PM +0300, Alexander Turenko wrote:
> * Implement SQL driver (#4123).
> * Support line carrying with backslash.
> * Support new result file format (for new tests).
> 
> See the link below for a full description of the new features.
> 
> https://github.com/tarantool/test-run/commit/a04b5b096c607172ce4fc86a84e3531c9f3a7304
> 
> Fixes #4123.
> ---
> 
> https://github.com/tarantool/tarantool/issues/4123
> 
>  test-run | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/test-run b/test-run
> index 5c71ff670..a04b5b096 160000
> --- a/test-run
> +++ b/test-run
> @@ -1 +1 @@
> -Subproject commit 5c71ff6707a96e9e06869b0336a153ae45456d77
> +Subproject commit a04b5b096c607172ce4fc86a84e3531c9f3a7304
> -- 
> 2.21.0
> 

Re: [PATCH] test: update test-run

[Konstantin Osipov]

* Alexander Turenko <alexander.turenko@tarantool.org> [19/07/01 14:05]:

This is great, now let's rewrite all tests :)


-- 
Konstantin Osipov, Moscow, Russia

Re: [PATCH] test: update test-run

[Vladimir Davydov]

On Wed, Dec 19, 2018 at 02:47:08AM +0300, Сергей Воронежский wrote:
> Did you guys update submodule?

Pushed to 2.1 and 1.10.

Re: [PATCH] test: update test-run

[Alexander Turenko]

Are not that should be on top of 1.10?

WBR, Alexander Turenko.

On Mon, Nov 26, 2018 at 02:18:40PM +0300, Sergei Voronezhskii wrote:
> BRANCH:  https://github.com/tarantool/tarantool/tree/sergw/update-test-run
> 
> Updated branch with new version test-run (fixed problem with file descriptors when use_unix_sockets=True)
> Also rebased on 2.1
> 
> 
> >Пятница, 16 ноября 2018, 18:41 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:
> >
> >Now LGTM.
> >
> >WBR, Alexander Turenko.
> >
> >On Fri, Nov 16, 2018 at 04:58:21PM +0300, Sergei Voronezhskii wrote:
> >> Build:  https://travis-ci.org/tarantool/tarantool/builds/455965753
> >> >Пятница, 16 ноября 2018, 12:27 +03:00 от Alexander Turenko < alexander.turenko@tarantool.org >:
> >> >
> >> >Please, investigate errors that were arise in CI:
> >> >
> >> >> TypeError: cleanup() got multiple values for keyword argument 'dirname'
> >> >
> >> > https://travis-ci.org/tarantool/tarantool/jobs/455877309#L2652
> >> > https://travis-ci.org/tarantool/tarantool/jobs/455877310#L2730
> >> Fixed:
> >
> >Ok.
> >
> >> diff --git a/test/replication-py/cluster.test.py b/test/replication-py/cluster.test.py
> >> index 14598b798..b5f59f40f 100644
> >> --- a/test/replication-py/cluster.test.py
> >> +++ b/test/replication-py/cluster.test.py
> >> @@ -209,7 +209,7 @@ master.admin('box.info.vclock[%d] == 2' % replica_id)
> >>  
> >>  master.admin("box.cfg{ replication = '' }")
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  
> >>  print '-------------------------------------------------------------'
> >>  print 'Start a new replica and check that server_id, LSN is re-used'
> >> @@ -238,7 +238,7 @@ master.admin('box.info.vclock[%d] == 2' % replica_id)
> >>  replica.admin('box.info.vclock[%d] == 2' % replica_id)
> >>  
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  
> >>  print '-------------------------------------------------------------'
> >>  print 'JOIN replica to read-only master'
> >> diff --git a/test/replication-py/conflict.test.py b/test/replication-py/conflict.test.py
> >> index 41c6fca73..f40148318 100644
> >> --- a/test/replication-py/conflict.test.py
> >> +++ b/test/replication-py/conflict.test.py
> >> @@ -43,12 +43,12 @@ def parallel_run(cmd1, cmd2, compare):
> >>  def prepare_cluster():
> >>      print 'reset master-master replication'
> >>      master.stop()
> >> -    master.cleanup(True)
> >> +    master.cleanup()
> >>      master.start()
> >>      master.admin("box.schema.user.grant('guest', 'replication')", silent=True)
> >>  
> >>      replica.stop()
> >> -    replica.cleanup(True)
> >> +    replica.cleanup()
> >>      replica.start()
> >>  
> >>      master.admin("box.cfg{replication='%s'}" % replica.iproto.uri, silent=True)
> >> @@ -114,7 +114,7 @@ check_replication([master, replica])
> >>  
> >>  # cleanup
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  server.stop()
> >> -server.cleanup(True)
> >> +server.cleanup()
> >>  server.deploy()
> >> diff --git a/test/replication-py/init_storage.test.py b/test/replication-py/init_storage.test.py
> >> index 32b4639f1..4be531f8d 100644
> >> --- a/test/replication-py/init_storage.test.py
> >> +++ b/test/replication-py/init_storage.test.py
> >> @@ -26,7 +26,7 @@ replica.admin('box.space.test:select()')
> >>  replica.restart()
> >>  replica.admin('box.space.test:select()')
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  
> >>  print '-------------------------------------------------------------'
> >>  print 'replica test 2 (must be ok)'
> >> @@ -49,7 +49,7 @@ for i in range(1, 20):
> >>      replica.admin('space:get{%d}' % i)
> >>  
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  
> >>  print '-------------------------------------------------------------'
> >>  print 'reconnect on JOIN/SUBSCRIBE'
> >> @@ -77,7 +77,7 @@ replica.wait_until_started()
> >>  print 'ok'
> >>  
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  
> >>  server.stop()
> >>  server.deploy()
> >> diff --git a/test/replication-py/swap.test.py b/test/replication-py/swap.test.py
> >> index b18940517..98eeeea6d 100644
> >> --- a/test/replication-py/swap.test.py
> >> +++ b/test/replication-py/swap.test.py
> >> @@ -119,6 +119,6 @@ for i in range(REPEAT):
> >>  
> >>  # Cleanup.
> >>  replica.stop()
> >> -replica.cleanup(True)
> >> +replica.cleanup()
> >>  server.stop()
> >>  server.deploy()
> >> 
> >> 
> >> >
> >> >
> >> >> RuntimeError: Broken tarantool console handshake
> >> >
> >> > https://travis-ci.org/tarantool/tarantool/jobs/455877312#L2655
> >> Looks like this error is not related.
> >> >
> >
> >Ok.
> >
> >> >WBR, Alexander Turenko.
> >> >
> >> >On Fri, Nov 16, 2018 at 11:58:36AM +0300, Sergei Voronezhskii wrote:
> >> >> * Perform the clean up if param `pre_cleanup = True` in suite.ini.
> >> >>   It cleans checkpoints in workdir for tap tests. And for app tests
> >> >>   cleans spaces/users/roles/globals/functions and checks count of
> >> >>   `box.space._cluster`
> >> >> 
> >> >> Fixes #1961
> >> >> ---
> >> >> BRANCH:  https://github.com/tarantool/tarantool/tree/sergw/update-test-run
> >> >> ISSUE:  https://github.com/tarantool/tarantool/issues/1961
> >> >> PR:  https://github.com/tarantool/test-run/pull/124
> >> >>  test-run | 2 +-
> >> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >> >> 
> >> >> diff --git a/test-run b/test-run
> >> >> index 670f330aa..d804b6a6e 160000
> >> >> --- a/test-run
> >> >> +++ b/test-run
> >> >> @@ -1 +1 @@
> >> >> -Subproject commit 670f330aacaf44bc8b1f969fa0cd5f811c5ceb1b
> >> >> +Subproject commit d804b6a6ead6fec466622435fddf2218966d7c06
> >> >> -- 
> >> >> 2.18.0
> >> >> 
> >> 
> >> 
> >> -- 
> >> Sergei Voronezhskii
> >
> 
> 
> -- 
> Sergei Voronezhskii

Re: [PATCH] test: update test-run

[Alexander Turenko]

Now LGTM.

WBR, Alexander Turenko.

On Fri, Nov 16, 2018 at 04:58:21PM +0300, Sergei Voronezhskii wrote:
> Build: https://travis-ci.org/tarantool/tarantool/builds/455965753
> >Пятница, 16 ноября 2018, 12:27 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:
> >
> >Please, investigate errors that were arise in CI:
> >
> >> TypeError: cleanup() got multiple values for keyword argument 'dirname'
> >
> >https://travis-ci.org/tarantool/tarantool/jobs/455877309#L2652
> >https://travis-ci.org/tarantool/tarantool/jobs/455877310#L2730
> Fixed:

Ok.

> diff --git a/test/replication-py/cluster.test.py b/test/replication-py/cluster.test.py
> index 14598b798..b5f59f40f 100644
> --- a/test/replication-py/cluster.test.py
> +++ b/test/replication-py/cluster.test.py
> @@ -209,7 +209,7 @@ master.admin('box.info.vclock[%d] == 2' % replica_id)
>  
>  master.admin("box.cfg{ replication = '' }")
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  
>  print '-------------------------------------------------------------'
>  print 'Start a new replica and check that server_id, LSN is re-used'
> @@ -238,7 +238,7 @@ master.admin('box.info.vclock[%d] == 2' % replica_id)
>  replica.admin('box.info.vclock[%d] == 2' % replica_id)
>  
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  
>  print '-------------------------------------------------------------'
>  print 'JOIN replica to read-only master'
> diff --git a/test/replication-py/conflict.test.py b/test/replication-py/conflict.test.py
> index 41c6fca73..f40148318 100644
> --- a/test/replication-py/conflict.test.py
> +++ b/test/replication-py/conflict.test.py
> @@ -43,12 +43,12 @@ def parallel_run(cmd1, cmd2, compare):
>  def prepare_cluster():
>      print 'reset master-master replication'
>      master.stop()
> -    master.cleanup(True)
> +    master.cleanup()
>      master.start()
>      master.admin("box.schema.user.grant('guest', 'replication')", silent=True)
>  
>      replica.stop()
> -    replica.cleanup(True)
> +    replica.cleanup()
>      replica.start()
>  
>      master.admin("box.cfg{replication='%s'}" % replica.iproto.uri, silent=True)
> @@ -114,7 +114,7 @@ check_replication([master, replica])
>  
>  # cleanup
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  server.stop()
> -server.cleanup(True)
> +server.cleanup()
>  server.deploy()
> diff --git a/test/replication-py/init_storage.test.py b/test/replication-py/init_storage.test.py
> index 32b4639f1..4be531f8d 100644
> --- a/test/replication-py/init_storage.test.py
> +++ b/test/replication-py/init_storage.test.py
> @@ -26,7 +26,7 @@ replica.admin('box.space.test:select()')
>  replica.restart()
>  replica.admin('box.space.test:select()')
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  
>  print '-------------------------------------------------------------'
>  print 'replica test 2 (must be ok)'
> @@ -49,7 +49,7 @@ for i in range(1, 20):
>      replica.admin('space:get{%d}' % i)
>  
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  
>  print '-------------------------------------------------------------'
>  print 'reconnect on JOIN/SUBSCRIBE'
> @@ -77,7 +77,7 @@ replica.wait_until_started()
>  print 'ok'
>  
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  
>  server.stop()
>  server.deploy()
> diff --git a/test/replication-py/swap.test.py b/test/replication-py/swap.test.py
> index b18940517..98eeeea6d 100644
> --- a/test/replication-py/swap.test.py
> +++ b/test/replication-py/swap.test.py
> @@ -119,6 +119,6 @@ for i in range(REPEAT):
>  
>  # Cleanup.
>  replica.stop()
> -replica.cleanup(True)
> +replica.cleanup()
>  server.stop()
>  server.deploy()
> 
> 
> >
> >
> >> RuntimeError: Broken tarantool console handshake
> >
> >https://travis-ci.org/tarantool/tarantool/jobs/455877312#L2655
> Looks like this error is not related.
> >

Ok.

> >WBR, Alexander Turenko.
> >
> >On Fri, Nov 16, 2018 at 11:58:36AM +0300, Sergei Voronezhskii wrote:
> >> * Perform the clean up if param `pre_cleanup = True` in suite.ini.
> >>   It cleans checkpoints in workdir for tap tests. And for app tests
> >>   cleans spaces/users/roles/globals/functions and checks count of
> >>   `box.space._cluster`
> >> 
> >> Fixes #1961
> >> ---
> >> BRANCH:  https://github.com/tarantool/tarantool/tree/sergw/update-test-run
> >> ISSUE:  https://github.com/tarantool/tarantool/issues/1961
> >> PR:  https://github.com/tarantool/test-run/pull/124
> >>  test-run | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >> 
> >> diff --git a/test-run b/test-run
> >> index 670f330aa..d804b6a6e 160000
> >> --- a/test-run
> >> +++ b/test-run
> >> @@ -1 +1 @@
> >> -Subproject commit 670f330aacaf44bc8b1f969fa0cd5f811c5ceb1b
> >> +Subproject commit d804b6a6ead6fec466622435fddf2218966d7c06
> >> -- 
> >> 2.18.0
> >> 
> 
> 
> -- 
> Sergei Voronezhskii

Re: [PATCH] test: update test-run

[Alexander Turenko]

Please, investigate errors that were arise in CI:

> TypeError: cleanup() got multiple values for keyword argument 'dirname'

https://travis-ci.org/tarantool/tarantool/jobs/455877309#L2652
https://travis-ci.org/tarantool/tarantool/jobs/455877310#L2730

> RuntimeError: Broken tarantool console handshake

https://travis-ci.org/tarantool/tarantool/jobs/455877312#L2655

WBR, Alexander Turenko.

On Fri, Nov 16, 2018 at 11:58:36AM +0300, Sergei Voronezhskii wrote:
> * Perform the clean up if param `pre_cleanup = True` in suite.ini.
>   It cleans checkpoints in workdir for tap tests. And for app tests
>   cleans spaces/users/roles/globals/functions and checks count of
>   `box.space._cluster`
> 
> Fixes #1961
> ---
> BRANCH: https://github.com/tarantool/tarantool/tree/sergw/update-test-run
> ISSUE: https://github.com/tarantool/tarantool/issues/1961
> PR: https://github.com/tarantool/test-run/pull/124
>  test-run | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/test-run b/test-run
> index 670f330aa..d804b6a6e 160000
> --- a/test-run
> +++ b/test-run
> @@ -1 +1 @@
> -Subproject commit 670f330aacaf44bc8b1f969fa0cd5f811c5ceb1b
> +Subproject commit d804b6a6ead6fec466622435fddf2218966d7c06
> -- 
> 2.18.0
> 

[PATCH] test: update test-run

[Sergei Voronezhskii]

* Perform the clean up if param `pre_cleanup = True` in suite.ini.
  It cleans checkpoints in workdir for tap tests. And for app tests
  cleans spaces/users/roles/globals/functions and checks count of
  `box.space._cluster`

Fixes #1961
---
BRANCH: https://github.com/tarantool/tarantool/tree/sergw/update-test-run
ISSUE: https://github.com/tarantool/tarantool/issues/1961
PR: https://github.com/tarantool/test-run/pull/124
 test-run | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test-run b/test-run
index 670f330aa..d804b6a6e 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit 670f330aacaf44bc8b1f969fa0cd5f811c5ceb1b
+Subproject commit d804b6a6ead6fec466622435fddf2218966d7c06
-- 
2.18.0