Tarantool development patches archive
 help / color / mirror / Atom feed
From: "Alexander V. Tikhonov" <avtikhon@tarantool.org>
To: sergeyb@tarantool.org
Cc: tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH v2] Add option to update file with reference output
Date: Wed, 20 May 2020 11:03:00 +0300	[thread overview]
Message-ID: <20200520080259.GA15557@hpalx> (raw)
In-Reply-To: <e0f00f4f70d21f8ae251ed5dd73b403257c65440.1589883920.git.sergeyb@tarantool.org>

Hi Sergey, thanks for the patch. In general LGTM, but want to ask if
it is needed to mention in help message of the new --update-result
option that the new result file will be in the new result style, I
mean the following difference, please check:
  mv replication/misc.result replication/misc.saved
  ./test-run.py replication/misc.test.lua --update-result
  diff replication/misc.result replication/misc.saved

On Tue, May 19, 2020 at 01:25:46PM +0300, sergeyb@tarantool.org wrote:
> From: Sergey Bronnikov <sergeyb@tarantool.org>
> 
> New option covers two cases:
> - in case of test failure test-run.py create a file .reject with
> actual test output and one need to move .reject file to .result manually
> when test has a valid behaviour. With option --update-result test-run.py
> will do it automatically.
> - in case of abcense reference result file test-run.py forced to create
> such file in a source directory and set test status "new". This patch
> changes behaviour. With option --update-result test status is "new"
> and result file is created, without option test status is "fail" and
> result file is not created.
> 
> Fixes https://github.com/tarantool/tarantool/issues/4654
> Fixes https://github.com/tarantool/tarantool/issues/4258
> Closes #194
> ---
> GH branch: https://github.com/tarantool/test-run/tree/ligurio/gh-4654-update-ref-output
> 
>  lib/options.py |  8 ++++++++
>  lib/test.py    | 21 +++++++++++++++++----
>  2 files changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/options.py b/lib/options.py
> index 8bacb4a..47bbc0f 100644
> --- a/lib/options.py
> +++ b/lib/options.py
> @@ -201,6 +201,14 @@ class Options:
>                  help="""Run the server under 'luacov'.
>                  Default: false.""")
>  
> +        parser.add_argument(
> +                "--update-result",
> +                dest="update_result",
> +                action="store_true",
> +                default=False,
> +                help="""Update or create file with reference output (.result).
> +                Default: false.""")
> +
>          # XXX: We can use parser.parse_intermixed_args() on
>          # Python 3.7 to understand commands like
>          # ./test-run.py foo --exclude bar baz
> diff --git a/lib/test.py b/lib/test.py
> index 8bc1feb..733b90c 100644
> --- a/lib/test.py
> +++ b/lib/test.py
> @@ -15,6 +15,7 @@ except ImportError:
>      from StringIO import StringIO
>  
>  import lib
> +from lib.options import Options
>  from lib.colorer import color_stdout
>  from lib.utils import non_empty_valgrind_logs
>  from lib.utils import print_tail_n
> @@ -152,7 +153,7 @@ class Test(object):
>              it to stdout.
>  
>              Returns short status of the test as a string: 'skip', 'pass',
> -            'new', or 'fail'. There is also one possible value for
> +            'new', 'updated' or 'fail'. There is also one possible value for
>              short_status, 'disabled', but it returned in the caller,
>              TestSuite.run_test().
>          """
> @@ -235,9 +236,20 @@ class Test(object):
>          elif (self.is_executed_ok and not
>                self.is_equal_result and not
>                os.path.isfile(self.result)) and not is_tap:
> +            if lib.Options().args.update_result:
> +                shutil.copy(self.tmp_result, self.result)
> +                short_status = 'new'
> +                color_stdout("[ new ]\n", schema='test_new')
> +            else:
> +                short_status = 'fail'
> +                color_stdout("[ fail ]\n", schema='test_fail')
> +        elif (self.is_executed_ok and
> +              not self.is_equal_result and
> +              os.path.isfile(self.result) and
> +              lib.Options().args.update_result):
>              shutil.copy(self.tmp_result, self.result)
> -            short_status = 'new'
> -            color_stdout("[ new ]\n", schema='test_new')
> +            short_status = 'updated'
> +            color_stdout("[ updated ]\n", schema='test_new')
>          else:
>              has_result = os.path.exists(self.tmp_result)
>              if has_result:
> @@ -256,7 +268,8 @@ class Test(object):
>                  server.print_log(15)
>                  where = ": test execution aborted, reason " \
>                          "'{0}'".format(diagnostics)
> -            elif not self.is_crash_reported and not self.is_equal_result:
> +            elif (not self.is_crash_reported and not self.is_equal_result and
> +                not lib.Options().args.update_result):
>                  self.print_unidiff()
>                  server.print_log(15)
>                  where = ": wrong test output"
> -- 
> 2.23.0
> 

  reply	other threads:[~2020-05-20  8:03 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-19 10:25 sergeyb
2020-05-20  8:03 ` Alexander V. Tikhonov [this message]
2020-05-20  8:23   ` Sergey Bronnikov
2020-06-03 17:27 ` Alexander Turenko
2020-06-04 13:07   ` Sergey Bronnikov
2020-06-08  9:19     ` Alexander Turenko
2020-06-08 15:31       ` Sergey Bronnikov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200520080259.GA15557@hpalx \
    --to=avtikhon@tarantool.org \
    --cc=sergeyb@tarantool.org \
    --cc=tarantool-patches@dev.tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v2] Add option to update file with reference output' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox