From: "Alexander Tikhonov" <avtikhon@tarantool.org>
To: "Sergey Bronnikov" <sergeyb@tarantool.org>
Cc: "Oleg Piskunov" <o.piskunov@tarantool.org>,
tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH v1] Add option to update file with reference output
Date: Mon, 30 Mar 2020 17:34:59 +0300 [thread overview]
Message-ID: <1585578899.878336951@f139.i.mail.ru> (raw)
In-Reply-To: <20200324080124.GA67461@pony.bronevichok.ru>
[-- Attachment #1: Type: text/plain, Size: 3866 bytes --]
Hi Sergey, thank you for the patch, LGTM, just minor corrections in comments
behaviout → behaviour
automagically → automaically
>Вторник, 24 марта 2020, 11:01 +03:00 от Sergey Bronnikov <sergeyb@tarantool.org>:
>
>In case of test failure test-run.py create a file .reject with actual
>test output and one need to move .reject file to .result manually when
>test has a valid behaviout. With option --update-ref-output test-run.py
>will do it automagically.
>
>Fixes: #4654
>
>GitHub branch: https://github.com/tarantool/test-run/tree/ligurio/gh-4654-update-ref-output
>
>---
> lib/options.py | 8 ++++++++
> lib/test.py | 19 +++++++++++++++----
> 2 files changed, 23 insertions(+), 4 deletions(-)
>
>diff --git a/lib/options.py b/lib/options.py
>index 8bacb4a..174a62f 100644
>--- a/lib/options.py
>+++ b/lib/options.py
>@@ -201,6 +201,14 @@ class Options:
> help="""Run the server under 'luacov'.
> Default: false.""")
>
>+ parser.add_argument(
>+ "--update-ref-output",
>+ dest="update_reference_output",
>+ action="store_true",
>+ default=False,
>+ help="""Update file with reference output (.reject) in case of fail
>+ and set status pass. Default: false.""")
>+
> # XXX: We can use parser.parse_intermixed_args() on
> # Python 3.7 to understand commands like
> # ./test-run.py foo --exclude bar baz
>diff --git a/lib/test.py b/lib/test.py
>index 3e93af3..396bb89 100644
>--- a/lib/test.py
>+++ b/lib/test.py
>@@ -15,6 +15,7 @@ except ImportError:
> from StringIO import StringIO
>
> import lib
>+from lib.options import Options
> from lib.colorer import color_stdout
> from lib.utils import non_empty_valgrind_logs
> from lib.utils import print_tail_n
>@@ -242,23 +243,33 @@ class Test(object):
> color_stdout("[ new ]\n", schema='test_new')
> else:
> has_result = os.path.exists(self.tmp_result)
>+ update_reference = lib.Options().args.update_reference_output
> if has_result:
>- shutil.copy(self.tmp_result, self.reject)
>- short_status = 'fail'
>+ if update_reference:
>+ reject_dest = self.result
>+ else:
>+ reject_dest = self.reject
>+ shutil.copy(self.tmp_result, reject_dest)
>+ if update_reference:
>+ short_status = 'pass'
>+ else:
>+ short_status = 'fail'
> color_stdout("[ fail ]\n", schema='test_fail')
>
> where = ""
> if not self.is_crash_reported and not has_result:
> color_stdout('\nCannot open %s\n' % self.tmp_result,
> schema='error')
>- elif not self.is_crash_reported and not self.is_executed_ok:
>+ elif not self.is_crash_reported and not self.is_executed_ok and \
>+ not update_reference:
> self.print_diagnostics(self.reject,
> "Test failed! Output from reject file "
> "{0}:\n".format(self.reject))
> server.print_log(15)
> where = ": test execution aborted, reason " \
> "'{0}'".format(diagnostics)
>- elif not self.is_crash_reported and not self.is_equal_result:
>+ elif not self.is_crash_reported and not self.is_equal_result and \
>+ not update_reference:
> self.print_unidiff()
> server.print_log(15)
> where = ": wrong test output"
>--
>2.23.0
>
>
>--
>sergeyb@
--
Alexander Tikhonov
[-- Attachment #2: Type: text/html, Size: 6280 bytes --]
next prev parent reply other threads:[~2020-03-30 14:35 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-24 8:01 Sergey Bronnikov
2020-03-25 7:07 ` Cyrill Gorcunov
2020-03-30 10:50 ` Sergey Bronnikov
2020-03-30 14:21 ` Sergey Bronnikov
2020-03-30 14:34 ` Alexander Tikhonov [this message]
2020-03-30 14:45 ` Sergey Bronnikov
2020-04-24 9:18 ` Oleg Piskunov
2020-05-15 22:21 ` Alexander Turenko
2020-05-19 10:23 ` Sergey Bronnikov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1585578899.878336951@f139.i.mail.ru \
--to=avtikhon@tarantool.org \
--cc=o.piskunov@tarantool.org \
--cc=sergeyb@tarantool.org \
--cc=tarantool-patches@dev.tarantool.org \
--subject='Re: [Tarantool-patches] [PATCH v1] Add option to update file with reference output' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox