From: Sergey Bronnikov <sergeyb@tarantool.org> To: tarantool-patches@dev.tarantool.org Cc: Oleg Piskunov <o.piskunov@tarantool.org> Subject: [Tarantool-patches] [PATCH v1] Add option to update file with reference output Date: Tue, 24 Mar 2020 11:01:24 +0300 [thread overview] Message-ID: <20200324080124.GA67461@pony.bronevichok.ru> (raw) In case of test failure test-run.py create a file .reject with actual test output and one need to move .reject file to .result manually when test has a valid behaviout. With option --update-ref-output test-run.py will do it automagically. Fixes: #4654 GitHub branch: https://github.com/tarantool/test-run/tree/ligurio/gh-4654-update-ref-output --- lib/options.py | 8 ++++++++ lib/test.py | 19 +++++++++++++++---- 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/lib/options.py b/lib/options.py index 8bacb4a..174a62f 100644 --- a/lib/options.py +++ b/lib/options.py @@ -201,6 +201,14 @@ class Options: help="""Run the server under 'luacov'. Default: false.""") + parser.add_argument( + "--update-ref-output", + dest="update_reference_output", + action="store_true", + default=False, + help="""Update file with reference output (.reject) in case of fail + and set status pass. Default: false.""") + # XXX: We can use parser.parse_intermixed_args() on # Python 3.7 to understand commands like # ./test-run.py foo --exclude bar baz diff --git a/lib/test.py b/lib/test.py index 3e93af3..396bb89 100644 --- a/lib/test.py +++ b/lib/test.py @@ -15,6 +15,7 @@ except ImportError: from StringIO import StringIO import lib +from lib.options import Options from lib.colorer import color_stdout from lib.utils import non_empty_valgrind_logs from lib.utils import print_tail_n @@ -242,23 +243,33 @@ class Test(object): color_stdout("[ new ]\n", schema='test_new') else: has_result = os.path.exists(self.tmp_result) + update_reference = lib.Options().args.update_reference_output if has_result: - shutil.copy(self.tmp_result, self.reject) - short_status = 'fail' + if update_reference: + reject_dest = self.result + else: + reject_dest = self.reject + shutil.copy(self.tmp_result, reject_dest) + if update_reference: + short_status = 'pass' + else: + short_status = 'fail' color_stdout("[ fail ]\n", schema='test_fail') where = "" if not self.is_crash_reported and not has_result: color_stdout('\nCannot open %s\n' % self.tmp_result, schema='error') - elif not self.is_crash_reported and not self.is_executed_ok: + elif not self.is_crash_reported and not self.is_executed_ok and \ + not update_reference: self.print_diagnostics(self.reject, "Test failed! Output from reject file " "{0}:\n".format(self.reject)) server.print_log(15) where = ": test execution aborted, reason " \ "'{0}'".format(diagnostics) - elif not self.is_crash_reported and not self.is_equal_result: + elif not self.is_crash_reported and not self.is_equal_result and \ + not update_reference: self.print_unidiff() server.print_log(15) where = ": wrong test output" -- 2.23.0 -- sergeyb@
next reply other threads:[~2020-03-24 8:01 UTC|newest] Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-24 8:01 Sergey Bronnikov [this message] 2020-03-25 7:07 ` Cyrill Gorcunov 2020-03-30 10:50 ` Sergey Bronnikov 2020-03-30 14:21 ` Sergey Bronnikov 2020-03-30 14:34 ` Alexander Tikhonov 2020-03-30 14:45 ` Sergey Bronnikov 2020-04-24 9:18 ` Oleg Piskunov 2020-05-15 22:21 ` Alexander Turenko 2020-05-19 10:23 ` Sergey Bronnikov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200324080124.GA67461@pony.bronevichok.ru \ --to=sergeyb@tarantool.org \ --cc=o.piskunov@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v1] Add option to update file with reference output' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox