Re: [Tarantool-patches] [PATCH v1] Add option to update file with reference output

["Oleg Piskunov" <o.piskunov@tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 4439 bytes --]


LGTM.
  
>Понедельник, 30 марта 2020, 17:45 +03:00 от Sergey Bronnikov <sergeyb@tarantool.org>:
> 
>Thanks for review!
>I have updated typos in the branch.
>
>On 17:34 Mon 30 Mar , Alexander Tikhonov wrote:
>>
>> Hi Sergey, thank you for the patch, LGTM, just minor corrections in comments
>> behaviout → behaviour
>> automagically → automaically
>>
>>  
>> >Вторник, 24 марта 2020, 11:01 +03:00 от Sergey Bronnikov < sergeyb@tarantool.org >:
>> > 
>> >In case of test failure test-run.py create a file .reject with actual
>> >test output and one need to move .reject file to .result manually when
>> >test has a valid behaviout. With option --update-ref-output test-run.py
>> >will do it automagically.
>> >
>> >Fixes: #4654
>> >
>> >GitHub branch:  https://github.com/tarantool/test-run/tree/ligurio/gh-4654-update-ref-output
>> >
>> >---
>> > lib/options.py | 8 ++++++++
>> > lib/test.py | 19 +++++++++++++++----
>> > 2 files changed, 23 insertions(+), 4 deletions(-)
>> >
>> >diff --git a/lib/options.py b/lib/options.py
>> >index 8bacb4a..174a62f 100644
>> >--- a/lib/options.py
>> >+++ b/lib/options.py
>> >@@ -201,6 +201,14 @@ class Options:
>> >                 help="""Run the server under 'luacov'.
>> >                 Default: false.""")
>> > 
>> >+ parser.add_argument(
>> >+ "--update-ref-output",
>> >+ dest="update_reference_output",
>> >+ action="store_true",
>> >+ default=False,
>> >+ help="""Update file with reference output (.reject) in case of fail
>> >+ and set status pass. Default: false.""")
>> >+
>> >         # XXX: We can use parser.parse_intermixed_args() on
>> >         # Python 3.7 to understand commands like
>> >         # ./test-run.py foo --exclude bar baz
>> >diff --git a/lib/test.py b/lib/test.py
>> >index 3e93af3..396bb89 100644
>> >--- a/lib/test.py
>> >+++ b/lib/test.py
>> >@@ -15,6 +15,7 @@ except ImportError:
>> >     from StringIO import StringIO
>> > 
>> > import lib
>> >+from lib.options import Options
>> > from lib.colorer import color_stdout
>> > from lib.utils import non_empty_valgrind_logs
>> > from lib.utils import print_tail_n
>> >@@ -242,23 +243,33 @@ class Test(object):
>> >             color_stdout("[ new ]\n", schema='test_new')
>> >         else:
>> >             has_result = os.path.exists(self.tmp_result)
>> >+ update_reference = lib.Options().args.update_reference_output
>> >             if has_result:
>> >- shutil.copy(self.tmp_result, self.reject)
>> >- short_status = 'fail'
>> >+ if update_reference:
>> >+ reject_dest = self.result
>> >+ else:
>> >+ reject_dest = self.reject
>> >+ shutil.copy(self.tmp_result, reject_dest)
>> >+ if update_reference:
>> >+ short_status = 'pass'
>> >+ else:
>> >+ short_status = 'fail'
>> >             color_stdout("[ fail ]\n", schema='test_fail')
>> > 
>> >             where = ""
>> >             if not self.is_crash_reported and not has_result:
>> >                 color_stdout('\nCannot open %s\n' % self.tmp_result,
>> >                              schema='error')
>> >- elif not self.is_crash_reported and not self.is_executed_ok:
>> >+ elif not self.is_crash_reported and not self.is_executed_ok and \
>> >+ not update_reference:
>> >                 self.print_diagnostics(self.reject,
>> >                                        "Test failed! Output from reject file "
>> >                                        "{0}:\n".format(self.reject))
>> >                 server.print_log(15)
>> >                 where = ": test execution aborted, reason " \
>> >                         "'{0}'".format(diagnostics)
>> >- elif not self.is_crash_reported and not self.is_equal_result:
>> >+ elif not self.is_crash_reported and not self.is_equal_result and \
>> >+ not update_reference:
>> >                 self.print_unidiff()
>> >                 server.print_log(15)
>> >                 where = ": wrong test output"
>> >--
>> >2.23.0
>> >
>> >
>> >--
>> >sergeyb@
>>  
>>  
>> --
>> Alexander Tikhonov
>>  
>--
>sergeyb@ 
 
 
--
Oleg Piskunov
 

[-- Attachment #2: Type: text/html, Size: 7568 bytes --]