[Tarantool-patches] [PATCH v1] Add option to update file with reference output

[Sergey Bronnikov]

Thanks for review!
I have updated typos in the branch.

On 17:34 Mon 30 Mar , Alexander Tikhonov wrote:
> 
> Hi Sergey, thank you for the patch, LGTM, just minor corrections in comments
> behaviout → behaviour
> automagically → automaically
> 
>   
> >Вторник, 24 марта 2020, 11:01 +03:00 от Sergey Bronnikov <sergeyb at tarantool.org>:
> > 
> >In case of test failure test-run.py create a file .reject with actual
> >test output and one need to move .reject file to .result manually when
> >test has a valid behaviout. With option --update-ref-output test-run.py
> >will do it automagically.
> >
> >Fixes: #4654
> >
> >GitHub branch:  https://github.com/tarantool/test-run/tree/ligurio/gh-4654-update-ref-output
> >
> >---
> > lib/options.py | 8 ++++++++
> > lib/test.py | 19 +++++++++++++++----
> > 2 files changed, 23 insertions(+), 4 deletions(-)
> >
> >diff --git a/lib/options.py b/lib/options.py
> >index 8bacb4a..174a62f 100644
> >--- a/lib/options.py
> >+++ b/lib/options.py
> >@@ -201,6 +201,14 @@ class Options:
> >                 help="""Run the server under 'luacov'.
> >                 Default: false.""")
> > 
> >+ parser.add_argument(
> >+ "--update-ref-output",
> >+ dest="update_reference_output",
> >+ action="store_true",
> >+ default=False,
> >+ help="""Update file with reference output (.reject) in case of fail
> >+ and set status pass. Default: false.""")
> >+
> >         # XXX: We can use parser.parse_intermixed_args() on
> >         # Python 3.7 to understand commands like
> >         # ./test-run.py foo --exclude bar baz
> >diff --git a/lib/test.py b/lib/test.py
> >index 3e93af3..396bb89 100644
> >--- a/lib/test.py
> >+++ b/lib/test.py
> >@@ -15,6 +15,7 @@ except ImportError:
> >     from StringIO import StringIO
> > 
> > import lib
> >+from lib.options import Options
> > from lib.colorer import color_stdout
> > from lib.utils import non_empty_valgrind_logs
> > from lib.utils import print_tail_n
> >@@ -242,23 +243,33 @@ class Test(object):
> >             color_stdout("[ new ]\n", schema='test_new')
> >         else:
> >             has_result = os.path.exists(self.tmp_result)
> >+ update_reference = lib.Options().args.update_reference_output
> >             if has_result:
> >- shutil.copy(self.tmp_result, self.reject)
> >- short_status = 'fail'
> >+ if update_reference:
> >+ reject_dest = self.result
> >+ else:
> >+ reject_dest = self.reject
> >+ shutil.copy(self.tmp_result, reject_dest)
> >+ if update_reference:
> >+ short_status = 'pass'
> >+ else:
> >+ short_status = 'fail'
> >             color_stdout("[ fail ]\n", schema='test_fail')
> > 
> >             where = ""
> >             if not self.is_crash_reported and not has_result:
> >                 color_stdout('\nCannot open %s\n' % self.tmp_result,
> >                              schema='error')
> >- elif not self.is_crash_reported and not self.is_executed_ok:
> >+ elif not self.is_crash_reported and not self.is_executed_ok and \
> >+ not update_reference:
> >                 self.print_diagnostics(self.reject,
> >                                        "Test failed! Output from reject file "
> >                                        "{0}:\n".format(self.reject))
> >                 server.print_log(15)
> >                 where = ": test execution aborted, reason " \
> >                         "'{0}'".format(diagnostics)
> >- elif not self.is_crash_reported and not self.is_equal_result:
> >+ elif not self.is_crash_reported and not self.is_equal_result and \
> >+ not update_reference:
> >                 self.print_unidiff()
> >                 server.print_log(15)
> >                 where = ": wrong test output"
> >--
> >2.23.0
> >
> >
> >--
> >sergeyb@ 
>  
>  
> --
> Alexander Tikhonov
>  

-- 
sergeyb@