Re: [Tarantool-patches] [PATCH v3 2/2] Add option to update file with reference output

[Alexander Turenko <alexander.turenko@tarantool.org>]

Made several changes, see below.

Pushed the patchset to test-run's master.

Updated the submodule in tarantool and pushed the update to master, 2.4,
2.3 and 1.10 branches.

WBR, Alexander Turenko.

> -        elif (self.is_executed_ok and not
> -              self.is_equal_result and not
> -              os.path.isfile(self.result)) and not is_tap:
> +        elif (self.is_executed_ok and
> +              not self.is_equal_result and
> +              not os.path.isfile(self.result) and not is_tap):
>              shutil.copy(self.tmp_result, self.result)
>              short_status = 'new'
>              color_stdout("[ new ]\n", schema='test_new')

We should write new result files only under --update-result. Added the check
for the option:

 |                  os.remove(self.tmp_result)
 |          elif (self.is_executed_ok and
 |                not self.is_equal_result and
 | -              not os.path.isfile(self.result) and not is_tap):
 | +              not os.path.isfile(self.result) and
 | +              not is_tap and
 | +              lib.Options().args.update_result):
 |              shutil.copy(self.tmp_result, self.result)
 |              short_status = 'new'
 |              color_stdout("[ new ]\n", schema='test_new')

> +        elif (self.is_executed_ok and
> +              not self.is_equal_result and
> +              lib.Options().args.update_result):
> +            shutil.copy(self.tmp_result, self.result)
> +            short_status = 'updated'
> +            color_stdout("[ updated ]\n", schema='test_new')

It will write a new result file for a failed TAP13 test under the
option. I guess it is not the expected behaviour in most of cases. Say,
I changed something in the net.box implementation and want to update
result files for all related tests in a batch (`./test-run.py
--update-result net.box`), but it should not write a result file (and
report successful testing) if some TAP13 test fails. Added the check:

 |              color_stdout("[ new ]\n", schema='test_new')
 |          elif (self.is_executed_ok and
 |                not self.is_equal_result and
 | +              os.path.isfile(self.result) and
 | +              not is_tap and
 |                lib.Options().args.update_result):
 |              shutil.copy(self.tmp_result, self.result)
 |              short_status = 'updated'