[Tarantool-patches] [PATCH 2/4] test: out-of-source testing

[Igor Munkin]

Max,

Thanks for the patch! I left several comments below, please consider
them in addition to the main comments in the cover letter.

On 05.07.21, Maxim Kokryashkin wrote:
> From: Maxim Kokryashkin <m.kokryashkin at tarantool.org>
> 
> Part of tarantool/tarantool#5970
> 
> ---
> The patch `fperrad/lua-Harness at 60da289` is similar to the patch 
> `tarantool/luajit at d11c5bb`, but `make_specific_checks()` was 
> renamed to `_dofile()` by maintainer. Another difference is that 
> it seems like there is no definition for `_dofile()` in mainline 
> lua-Harness, so it should be user-defined somewhere. If it is not, 
> then `_dofile()` will act like `dofile()`. Considering this, we 
> should keep `make_specific_checks` implementation from 
> `tarantool/luajit at d11c5bb`, but rename it to `_dofile` and move 
> to `tap.lua`.

In addition to all comments for the first patch, I would like to clarify
the semantics of the change. The general idea was to give the user an
opportunity to *overload* <_dofile> for his own purposes, but the
default behaviour should be the same as <dofile>. We have been already
using this technique for PUC-Rio Lua 5.1 test suite and <_dofile> is
already defined in luajit-test-init.lua (please adjust the comment,
since as a result of the patch, <_dofile> is also used in lua-Harness
suite). Considering all comments, I propose the following rewording:

| The patch[1] from lua-Harness suite is quite similar to commit
| d11c5bbc08a118b11167c5d455d4024607dab662 ('test: adjust lua-Harness
| tests that use dofile'), with the one difference (except the naming):
| `make_specific_checks()` provides the particular behaviour to run the
| tests for out-of-source build, however `_dofile()` allows user to
| overload this function considering his purposes. If user leave
| `_dofile()` untouched, it defaults to `dofile()` Lua standard library
| function.
|
| It's worth to mention, that LuaJIT testing machinery has been using
| such approach since commit 734cc0f1d508429c2c3c46bd1860398db0051060
| ('test: adapt PUC-Rio suite for out-of-source build') and `_dofile()`
| behaviour for running tests when out of source build is used is
| already defined in luajit-test-init.lua.
|
| [1]: https://framagit.org/fperrad/lua-Harness/-/commit/60da289
|
| Part of tarantool/tarantool#5970
| Part of tarantool/tarantool#4473

If you are OK with the part above, feel free to paste it as is into the
commit message for the next version of the patch.

> 
>  test/lua-Harness-tests/101-boolean.t   | 2 +-
>  test/lua-Harness-tests/102-function.t  | 2 +-
>  test/lua-Harness-tests/103-nil.t       | 2 +-
>  test/lua-Harness-tests/104-number.t    | 2 +-
>  test/lua-Harness-tests/105-string.t    | 2 +-
>  test/lua-Harness-tests/106-table.t     | 2 +-
>  test/lua-Harness-tests/107-thread.t    | 2 +-
>  test/lua-Harness-tests/108-userdata.t  | 2 +-
>  test/lua-Harness-tests/203-lexico.t    | 8 ++++----
>  test/lua-Harness-tests/231-metatable.t | 2 +-
>  test/lua-Harness-tests/301-basic.t     | 2 +-
>  test/lua-Harness-tests/305-utf8.t      | 2 +-
>  test/lua-Harness-tests/404-ext.t       | 2 +-
>  test/lua-Harness-tests/tap.lua         | 4 ++++
>  14 files changed, 20 insertions(+), 16 deletions(-)
> 

<snipped>

> diff --git a/test/lua-Harness-tests/305-utf8.t b/test/lua-Harness-tests/305-utf8.t
> index 6c125388..1e975e6d 100755
> --- a/test/lua-Harness-tests/305-utf8.t
> +++ b/test/lua-Harness-tests/305-utf8.t
> @@ -40,7 +40,7 @@ if not utf8 then
>      nok(has_utf8, "no has_utf8")
>  else
>      plan'no_plan'
> -    make_specific_checks'lexico53/utf8.t'
> +    _dofile'lexico53/utf8.t'
>      if _VERSION >= 'Lua 5.4' then
>          make_specific_checks'lexico54/utf8.t'

This one should be also changed to <_dofile>.

>      end

<snipped>

> diff --git a/test/lua-Harness-tests/tap.lua b/test/lua-Harness-tests/tap.lua
> index 86cca4e0..b17709e5 100644
> --- a/test/lua-Harness-tests/tap.lua
> +++ b/test/lua-Harness-tests/tap.lua
> @@ -9,6 +9,10 @@
>  
>  ]]
>  
> +function _dofile (filename)
> +    return dofile(arg[0]:gsub('([^/]+)%.t$', '') .. filename)
> +end

At first, <_dofile> should use <dofile> for the default behaviour.
Furthermore, such definition can redefine the function specified by
user, if this function is created after the user-defined one. So, you
need to check whether <_dofile> is defined or use <dofile> if it is not.
BTW, please drop <make_specific_checks> routine from tap.lua, since it's
not used anymore.

> +
>  function _retrieve_progname ()
>      local i = 0
>      while arg[i] do
> -- 
> 2.31.1
> 

-- 
Best regards,
IM