[Tarantool-patches] [PATCH luajit 0/5] Tarantool tests enhancements

[Tarantool-patches] [PATCH luajit 0/5] Tarantool tests enhancements

[Igor Munkin via Tarantool-patches]

I decided to split the original series[1], adjusting JIT-related tests
in tarantool-tests suite into two separate patchsets: one with
enhancements for test suite and another one for JIT-related tweaks.

This series consists of the following changes:

1. The first one is a simple tiny maintenance patch to remove excess
   checks in testing workflows.
2. The second one is epic one: unfortunately, <utils.selfrun> is too
   complex to be maintained, so the corresponding tests are split
   into two files: the test itself and the script to be run by the
   test. There is a new helper introduced within this patch, and
   three tests are refactored.
3. The third and the fourth patches introduce <tap.skipall> and
   <tap.skiprest> helpers.
4. The fourth patch makes skipcond helper more convenient: it becomes
   multi-conditional and it yields the test object, so we can organize
   more readable chains of skip conditions.

[1]: https://lists.tarantool.org/tarantool-patches/cover.1676304797.git.imun@tarantool.org/T/#t

Branch: https://github.com/tarantool/luajit/tree/imun/tap-enhancements
CI: https://github.com/tarantool/luajit/commit/4e6d8b4
Tarantool PR: https://github.com/tarantool/tarantool/pull/8288

Igor Munkin (5):
  ci: use LuaJIT-test target in testing workflows
  test: stop using utils.selfrun in tests
  test: introduce test:skipall TAP helper
  test: introduce test:skiprest TAP helper
  test: make skipcond helper more convenient

 .github/workflows/exotic-builds-testing.yml   |  2 +-
 .github/workflows/testing.yml                 |  2 +-
 .../gh-4199-gc64-fuse.test.lua                | 11 ++-
 .../gh-4427-ffi-sandwich.test.lua             | 95 ++++++++++---------
 .../gh-4427-ffi-sandwich/script.lua           | 25 +++++
 .../gh-5813-resolving-of-c-symbols.test.lua   | 19 ++--
 ...4-add-proto-trace-sysprof-default.test.lua | 15 ++-
 .../lj-351-print-tostring-number.test.lua     | 34 +++----
 .../lj-351-print-tostring-number/script.lua   |  9 ++
 .../lj-430-maxirconst.test.lua                | 10 +-
 .../lj-586-debug-non-string-error.test.lua    |  2 +-
 .../lj-603-err-snap-restore.test.lua          | 19 ++--
 ...lj-672-cdata-allocation-recording.test.lua | 12 +--
 .../lj-906-fix-err-mem.test.lua               | 12 +--
 .../lj-flush-on-trace.test.lua                | 94 +++++++++---------
 .../lj-flush-on-trace/script.lua              | 23 +++++
 .../misclib-getmetrics-capi.test.lua          | 17 ++--
 .../misclib-getmetrics-lapi.test.lua          | 13 +--
 .../misclib-memprof-lapi.test.lua             | 28 +++---
 .../misclib-sysprof-capi.test.lua             | 19 ++--
 .../misclib-sysprof-lapi.test.lua             | 19 ++--
 test/tarantool-tests/tap.lua                  | 42 +++++++-
 test/tarantool-tests/utils.lua                | 88 +++++++----------
 23 files changed, 333 insertions(+), 277 deletions(-)
 create mode 100644 test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
 create mode 100644 test/tarantool-tests/lj-351-print-tostring-number/script.lua
 create mode 100644 test/tarantool-tests/lj-flush-on-trace/script.lua

-- 
2.30.2

[Tarantool-patches] [PATCH luajit 1/5] ci: use LuaJIT-test target in testing workflows

[Igor Munkin via Tarantool-patches]

There is no need to run linter checks (such as luacheck) in scope of the
testing workflows, since there is a separate one for this. Hence, only
<LuaJIT-test> target should be used in these workflows.

Signed-off-by: Igor Munkin <imun@tarantool.org>
---
 .github/workflows/exotic-builds-testing.yml | 2 +-
 .github/workflows/testing.yml               | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/exotic-builds-testing.yml b/.github/workflows/exotic-builds-testing.yml
index 67ab9cc3..62b83931 100644
--- a/.github/workflows/exotic-builds-testing.yml
+++ b/.github/workflows/exotic-builds-testing.yml
@@ -74,5 +74,5 @@ jobs:
         run: cmake --build . --parallel
         working-directory: ${{ env.BUILDDIR }}
       - name: test
-        run: cmake --build . --parallel --target test
+        run: cmake --build . --parallel --target LuaJIT-test
         working-directory: ${{ env.BUILDDIR }}
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
index b7a6cfe2..8300c443 100644
--- a/.github/workflows/testing.yml
+++ b/.github/workflows/testing.yml
@@ -74,7 +74,7 @@ jobs:
         run: cmake --build . --parallel
         working-directory: ${{ env.BUILDDIR }}
       - name: test
-        run: cmake --build . --parallel --target test
+        run: cmake --build . --parallel --target LuaJIT-test
         working-directory: ${{ env.BUILDDIR }}
 
 
-- 
2.30.2

Re: [Tarantool-patches] [PATCH luajit 1/5] ci: use LuaJIT-test target in testing workflows

[Sergey Kaplun via Tarantool-patches]

Hi, Igor!
Thanks for the patch!
LGTM!

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 1/5] ci: use LuaJIT-test target in testing workflows

[Maxim Kokryashkin via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 1700 bytes --]


Hi, Igor!
Thanks for the patch!
LGTM as trivial.
--
Best regards,
Maxim Kokryashkin
 
  
>Понедельник, 27 февраля 2023, 12:10 +03:00 от Igor Munkin <imun@tarantool.org>:
> 
>There is no need to run linter checks (such as luacheck) in scope of the
>testing workflows, since there is a separate one for this. Hence, only
><LuaJIT-test> target should be used in these workflows.
>
>Signed-off-by: Igor Munkin < imun@tarantool.org >
>---
> .github/workflows/exotic-builds-testing.yml | 2 +-
> .github/workflows/testing.yml | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
>diff --git a/.github/workflows/exotic-builds-testing.yml b/.github/workflows/exotic-builds-testing.yml
>index 67ab9cc3..62b83931 100644
>--- a/.github/workflows/exotic-builds-testing.yml
>+++ b/.github/workflows/exotic-builds-testing.yml
>@@ -74,5 +74,5 @@ jobs:
>         run: cmake --build . --parallel
>         working-directory: ${{ env.BUILDDIR }}
>       - name: test
>- run: cmake --build . --parallel --target test
>+ run: cmake --build . --parallel --target LuaJIT-test
>         working-directory: ${{ env.BUILDDIR }}
>diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
>index b7a6cfe2..8300c443 100644
>--- a/.github/workflows/testing.yml
>+++ b/.github/workflows/testing.yml
>@@ -74,7 +74,7 @@ jobs:
>         run: cmake --build . --parallel
>         working-directory: ${{ env.BUILDDIR }}
>       - name: test
>- run: cmake --build . --parallel --target test
>+ run: cmake --build . --parallel --target LuaJIT-test
>         working-directory: ${{ env.BUILDDIR }}
> 
> 
>--
>2.30.2
 

[-- Attachment #2: Type: text/html, Size: 2552 bytes --]

[Tarantool-patches] [PATCH luajit 2/5] test: stop using utils.selfrun in tests

[Igor Munkin via Tarantool-patches]

Unfortunately, <utils.selfrun> is too complex to be maintained, so the
corresponding tests are split into two files: the test itself and the
script to be run by the test. As a result of the patch <utils.makecmd>
helper is introduced: it inherits some approaches from <utils.selfrun>,
but it's considered for more general use.

Signed-off-by: Igor Munkin <imun@tarantool.org>
---
 .../gh-4427-ffi-sandwich.test.lua             | 88 +++++++++++--------
 .../gh-4427-ffi-sandwich/script.lua           | 25 ++++++
 .../lj-351-print-tostring-number.test.lua     | 34 +++----
 .../lj-351-print-tostring-number/script.lua   |  9 ++
 .../lj-586-debug-non-string-error.test.lua    |  2 +-
 .../lj-flush-on-trace.test.lua                | 87 ++++++++++--------
 .../lj-flush-on-trace/script.lua              | 23 +++++
 test/tarantool-tests/utils.lua                | 80 +++++++----------
 8 files changed, 200 insertions(+), 148 deletions(-)
 create mode 100644 test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
 create mode 100644 test/tarantool-tests/lj-351-print-tostring-number/script.lua
 create mode 100644 test/tarantool-tests/lj-flush-on-trace/script.lua

diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
index dd02130c..06985dcd 100644
--- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
@@ -3,52 +3,62 @@ local utils = require('utils')
 -- Disabled on *BSD due to #4819.
 utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
 
-utils.selfrun(arg, {
-  {
-    arg = {
-      1, -- hotloop (arg[1])
-      1, -- trigger (arg[2])
-    },
-    msg = 'Trace is aborted',
-    res = tostring(3), -- hotloop + trigger + 1
-    test = 'is',
-  },
-  {
-    arg = {
-      1, -- hotloop (arg[1])
-      2, -- trigger (arg[2])
-    },
-    msg = 'Trace is recorded',
-    res = 'Lua VM re%-entrancy is detected while executing the trace',
-    test = 'like',
-  },
-})
-
------ Test payload. ----------------------------------------------
-
-local cfg = {
-  hotloop = arg[1] or 1,
-  trigger = arg[2] or 1,
-}
+local tap = require('tap')
 
-local ffi = require('ffi')
-local ffisandwich = ffi.load('libsandwich')
-ffi.cdef('int increment(struct sandwich *state, int i)')
+local test = tap.test('gh-4427-ffi-sandwich')
+test:plan(2)
 
--- Save the current coroutine and set the value to trigger
--- <increment> call the Lua routine instead of C implementation.
-local sandwich = require('libsandwich')(cfg.trigger)
+-- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
+-- with the given environment, launch options and CLI arguments.
+local script = utils.makecmd(arg, {
+  -- XXX: Apple tries their best to "protect their users from
+  -- malware". As a result SIP (see the link[1] below) has been
+  -- designed and released. Now, Apple developers are so
+  -- protected, that they can load nothing being not installed in
+  -- the system, since the environment is sanitized before the
+  -- child process is launched. In particular, environment
+  -- variables starting with DYLD_ and LD_ are unset for child
+  -- process. For more info, see the docs[2] below.
+  --
+  -- The environment variable below is used by FFI machinery to
+  -- find the proper shared library.
+  --
+  -- luacheck: push no max comment line length
+  --
+  -- [1]: https://support.apple.com/en-us/HT204899
+  -- [2]: https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html
+  --
+  -- luacheck: pop
+  env = { DYLD_LIBRARY_PATH = os.getenv('DYLD_LIBRARY_PATH') },
+  redirect = '2>&1',
+})
 
 -- Depending on trigger and hotloop values the following contexts
 -- are possible:
 -- * if trigger <= hotloop -> trace recording is aborted
 -- * if trigger >  hotloop -> trace is recorded but execution
 --   leads to panic
-jit.opt.start("3", string.format("hotloop=%d", cfg.hotloop))
+local hotloop = 1
+local cases = {
+  abort = {
+    trigger = hotloop,
+    expected = '#4427 still works',
+    test = 'is',
+    message = 'Trace is aborted',
+  },
+  panic = {
+    trigger = hotloop + 1,
+    expected = 'Lua VM re%-entrancy is detected while executing the trace',
+    test = 'like',
+    message = 'Trace is compiled',
+  },
+}
 
-local res
-for i = 0, cfg.trigger + cfg.hotloop do
-  res = ffisandwich.increment(sandwich, i)
+for _, subtest in pairs(cases) do
+  local output = script(hotloop, subtest.trigger)
+  -- XXX: explicitly pass <test> as an argument to <testf>
+  -- to emulate test:is(...), test:like(...), etc.
+  test[subtest.test](test, output, subtest.expected, subtest.message)
 end
--- Check the resulting value if panic didn't occur earlier.
-print(res)
+
+os.exit(test:check() and 0 or 1)
diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
new file mode 100644
index 00000000..9ecd964e
--- /dev/null
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
@@ -0,0 +1,25 @@
+local hotloop = assert(arg[1], 'hotloop argument is missing')
+local trigger = assert(arg[2], 'trigger argument is missing')
+
+local ffi = require('ffi')
+local ffisandwich = ffi.load('libsandwich')
+ffi.cdef('int increment(struct sandwich *state, int i)')
+
+-- Save the current coroutine and set the value to trigger
+-- <increment> call the Lua routine instead of C implementation.
+local sandwich = require('libsandwich')(trigger)
+
+-- Depending on trigger and hotloop values the following contexts
+-- are possible:
+-- * if trigger <= hotloop -> trace recording is aborted
+-- * if trigger >  hotloop -> trace is recorded but execution
+--   leads to panic
+jit.opt.start("3", string.format("hotloop=%d", hotloop))
+
+local res
+for i = 0, hotloop + trigger do
+  res = ffisandwich.increment(sandwich, i)
+end
+-- Check the resulting value if panic didn't occur earlier.
+assert(res == hotloop + trigger + 1, 'res is calculated correctly')
+io.write('#4427 still works')
diff --git a/test/tarantool-tests/lj-351-print-tostring-number.test.lua b/test/tarantool-tests/lj-351-print-tostring-number.test.lua
index da5b31be..72a9ec2b 100644
--- a/test/tarantool-tests/lj-351-print-tostring-number.test.lua
+++ b/test/tarantool-tests/lj-351-print-tostring-number.test.lua
@@ -1,4 +1,9 @@
-local utils = require('utils')
+local tap = require('tap')
+
+local test = tap.test('lj-351-print-tostring-number')
+test:plan(8)
+
+local script = require('utils').makecmd(arg)
 
 local cases = {
   {typename = 'nil', value = 'nil'},
@@ -15,27 +20,10 @@ local cases = {
   {typename = 'cdata', value = '1ULL'}
 }
 
-local checks = {}
-
-for i, case in pairs(cases) do
-  checks[i] = {
-    arg = {('"%s"'):format(case.value), case.typename},
-    msg = ('%s'):format(case.typename),
-    res = ('__tostring is reloaded for %s'):format(case.typename),
-    test = 'is',
-  }
+for _, subtest in pairs(cases) do
+  local output = script(('"%s"'):format(subtest.value), subtest.typename)
+  test:is(output, ('__tostring is reloaded for %s'):format(subtest.typename),
+          ('subtest is OK for %s type'):format(subtest.typename))
 end
 
-utils.selfrun(arg, checks)
-
------ Test payload. ----------------------------------------------
-
-local test = [[
-  local testvar = %s
-  debug.setmetatable(testvar, {__tostring = function(o)
-    return ('__tostring is reloaded for %s'):format(type(o))
-  end})
-  print(testvar)
-]]
-
-pcall(load(test:format(unpack(arg))))
+os.exit(test:check() and 0 or 1)
diff --git a/test/tarantool-tests/lj-351-print-tostring-number/script.lua b/test/tarantool-tests/lj-351-print-tostring-number/script.lua
new file mode 100644
index 00000000..c3066f49
--- /dev/null
+++ b/test/tarantool-tests/lj-351-print-tostring-number/script.lua
@@ -0,0 +1,9 @@
+local test = [[
+  local testvar = %s
+  debug.setmetatable(testvar, {__tostring = function(o)
+    return ('__tostring is reloaded for %s'):format(type(o))
+  end})
+  print(testvar)
+]]
+
+pcall(load(test:format(unpack(arg))))
diff --git a/test/tarantool-tests/lj-586-debug-non-string-error.test.lua b/test/tarantool-tests/lj-586-debug-non-string-error.test.lua
index f02353fe..dcb730a2 100644
--- a/test/tarantool-tests/lj-586-debug-non-string-error.test.lua
+++ b/test/tarantool-tests/lj-586-debug-non-string-error.test.lua
@@ -8,7 +8,7 @@ test:plan(1)
 -- that testing the debug interactive interface always ends with
 -- sending commands to another instance via stdin. However, the
 -- module with test helpers lacks the suitable routine.
--- `utils.selfrun()` doesn't fit for this, since `debug.debug()`
+-- `utils.makecmd()` doesn't fit for this, since `debug.debug()`
 -- captures `io.stdin` and waits at `fgets()` in debug busy loop.
 -- As it's already mentioned, such tests are not usual, so there
 -- is no need to introduce a new helper to utils module (at least
diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua
index c46b93f0..3351cc5a 100644
--- a/test/tarantool-tests/lj-flush-on-trace.test.lua
+++ b/test/tarantool-tests/lj-flush-on-trace.test.lua
@@ -3,51 +3,62 @@ local utils = require('utils')
 -- Disabled on *BSD due to #4819.
 utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
 
-utils.selfrun(arg, {
-  {
-    arg = {
-      1, -- hotloop (arg[1])
-      1, -- trigger (arg[2])
-    },
-    msg = 'Trace is aborted',
-    res = 'OK',
-    test = 'is',
-  },
-  {
-    arg = {
-      1, -- hotloop (arg[1])
-      2, -- trigger (arg[2])
-    },
-    msg = 'Trace is recorded',
-    res = 'JIT mode change is detected while executing the trace',
-    test = 'like',
-  },
-})
-
------ Test payload. ----------------------------------------------
-
-local cfg = {
-  hotloop = arg[1] or 1,
-  trigger = arg[2] or 1,
-}
+local tap = require('tap')
 
-local ffi = require('ffi')
-local ffiflush = ffi.load('libflush')
-ffi.cdef('void flush(struct flush *state, int i)')
+local test = tap.test('lj-flush-on-trace')
+test:plan(2)
 
--- Save the current coroutine and set the value to trigger
--- <flush> call the Lua routine instead of C implementation.
-local flush = require('libflush')(cfg.trigger)
+-- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
+-- with the given environment, launch options and CLI arguments.
+local script = utils.makecmd(arg, {
+  -- XXX: Apple tries their best to "protect their users from
+  -- malware". As a result SIP (see the link[1] below) has been
+  -- designed and released. Now, Apple developers are so
+  -- protected, that they can load nothing being not installed in
+  -- the system, since the environment is sanitized before the
+  -- child process is launched. In particular, environment
+  -- variables starting with DYLD_ and LD_ are unset for child
+  -- process. For more info, see the docs[2] below.
+  --
+  -- The environment variable below is used by FFI machinery to
+  -- find the proper shared library.
+  --
+  -- luacheck: push no max comment line length
+  --
+  -- [1]: https://support.apple.com/en-us/HT204899
+  -- [2]: https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html
+  --
+  -- luacheck: pop
+  env = { DYLD_LIBRARY_PATH = os.getenv('DYLD_LIBRARY_PATH') },
+  redirect = '2>&1',
+})
 
 -- Depending on trigger and hotloop values the following contexts
 -- are possible:
 -- * if trigger <= hotloop -> trace recording is aborted
 -- * if trigger >  hotloop -> trace is recorded but execution
 --   leads to panic
-jit.opt.start("3", string.format("hotloop=%d", cfg.hotloop))
+local hotloop = 1
+local cases = {
+  abort = {
+    trigger = hotloop,
+    expected = 'LJ flush still works',
+    test = 'is',
+    message = 'Trace is aborted',
+  },
+  panic = {
+    trigger = hotloop + 1,
+    expected = 'JIT mode change is detected while executing the trace',
+    test = 'like',
+    message = 'Trace is compiled',
+  },
+}
 
-for i = 0, cfg.trigger + cfg.hotloop do
-  ffiflush.flush(flush, i)
+for _, subtest in pairs(cases) do
+  local output = script(hotloop, subtest.trigger)
+  -- XXX: explicitly pass <test> as an argument to <testf>
+  -- to emulate test:is(...), test:like(...), etc.
+  test[subtest.test](test, output, subtest.expected, subtest.message)
 end
--- Panic didn't occur earlier.
-print('OK')
+
+os.exit(test:check() and 0 or 1)
diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
new file mode 100644
index 00000000..d2c35534
--- /dev/null
+++ b/test/tarantool-tests/lj-flush-on-trace/script.lua
@@ -0,0 +1,23 @@
+local hotloop = assert(arg[1], 'hotloop argument is missing')
+local trigger = assert(arg[2], 'trigger argument is missing')
+
+local ffi = require('ffi')
+local ffiflush = ffi.load('libflush')
+ffi.cdef('void flush(struct flush *state, int i)')
+
+-- Save the current coroutine and set the value to trigger
+-- <flush> call the Lua routine instead of C implementation.
+local flush = require('libflush')(trigger)
+
+-- Depending on trigger and hotloop values the following contexts
+-- are possible:
+-- * if trigger <= hotloop -> trace recording is aborted
+-- * if trigger >  hotloop -> trace is recorded but execution
+--   leads to panic
+jit.opt.start("3", string.format("hotloop=%d", hotloop))
+
+for i = 0, trigger + hotloop do
+  ffiflush.flush(flush, i)
+end
+-- Panic didn't occur earlier.
+io.write('LJ flush still works')
diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
index eb11d40d..8355149b 100644
--- a/test/tarantool-tests/utils.lua
+++ b/test/tarantool-tests/utils.lua
@@ -1,7 +1,6 @@
 local M = {}
 
 local ffi = require('ffi')
-local tap = require('tap')
 local bc = require('jit.bc')
 local bit = require('bit')
 
@@ -44,55 +43,42 @@ function M.luacmd(args)
   return table.concat(args, ' ', idx + 1, -1)
 end
 
-local function unshiftenv(variable, value, sep)
-  local envvar = os.getenv(variable)
-  return ('%s="%s%s"'):format(variable, value,
-                              envvar and ('%s%s'):format(sep, envvar) or '')
+local function makeenv(tabenv)
+  if tabenv == nil then return '' end
+  local flatenv = {}
+  for var, value in pairs(tabenv) do
+    table.insert(flatenv, ('%s=%s'):format(var, value))
+  end
+  return table.concat(flatenv, ' ')
 end
 
-function M.selfrun(arg, checks)
-  -- If TEST_SELFRUN is set, it means the test has been run via
-  -- <io.popen>, so just return from this routine and proceed
-  -- the execution to the test payload, ...
-  if os.getenv('TEST_SELFRUN') then return end
-
-  -- ... otherwise initialize <tap>, setup testing environment
-  -- and run this chunk via <io.popen> for each case in <checks>.
-  -- XXX: The function doesn't return back from this moment. It
-  -- checks whether all assertions are fine and exits.
-
-  local test = tap.test(arg[0]:match('/?(.+)%.test%.lua'))
-
-  test:plan(#checks)
-
-  local libext = package.cpath:match('?.(%a+);')
-  local vars = {
+-- <makecmd> creates a command that runs %testname%/script.lua by
+-- <LUAJIT_TEST_BINARY> with the given environment, launch options
+-- and CLI arguments. The function yields an object (i.e. table)
+-- with the aforementioned parameters. To launch the command just
+-- call the object.
+function M.makecmd(arg, opts)
+  return setmetatable({
     LUABIN = M.luacmd(arg),
-    SCRIPT = arg[0],
-    PATH   = arg[0]:gsub('%.test%.lua', ''),
-    SUFFIX = libext,
-    ENV = table.concat({
-      unshiftenv('LUA_PATH', '<PATH>/?.lua', ';'),
-      unshiftenv('LUA_CPATH', '<PATH>/?.<SUFFIX>', ';'),
-      unshiftenv((libext == 'dylib' and 'DYLD' or 'LD') .. '_LIBRARY_PATH',
-                 '<PATH>', ':'),
-      'TEST_SELFRUN=1',
-    }, ' '),
-  }
-
-  local cmd = string.gsub('<ENV> <LUABIN> 2>&1 <SCRIPT>', '%<(%w+)>', vars)
-
-  for _, ch in pairs(checks) do
-    local testf = test[ch.test]
-    assert(testf, ("tap doesn't provide test.%s function"):format(ch.test))
-    local proc = io.popen((cmd .. (' %s'):rep(#ch.arg)):format(unpack(ch.arg)))
-    local res = proc:read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
-    -- XXX: explicitly pass <test> as an argument to <testf>
-    -- to emulate test:is(...), test:like(...), etc.
-    testf(test, res, ch.res, ch.msg)
-  end
-
-  os.exit(test:check() and 0 or 1)
+    SCRIPT = opts and opts.script or arg[0]:gsub('%.test%.lua$', '/script.lua'),
+    ENV = opts and makeenv(opts.env) or '',
+    REDIRECT = opts and opts.redirect or '',
+  }, {
+    __call = function(self, ...)
+      -- This line just makes the command for <io.popen> by the
+      -- following steps:
+      -- 1. Replace the placeholders with the corresponding values
+      --    given to the command constructor (e.g. script, env)
+      -- 2. Join all CLI arguments given to the __call metamethod
+      -- 3. Concatenate the results of step 1 and step 2 to obtain
+      --    the resulting command.
+      local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)
+                  .. (' %s'):rep(select('#', ...)):format(...)
+      -- Trim both leading and trailing whitespace from the output
+      -- produced by the child process.
+      return io.popen(cmd):read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
+    end
+  })
 end
 
 function M.skipcond(condition, message)
-- 
2.30.2

Re: [Tarantool-patches] [PATCH luajit 2/5] test: stop using utils.selfrun in tests

[Sergey Kaplun via Tarantool-patches]

Hi, Igor!
Thanks for the fixes!
LGTM, just a few minor nits below.

On 27.02.23, Igor Munkin wrote:
> Unfortunately, <utils.selfrun> is too complex to be maintained, so the
> corresponding tests are split into two files: the test itself and the
> script to be run by the test. As a result of the patch <utils.makecmd>
> helper is introduced: it inherits some approaches from <utils.selfrun>,
> but it's considered for more general use.
> 
> Signed-off-by: Igor Munkin <imun@tarantool.org>
> ---
>  .../gh-4427-ffi-sandwich.test.lua             | 88 +++++++++++--------
>  .../gh-4427-ffi-sandwich/script.lua           | 25 ++++++
>  .../lj-351-print-tostring-number.test.lua     | 34 +++----
>  .../lj-351-print-tostring-number/script.lua   |  9 ++
>  .../lj-586-debug-non-string-error.test.lua    |  2 +-
>  .../lj-flush-on-trace.test.lua                | 87 ++++++++++--------
>  .../lj-flush-on-trace/script.lua              | 23 +++++
>  test/tarantool-tests/utils.lua                | 80 +++++++----------
>  8 files changed, 200 insertions(+), 148 deletions(-)
>  create mode 100644 test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>  create mode 100644 test/tarantool-tests/lj-351-print-tostring-number/script.lua
>  create mode 100644 test/tarantool-tests/lj-flush-on-trace/script.lua
> 
> diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
> index dd02130c..06985dcd 100644
> --- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
> +++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua

<snipped>

> diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
> new file mode 100644
> index 00000000..9ecd964e
> --- /dev/null
> +++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua

<snipped>

> +jit.opt.start("3", string.format("hotloop=%d", hotloop))

Do we need set level 3 direct here?
Also, I suggest to use single quotes according to our coding style in
tests.

> +

<snipped>

> diff --git a/test/tarantool-tests/lj-351-print-tostring-number.test.lua b/test/tarantool-tests/lj-351-print-tostring-number.test.lua
> index da5b31be..72a9ec2b 100644
> --- a/test/tarantool-tests/lj-351-print-tostring-number.test.lua
> +++ b/test/tarantool-tests/lj-351-print-tostring-number.test.lua

<snipped>

> diff --git a/test/tarantool-tests/lj-351-print-tostring-number/script.lua b/test/tarantool-tests/lj-351-print-tostring-number/script.lua
> new file mode 100644
> index 00000000..c3066f49
> --- /dev/null
> +++ b/test/tarantool-tests/lj-351-print-tostring-number/script.lua

<snipped>

> diff --git a/test/tarantool-tests/lj-586-debug-non-string-error.test.lua b/test/tarantool-tests/lj-586-debug-non-string-error.test.lua
> index f02353fe..dcb730a2 100644
> --- a/test/tarantool-tests/lj-586-debug-non-string-error.test.lua
> +++ b/test/tarantool-tests/lj-586-debug-non-string-error.test.lua

<snipped>

> diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua
> index c46b93f0..3351cc5a 100644
> --- a/test/tarantool-tests/lj-flush-on-trace.test.lua
> +++ b/test/tarantool-tests/lj-flush-on-trace.test.lua

<snipped>

> diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
> new file mode 100644
> index 00000000..d2c35534
> --- /dev/null
> +++ b/test/tarantool-tests/lj-flush-on-trace/script.lua

<snipped>

> +jit.opt.start("3", string.format("hotloop=%d", hotloop))

Do we need set level 3 direct here?
Also, I suggest to use single quotes according to our coding style in
tests.

> +

<snipped>

> diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
> index eb11d40d..8355149b 100644
> --- a/test/tarantool-tests/utils.lua
> +++ b/test/tarantool-tests/utils.lua

<snipped>

> +    SCRIPT = opts and opts.script or arg[0]:gsub('%.test%.lua$', '/script.lua'),
> +    ENV = opts and makeenv(opts.env) or '',
> +    REDIRECT = opts and opts.redirect or '',
> +  }, {
> +    __call = function(self, ...)
> +      -- This line just makes the command for <io.popen> by the
> +      -- following steps:
> +      -- 1. Replace the placeholders with the corresponding values
> +      --    given to the command constructor (e.g. script, env)
> +      -- 2. Join all CLI arguments given to the __call metamethod
> +      -- 3. Concatenate the results of step 1 and step 2 to obtain
> +      --    the resulting command.

Extra dot at the end of the sentence (or missing dots for previous
bullets).

> +      local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)
> +                  .. (' %s'):rep(select('#', ...)):format(...)
> +      -- Trim both leading and trailing whitespace from the output
> +      -- produced by the child process.
> +      return io.popen(cmd):read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
> +    end
> +  })
>  end
>  
>  function M.skipcond(condition, message)
> -- 
> 2.30.2
> 

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 2/5] test: stop using utils.selfrun in tests

[Igor Munkin via Tarantool-patches]

Sergey,

Thanks for your review! See my answes below.

On 27.02.23, Sergey Kaplun wrote:
> Hi, Igor!
> Thanks for the fixes!
> LGTM, just a few minor nits below.
> 
> On 27.02.23, Igor Munkin wrote:
> > Unfortunately, <utils.selfrun> is too complex to be maintained, so the
> > corresponding tests are split into two files: the test itself and the
> > script to be run by the test. As a result of the patch <utils.makecmd>
> > helper is introduced: it inherits some approaches from <utils.selfrun>,
> > but it's considered for more general use.
> > 
> > Signed-off-by: Igor Munkin <imun@tarantool.org>
> > ---
> >  .../gh-4427-ffi-sandwich.test.lua             | 88 +++++++++++--------
> >  .../gh-4427-ffi-sandwich/script.lua           | 25 ++++++
> >  .../lj-351-print-tostring-number.test.lua     | 34 +++----
> >  .../lj-351-print-tostring-number/script.lua   |  9 ++
> >  .../lj-586-debug-non-string-error.test.lua    |  2 +-
> >  .../lj-flush-on-trace.test.lua                | 87 ++++++++++--------
> >  .../lj-flush-on-trace/script.lua              | 23 +++++
> >  test/tarantool-tests/utils.lua                | 80 +++++++----------
> >  8 files changed, 200 insertions(+), 148 deletions(-)
> >  create mode 100644 test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
> >  create mode 100644 test/tarantool-tests/lj-351-print-tostring-number/script.lua
> >  create mode 100644 test/tarantool-tests/lj-flush-on-trace/script.lua
> > 

<snipped>

> > diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
> > new file mode 100644
> > index 00000000..9ecd964e
> > --- /dev/null
> > +++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
> 
> <snipped>
> 
> > +jit.opt.start("3", string.format("hotloop=%d", hotloop))
> 
> Do we need set level 3 direct here?
> Also, I suggest to use single quotes according to our coding style in
> tests.

I don't remember, so removed. Quotes are fixed. The diff is below:

================================================================================

diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
index 9ecd964e..a7445e92 100644
--- a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
@@ -14,7 +14,7 @@ local sandwich = require('libsandwich')(trigger)
 -- * if trigger <= hotloop -> trace recording is aborted
 -- * if trigger >  hotloop -> trace is recorded but execution
 --   leads to panic
-jit.opt.start("3", string.format("hotloop=%d", hotloop))
+jit.opt.start(string.format('hotloop=%d', hotloop))
 
 local res
 for i = 0, hotloop + trigger do

================================================================================

> 
> > +

<snipped>

> > diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
> > new file mode 100644
> > index 00000000..d2c35534
> > --- /dev/null
> > +++ b/test/tarantool-tests/lj-flush-on-trace/script.lua
> 
> <snipped>
> 
> > +jit.opt.start("3", string.format("hotloop=%d", hotloop))
> 
> Do we need set level 3 direct here?
> Also, I suggest to use single quotes according to our coding style in
> tests.

Ditto.

================================================================================

diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
index d2c35534..d25c4a33 100644
--- a/test/tarantool-tests/lj-flush-on-trace/script.lua
+++ b/test/tarantool-tests/lj-flush-on-trace/script.lua
@@ -14,7 +14,7 @@ local flush = require('libflush')(trigger)
 -- * if trigger <= hotloop -> trace recording is aborted
 -- * if trigger >  hotloop -> trace is recorded but execution
 --   leads to panic
-jit.opt.start("3", string.format("hotloop=%d", hotloop))
+jit.opt.start(string.format('hotloop=%d', hotloop))
 
 for i = 0, trigger + hotloop do
   ffiflush.flush(flush, i)

================================================================================

> 
> > +
> 
> <snipped>
> 
> > diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
> > index eb11d40d..8355149b 100644
> > --- a/test/tarantool-tests/utils.lua
> > +++ b/test/tarantool-tests/utils.lua
> 
> <snipped>
> 
> > +    SCRIPT = opts and opts.script or arg[0]:gsub('%.test%.lua$', '/script.lua'),
> > +    ENV = opts and makeenv(opts.env) or '',
> > +    REDIRECT = opts and opts.redirect or '',
> > +  }, {
> > +    __call = function(self, ...)
> > +      -- This line just makes the command for <io.popen> by the
> > +      -- following steps:
> > +      -- 1. Replace the placeholders with the corresponding values
> > +      --    given to the command constructor (e.g. script, env)
> > +      -- 2. Join all CLI arguments given to the __call metamethod
> > +      -- 3. Concatenate the results of step 1 and step 2 to obtain
> > +      --    the resulting command.
> 
> Extra dot at the end of the sentence (or missing dots for previous
> bullets).

Thanks, fixed! The diff is below:

================================================================================

diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
index 83716bb3..8c1538d6 100644
--- a/test/tarantool-tests/utils.lua
+++ b/test/tarantool-tests/utils.lua
@@ -68,8 +68,8 @@ function M.makecmd(arg, opts)
       -- This line just makes the command for <io.popen> by the
       -- following steps:
       -- 1. Replace the placeholders with the corresponding values
-      --    given to the command constructor (e.g. script, env)
-      -- 2. Join all CLI arguments given to the __call metamethod
+      --    given to the command constructor (e.g. script, env).
+      -- 2. Join all CLI arguments given to the __call metamethod.
       -- 3. Concatenate the results of step 1 and step 2 to obtain
       --    the resulting command.
       local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)

================================================================================

> 
> > +      local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)
> > +                  .. (' %s'):rep(select('#', ...)):format(...)
> > +      -- Trim both leading and trailing whitespace from the output
> > +      -- produced by the child process.
> > +      return io.popen(cmd):read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
> > +    end
> > +  })
> >  end
> >  
> >  function M.skipcond(condition, message)
> > -- 
> > 2.30.2
> > 
> 
> -- 
> Best regards,
> Sergey Kaplun

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH luajit 2/5] test: stop using utils.selfrun in tests

[Maxim Kokryashkin via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 6802 bytes --]


Hi, Igor!
Thanks for the patch!
LGTM
--
Best regards,
Maxim Kokryashkin
 
  
>Понедельник, 27 февраля 2023, 21:07 +03:00 от Igor Munkin <imun@tarantool.org>:
> 
>Sergey,
>
>Thanks for your review! See my answes below.
>
>On 27.02.23, Sergey Kaplun wrote:
>> Hi, Igor!
>> Thanks for the fixes!
>> LGTM, just a few minor nits below.
>>
>> On 27.02.23, Igor Munkin wrote:
>> > Unfortunately, <utils.selfrun> is too complex to be maintained, so the
>> > corresponding tests are split into two files: the test itself and the
>> > script to be run by the test. As a result of the patch <utils.makecmd>
>> > helper is introduced: it inherits some approaches from <utils.selfrun>,
>> > but it's considered for more general use.
>> >
>> > Signed-off-by: Igor Munkin < imun@tarantool.org >
>> > ---
>> > .../gh-4427-ffi-sandwich.test.lua | 88 +++++++++++--------
>> > .../gh-4427-ffi-sandwich/script.lua | 25 ++++++
>> > .../lj-351-print-tostring-number.test.lua | 34 +++----
>> > .../lj-351-print-tostring-number/script.lua | 9 ++
>> > .../lj-586-debug-non-string-error.test.lua | 2 +-
>> > .../lj-flush-on-trace.test.lua | 87 ++++++++++--------
>> > .../lj-flush-on-trace/script.lua | 23 +++++
>> > test/tarantool-tests/utils.lua | 80 +++++++----------
>> > 8 files changed, 200 insertions(+), 148 deletions(-)
>> > create mode 100644 test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>> > create mode 100644 test/tarantool-tests/lj-351-print-tostring-number/script.lua
>> > create mode 100644 test/tarantool-tests/lj-flush-on-trace/script.lua
>> >
>
><snipped>
>
>> > diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>> > new file mode 100644
>> > index 00000000..9ecd964e
>> > --- /dev/null
>> > +++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>>
>> <snipped>
>>
>> > +jit.opt.start("3", string.format("hotloop=%d", hotloop))
>>
>> Do we need set level 3 direct here?
>> Also, I suggest to use single quotes according to our coding style in
>> tests.
>
>I don't remember, so removed. Quotes are fixed. The diff is below:
>
>================================================================================
>
>diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>index 9ecd964e..a7445e92 100644
>--- a/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>+++ b/test/tarantool-tests/gh-4427-ffi-sandwich/script.lua
>@@ -14,7 +14,7 @@ local sandwich = require('libsandwich')(trigger)
> -- * if trigger <= hotloop -> trace recording is aborted
> -- * if trigger > hotloop -> trace is recorded but execution
> -- leads to panic
>-jit.opt.start("3", string.format("hotloop=%d", hotloop))
>+jit.opt.start(string.format('hotloop=%d', hotloop))
> 
> local res
> for i = 0, hotloop + trigger do
>
>================================================================================
>
>>
>> > +
>
><snipped>
>
>> > diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
>> > new file mode 100644
>> > index 00000000..d2c35534
>> > --- /dev/null
>> > +++ b/test/tarantool-tests/lj-flush-on-trace/script.lua
>>
>> <snipped>
>>
>> > +jit.opt.start("3", string.format("hotloop=%d", hotloop))
>>
>> Do we need set level 3 direct here?
>> Also, I suggest to use single quotes according to our coding style in
>> tests.
>
>Ditto.
>
>================================================================================
>
>diff --git a/test/tarantool-tests/lj-flush-on-trace/script.lua b/test/tarantool-tests/lj-flush-on-trace/script.lua
>index d2c35534..d25c4a33 100644
>--- a/test/tarantool-tests/lj-flush-on-trace/script.lua
>+++ b/test/tarantool-tests/lj-flush-on-trace/script.lua
>@@ -14,7 +14,7 @@ local flush = require('libflush')(trigger)
> -- * if trigger <= hotloop -> trace recording is aborted
> -- * if trigger > hotloop -> trace is recorded but execution
> -- leads to panic
>-jit.opt.start("3", string.format("hotloop=%d", hotloop))
>+jit.opt.start(string.format('hotloop=%d', hotloop))
> 
> for i = 0, trigger + hotloop do
>   ffiflush.flush(flush, i)
>
>================================================================================
>
>>
>> > +
>>
>> <snipped>
>>
>> > diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
>> > index eb11d40d..8355149b 100644
>> > --- a/test/tarantool-tests/utils.lua
>> > +++ b/test/tarantool-tests/utils.lua
>>
>> <snipped>
>>
>> > + SCRIPT = opts and opts.script or arg[0]:gsub('%.test%.lua$', '/script.lua'),
>> > + ENV = opts and makeenv(opts.env) or '',
>> > + REDIRECT = opts and opts.redirect or '',
>> > + }, {
>> > + __call = function(self, ...)
>> > + -- This line just makes the command for <io.popen> by the
>> > + -- following steps:
>> > + -- 1. Replace the placeholders with the corresponding values
>> > + -- given to the command constructor (e.g. script, env)
>> > + -- 2. Join all CLI arguments given to the __call metamethod
>> > + -- 3. Concatenate the results of step 1 and step 2 to obtain
>> > + -- the resulting command.
>>
>> Extra dot at the end of the sentence (or missing dots for previous
>> bullets).
>
>Thanks, fixed! The diff is below:
>
>================================================================================
>
>diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
>index 83716bb3..8c1538d6 100644
>--- a/test/tarantool-tests/utils.lua
>+++ b/test/tarantool-tests/utils.lua
>@@ -68,8 +68,8 @@ function M.makecmd(arg, opts)
>       -- This line just makes the command for <io.popen> by the
>       -- following steps:
>       -- 1. Replace the placeholders with the corresponding values
>- -- given to the command constructor (e.g. script, env)
>- -- 2. Join all CLI arguments given to the __call metamethod
>+ -- given to the command constructor (e.g. script, env).
>+ -- 2. Join all CLI arguments given to the __call metamethod.
>       -- 3. Concatenate the results of step 1 and step 2 to obtain
>       -- the resulting command.
>       local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)
>
>================================================================================
>
>>
>> > + local cmd = ('<ENV> <LUABIN> <REDIRECT> <SCRIPT>'):gsub('%<(%w+)>', self)
>> > + .. (' %s'):rep(select('#', ...)):format(...)
>> > + -- Trim both leading and trailing whitespace from the output
>> > + -- produced by the child process.
>> > + return io.popen(cmd):read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
>> > + end
>> > + })
>> > end
>> >
>> > function M.skipcond(condition, message)
>> > --
>> > 2.30.2
>> >
>>
>> --
>> Best regards,
>> Sergey Kaplun
>
>--
>Best regards,
>IM
 

[-- Attachment #2: Type: text/html, Size: 8505 bytes --]

[Tarantool-patches] [PATCH luajit 3/5] test: introduce test:skipall TAP helper

[Igor Munkin via Tarantool-patches]

Sometimes one need to skip all tests in the script. TAP13 protocol
supports this via the following line:
| 1..0 # SKIP <reason>

Within this commit <test:skipall> helper is introduced that yields the
aforementined line to the TAP consumer and exits the test script. It's
worth to mention, that skipall helper is implemented only for root (i.e.
test with no parent) tests and subtests are not supported at the moment.
The corresponding assertion with FIXME reason is added to <finalize>.

Signed-off-by: Igor Munkin <imun@tarantool.org>
---
 test/tarantool-tests/tap.lua | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index a1f54d20..92d59ce9 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -7,6 +7,12 @@
 local ffi = require("ffi")
 local NULL = ffi.new("void *")
 
+local function finalize(test)
+  -- TODO: implement finalization for subtests too.
+  assert(test.parent == nil, 'FIXME: There is no way to use finalize subtest')
+  os.exit(test:check() and 0 or 1)
+end
+
 local function indent(level, size)
   -- Use the default indent size if none is specified.
   size = tonumber(size) or 4
@@ -90,6 +96,11 @@ local function skip(test, message, extra)
   ok(test, true, message .. " # skip", extra)
 end
 
+local function skipall(test, reason)
+  test:plan(0, reason)
+  finalize(test)
+end
+
 local function like(test, got, pattern, message, extra)
   extra = extra or {}
   extra.got = got
@@ -246,7 +257,7 @@ local function new(parent, name, fun, ...)
     level   = level,
     total   = 0,
     failed  = 0,
-    planned = 0,
+    planned = nil,
     trace   = parent == nil and true or parent.trace,
     -- Set test.strict = true if test:is, test:isnt and
     -- test:is_deeply must be compared strictly with nil.
@@ -266,9 +277,14 @@ local function new(parent, name, fun, ...)
   return test:check()
 end
 
-local function plan(test, planned)
+local function plan(test, planned, reason)
+  if test.planned then
+    -- Use <plan> call location in the error message.
+    error("plan called twice", 2)
+  end
   test.planned = planned
-  io.write(indent(test.level), ("1..%d\n"):format(planned))
+  local tail = planned == 0 and (" # SKIP %s"):format(reason) or ""
+  io.write(indent(test.level), ("1..%d%s\n"):format(planned, tail))
 end
 
 local function check(test)
@@ -313,6 +329,7 @@ test_mt = {
     ok         = ok,
     fail       = fail,
     skip       = skip,
+    skipall    = skipall,
     is         = is,
     isnt       = isnt,
     isnil      = isnil,
-- 
2.30.2

Re: [Tarantool-patches] [PATCH luajit 3/5] test: introduce test:skipall TAP helper

[Sergey Kaplun via Tarantool-patches]

Hi, Igor!

Thanks for the patch!
LGTM, just a few nits regarding the commit message.

On 27.02.23, Igor Munkin wrote:
> Sometimes one need to skip all tests in the script. TAP13 protocol

Typo: s/need/needs/

> supports this via the following line:
> | 1..0 # SKIP <reason>
> 
> Within this commit <test:skipall> helper is introduced that yields the
> aforementined line to the TAP consumer and exits the test script. It's

Typo: s/aforementined/aforementioned/

> worth to mention, that skipall helper is implemented only for root (i.e.
> test with no parent) tests and subtests are not supported at the moment.
> The corresponding assertion with FIXME reason is added to <finalize>.
> 
> Signed-off-by: Igor Munkin <imun@tarantool.org>
> ---
>  test/tarantool-tests/tap.lua | 23 ++++++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
> index a1f54d20..92d59ce9 100644
> --- a/test/tarantool-tests/tap.lua
> +++ b/test/tarantool-tests/tap.lua

<snipped>


-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 3/5] test: introduce test:skipall TAP helper

[Igor Munkin via Tarantool-patches]

Sergey,

Thanks for your review! I've fixed all your comments and force-pushed
the branch.

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH luajit 3/5] test: introduce test:skipall TAP helper

[Maxim Kokryashkin via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 3045 bytes --]


Hi, Igor!
Thanks for the patch!
LGTM, except for a few nits regarding the commit message.
 
> 
>>Sometimes one need to skip all tests in the script. TAP13 protocol
>Typo: s/TAP 13/The TAP 13/
>>supports this via the following line:
>>| 1..0 # SKIP <reason>
>>
>>Within this commit <test:skipall> helper is introduced that yields the
>>aforementined line to the TAP consumer and exits the test script. It's
>>worth to mention, that skipall helper is implemented only for root (i.e.
>Typo: s/worth to mention/worth mentioning/
>>test with no parent) tests and subtests are not supported at the moment.
>>The corresponding assertion with FIXME reason is added to <finalize>.
>Typo: s/with/with a/
>>
>>Signed-off-by: Igor Munkin < imun@tarantool.org >
>>---
>> test/tarantool-tests/tap.lua | 23 ++++++++++++++++++++---
>> 1 file changed, 20 insertions(+), 3 deletions(-)
>>
>>diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
>>index a1f54d20..92d59ce9 100644
>>--- a/test/tarantool-tests/tap.lua
>>+++ b/test/tarantool-tests/tap.lua
>>@@ -7,6 +7,12 @@
>> local ffi = require("ffi")
>> local NULL = ffi.new("void *")
>> 
>>+local function finalize(test)
>>+ -- TODO: implement finalization for subtests too.
>>+ assert(test.parent == nil, 'FIXME: There is no way to use finalize subtest')
>>+ os.exit(test:check() and 0 or 1)
>>+end
>>+
>> local function indent(level, size)
>>   -- Use the default indent size if none is specified.
>>   size = tonumber(size) or 4
>>@@ -90,6 +96,11 @@ local function skip(test, message, extra)
>>   ok(test, true, message .. " # skip", extra)
>> end
>> 
>>+local function skipall(test, reason)
>>+ test:plan(0, reason)
>>+ finalize(test)
>>+end
>>+
>> local function like(test, got, pattern, message, extra)
>>   extra = extra or {}
>>   extra.got = got
>>@@ -246,7 +257,7 @@ local function new(parent, name, fun, ...)
>>     level = level,
>>     total = 0,
>>     failed = 0,
>>- planned = 0,
>>+ planned = nil,
>>     trace = parent == nil and true or parent.trace,
>>     -- Set test.strict = true if test:is, test:isnt and
>>     -- test:is_deeply must be compared strictly with nil.
>>@@ -266,9 +277,14 @@ local function new(parent, name, fun, ...)
>>   return test:check()
>> end
>> 
>>-local function plan(test, planned)
>>+local function plan(test, planned, reason)
>>+ if test.planned then
>>+ -- Use <plan> call location in the error message.
>>+ error("plan called twice", 2)
>>+ end
>>   test.planned = planned
>>- io.write(indent(test.level), ("1..%d\n"):format(planned))
>>+ local tail = planned == 0 and (" # SKIP %s"):format(reason) or ""
>>+ io.write(indent(test.level), ("1..%d%s\n"):format(planned, tail))
>> end
>> 
>> local function check(test)
>>@@ -313,6 +329,7 @@ test_mt = {
>>     ok = ok,
>>     fail = fail,
>>     skip = skip,
>>+ skipall = skipall,
>>     is = is,
>>     isnt = isnt,
>>     isnil = isnil,
>>--
>>2.30.2
>--
>Best regards,
>Maxim Kokryashkin
> 

[-- Attachment #2: Type: text/html, Size: 4554 bytes --]

Re: [Tarantool-patches] [PATCH luajit 3/5] test: introduce test:skipall TAP helper

[Igor Munkin via Tarantool-patches]

Max,

Thanks for your review! I've fixed all your comments and force-pushed
the branch.

-- 
Best regards,
IM

[Tarantool-patches] [PATCH luajit 4/5] test: introduce test:skiprest TAP helper

[Igor Munkin via Tarantool-patches]

Sometimes one need to skip the remaining tests in the script. There is
no special stub in TAP13 protocol for this, but this could be done via
skipping all the remaining planned tests.

Within this commit <test:skiprest> helper is introduced that calls
<test:skip> (planned - total) times and exits the test script. As for
the previous commit it's worth to mention, that skiprest helper is
implemented only for root (i.e. test with no parent) tests and subtests
are not supported at the moment. The corresponding assertion with FIXME
reason can be found in <finalize> routine.

Signed-off-by: Igor Munkin <imun@tarantool.org>
---
 test/tarantool-tests/tap.lua | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index 92d59ce9..343f97e3 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -101,6 +101,13 @@ local function skipall(test, reason)
   finalize(test)
 end
 
+local function skiprest(test, reason)
+  for _ = 1, test.planned - test.total do
+    test:skip(reason)
+  end
+  finalize(test)
+end
+
 local function like(test, got, pattern, message, extra)
   extra = extra or {}
   extra.got = got
@@ -330,6 +337,7 @@ test_mt = {
     fail       = fail,
     skip       = skip,
     skipall    = skipall,
+    skiprest   = skiprest,
     is         = is,
     isnt       = isnt,
     isnil      = isnil,
-- 
2.30.2

Re: [Tarantool-patches] [PATCH luajit 4/5] test: introduce test:skiprest TAP helper

[Sergey Kaplun via Tarantool-patches]

Hi, Igor!

Thanks for the patch!
LGTM, except a single nit regarding the commit message.

On 27.02.23, Igor Munkin wrote:
> Sometimes one need to skip the remaining tests in the script. There is

Typo: s/need/needs/

> no special stub in TAP13 protocol for this, but this could be done via
> skipping all the remaining planned tests.
> 
> Within this commit <test:skiprest> helper is introduced that calls
> <test:skip> (planned - total) times and exits the test script. As for
> the previous commit it's worth to mention, that skiprest helper is
> implemented only for root (i.e. test with no parent) tests and subtests
> are not supported at the moment. The corresponding assertion with FIXME
> reason can be found in <finalize> routine.
> 
> Signed-off-by: Igor Munkin <imun@tarantool.org>

<snipped>

-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 4/5] test: introduce test:skiprest TAP helper

[Igor Munkin via Tarantool-patches]

Sergey,

Thanks for your review! I've fixed all your comments and force-pushed
the branch.

-- 
Best regards,
IM

Re: [Tarantool-patches] [PATCH luajit 4/5] test: introduce test:skiprest TAP helper

[Maxim Kokryashkin via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 1768 bytes --]


Hi, Igor!
Thanks for the patch!
LGTM, except for a few nits regarding the commit message:
  
>Понедельник, 27 февраля 2023, 12:10 +03:00 от Igor Munkin <imun@tarantool.org>:
> 
>Sometimes one need to skip the remaining tests in the script. There is
>no special stub in TAP13 protocol for this, but this could be done via
Typo: s/TAP 13/the TAP 13/
>skipping all the remaining planned tests.
>
>Within this commit <test:skiprest> helper is introduced that calls
><test:skip> (planned - total) times and exits the test script. As for
>the previous commit it's worth to mention, that skiprest helper is
Typo: s/worth to mention/worth mentioning/
>implemented only for root (i.e. test with no parent) tests and subtests
>are not supported at the moment. The corresponding assertion with FIXME
>reason can be found in <finalize> routine.
>
>Signed-off-by: Igor Munkin < imun@tarantool.org >
>---
> test/tarantool-tests/tap.lua | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
>diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
>index 92d59ce9..343f97e3 100644
>--- a/test/tarantool-tests/tap.lua
>+++ b/test/tarantool-tests/tap.lua
>@@ -101,6 +101,13 @@ local function skipall(test, reason)
>   finalize(test)
> end
> 
>+local function skiprest(test, reason)
>+ for _ = 1, test.planned - test.total do
>+ test:skip(reason)
>+ end
>+ finalize(test)
>+end
>+
> local function like(test, got, pattern, message, extra)
>   extra = extra or {}
>   extra.got = got
>@@ -330,6 +337,7 @@ test_mt = {
>     fail = fail,
>     skip = skip,
>     skipall = skipall,
>+ skiprest = skiprest,
>     is = is,
>     isnt = isnt,
>     isnil = isnil,
>--
>2.30.2
--
Best regards,
Maxim Kokryashkin
 

[-- Attachment #2: Type: text/html, Size: 2801 bytes --]

Re: [Tarantool-patches] [PATCH luajit 4/5] test: introduce test:skiprest TAP helper

[Igor Munkin via Tarantool-patches]

Max,

Thanks for your review! I've fixed all your comments and force-pushed
the branch.

-- 
Best regards,
IM

[Tarantool-patches] [PATCH luajit 5/5] test: make skipcond helper more convenient

[Igor Munkin via Tarantool-patches]

This patch provides two enhancements for <utils.skipcond>:
1. As a result of the patch, <utils.skipcond> becomes multi-conditional,
   so there is no need to concatenate one huge and complex condition
   with one huge unreadable reason. Now skipcond receives the table with
   conditions and skips the test if one of the given conditions is met.
2. <utils.skipcond> yields the test object, that allows to chain
   skipcond call right next to the test constructor.

Finally as a result of the aforementioned changes <utils.skipcond> is
moved to tap.lua.

Co-authored-by: Sergey Kaplun <skaplun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>
---
 .../gh-4199-gc64-fuse.test.lua                | 11 ++++----
 .../gh-4427-ffi-sandwich.test.lua             | 11 +++-----
 .../gh-5813-resolving-of-c-symbols.test.lua   | 19 ++++++-------
 ...4-add-proto-trace-sysprof-default.test.lua | 15 ++++------
 .../lj-430-maxirconst.test.lua                | 10 +++----
 .../lj-603-err-snap-restore.test.lua          | 19 ++++++++-----
 ...lj-672-cdata-allocation-recording.test.lua | 12 ++++----
 .../lj-906-fix-err-mem.test.lua               | 12 ++++----
 .../lj-flush-on-trace.test.lua                | 11 +++-----
 .../misclib-getmetrics-capi.test.lua          | 17 +++++------
 .../misclib-getmetrics-lapi.test.lua          | 13 ++++-----
 .../misclib-memprof-lapi.test.lua             | 28 ++++++++-----------
 .../misclib-sysprof-capi.test.lua             | 19 ++++++-------
 .../misclib-sysprof-lapi.test.lua             | 19 ++++++-------
 test/tarantool-tests/tap.lua                  | 11 ++++++++
 test/tarantool-tests/utils.lua                |  8 ------
 16 files changed, 107 insertions(+), 128 deletions(-)

diff --git a/test/tarantool-tests/gh-4199-gc64-fuse.test.lua b/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
index 4d69250f..65f9faac 100644
--- a/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
+++ b/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
@@ -1,11 +1,12 @@
--- The test is GC64 only.
-local ffi = require('ffi')
-require('utils').skipcond(not ffi.abi('gc64'), 'test is GC64 only')
-
 local tap = require('tap')
-local test = tap.test('gh-4199-gc64-fuse')
+local test = tap.test('gh-4199-gc64-fuse'):skipcond({
+  ['Test requires GC64 mode enabled'] = not require('ffi').abi('gc64'),
+})
+
 test:plan(1)
 
+local ffi = require('ffi')
+
 collectgarbage()
 -- Chomp memory in currently allocated GC space.
 collectgarbage('stop')
diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
index 06985dcd..ed3f50d1 100644
--- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
@@ -1,16 +1,13 @@
-local utils = require('utils')
-
--- Disabled on *BSD due to #4819.
-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
-
 local tap = require('tap')
+local test = tap.test('gh-4427-ffi-sandwich'):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
 
-local test = tap.test('gh-4427-ffi-sandwich')
 test:plan(2)
 
 -- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
 -- with the given environment, launch options and CLI arguments.
-local script = utils.makecmd(arg, {
+local script = require('utils').makecmd(arg, {
   -- XXX: Apple tries their best to "protect their users from
   -- malware". As a result SIP (see the link[1] below) has been
   -- designed and released. Now, Apple developers are so
diff --git a/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua b/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
index e0b6d86d..3c6833fc 100644
--- a/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
+++ b/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
@@ -1,14 +1,10 @@
--- Memprof is implemented for x86 and x64 architectures only.
-local utils = require("utils")
-
-utils.skipcond(
-  jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
-  jit.arch.." architecture or "..jit.os..
-  " OS is NIY for memprof c symbols resolving"
-)
-
 local tap = require("tap")
-local test = tap.test("gh-5813-resolving-of-c-symbols")
+local test = tap.test("gh-5813-resolving-of-c-symbols"):skipcond({
+  ["Memprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
+                                               jit.arch ~= "x64",
+  ["Memprof is implemented for Linux only"] = jit.os ~= "Linux",
+})
+
 test:plan(5)
 
 jit.off()
@@ -19,8 +15,9 @@ local symtab = require "utils.symtab"
 local testboth = require "resboth"
 local testhash = require "reshash"
 local testgnuhash = require "resgnuhash"
+local profilename = require("utils").profilename
 
-local TMP_BINFILE = utils.profilename("memprofdata.tmp.bin")
+local TMP_BINFILE = profilename("memprofdata.tmp.bin")
 
 local function tree_contains(node, name)
   if node == nil then
diff --git a/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua b/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
index 15bd0a8b..472bc2d1 100644
--- a/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
+++ b/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
@@ -1,13 +1,10 @@
--- Sysprof is implemented for x86 and x64 architectures only.
-require('utils').skipcond(
-  jit.arch ~= 'x86' and jit.arch ~= 'x64' or jit.os ~= 'Linux'
-    or require('ffi').abi('gc64'),
-  jit.arch..' architecture or '..jit.os..
-  ' OS is NIY for sysprof'
-)
-
 local tap = require('tap')
-local test = tap.test('gh-7264-add-proto-trace-sysprof-default.test.lua')
+local test = tap.test('gh-7264-add-proto-trace-sysprof-default'):skipcond({
+  ['Sysprof is implemented for x86_64 only'] = jit.arch ~= 'x86' and
+                                               jit.arch ~= 'x64',
+  ['Sysprof is implemented for Linux only'] = jit.os ~= 'Linux',
+})
+
 test:plan(2)
 
 local chunk = [[
diff --git a/test/tarantool-tests/lj-430-maxirconst.test.lua b/test/tarantool-tests/lj-430-maxirconst.test.lua
index 8bc523c2..633ab676 100644
--- a/test/tarantool-tests/lj-430-maxirconst.test.lua
+++ b/test/tarantool-tests/lj-430-maxirconst.test.lua
@@ -1,12 +1,12 @@
--- Disabled on *BSD due to #4819.
-require('utils').skipcond(jit.os == 'BSD', 'Disabled due to #4819')
-
 local tap = require('tap')
-local traceinfo = require('jit.util').traceinfo
+local test = tap.test('lj-430-maxirconst'):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
 
-local test = tap.test('lj-430-maxirconst')
 test:plan(2)
 
+local traceinfo = require('jit.util').traceinfo
+
 -- This function has only 3 IR constant.
 local function irconst3()
 end
diff --git a/test/tarantool-tests/lj-603-err-snap-restore.test.lua b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
index b5353e85..be54a5f3 100644
--- a/test/tarantool-tests/lj-603-err-snap-restore.test.lua
+++ b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
@@ -1,9 +1,9 @@
 local tap = require('tap')
-
 -- Test to demonstrate the incorrect JIT behaviour when an error
 -- is raised on restoration from the snapshot.
 -- See also https://github.com/LuaJIT/LuaJIT/issues/603.
-local test = tap.test('lj-603-err-snap-restore.test.lua')
+local test = tap.test('lj-603-err-snap-restore')
+
 test:plan(2)
 
 -- XXX: This is fragile. We need a specific amount of Lua stack
@@ -38,11 +38,16 @@ end
 recursive_f()
 
 test:ok(true)
--- Disabled on *BSD due to #4819.
--- XXX: The different amount of stack slots is in-use for
--- Tarantool at start, so just skip test for it.
--- luacheck: no global
-test:ok(jit.os == 'BSD' or _TARANTOOL or not handler_is_called)
+
+test:skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+  -- XXX: The different amount of stack slots is in-use for
+  -- Tarantool at start, so just skip test for it.
+  -- luacheck: no global
+  ['Disable test for Tarantool'] = _TARANTOOL,
+})
+
+test:ok(not handler_is_called)
 
 -- XXX: Don't use `os.exit()` here by intention. When error on
 -- snap restoration is raised, `err_unwind()` doesn't stop on
diff --git a/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua b/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
index 1dc741d8..2165afe3 100644
--- a/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
+++ b/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
@@ -1,13 +1,13 @@
--- Disabled on *BSD due to #4819.
-require('utils').skipcond(jit.os == 'BSD', 'Disabled due to #4819')
+local tap = require('tap')
+local test = tap.test('lj-672-cdata-allocation-recording'):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
+
+test:plan(1)
 
 local ffi = require('ffi')
 local traceinfo = require('jit.util').traceinfo
 
-local tap = require('tap')
-local test = tap.test('lj-672-cdata-allocation-recording')
-test:plan(1)
-
 -- Structure with array.
 ffi.cdef('struct my_struct {int a; char d[8];}')
 
diff --git a/test/tarantool-tests/lj-906-fix-err-mem.test.lua b/test/tarantool-tests/lj-906-fix-err-mem.test.lua
index 450beeff..6c6df338 100644
--- a/test/tarantool-tests/lj-906-fix-err-mem.test.lua
+++ b/test/tarantool-tests/lj-906-fix-err-mem.test.lua
@@ -1,13 +1,13 @@
 local tap = require('tap')
-local ffi = require('ffi')
-local table_new = require('table.new')
-
--- Avoid test to be killed.
-require('utils').skipcond(ffi.abi('gc64'), 'test is not GC64 only')
+local test = tap.test('lj-906-fix-err-mem'):skipcond({
+  ['Test requires GC64 mode disabled'] = require('ffi').abi('gc64'),
+})
 
-local test = tap.test('lj-906-fix-err-mem')
 test:plan(1)
 
+local ffi = require('ffi')
+local table_new = require('table.new')
+
 local KB = 1024
 local MB = 1024 * KB
 local sizes = {8 * MB, 8 * KB, 8}
diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua
index 3351cc5a..099e9650 100644
--- a/test/tarantool-tests/lj-flush-on-trace.test.lua
+++ b/test/tarantool-tests/lj-flush-on-trace.test.lua
@@ -1,16 +1,13 @@
-local utils = require('utils')
-
--- Disabled on *BSD due to #4819.
-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
-
 local tap = require('tap')
+local test = tap.test('lj-flush-on-trace'):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
 
-local test = tap.test('lj-flush-on-trace')
 test:plan(2)
 
 -- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
 -- with the given environment, launch options and CLI arguments.
-local script = utils.makecmd(arg, {
+local script = require('utils').makecmd(arg, {
   -- XXX: Apple tries their best to "protect their users from
   -- malware". As a result SIP (see the link[1] below) has been
   -- designed and released. Now, Apple developers are so
diff --git a/test/tarantool-tests/misclib-getmetrics-capi.test.lua b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
index 42a9adf9..c5a91955 100644
--- a/test/tarantool-tests/misclib-getmetrics-capi.test.lua
+++ b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
@@ -1,12 +1,15 @@
-local utils = require('utils')
+local tap = require('tap')
+local test = tap.test("clib-misc-getmetrics"):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
 
--- Disabled on *BSD due to #4819.
-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
+test:plan(11)
 
 local path = arg[0]:gsub('%.test%.lua', '')
 local suffix = package.cpath:match('?.(%a+);')
 package.cpath = ('%s/?.%s;'):format(path, suffix)..package.cpath
 
+local MAXNINS = require('utils').const.maxnins
 local jit_opt_default = {
     3, -- level
     "hotloop=56",
@@ -14,11 +17,6 @@ local jit_opt_default = {
     "minstitch=0",
 }
 
-local tap = require('tap')
-
-local test = tap.test("clib-misc-getmetrics")
-test:plan(11)
-
 local testgetmetrics = require("testgetmetrics")
 
 test:ok(testgetmetrics.base())
@@ -96,8 +94,7 @@ end))
 
 -- Compiled loop with a side exit which does not get compiled.
 test:ok(testgetmetrics.snap_restores(function()
-    jit.opt.start(0, "hotloop=1", "hotexit=2",
-                  ("minstitch=%d"):format(utils.const.maxnins))
+    jit.opt.start(0, "hotloop=1", "hotexit=2", ("minstitch=%d"):format(MAXNINS))
 
     local function foo(i)
         -- math.fmod is not yet compiled!
diff --git a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
index 0c170d0c..e71bc239 100644
--- a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
+++ b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
@@ -2,16 +2,14 @@
 -- Major portions taken verbatim or adapted from the LuaVela testing suite.
 -- Copyright (C) 2015-2019 IPONWEB Ltd.
 
-local utils = require('utils')
-
--- Disabled on *BSD due to #4819.
-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
-
 local tap = require('tap')
+local test = tap.test("lib-misc-getmetrics"):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+})
 
-local test = tap.test("lib-misc-getmetrics")
 test:plan(10)
 
+local MAXNINS = require('utils').const.maxnins
 local jit_opt_default = {
     3, -- level
     "hotloop=56",
@@ -281,8 +279,7 @@ test:test("snap-restores-loop-side-exit-non-compiled", function(subtest)
     -- Compiled loop with a side exit which does not get compiled.
     subtest:plan(1)
 
-    jit.opt.start(0, "hotloop=1", "hotexit=2",
-                  ("minstitch=%d"):format(utils.const.maxnins))
+    jit.opt.start(0, "hotloop=1", "hotexit=2", ("minstitch=%d"):format(MAXNINS))
 
     local function foo(i)
         -- math.fmod is not yet compiled!
diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
index bae0c27c..18c8aaab 100644
--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
@@ -1,14 +1,13 @@
--- Memprof is implemented for x86 and x64 architectures only.
-local utils = require("utils")
-
-utils.skipcond(
-  jit.arch ~= "x86" and jit.arch ~= "x64",
-  jit.arch.." architecture is NIY for memprof"
-)
-
+-- XXX: This comment is a reminder to reimplement memprof tests
+-- assertions to make them more indepentent to the changes made.
+-- Now I just leave this 3 lines comment to preserve line numbers.
 local tap = require("tap")
+local test = tap.test("misc-memprof-lapi"):skipcond({
+  ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
+  ["Memprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
+                                               jit.arch ~= "x64",
+})
 
-local test = tap.test("misc-memprof-lapi")
 test:plan(5)
 
 local jit_opt_default = {
@@ -27,9 +26,10 @@ local bufread = require "utils.bufread"
 local memprof = require "memprof.parse"
 local process = require "memprof.process"
 local symtab = require "utils.symtab"
+local profilename = require("utils").profilename
 
-local TMP_BINFILE = utils.profilename("memprofdata.tmp.bin")
-local BAD_PATH = utils.profilename("memprofdata/tmp.bin")
+local TMP_BINFILE = profilename("memprofdata.tmp.bin")
+local BAD_PATH = profilename("memprofdata/tmp.bin")
 local SRC_PATH = "@"..arg[0]
 
 local function default_payload()
@@ -267,12 +267,6 @@ end)
 jit.on()
 
 test:test("jit-output", function(subtest)
-  -- Disabled on *BSD due to #4819.
-  if jit.os == 'BSD' then
-    subtest:plan(1)
-    subtest:skip('Disabled due to #4819')
-    return
-  end
 
   subtest:plan(4)
 
diff --git a/test/tarantool-tests/misclib-sysprof-capi.test.lua b/test/tarantool-tests/misclib-sysprof-capi.test.lua
index dad0fe4a..a9b712a5 100644
--- a/test/tarantool-tests/misclib-sysprof-capi.test.lua
+++ b/test/tarantool-tests/misclib-sysprof-capi.test.lua
@@ -1,21 +1,18 @@
--- Sysprof is implemented for x86 and x64 architectures only.
-local utils = require("utils")
-utils.skipcond(
-  jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
-  jit.arch.." architecture or "..jit.os..
-  " OS is NIY for sysprof"
-)
+local tap = require("tap")
+local test = tap.test("clib-misc-sysprof"):skipcond({
+  ["Sysprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
+                                               jit.arch ~= "x64",
+  ["Sysprof is implemented for Linux only"] = jit.os ~= "Linux",
+})
+
+test:plan(2)
 
 local testsysprof = require("testsysprof")
 
-local tap = require("tap")
 local jit = require('jit')
 
 jit.off()
 
-local test = tap.test("clib-misc-sysprof")
-test:plan(2)
-
 test:ok(testsysprof.base())
 test:ok(testsysprof.validation())
 
diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua
index 4bf10e8d..fff89dfd 100644
--- a/test/tarantool-tests/misclib-sysprof-lapi.test.lua
+++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua
@@ -1,14 +1,10 @@
--- Sysprof is implemented for x86 and x64 architectures only.
-local utils = require("utils")
-utils.skipcond(
-  jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
-  jit.arch.." architecture or "..jit.os..
-  " OS is NIY for sysprof"
-)
-
 local tap = require("tap")
+local test = tap.test("misc-sysprof-lapi"):skipcond({
+  ["Sysprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
+                                               jit.arch ~= "x64",
+  ["Sysprof is implemented for Linux only"] = jit.os ~= "Linux",
+})
 
-local test = tap.test("misc-sysprof-lapi")
 test:plan(19)
 
 jit.off()
@@ -17,9 +13,10 @@ jit.flush()
 local bufread = require("utils.bufread")
 local symtab = require("utils.symtab")
 local sysprof = require("sysprof.parse")
+local profilename = require("utils").profilename
 
-local TMP_BINFILE = utils.profilename("sysprofdata.tmp.bin")
-local BAD_PATH = utils.profilename("sysprofdata/tmp.bin")
+local TMP_BINFILE = profilename("sysprofdata.tmp.bin")
+local BAD_PATH = profilename("sysprofdata/tmp.bin")
 
 local function payload()
   local function fib(n)
diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index 343f97e3..47a8fe87 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -327,6 +327,16 @@ local function check(test)
   return test.planned == test.total and test.failed == 0
 end
 
+local function skipcond(test, conditions)
+  for reason, condition in pairs(conditions) do
+    if condition then
+      local skipfunc = test.planned and skiprest or skipall
+      skipfunc(test, reason)
+    end
+  end
+  return test
+end
+
 test_mt = {
   __index = {
     test       = new,
@@ -338,6 +348,7 @@ test_mt = {
     skip       = skip,
     skipall    = skipall,
     skiprest   = skiprest,
+    skipcond   = skipcond,
     is         = is,
     isnt       = isnt,
     isnil      = isnil,
diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
index 8355149b..83716bb3 100644
--- a/test/tarantool-tests/utils.lua
+++ b/test/tarantool-tests/utils.lua
@@ -81,14 +81,6 @@ function M.makecmd(arg, opts)
   })
 end
 
-function M.skipcond(condition, message)
-  if not condition then return end
-  local test = tap.test(arg[0]:match('/?(.+)%.test%.lua'))
-  test:plan(1)
-  test:skip(message)
-  os.exit(test:check() and 0 or 1)
-end
-
 function M.hasbc(f, bytecode)
   assert(type(f) == 'function', 'argument #1 should be a function')
   assert(type(bytecode) == 'string', 'argument #2 should be a string')
-- 
2.30.2

Re: [Tarantool-patches] [PATCH luajit 5/5] test: make skipcond helper more convenient

[Sergey Kaplun via Tarantool-patches]

Hi, Igor!
Thanks for the fixes!
LGTM!


-- 
Best regards,
Sergey Kaplun

Re: [Tarantool-patches] [PATCH luajit 5/5] test: make skipcond helper more convenient

[Maxim Kokryashkin via Tarantool-patches]

[-- Attachment #1: Type: text/plain, Size: 19807 bytes --]


Hi, Igor!
Thanks for the patch!
LGTM
--
Best regards,
Maxim Kokryashkin
 
  
>Понедельник, 27 февраля 2023, 12:10 +03:00 от Igor Munkin <imun@tarantool.org>:
> 
>This patch provides two enhancements for <utils.skipcond>:
>1. As a result of the patch, <utils.skipcond> becomes multi-conditional,
>   so there is no need to concatenate one huge and complex condition
>   with one huge unreadable reason. Now skipcond receives the table with
>   conditions and skips the test if one of the given conditions is met.
>2. <utils.skipcond> yields the test object, that allows to chain
>   skipcond call right next to the test constructor.
>
>Finally as a result of the aforementioned changes <utils.skipcond> is
>moved to tap.lua.
>
>Co-authored-by: Sergey Kaplun < skaplun@tarantool.org >
>Signed-off-by: Igor Munkin < imun@tarantool.org >
>---
> .../gh-4199-gc64-fuse.test.lua | 11 ++++----
> .../gh-4427-ffi-sandwich.test.lua | 11 +++-----
> .../gh-5813-resolving-of-c-symbols.test.lua | 19 ++++++-------
> ...4-add-proto-trace-sysprof-default.test.lua | 15 ++++------
> .../lj-430-maxirconst.test.lua | 10 +++----
> .../lj-603-err-snap-restore.test.lua | 19 ++++++++-----
> ...lj-672-cdata-allocation-recording.test.lua | 12 ++++----
> .../lj-906-fix-err-mem.test.lua | 12 ++++----
> .../lj-flush-on-trace.test.lua | 11 +++-----
> .../misclib-getmetrics-capi.test.lua | 17 +++++------
> .../misclib-getmetrics-lapi.test.lua | 13 ++++-----
> .../misclib-memprof-lapi.test.lua | 28 ++++++++-----------
> .../misclib-sysprof-capi.test.lua | 19 ++++++-------
> .../misclib-sysprof-lapi.test.lua | 19 ++++++-------
> test/tarantool-tests/tap.lua | 11 ++++++++
> test/tarantool-tests/utils.lua | 8 ------
> 16 files changed, 107 insertions(+), 128 deletions(-)
>
>diff --git a/test/tarantool-tests/gh-4199-gc64-fuse.test.lua b/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
>index 4d69250f..65f9faac 100644
>--- a/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
>+++ b/test/tarantool-tests/gh-4199-gc64-fuse.test.lua
>@@ -1,11 +1,12 @@
>--- The test is GC64 only.
>-local ffi = require('ffi')
>-require('utils').skipcond(not ffi.abi('gc64'), 'test is GC64 only')
>-
> local tap = require('tap')
>-local test = tap.test('gh-4199-gc64-fuse')
>+local test = tap.test('gh-4199-gc64-fuse'):skipcond({
>+ ['Test requires GC64 mode enabled'] = not require('ffi').abi('gc64'),
>+})
>+
> test:plan(1)
> 
>+local ffi = require('ffi')
>+
> collectgarbage()
> -- Chomp memory in currently allocated GC space.
> collectgarbage('stop')
>diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
>index 06985dcd..ed3f50d1 100644
>--- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
>+++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
>@@ -1,16 +1,13 @@
>-local utils = require('utils')
>-
>--- Disabled on *BSD due to #4819.
>-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>-
> local tap = require('tap')
>+local test = tap.test('gh-4427-ffi-sandwich'):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
> 
>-local test = tap.test('gh-4427-ffi-sandwich')
> test:plan(2)
> 
> -- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
> -- with the given environment, launch options and CLI arguments.
>-local script = utils.makecmd(arg, {
>+local script = require('utils').makecmd(arg, {
>   -- XXX: Apple tries their best to "protect their users from
>   -- malware". As a result SIP (see the link[1] below) has been
>   -- designed and released. Now, Apple developers are so
>diff --git a/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua b/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
>index e0b6d86d..3c6833fc 100644
>--- a/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
>+++ b/test/tarantool-tests/gh-5813-resolving-of-c-symbols.test.lua
>@@ -1,14 +1,10 @@
>--- Memprof is implemented for x86 and x64 architectures only.
>-local utils = require("utils")
>-
>-utils.skipcond(
>- jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
>- jit.arch.." architecture or "..jit.os..
>- " OS is NIY for memprof c symbols resolving"
>-)
>-
> local tap = require("tap")
>-local test = tap.test("gh-5813-resolving-of-c-symbols")
>+local test = tap.test("gh-5813-resolving-of-c-symbols"):skipcond({
>+ ["Memprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
>+ jit.arch ~= "x64",
>+ ["Memprof is implemented for Linux only"] = jit.os ~= "Linux",
>+})
>+
> test:plan(5)
> 
> jit.off()
>@@ -19,8 +15,9 @@ local symtab = require "utils.symtab"
> local testboth = require "resboth"
> local testhash = require "reshash"
> local testgnuhash = require "resgnuhash"
>+local profilename = require("utils").profilename
> 
>-local TMP_BINFILE = utils.profilename("memprofdata.tmp.bin")
>+local TMP_BINFILE = profilename("memprofdata.tmp.bin")
> 
> local function tree_contains(node, name)
>   if node == nil then
>diff --git a/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua b/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
>index 15bd0a8b..472bc2d1 100644
>--- a/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
>+++ b/test/tarantool-tests/gh-7264-add-proto-trace-sysprof-default.test.lua
>@@ -1,13 +1,10 @@
>--- Sysprof is implemented for x86 and x64 architectures only.
>-require('utils').skipcond(
>- jit.arch ~= 'x86' and jit.arch ~= 'x64' or jit.os ~= 'Linux'
>- or require('ffi').abi('gc64'),
>- jit.arch..' architecture or '..jit.os..
>- ' OS is NIY for sysprof'
>-)
>-
> local tap = require('tap')
>-local test = tap.test('gh-7264-add-proto-trace-sysprof-default.test.lua')
>+local test = tap.test('gh-7264-add-proto-trace-sysprof-default'):skipcond({
>+ ['Sysprof is implemented for x86_64 only'] = jit.arch ~= 'x86' and
>+ jit.arch ~= 'x64',
>+ ['Sysprof is implemented for Linux only'] = jit.os ~= 'Linux',
>+})
>+
> test:plan(2)
> 
> local chunk = [[
>diff --git a/test/tarantool-tests/lj-430-maxirconst.test.lua b/test/tarantool-tests/lj-430-maxirconst.test.lua
>index 8bc523c2..633ab676 100644
>--- a/test/tarantool-tests/lj-430-maxirconst.test.lua
>+++ b/test/tarantool-tests/lj-430-maxirconst.test.lua
>@@ -1,12 +1,12 @@
>--- Disabled on *BSD due to #4819.
>-require('utils').skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>-
> local tap = require('tap')
>-local traceinfo = require('jit.util').traceinfo
>+local test = tap.test('lj-430-maxirconst'):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
> 
>-local test = tap.test('lj-430-maxirconst')
> test:plan(2)
> 
>+local traceinfo = require('jit.util').traceinfo
>+
> -- This function has only 3 IR constant.
> local function irconst3()
> end
>diff --git a/test/tarantool-tests/lj-603-err-snap-restore.test.lua b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
>index b5353e85..be54a5f3 100644
>--- a/test/tarantool-tests/lj-603-err-snap-restore.test.lua
>+++ b/test/tarantool-tests/lj-603-err-snap-restore.test.lua
>@@ -1,9 +1,9 @@
> local tap = require('tap')
>-
> -- Test to demonstrate the incorrect JIT behaviour when an error
> -- is raised on restoration from the snapshot.
> -- See also  https://github.com/LuaJIT/LuaJIT/issues/603 .
>-local test = tap.test('lj-603-err-snap-restore.test.lua')
>+local test = tap.test('lj-603-err-snap-restore')
>+
> test:plan(2)
> 
> -- XXX: This is fragile. We need a specific amount of Lua stack
>@@ -38,11 +38,16 @@ end
> recursive_f()
> 
> test:ok(true)
>--- Disabled on *BSD due to #4819.
>--- XXX: The different amount of stack slots is in-use for
>--- Tarantool at start, so just skip test for it.
>--- luacheck: no global
>-test:ok(jit.os == 'BSD' or _TARANTOOL or not handler_is_called)
>+
>+test:skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+ -- XXX: The different amount of stack slots is in-use for
>+ -- Tarantool at start, so just skip test for it.
>+ -- luacheck: no global
>+ ['Disable test for Tarantool'] = _TARANTOOL,
>+})
>+
>+test:ok(not handler_is_called)
> 
> -- XXX: Don't use `os.exit()` here by intention. When error on
> -- snap restoration is raised, `err_unwind()` doesn't stop on
>diff --git a/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua b/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
>index 1dc741d8..2165afe3 100644
>--- a/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
>+++ b/test/tarantool-tests/lj-672-cdata-allocation-recording.test.lua
>@@ -1,13 +1,13 @@
>--- Disabled on *BSD due to #4819.
>-require('utils').skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>+local tap = require('tap')
>+local test = tap.test('lj-672-cdata-allocation-recording'):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
>+
>+test:plan(1)
> 
> local ffi = require('ffi')
> local traceinfo = require('jit.util').traceinfo
> 
>-local tap = require('tap')
>-local test = tap.test('lj-672-cdata-allocation-recording')
>-test:plan(1)
>-
> -- Structure with array.
> ffi.cdef('struct my_struct {int a; char d[8];}')
> 
>diff --git a/test/tarantool-tests/lj-906-fix-err-mem.test.lua b/test/tarantool-tests/lj-906-fix-err-mem.test.lua
>index 450beeff..6c6df338 100644
>--- a/test/tarantool-tests/lj-906-fix-err-mem.test.lua
>+++ b/test/tarantool-tests/lj-906-fix-err-mem.test.lua
>@@ -1,13 +1,13 @@
> local tap = require('tap')
>-local ffi = require('ffi')
>-local table_new = require('table.new')
>-
>--- Avoid test to be killed.
>-require('utils').skipcond(ffi.abi('gc64'), 'test is not GC64 only')
>+local test = tap.test('lj-906-fix-err-mem'):skipcond({
>+ ['Test requires GC64 mode disabled'] = require('ffi').abi('gc64'),
>+})
> 
>-local test = tap.test('lj-906-fix-err-mem')
> test:plan(1)
> 
>+local ffi = require('ffi')
>+local table_new = require('table.new')
>+
> local KB = 1024
> local MB = 1024 * KB
> local sizes = {8 * MB, 8 * KB, 8}
>diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua
>index 3351cc5a..099e9650 100644
>--- a/test/tarantool-tests/lj-flush-on-trace.test.lua
>+++ b/test/tarantool-tests/lj-flush-on-trace.test.lua
>@@ -1,16 +1,13 @@
>-local utils = require('utils')
>-
>--- Disabled on *BSD due to #4819.
>-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>-
> local tap = require('tap')
>+local test = tap.test('lj-flush-on-trace'):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
> 
>-local test = tap.test('lj-flush-on-trace')
> test:plan(2)
> 
> -- <makecmd> runs %testname%/script.lua by <LUAJIT_TEST_BINARY>
> -- with the given environment, launch options and CLI arguments.
>-local script = utils.makecmd(arg, {
>+local script = require('utils').makecmd(arg, {
>   -- XXX: Apple tries their best to "protect their users from
>   -- malware". As a result SIP (see the link[1] below) has been
>   -- designed and released. Now, Apple developers are so
>diff --git a/test/tarantool-tests/misclib-getmetrics-capi.test.lua b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
>index 42a9adf9..c5a91955 100644
>--- a/test/tarantool-tests/misclib-getmetrics-capi.test.lua
>+++ b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
>@@ -1,12 +1,15 @@
>-local utils = require('utils')
>+local tap = require('tap')
>+local test = tap.test("clib-misc-getmetrics"):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
> 
>--- Disabled on *BSD due to #4819.
>-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>+test:plan(11)
> 
> local path = arg[0]:gsub('%.test%.lua', '')
> local suffix = package.cpath:match('?.(%a+);')
> package.cpath = ('%s/?.%s;'):format(path, suffix)..package.cpath
> 
>+local MAXNINS = require('utils').const.maxnins
> local jit_opt_default = {
>     3, -- level
>     "hotloop=56",
>@@ -14,11 +17,6 @@ local jit_opt_default = {
>     "minstitch=0",
> }
> 
>-local tap = require('tap')
>-
>-local test = tap.test("clib-misc-getmetrics")
>-test:plan(11)
>-
> local testgetmetrics = require("testgetmetrics")
> 
> test:ok(testgetmetrics.base())
>@@ -96,8 +94,7 @@ end))
> 
> -- Compiled loop with a side exit which does not get compiled.
> test:ok(testgetmetrics.snap_restores(function()
>- jit.opt.start(0, "hotloop=1", "hotexit=2",
>- ("minstitch=%d"):format(utils.const.maxnins))
>+ jit.opt.start(0, "hotloop=1", "hotexit=2", ("minstitch=%d"):format(MAXNINS))
> 
>     local function foo(i)
>         -- math.fmod is not yet compiled!
>diff --git a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
>index 0c170d0c..e71bc239 100644
>--- a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
>+++ b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
>@@ -2,16 +2,14 @@
> -- Major portions taken verbatim or adapted from the LuaVela testing suite.
> -- Copyright (C) 2015-2019 IPONWEB Ltd.
> 
>-local utils = require('utils')
>-
>--- Disabled on *BSD due to #4819.
>-utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
>-
> local tap = require('tap')
>+local test = tap.test("lib-misc-getmetrics"):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+})
> 
>-local test = tap.test("lib-misc-getmetrics")
> test:plan(10)
> 
>+local MAXNINS = require('utils').const.maxnins
> local jit_opt_default = {
>     3, -- level
>     "hotloop=56",
>@@ -281,8 +279,7 @@ test:test("snap-restores-loop-side-exit-non-compiled", function(subtest)
>     -- Compiled loop with a side exit which does not get compiled.
>     subtest:plan(1)
> 
>- jit.opt.start(0, "hotloop=1", "hotexit=2",
>- ("minstitch=%d"):format(utils.const.maxnins))
>+ jit.opt.start(0, "hotloop=1", "hotexit=2", ("minstitch=%d"):format(MAXNINS))
> 
>     local function foo(i)
>         -- math.fmod is not yet compiled!
>diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
>index bae0c27c..18c8aaab 100644
>--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
>+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
>@@ -1,14 +1,13 @@
>--- Memprof is implemented for x86 and x64 architectures only.
>-local utils = require("utils")
>-
>-utils.skipcond(
>- jit.arch ~= "x86" and jit.arch ~= "x64",
>- jit.arch.." architecture is NIY for memprof"
>-)
>-
>+-- XXX: This comment is a reminder to reimplement memprof tests
>+-- assertions to make them more indepentent to the changes made.
>+-- Now I just leave this 3 lines comment to preserve line numbers.
> local tap = require("tap")
>+local test = tap.test("misc-memprof-lapi"):skipcond({
>+ ['Disabled on *BSD due to #4819'] = jit.os == 'BSD',
>+ ["Memprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
>+ jit.arch ~= "x64",
>+})
> 
>-local test = tap.test("misc-memprof-lapi")
> test:plan(5)
> 
> local jit_opt_default = {
>@@ -27,9 +26,10 @@ local bufread = require "utils.bufread"
> local memprof = require "memprof.parse"
> local process = require "memprof.process"
> local symtab = require "utils.symtab"
>+local profilename = require("utils").profilename
> 
>-local TMP_BINFILE = utils.profilename("memprofdata.tmp.bin")
>-local BAD_PATH = utils.profilename("memprofdata/tmp.bin")
>+local TMP_BINFILE = profilename("memprofdata.tmp.bin")
>+local BAD_PATH = profilename("memprofdata/tmp.bin")
> local SRC_PATH = "@"..arg[0]
> 
> local function default_payload()
>@@ -267,12 +267,6 @@ end)
> jit.on()
> 
> test:test("jit-output", function(subtest)
>- -- Disabled on *BSD due to #4819.
>- if jit.os == 'BSD' then
>- subtest:plan(1)
>- subtest:skip('Disabled due to #4819')
>- return
>- end
> 
>   subtest:plan(4)
> 
>diff --git a/test/tarantool-tests/misclib-sysprof-capi.test.lua b/test/tarantool-tests/misclib-sysprof-capi.test.lua
>index dad0fe4a..a9b712a5 100644
>--- a/test/tarantool-tests/misclib-sysprof-capi.test.lua
>+++ b/test/tarantool-tests/misclib-sysprof-capi.test.lua
>@@ -1,21 +1,18 @@
>--- Sysprof is implemented for x86 and x64 architectures only.
>-local utils = require("utils")
>-utils.skipcond(
>- jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
>- jit.arch.." architecture or "..jit.os..
>- " OS is NIY for sysprof"
>-)
>+local tap = require("tap")
>+local test = tap.test("clib-misc-sysprof"):skipcond({
>+ ["Sysprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
>+ jit.arch ~= "x64",
>+ ["Sysprof is implemented for Linux only"] = jit.os ~= "Linux",
>+})
>+
>+test:plan(2)
> 
> local testsysprof = require("testsysprof")
> 
>-local tap = require("tap")
> local jit = require('jit')
> 
> jit.off()
> 
>-local test = tap.test("clib-misc-sysprof")
>-test:plan(2)
>-
> test:ok(testsysprof.base())
> test:ok(testsysprof.validation())
> 
>diff --git a/test/tarantool-tests/misclib-sysprof-lapi.test.lua b/test/tarantool-tests/misclib-sysprof-lapi.test.lua
>index 4bf10e8d..fff89dfd 100644
>--- a/test/tarantool-tests/misclib-sysprof-lapi.test.lua
>+++ b/test/tarantool-tests/misclib-sysprof-lapi.test.lua
>@@ -1,14 +1,10 @@
>--- Sysprof is implemented for x86 and x64 architectures only.
>-local utils = require("utils")
>-utils.skipcond(
>- jit.arch ~= "x86" and jit.arch ~= "x64" or jit.os ~= "Linux",
>- jit.arch.." architecture or "..jit.os..
>- " OS is NIY for sysprof"
>-)
>-
> local tap = require("tap")
>+local test = tap.test("misc-sysprof-lapi"):skipcond({
>+ ["Sysprof is implemented for x86_64 only"] = jit.arch ~= "x86" and
>+ jit.arch ~= "x64",
>+ ["Sysprof is implemented for Linux only"] = jit.os ~= "Linux",
>+})
> 
>-local test = tap.test("misc-sysprof-lapi")
> test:plan(19)
> 
> jit.off()
>@@ -17,9 +13,10 @@ jit.flush()
> local bufread = require("utils.bufread")
> local symtab = require("utils.symtab")
> local sysprof = require("sysprof.parse")
>+local profilename = require("utils").profilename
> 
>-local TMP_BINFILE = utils.profilename("sysprofdata.tmp.bin")
>-local BAD_PATH = utils.profilename("sysprofdata/tmp.bin")
>+local TMP_BINFILE = profilename("sysprofdata.tmp.bin")
>+local BAD_PATH = profilename("sysprofdata/tmp.bin")
> 
> local function payload()
>   local function fib(n)
>diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
>index 343f97e3..47a8fe87 100644
>--- a/test/tarantool-tests/tap.lua
>+++ b/test/tarantool-tests/tap.lua
>@@ -327,6 +327,16 @@ local function check(test)
>   return test.planned == test.total and test.failed == 0
> end
> 
>+local function skipcond(test, conditions)
>+ for reason, condition in pairs(conditions) do
>+ if condition then
>+ local skipfunc = test.planned and skiprest or skipall
>+ skipfunc(test, reason)
>+ end
>+ end
>+ return test
>+end
>+
> test_mt = {
>   __index = {
>     test = new,
>@@ -338,6 +348,7 @@ test_mt = {
>     skip = skip,
>     skipall = skipall,
>     skiprest = skiprest,
>+ skipcond = skipcond,
>     is = is,
>     isnt = isnt,
>     isnil = isnil,
>diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
>index  8355149 b..83716bb3 100644
>--- a/test/tarantool-tests/utils.lua
>+++ b/test/tarantool-tests/utils.lua
>@@ -81,14 +81,6 @@ function M.makecmd(arg, opts)
>   })
> end
> 
>-function M.skipcond(condition, message)
>- if not condition then return end
>- local test = tap.test(arg[0]:match('/?(.+)%.test%.lua'))
>- test:plan(1)
>- test:skip(message)
>- os.exit(test:check() and 0 or 1)
>-end
>-
> function M.hasbc(f, bytecode)
>   assert(type(f) == 'function', 'argument #1 should be a function')
>   assert(type(bytecode) == 'string', 'argument #2 should be a string')
>--
>2.30.2
 

[-- Attachment #2: Type: text/html, Size: 22768 bytes --]

Re: [Tarantool-patches] [PATCH luajit 0/5] Tarantool tests enhancements

[Igor Munkin via Tarantool-patches]

I've checked the patchset into all long-term branches in
tarantool/luajit and bumped a new version in master, 2.11 and 2.10.

On 27.02.23, Igor Munkin wrote:
> I decided to split the original series[1], adjusting JIT-related tests
> in tarantool-tests suite into two separate patchsets: one with
> enhancements for test suite and another one for JIT-related tweaks.
> 
> This series consists of the following changes:
> 
> 1. The first one is a simple tiny maintenance patch to remove excess
>    checks in testing workflows.
> 2. The second one is epic one: unfortunately, <utils.selfrun> is too
>    complex to be maintained, so the corresponding tests are split
>    into two files: the test itself and the script to be run by the
>    test. There is a new helper introduced within this patch, and
>    three tests are refactored.
> 3. The third and the fourth patches introduce <tap.skipall> and
>    <tap.skiprest> helpers.
> 4. The fourth patch makes skipcond helper more convenient: it becomes
>    multi-conditional and it yields the test object, so we can organize
>    more readable chains of skip conditions.
> 
> [1]: https://lists.tarantool.org/tarantool-patches/cover.1676304797.git.imun@tarantool.org/T/#t
> 
> Branch: https://github.com/tarantool/luajit/tree/imun/tap-enhancements
> CI: https://github.com/tarantool/luajit/commit/4e6d8b4
> Tarantool PR: https://github.com/tarantool/tarantool/pull/8288
> 
> Igor Munkin (5):
>   ci: use LuaJIT-test target in testing workflows
>   test: stop using utils.selfrun in tests
>   test: introduce test:skipall TAP helper
>   test: introduce test:skiprest TAP helper
>   test: make skipcond helper more convenient
> 

<snipped>

> 
> -- 
> 2.30.2
> 

-- 
Best regards,
IM