Tarantool development patches archive
 help / color / mirror / Atom feed
From: Igor Munkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Sergey Kaplun <skaplun@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH luajit 3/5] test: run LuaJIT tests via CMake
Date: Fri, 19 Feb 2021 22:04:30 +0300	[thread overview]
Message-ID: <20210219190430.GO5448@tarantool.org> (raw)
In-Reply-To: <20210214184858.GE9361@root>

Sergey,

Thanks for the patch!

On 14.02.21, Sergey Kaplun wrote:
> Hi, Igor!
> 

<snipped>

> > CMake can't expand the generator expression used in DEPENDS section of
> 
> Typo? s/in DEPENDS section/in the DEPENDS section/

This is the common section name, not the particular section.

> 

<snipped>

> > Tarantool tests are implemented using Tarantool on-board TAP module[2],
> > that is moved to LuaJIT repository with a little changes to save Lua
> 
> Typo: s/a little changes/little changes/

No, I don't want to emphasize the amount, but rather the fact the chunks
are almost untouched.

> 

<snipped>

> 
> Side note: What kind of quotes is preferable for our code style in
> LuaJIT?

This is an open question for the LuaJIT style guide. I personally prefer
single until I have to use double ones.

> The new <tap.lua> module is written with single quotes, but a lot of
> code inside LuaJIT uses double quotes (see <src/jit/*.lua>, for
> example). Also, as far as you've already changed indentation it will be
> nice do not stop there and fix quotes and change `{ }` to `{}` as usual.

Fixed all new occurrences.

> 
> Side note: (this is more like discussion question) -- do we really need
> to use Tarantool's <tap.lua> module here? I provide some
> dissatisfactions, but as I say below, we can't just change this
> module API because we want (if you remove this module from Tarantool
> and save it only here).
> If it is saved untouchable in Tarantool repo I see no reason to
> avoid replacing it with another one more suitable (if we want) or
> improving it. Amount of tests is really small now and it can be easily
> adapted. May be we should take a look at Lua test-modules like [1] and
> [2].

I'm totally for lua-TestMore. We can try to port everything to its tap,
when it has been incorporated to the trunk.

> 

<snipped>

> > diff --git a/etc/CMakeLists.txt b/etc/CMakeLists.txt
> > index 4a4c3cd..d54fa79 100644
> > --- a/etc/CMakeLists.txt
> > +++ b/etc/CMakeLists.txt
> > @@ -1,6 +1,7 @@
> >  # Building supplementary materials for LuaJIT.
> >  
> > -cmake_minimum_required(VERSION 2.8.12 FATAL_ERROR)
> > +# See the rationale in the root CMakeLists.txt.
> > +cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
> 
> Why is the minimum required version for this module/subdirectory
> increased?

To make it consistent project-wide, obviously. I totally don't want to
check every damn shit per-CMakeLists.txt like genex support, <continue>
and others, so now we can freely consider only 3.1.3 docs on cmake.org.

> 
> >  
> >  set(LUAJIT_PC_PREFIX ${CMAKE_INSTALL_PREFIX})
> >  if(CMAKE_LIBRARY_ARCHITECTURE)

<snipped>

> > diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
> > new file mode 100644
> > index 0000000..0be4b34
> > --- /dev/null
> > +++ b/test/tarantool-tests/CMakeLists.txt
> > @@ -0,0 +1,92 @@
> > +# Test suite that has been moved from Tarantool repository in
> > +# scope of https://github.com/tarantool/tarantool/issues/4478.
> > +
> > +# See the rationale in the root CMakeLists.txt.
> > +cmake_minimum_required(VERSION 3.1 FATAL_ERROR)
> > +
> > +find_program(PROVE prove)
> > +if(NOT PROVE)
> > +  message(WARNING "`prove' is not found, so tarantool-tests target is not generated")
> > +  return()
> > +endif()
> > +
> > +macro(BuildTestLib lib sources)
> 
> Nit: BuildTestCLib or BuildTestLuaCLib is more verbose for me.
> Feel free to ignore.

Technicaly these libraries are not fully implemented via Lua C API, so I
chose the first one. Diff is below:

================================================================================

diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
index a7af807..c7a4ccc 100644
--- a/test/tarantool-tests/CMakeLists.txt
+++ b/test/tarantool-tests/CMakeLists.txt
@@ -10,7 +10,7 @@ if(NOT PROVE)
   return()
 endif()
 
-macro(BuildTestLib lib sources)
+macro(BuildTestCLib lib sources)
   add_library(${lib} SHARED EXCLUDE_FROM_ALL ${sources})
   target_include_directories(${lib} PRIVATE
     ${LUAJIT_SOURCE_DIR}
diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich/CMakeLists.txt b/test/tarantool-tests/gh-4427-ffi-sandwich/CMakeLists.txt
index 5515567..f2be549 100644
--- a/test/tarantool-tests/gh-4427-ffi-sandwich/CMakeLists.txt
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich/CMakeLists.txt
@@ -1 +1 @@
-BuildTestLib(libsandwich libsandwich.c)
+BuildTestCLib(libsandwich libsandwich.c)
diff --git a/test/tarantool-tests/lj-flush-on-trace/CMakeLists.txt b/test/tarantool-tests/lj-flush-on-trace/CMakeLists.txt
index 91a18a6..16f1aa8 100644
--- a/test/tarantool-tests/lj-flush-on-trace/CMakeLists.txt
+++ b/test/tarantool-tests/lj-flush-on-trace/CMakeLists.txt
@@ -1 +1 @@
-BuildTestLib(libflush libflush.c)
+BuildTestCLib(libflush libflush.c)
diff --git a/test/tarantool-tests/misclib-getmetrics-capi/CMakeLists.txt b/test/tarantool-tests/misclib-getmetrics-capi/CMakeLists.txt
index cff0096..60eb5bb 100644
--- a/test/tarantool-tests/misclib-getmetrics-capi/CMakeLists.txt
+++ b/test/tarantool-tests/misclib-getmetrics-capi/CMakeLists.txt
@@ -1 +1 @@
-BuildTestLib(testgetmetrics testgetmetrics.c)
+BuildTestCLib(testgetmetrics testgetmetrics.c)

================================================================================

> 

<snipped>

> > +  # Unfortunately, CMake is a crap and there is no other way to
> > +  # extend the list in parent scope but join two strings with
> > +  # semicolon. If one finds the normal way to make it work, feel
> > +  # free to reach me.
> > +  set(TESTLIBS "${lib};${TESTLIBS}" PARENT_SCOPE)
> > +  # Add the directory where the lib is built to the LUA_CPATH
> > +  # environment variable, so interpreter can find and load it.
> 
> Typo: s/interpreter/the interpreter/

Changed to LuaJIT.

> 

<snipped>

> > +# LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list
> > +# with dependecies are set in scope of BuildTestLib macro.
> > +add_custom_command(
> > +  COMMENT "Running Tarantool tests"
> > +  OUTPUT tests.ok
> > +  DEPENDS ${LUAJIT_TEST_BINARY} ${TESTLIBS} ${TEST_DEPS}
> > +  COMMAND
> > +  env
> > +    LUA_PATH="${LUA_PATH}\;\;"
> > +    LUA_CPATH="${LUA_CPATH}\;\;"
> > +    LD_LIBRARY_PATH="${LD_LIBRARY_PATH}"
> > +    ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR}
> > +      --exec ${LUAJIT_TEST_BINARY}
> > +      --ext ${LUA_TEST_SUFFIX}
> > +      --failures --shuffle
> 
> See nothing bad to add `--verbose` option here. It's better to see the
> name of the single test, not the test file only.

No. It spoils the output too much. I propose the following change:

================================================================================

diff --git a/test/tarantool-tests/CMakeLists.txt b/test/tarantool-tests/CMakeLists.txt
index c7a4ccc..9c9dabf 100644
--- a/test/tarantool-tests/CMakeLists.txt
+++ b/test/tarantool-tests/CMakeLists.txt
@@ -68,8 +68,13 @@ set(LUA_PATH
   "${CMAKE_CURRENT_SOURCE_DIR}/?.lua\;${PROJECT_SOURCE_DIR}/tools/?.lua"
 )
 set(LUA_TEST_SUFFIX .test.lua)
+set(LUA_TEST_FLAGS --failures --shuffle)
 file(GLOB TEST_DEPS ${CMAKE_CURRENT_SOURCE_DIR}/*${LUA_TEST_SUFFIX})
 
+if(CMAKE_VERBOSE_MAKEFILE)
+  list(APPEND LUA_TEST_FLAGS --verbose)
+endif()
+
 # LUA_CPATH and LD_LIBRARY_PATH variables and also TESTLIBS list
 # with dependecies are set in scope of BuildTestLib macro.
 add_custom_command(
@@ -84,7 +89,7 @@ add_custom_command(
     ${PROVE} ${CMAKE_CURRENT_SOURCE_DIR}
       --exec ${LUAJIT_TEST_BINARY}
       --ext ${LUA_TEST_SUFFIX}
-      --failures --shuffle
+      ${LUA_TEST_FLAGS}
     && touch tests.ok
   WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
 )

================================================================================

If you're OK with it, I'll apply these changes to the branch.

> 
> > +    && touch tests.ok
> > +  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
> > +)
> > +
> > +add_custom_target(tarantool-tests DEPENDS tests.ok)
> > diff --git a/test/gh-3196-incorrect-string-length.test.lua b/test/tarantool-tests/gh-3196-incorrect-string-length.test.lua
> > similarity index 94%
> > rename from test/gh-3196-incorrect-string-length.test.lua
> > rename to test/tarantool-tests/gh-3196-incorrect-string-length.test.lua
> > index f135de7..edb728f 100755
> > --- a/test/gh-3196-incorrect-string-length.test.lua
> > +++ b/test/tarantool-tests/gh-3196-incorrect-string-length.test.lua
> > @@ -1,4 +1,4 @@
> > -#!/usr/bin/env tarantool
> > +#!/usr/bin/env luajit
> 
> I think, that this line unnecessary, as much as an executable format of
> the file. Same thoughts about all test files below.

Removed all shebangs and x permissions.

> 
> >  
> >  -- Miscellaneous test for LuaJIT bugs
> >  local tap = require('tap')
> > diff --git a/test/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
> > similarity index 70%
> > rename from test/gh-4427-ffi-sandwich.test.lua
> > rename to test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
> > index 9d5e50f..e9771e8 100755
> > --- a/test/gh-4427-ffi-sandwich.test.lua
> > +++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
> > @@ -1,22 +1,30 @@
> > -#!/usr/bin/env tarantool
> > +#!/usr/bin/env luajit
> >  
> >  if #arg == 0 then
> > -  require('utils').selfrun(arg, {
> > +
> > +  local utils = require('utils')
> > +
> > +  -- Disabled on *BSD due to #4819.
> > +  utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
> > +
> > +  utils.selfrun(arg, {
> >      {
> > -      arg = {
> > +      arg  = {
> 
> This alignment is beauty, but unnecessary, please drop these changes.
> Same remarks for the changes below.

Removed.

> 
> >          1, -- hotloop (arg[1])
> 
> Side note: this may be not arg[1] or so on for Tarantool or pure LuaJIT
> with additional flags. But good for now, so whatever.
> 
> >          1, -- trigger (arg[2])
> >        },
> > -      res = tostring(3), -- hotloop + trigger + 1
> > -      msg = 'Trace is aborted',
> > +      test = 'is',
> > +      res  = tostring(3), -- hotloop + trigger + 1
> > +      msg  = 'Trace is aborted',
> >      },
> >      {
> > -      arg = {
> > +      arg  = {
> >          1, -- hotloop (arg[1])
> >          2, -- trigger (arg[2])
> >        },
> > -      res = 'Lua VM re-entrancy is detected while executing the trace',
> > -      msg = 'Trace is recorded',
> > +      test = 'like',
> 
> What does it mean?

The type of test used for verification: either pattern matching ('like')
or exact match ('is').

> 
> > +      res  = 'Lua VM re%-entrancy is detected while executing the trace',
> 
> This % is redundant.

Any proof for it?

> 
> > +      msg  = 'Trace is recorded',
> >      },
> >    })
> >  end

<snipped>

> > diff --git a/test/misclib-getmetrics-lapi.test.lua b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
> > similarity index 98%
> > rename from test/misclib-getmetrics-lapi.test.lua
> > rename to test/tarantool-tests/misclib-getmetrics-lapi.test.lua
> > index 3b3d1f8..959293d 100755
> > --- a/test/misclib-getmetrics-lapi.test.lua
> > +++ b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua

<snipped>

> > @@ -49,7 +52,10 @@ test:test("gc-allocated-freed", function(subtest)
> >      subtest:plan(1)
> >  
> >      -- Force up garbage collect all dead objects.
> 
> Please, add comments about testing as a third party for the Tarantool
> and GC finalizers to describe this change.

Well, isn't everything described above? We are forcing GC to collect
*all* dead objects. In other words until collectgarbage("count") return
value changes.

> 
> > -    collectgarbage("collect")
> > +    repeat
> > +        local count = collectgarbage("count")
> > +        collectgarbage("collect")
> > +    until collectgarbage("count") == count
> >  
> >      -- Bump getmetrics table and string keys allocation.
> >      local old_metrics = misc.getmetrics()

<snipped>

> > diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
> 
> Side note: firstly I thought that <tap.lua> will be clashing with tap
> inside other suites, but these conflicts will be easily resolved by
> LUA_PATH.
> 
> > new file mode 100644
> > index 0000000..fd03132
> > --- /dev/null
> > +++ b/test/tarantool-tests/tap.lua
> 
> Feel free to stop me from refactoring anytime, but I try to dump all my
> thoughts about this module now to be referred later.
> 
> Side note: It's a pity that there is no such module as Perl's Test::Deep [3]
> in Lua (or at least I haven't seen one).
> 
> > @@ -0,0 +1,306 @@
> > +--- tap.lua internal file.
> > +---
> > +--- The Test Anything Protocol vesion 13 producer.
> > +---
> > +
> > +-- Initializer FFI for <iscdata> check.
> 
> This comment is misleading a bit, NULL usage is important too.

Removed.

> 
> > +local ffi = require('ffi')
> > +local NULL = ffi.new('void *')
> 
> Nit: What if LuaJIT is built without FFI?

Lol, these tests do not respect such configuration. I have the following
change to support testing with FFI disabled, but most of the tests are
simply skipped.

================================================================================

diff --git a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
index 149bd9a..17b4852 100644
--- a/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
+++ b/test/tarantool-tests/gh-4427-ffi-sandwich.test.lua
@@ -5,6 +5,9 @@ if #arg == 0 then
   -- Disabled on *BSD due to #4819.
   utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
 
+  utils.skipcond(not pcall(require, 'ffi'),
+                 'Disabled due to the lack of FFI support')
+
   utils.selfrun(arg, {
     {
       arg = {
diff --git a/test/tarantool-tests/lj-494-table-chain-infinite-loop.test.lua b/test/tarantool-tests/lj-494-table-chain-infinite-loop.test.lua
index 9c04ef6..f478516 100644
--- a/test/tarantool-tests/lj-494-table-chain-infinite-loop.test.lua
+++ b/test/tarantool-tests/lj-494-table-chain-infinite-loop.test.lua
@@ -1,5 +1,8 @@
 #!/usr/bin/env luajit
 
+require('utils').skipcond(not pcall(require, 'ffi'),
+                          'Disabled due to the lack of FFI support')
+
 local tap = require('tap')
 
 local test = tap.test("lj-494-table-chain-infinite-loop")
diff --git a/test/tarantool-tests/lj-524-fold-conv-respect-src-irt.test.lua b/test/tarantool-tests/lj-524-fold-conv-respect-src-irt.test.lua
index cd0f0f0..adce712 100644
--- a/test/tarantool-tests/lj-524-fold-conv-respect-src-irt.test.lua
+++ b/test/tarantool-tests/lj-524-fold-conv-respect-src-irt.test.lua
@@ -1,3 +1,5 @@
+require('utils').skipcond(not pcall(require, 'ffi'),
+                          'Disabled due to the lack of FFI support')
 local tap = require('tap')
 local ffi = require('ffi')
 
diff --git a/test/tarantool-tests/lj-flush-on-trace.test.lua b/test/tarantool-tests/lj-flush-on-trace.test.lua
index 46fee53..4c665f2 100644
--- a/test/tarantool-tests/lj-flush-on-trace.test.lua
+++ b/test/tarantool-tests/lj-flush-on-trace.test.lua
@@ -5,6 +5,9 @@ if #arg == 0 then
   -- Disabled on *BSD due to #4819.
   utils.skipcond(jit.os == 'BSD', 'Disabled due to #4819')
 
+  utils.skipcond(not pcall(require, 'ffi'),
+                 'Disabled due to the lack of FFI support')
+
   utils.selfrun(arg, {
     {
       arg = {
diff --git a/test/tarantool-tests/misclib-getmetrics-capi.test.lua b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
index ef93597..785b2c6 100644
--- a/test/tarantool-tests/misclib-getmetrics-capi.test.lua
+++ b/test/tarantool-tests/misclib-getmetrics-capi.test.lua
@@ -23,45 +23,49 @@ test:ok(testgetmetrics.base())
 test:ok(testgetmetrics.gc_allocated_freed())
 test:ok(testgetmetrics.gc_steps())
 
-test:ok(testgetmetrics.objcount(function(iterations)
-    local ffi = require("ffi")
-
-    jit.opt.start(0)
-
-    local placeholder = {
-        str = {},
-        tab = {},
-        udata = {},
-        cdata = {},
-    }
-
-    -- Separate objects creations to separate jit traces.
-    for _ = 1, iterations do
-        table.insert(placeholder.str, tostring(_))
-    end
-
-    for _ = 1, iterations do
-        table.insert(placeholder.tab, {_})
-    end
-
-    for _ = 1, iterations do
-        table.insert(placeholder.udata, newproxy())
-    end
-
-    for _ = 1, iterations do
-        -- Check counting of VLA/VLS/aligned cdata.
-        table.insert(placeholder.cdata, ffi.new("char[?]", 4))
-    end
-
-    for _ = 1, iterations do
-        -- Check counting of non-VLA/VLS/aligned cdata.
-        table.insert(placeholder.cdata, ffi.new("uint64_t", _))
-    end
-
-    placeholder = nil -- luacheck: no unused
-    -- Restore default jit settings.
-    jit.opt.start(unpack(jit_opt_default))
-end))
+if pcall(require, "ffi") then
+    test:ok(testgetmetrics.objcount(function(iterations)
+        local ffi = require("ffi")
+
+        jit.opt.start(0)
+
+        local placeholder = {
+            str = {},
+            tab = {},
+            udata = {},
+            cdata = {},
+        }
+
+        -- Separate objects creations to separate jit traces.
+        for _ = 1, iterations do
+            table.insert(placeholder.str, tostring(_))
+        end
+
+        for _ = 1, iterations do
+            table.insert(placeholder.tab, {_})
+        end
+
+        for _ = 1, iterations do
+            table.insert(placeholder.udata, newproxy())
+        end
+
+        for _ = 1, iterations do
+            -- Check counting of VLA/VLS/aligned cdata.
+            table.insert(placeholder.cdata, ffi.new("char[?]", 4))
+        end
+
+        for _ = 1, iterations do
+            -- Check counting of non-VLA/VLS/aligned cdata.
+            table.insert(placeholder.cdata, ffi.new("uint64_t", _))
+        end
+
+        placeholder = nil -- luacheck: no unused
+        -- Restore default jit settings.
+        jit.opt.start(unpack(jit_opt_default))
+    end))
+else
+    test:skip('Disabled due to the lack of FFI support')
+end
 
 -- Compiled loop with a direct exit to the interpreter.
 test:ok(testgetmetrics.snap_restores(function()
diff --git a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
index a005781..faffe5c 100644
--- a/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
+++ b/test/tarantool-tests/misclib-getmetrics-lapi.test.lua
@@ -175,69 +175,73 @@ test:test("gc-steps", function(subtest)
     subtest:is(newm.gc_steps_finalize, 0)
 end)
 
-test:test("objcount", function(subtest)
-    subtest:plan(4)
-    local ffi = require("ffi")
+if pcall(require, "ffi") then
+    test:test("objcount", function(subtest)
+        subtest:plan(4)
+        local ffi = require("ffi")
 
-    jit.opt.start(0)
+        jit.opt.start(0)
 
-    -- Remove all dead objects.
-    collectgarbage("collect")
-
-    local old_metrics = misc.getmetrics()
-
-    local placeholder = {
-        str = {},
-        tab = {},
-        udata = {},
-        cdata = {},
-    }
+        -- Remove all dead objects.
+        collectgarbage("collect")
 
-    -- Separate objects creations to separate jit traces.
-    for _ = 1, 1000 do
-        table.insert(placeholder.str, tostring(_))
-    end
+        local old_metrics = misc.getmetrics()
 
-    for _ = 1, 1000 do
-        table.insert(placeholder.tab, {_})
-    end
+        local placeholder = {
+            str = {},
+            tab = {},
+            udata = {},
+            cdata = {},
+        }
 
-    for _ = 1, 1000 do
-        table.insert(placeholder.udata, newproxy())
-    end
+        -- Separate objects creations to separate jit traces.
+        for _ = 1, 1000 do
+            table.insert(placeholder.str, tostring(_))
+        end
 
-    for _ = 1, 1000 do
-        -- Check counting of VLA/VLS/aligned cdata.
-        table.insert(placeholder.cdata, ffi.new("char[?]", 4))
-    end
+        for _ = 1, 1000 do
+            table.insert(placeholder.tab, {_})
+        end
 
-    for _ = 1, 1000 do
-        -- Check counting of non-VLA/VLS/aligned cdata.
-        table.insert(placeholder.cdata, ffi.new("uint64_t", _))
-    end
+        for _ = 1, 1000 do
+            table.insert(placeholder.udata, newproxy())
+        end
 
-    placeholder = nil -- luacheck: no unused
-    collectgarbage("collect")
-    local new_metrics = misc.getmetrics()
+        for _ = 1, 1000 do
+            -- Check counting of VLA/VLS/aligned cdata.
+            table.insert(placeholder.cdata, ffi.new("char[?]", 4))
+        end
 
-    -- Check that amount of objects not increased.
-    subtest:is(new_metrics.gc_strnum, old_metrics.gc_strnum,
-               "strnum don't change")
-    -- When we call getmetrics, we create table for metrics first.
-    -- So, when we save old_metrics there are x + 1 tables,
-    -- when we save new_metrics there are x + 2 tables, because
-    -- old table hasn't been collected yet (it is still
-    -- reachable).
-    subtest:is(new_metrics.gc_tabnum - old_metrics.gc_tabnum, 1,
-               "tabnum don't change")
-    subtest:is(new_metrics.gc_udatanum, old_metrics.gc_udatanum,
-               "udatanum don't change")
-    subtest:is(new_metrics.gc_cdatanum, old_metrics.gc_cdatanum,
-               "cdatanum don't change")
+        for _ = 1, 1000 do
+            -- Check counting of non-VLA/VLS/aligned cdata.
+            table.insert(placeholder.cdata, ffi.new("uint64_t", _))
+        end
 
-    -- Restore default jit settings.
-    jit.opt.start(unpack(jit_opt_default))
-end)
+        placeholder = nil -- luacheck: no unused
+        collectgarbage("collect")
+        local new_metrics = misc.getmetrics()
+
+        -- Check that amount of objects not increased.
+        subtest:is(new_metrics.gc_strnum, old_metrics.gc_strnum,
+                   "strnum don't change")
+        -- When we call getmetrics, we create table for metrics first.
+        -- So, when we save old_metrics there are x + 1 tables,
+        -- when we save new_metrics there are x + 2 tables, because
+        -- old table hasn't been collected yet (it is still
+        -- reachable).
+        subtest:is(new_metrics.gc_tabnum - old_metrics.gc_tabnum, 1,
+                   "tabnum don't change")
+        subtest:is(new_metrics.gc_udatanum, old_metrics.gc_udatanum,
+                   "udatanum don't change")
+        subtest:is(new_metrics.gc_cdatanum, old_metrics.gc_cdatanum,
+                   "cdatanum don't change")
+
+        -- Restore default jit settings.
+        jit.opt.start(unpack(jit_opt_default))
+    end)
+else
+    test:skip('Disabled due to the lack of FFI support')
+end
 
 test:test("snap-restores-direct-loop", function(subtest)
     -- Compiled loop with a direct exit to the interpreter.
diff --git a/test/tarantool-tests/misclib-memprof-lapi.test.lua b/test/tarantool-tests/misclib-memprof-lapi.test.lua
index 7bfcb4d..f05d7e6 100644
--- a/test/tarantool-tests/misclib-memprof-lapi.test.lua
+++ b/test/tarantool-tests/misclib-memprof-lapi.test.lua
@@ -1,3 +1,8 @@
+-- XXX: utils.bufread unconditionally loads FFI, so the test can't
+-- proceed if FFI support is not enabled.
+require('utils').skipcond(not pcall(require, 'ffi'),
+                          'Disabled due to the lack of FFI support')
+
 local tap = require("tap")
 
 local test = tap.test("misc-memprof-lapi")
diff --git a/test/tarantool-tests/or-232-unsink-64-kptr.test.lua b/test/tarantool-tests/or-232-unsink-64-kptr.test.lua
index fc8acaa..b4e0778 100644
--- a/test/tarantool-tests/or-232-unsink-64-kptr.test.lua
+++ b/test/tarantool-tests/or-232-unsink-64-kptr.test.lua
@@ -1,3 +1,6 @@
+require('utils').skipcond(not pcall(require, 'ffi'),
+                          'Disabled due to the lack of FFI support')
+
 local tap = require('tap')
 
 local test = tap.test("or-232-unsink-64-kptr")
diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index f0730c0..46e3bd1 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -4,8 +4,8 @@
 ---
 
 -- Initializer FFI for <iscdata> check.
-local ffi = require('ffi')
-local NULL = ffi.new('void *')
+local hasffi, ffi = pcall(require, 'ffi')
+local NULL = hasffi and ffi.new('void *')
 
 local function indent(chars)
   return (' '):rep(chars)
@@ -204,6 +204,9 @@ end
 
 local function iscdata(test, v, ctype, message, extra)
   extra = extra or {}
+  if not hasffi then
+    return fail(test, message, extra)
+  end
   extra.expected = ffi.typeof(ctype)
   if type(v) ~= 'cdata' then
     extra.got = type(v)

================================================================================

BTW, nothing works also if LuaJIT is build with JIT support disabled.
Furthermore, I believe LuaJIT suite also doesn't fit to this particular
configuration. I propose to adjust the tests in scope of another issue,
if you want.

> 

<snipped>

> 
> > +  level = level or 3
> 
> Why 3? Please add the comment about this here.
> Also, please mention about LuaJIT behaviour here (traceback not
> reporting tail call) -- that's why traceback is one and the same for
> `ok()` and `fail()` functions.

Added the following comment:

================================================================================

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index 29af3b9..41e918c 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -15,6 +15,14 @@ end
 
 local function traceback(level)
   local trace = {}
+  -- The default level is 3 since at this point you have the
+  -- following frame layout
+  -- frame #0: <debug.getinfo> (its call is below)
+  -- frame #1: <traceback> (this function)
+  -- frame #2: <ok> (the "dominator" function for all assertions)
+  -- XXX: all exported assertions call <ok> function using tail
+  -- call (i.e. "return ok(...)"), so frame #3 contains the
+  -- assertion function used in the test chunk.
   level = level or 3
   while true do
     local info = debug.getinfo(level, "nSl")

================================================================================

> 

<snipped>

> > +    table.insert(trace, {
> > +      source   = info.source,
> > +      src      = info.short_src,
> > +      line     = info.linedefined or 0,
> > +      what     = info.what,
> > +      name     = info.name,
> > +      namewhat = info.namewhat,
> > +      filename = info.source:sub(1, 1) == '@' and info.source:sub(2) or 'eval',
> 
> What does 'eval' mean?

AFAICS, the code executed via -e flag: in that case source is the line
passed via this flag.

> 

<snipped>

> > +local function ok(test, cond, message, extra)
> > +  test.total = test.total + 1
> > +  if cond then
> > +    io.write(indent(4 * test.level), ('ok - %s\n'):format(message))
> > +    return true
> > +  end
> > +
> > +  test.failed = test.failed + 1
> > +  io.write(indent(4 * test.level), ('not ok - %s\n'):format(message))
> 
> Nit: 4 * test.level repeated several times. May be encapsulated inside
> `local cur_level` variable.
> Feel free to ignore.

I reimplemented <indent> function a bit. The changes are below:

================================================================================

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index 3fe6910..41e918c 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -7,8 +7,10 @@
 local ffi = require("ffi")
 local NULL = ffi.new("void *")
 
-local function indent(chars)
-  return (" "):rep(chars)
+local function indent(level, size)
+  -- Use the default indent size if none is specified.
+  size = tonumber(size) or 4
+  return (" "):rep(level * size)
 end
 
 local function traceback(level)
@@ -42,22 +44,23 @@ local function traceback(level)
 end
 
 local function diag(test, fmt, ...)
-  io.write(indent(4 * test.level), ("# %s\n"):format(fmt:format(...)))
+  io.write(indent(test.level), ("# %s\n"):format(fmt:format(...)))
 end
 
 local function ok(test, cond, message, extra)
   test.total = test.total + 1
   if cond then
-    io.write(indent(4 * test.level), ("ok - %s\n"):format(message))
+    io.write(indent(test.level), ("ok - %s\n"):format(message))
     return true
   end
 
   test.failed = test.failed + 1
-  io.write(indent(4 * test.level), ("not ok - %s\n"):format(message))
+  io.write(indent(test.level), ("not ok - %s\n"):format(message))
 
   -- Dump extra contents added in outer space.
   for key, value in pairs(extra or {}) do
-    io.write(indent(2 + 4 * test.level), ("%s:\t%s\n"):format(key, value))
+    -- XXX: Use "half indent" to dump <extra> fields.
+    io.write(indent(test.level + 0.5), ("%s:\t%s\n"):format(key, value))
   end
 
   if not test.trace then
@@ -65,12 +68,13 @@ local function ok(test, cond, message, extra)
   end
 
   local trace = traceback()
-  local tindent = indent(4 + 4 * test.level)
+  local tindent = indent(test.level + 1)
   io.write(tindent, ("filename:\t%s\n"):format(trace[#trace].filename))
   io.write(tindent, ("line:\t%s\n"):format(trace[#trace].line))
   for frameno, frame in ipairs(trace) do
     io.write(tindent, ("frame #%d\n"):format(frameno))
-    local findent = indent(2) .. tindent
+    -- XXX: Use "half indent" to dump <frame> fiels.
+    local findent = indent(0.5) .. tindent
     for key, value in pairs(frame) do
       io.write(findent, ("%s:\t%s\n"):format(key, value))
     end
@@ -257,7 +261,7 @@ end
 
 local function plan(test, planned)
   test.planned = planned
-  io.write(indent(4 * test.level), ("1..%d\n"):format(planned))
+  io.write(indent(test.level), ("1..%d\n"):format(planned))
 end
 
 local function check(test)

================================================================================

> 

<snipped>

> 
> > +    return false
> > +  end
> > +
> > +  local trace = traceback()
> 
> Nit: Looks like this chunk can be separated inside one function
> `dump_traceback()` back or whatever.
> Feel free to ignore.

Don't get this nit.

> 
> > +  local tindent = indent(4 + 4 * test.level)
> > +  io.write(tindent, ('filename:\t%s\n'):format(trace[#trace].filename))
> > +  io.write(tindent, ('line:\t%s\n'):format(trace[#trace].line))
> > +  for frameno, frame in ipairs(trace) do
> > +    io.write(tindent, ('frame #%d\n'):format(frameno))
> 
> Nit: Looks inconsistent with other output.
> 
> | io.write(tindent, ('frameno: #%d\n'):format(frameno))
> 
> hits better, in my opinion.
> 
> Looks like it will be good to add the function that does something like:

You missed there is a space instead of the tab in the output above. This
is done intentionally.

> 
> | io.write(indent, ('%s:\t%s\n'):format(key, value))
> 
> Feel free to ignore.

Ignoring.

> 
> > +    local findent = indent(2) .. tindent
> 
> Nit: s/ .. /../
> Here and below.

We have already discussed it: this is a binary op, so it should be
wrapped with the spaces like other binary ops. You argument "this is how
it is used in LuaJIT sources" doesn't work here, since concat is wrapped
with spaces in almost all test files (except ones introduced by you).

Ignoring.

> 

<snipped>

> 
> > +local function cmpdeeply(got, expected, extra)
> > +  if type(expected) == 'number' or type(got) == 'number' then
> > +    extra.got = got
> > +    extra.expected = expected
> > +    -- Handle NaN.
> > +    if got ~= got and expected ~= expected then
> > +      return true
> > +    end
> > +    return got == expected
> > +  end
> > +
> > +  if ffi.istype('bool', got) then got = (got == 1) end
> > +  if ffi.istype('bool', expected) then expected = (expected == 1) end
> > +
> > +  if extra.strict and type(got) ~= type(expected) then
> > +    extra.got = type(got)
> > +    extra.expected = type(expected)
> > +    return false
> > +  end
> > +
> > +  if type(got) ~= 'table' or type(expected) ~= 'table' then
> > +    extra.got = got
> > +    extra.expected = expected
> > +    return got == expected
> > +  end
> > +
> > +  local path = extra.path or '/'
> 
> This `path` in `extra` field looks unusable, please drop it.

Why do you think so? It allows to distinguish the path with missing keys
or invalid values. Anyway, I've reimplemented it a bit.

================================================================================

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index ce563fb..a5ac31a 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -159,12 +159,12 @@ local function is_deeply(test, got, expected, message, extra)
     end
     seen[got] = true
 
-    local path = extra.path or "/"
+    local path = extra.path or "obj"
     local has = {}
 
     for k, v in pairs(got) do
       has[k] = true
-      extra.path = path .. "/" .. k
+      extra.path = path .. "." .. k
       if not cmpdeeply(v, expected[k], extra) then
         return false
       end
@@ -173,6 +173,7 @@ local function is_deeply(test, got, expected, message, extra)
     -- Check if expected contains more keys then got.
     for k, v in pairs(expected) do
       if has[k] ~= true and (extra.strict or v ~= NULL) then
+        extra.path = path .. "." .. k
         extra.expected = "key " .. tostring(k)
         extra.got = "nil"
         return false

================================================================================

> 

<snipped>

> > +    visited_keys[i] = true

I believe this doesn't handle your case. This variable is introduced for
checking whether there are extra keys in <expected> table.

> 
> This is not good enough, visited keys should be cashed better:
> 
> | tarantool> local test = tap.test("<T>") local t1 = {1, 2, 3} t1[4] = {t1} local t2 = {1, 2, 3} t2[4] = {t2} return test:is_deeply(t1, t2), t1, t2
> | TAP version 13
> | ---
> | - error: stack overflow

Anyway, nice catch, thanks! Fixed this, diff is below:

================================================================================

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index f13e1b7..44e731f 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -88,58 +88,6 @@ local function skip(test, message, extra)
   ok(test, true, message .. " # skip", extra)
 end
 
-local function cmpdeeply(got, expected, extra)
-  if type(expected) == "number" or type(got) == "number" then
-    extra.got = got
-    extra.expected = expected
-    -- Handle NaN.
-    if got ~= got and expected ~= expected then
-      return true
-    end
-    return got == expected
-  end
-
-  if ffi.istype("bool", got) then got = (got == 1) end
-  if ffi.istype("bool", expected) then expected = (expected == 1) end
-
-  if extra.strict and type(got) ~= type(expected) then
-    extra.got = type(got)
-    extra.expected = type(expected)
-    return false
-  end
-
-  if type(got) ~= "table" or type(expected) ~= "table" then
-    extra.got = got
-    extra.expected = expected
-    return got == expected
-  end
-
-  local path = extra.path or "obj"
-  local visited_keys = {}
-
-  for i, v in pairs(got) do
-    visited_keys[i] = true
-    extra.path = path .. "." .. i
-    if not cmpdeeply(v, expected[i], extra) then
-      return false
-    end
-  end
-
-  -- Check if expected contains more keys then got.
-  for i, v in pairs(expected) do
-    if visited_keys[i] ~= true and (extra.strict or v ~= NULL) then
-      extra.path = path .. "." .. i
-      extra.expected = "key " .. tostring(i)
-      extra.got = "nil"
-      return false
-    end
-  end
-
-  extra.path = path
-
-  return true
-end
-
 local function like(test, got, pattern, message, extra)
   extra = extra or {}
   extra.got = got
@@ -177,6 +125,66 @@ local function is_deeply(test, got, expected, message, extra)
   extra.got = got
   extra.expected = expected
   extra.strict = test.strict
+
+  local seen = {}
+  local function cmpdeeply(got, expected, extra) --luacheck: ignore
+    if type(expected) == "number" or type(got) == "number" then
+      extra.got = got
+      extra.expected = expected
+      -- Handle NaN.
+      if got ~= got and expected ~= expected then
+        return true
+      end
+      return got == expected
+    end
+
+    if ffi.istype("bool", got) then got = (got == 1) end
+    if ffi.istype("bool", expected) then expected = (expected == 1) end
+
+    if extra.strict and type(got) ~= type(expected) then
+      extra.got = type(got)
+      extra.expected = type(expected)
+      return false
+    end
+
+    if type(got) ~= "table" or type(expected) ~= "table" then
+      extra.got = got
+      extra.expected = expected
+      return got == expected
+    end
+
+    -- Stop if tables are equal or <got> has been already seen.
+    if got == expected or seen[got] then
+      return true
+    end
+    seen[got] = true
+
+    local path = extra.path or "obj"
+    local has = {}
+
+    for k, v in pairs(got) do
+      has[k] = true
+      extra.path = path .. "." .. k
+      if not cmpdeeply(v, expected[k], extra) then
+        return false
+      end
+    end
+
+    -- Check if expected contains more keys then got.
+    for k, v in pairs(expected) do
+      if has[k] ~= true and (extra.strict or v ~= NULL) then
+        extra.path = path .. "." .. k
+        extra.expected = "key " .. tostring(k)
+        extra.got = "nil"
+        return false
+      end
+    end
+
+    extra.path = path
+
+    return true
+  end
+
   return ok(test, cmpdeeply(got, expected, extra), message, extra)
 end
 

================================================================================

> 
> > +    extra.path = path .. '/' .. i
> > +    if not cmpdeeply(v, expected[i], extra) then
> > +      return false
> > +    end
> > +  end
> > +
> > +  -- Check if expected contains more keys then got.
> > +  for i, v in pairs(expected) do

<snipped>

> > +local function is(test, got, expected, message, extra)
> > +  extra = extra or { }
> > +  extra.got = got
> > +  extra.expected = expected
> > +  local rc = (test.strict == false or type(got) == type(expected))
> 
> Nit: It is nice to leave the comment here to provide description why
> just `==` is not enough. Also, I don't like this behaviour -- it
> disrespects `eq` metamethod, and there is not mentioned in the doc [4].

In what sense this "disrespects" eq metamethod? Unfortunately, these
docs are quite bad...

> 
> > +             and got == expected
> > +  return ok(test, rc, message, extra)
> > +end
> > +
> > +local function isnt(test, got, unexpected, message, extra)
> > +  extra = extra or { }
> > +  extra.got = got
> > +  extra.unexpected = unexpected
> > +  local rc = (test.strict == true and type(got) ~= type(unexpected))
> 
> Ditto.
> 
> > +             or got ~= unexpected
> > +  return ok(test, rc, message, extra)
> > +end
> > +
> > +local function is_deeply(test, got, expected, message, extra)
> > +  extra = extra or { }
> > +  extra.got = got
> > +  extra.expected = expected
> > +  extra.strict = test.strict
> > +  return ok(test, cmpdeeply(got, expected, extra), message, extra)
> > +end
> > +
> > +local function isnil(test, v, message, extra)
> > +  return is(test, not v and v == nil and 'nil' or v, 'nil', message, extra)
> 
> Looks like type(v) 'nil' is enough. Or please add a comment why it is
> necessary.

You're right, these are just artefacts. Fixed, squashed, force-pushed to
the branch. Diff is below:

================================================================================

diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index 9aa823f..44e731f 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -189,7 +189,7 @@ local function is_deeply(test, got, expected, message, extra)
 end
 
 local function isnil(test, v, message, extra)
-  return is(test, not v and v == nil and "nil" or v, "nil", message, extra)
+  return is(test, type(v), "nil", message, extra)
 end
 
 local function isnumber(test, v, message, extra)

================================================================================

> 
> > +end
> > +
> > +local function isnumber(test, v, message, extra)
> > +  return is(test, type(v), 'number', message, extra)
> > +end
> > +
> > +local function isstring(test, v, message, extra)
> > +  return is(test, type(v), 'string', message, extra)
> > +end
> > +
> > +local function istable(test, v, message, extra)
> > +  return is(test, type(v), 'table', message, extra)
> > +end
> > +
> > +local function isboolean(test, v, message, extra)
> > +  return is(test, type(v), 'boolean', message, extra)
> > +end
> > +
> > +local function isfunction(test, v, message, extra)
> > +  return is(test, type(v), 'function', message, extra)
> > +end
> > +
> > +local function isudata(test, v, utype, message, extra)
> 
> Side note: Looks like it does not test type(v) equals "udata", that's

Em, what?..

> inconsistent with other checkers. But changing it equals to break
> backward compatibility.

With what? BTW, it is already broken after the changes in <isnil>.

> 
> > +  extra = extra or { }
> > +  extra.expected = ('userdata<%s>'):format(utype)
> > +  if type(v) ~= 'userdata' then

... Here it is.

> > +    extra.got = type(v)
> > +    return fail(test, message, extra)
> > +  end
> > +  extra.got = ('userdata<%s>'):format(getmetatable(v))
> > +  return ok(test, getmetatable(v) == utype, message, extra)
> 
> Does it assume that compared udata-s should have the same metatable?

Otherwise there is nothing else to compare. BTW, I strongly doubt this
is rather useful.

> 
> > +end
> > +
> > +local function iscdata(test, v, ctype, message, extra)
> 
> Ditto about inconsistency.

Still don't get what inconsistency you're talking about.

> 
> > +  extra = extra or { }
> > +  extra.expected = ffi.typeof(ctype)
> > +  if type(v) ~= 'cdata' then
> > +    extra.got = type(v)
> > +    return fail(test, message, extra)
> > +  end
> > +  extra.got = ffi.typeof(v)
> > +  return ok(test, ffi.istype(ctype, v), message, extra)
> > +end
> > +
> > +local test_mt
> 
> Nit: Why is it declared here, not inside the `new()` function?

It is used in new function as an upvalue and is filled with the methods
later. If I declared it inside the <new> function it would be either
global or not visible outside of the function.

> 
> > +
> > +local function new(parent, name, fun, ...)
> 
> Side note: Why so serious :}?
> 
> > +  local level = parent ~= nil and parent.level + 1 or 0
> > +  local test = setmetatable({
> > +    parent  = parent,
> > +    name    = name,
> > +    level   = level,
> > +    total   = 0,
> > +    failed  = 0,
> > +    planned = 0,
> > +    trace   = parent == nil and true or parent.trace,
> > +    strict  = false,
> 
> What does strict mean and how it can be changed?

Em, in a quite simple way?
| test.strict = true

> I see nothing bad to copy some rationale from documentation [5] here
> as comments for each field.

I found no rationale there, could you please specify the places you want
to clarify in a more accurate way?

> 
> > +  }, test_mt)
> > +  if fun == nil then
> > +    return test
> > +  end
> > +  test:diag('%s', test.name)
> > +  fun(test, ...)
> > +  test:diag('%s: end', test.name)
> > +  return test:check()
> > +end

<snipped>

> > +local function check(test)
> > +  if test.checked then
> > +    error('check called twice')
> 
> Nit: eror_level = 2 is better here in my opinion.
> Feel free to ignore.

Unfortunately, nothing's changed:

================================================================================

$ luajit t.lua
luajit: ./tap.lua:261: check called twice
stack traceback:
	[C]: in function 'error'
	./tap.lua:261: in function 'check'
	t.lua:9: in main chunk
	[C]: at 0x5563639c4eb0
TAP version 13
1..1
ok - nil
$ git stash pop
<snipped>
$ git diff
diff --git a/test/tarantool-tests/tap.lua b/test/tarantool-tests/tap.lua
index e2149a7..575e7bd 100644
--- a/test/tarantool-tests/tap.lua
+++ b/test/tarantool-tests/tap.lua
@@ -258,7 +258,7 @@ end
 
 local function check(test)
   if test.checked then
-    error('check called twice')
+    error('check called twice', 2)
   end
   test.checked = true
   if test.planned ~= test.total then
$ luajit t.lua
luajit: t.lua:9: check called twice
stack traceback:
	[C]: in function 'error'
	./tap.lua:261: in function 'check'
	t.lua:9: in main chunk
	[C]: at 0x55cea3cfceb0
TAP version 13
1..1
ok - nil

================================================================================

Please correct me if I've done something wrong. Ignoring for now.

> 
> > +  end
> > +  test.checked = true
> > +  if test.planned ~= test.total then
> > +    if test.parent ~= nil then
> > +      ok(test.parent, false, 'bad plan', {
> > +        planned = test.planned,
> > +        run = test.total,
> > +      })
> > +    else
> > +      diag(test, ('bad plan: planned %d run %d')
> > +        :format(test.planned, test.total))
> > +    end
> > +  elseif test.failed > 0 then
> > +    if test.parent ~= nil then
> > +      ok(test.parent, false, 'failed subtests', {
> > +        failed = test.failed,
> > +        planned = test.planned,
> > +      })
> > +    else
> > +      diag(test, 'failed subtest: %d', test.failed)
> > +    end
> > +  else
> > +    if test.parent ~= nil then
> > +      ok(test.parent, true, test.name)
> > +    end
> > +  end
> 
> Nit: looks like `if test.parent ~= nil` can be taken out for branches.
> Feel free to ignore.

It can, but such change doesn't worth it.

Ignoring.

> 
> > +  return test.planned == test.total and test.failed == 0
> > +end
> > +
> > +test_mt = {
> > +  __index = {
> > +    test       = new,
> > +    plan       = plan,
> > +    check      = check,
> > +    diag       = diag,
> > +    ok         = ok,
> > +    fail       = fail,
> > +    skip       = skip,
> > +    is         = is,
> > +    isnt       = isnt,
> > +    isnil      = isnil,
> > +    isnumber   = isnumber,
> > +    isstring   = isstring,
> > +    istable    = istable,
> > +    isboolean  = isboolean,
> > +    isfunction = isfunction,
> > +    isudata    = isudata,
> > +    iscdata    = iscdata,
> > +    is_deeply  = is_deeply,
> > +    like       = like,
> > +    unlike     = unlike,
> > +  }
> > +}
> > +
> > +return {
> > +  test = function(...)
> > +    io.write('TAP version 13\n')
> > +    return new(nil, ...)
> > +  end
> > +}
> > diff --git a/test/tarantool-tests/utils.lua b/test/tarantool-tests/utils.lua
> > new file mode 100644
> > index 0000000..197b138
> > --- /dev/null
> > +++ b/test/tarantool-tests/utils.lua
> > @@ -0,0 +1,43 @@
> > +local M = { }
> > +
> > +local tap = require('tap')
> > +
> > +function M.selfrun(arg, checks)
> > +  local test = tap.test(arg[0]:match('/?(.+)%.test%.lua'))
> > +
> > +  test:plan(#checks)
> > +
> > +  local vars = {
> > +    LUABIN = arg[-1],
> > +    SCRIPT = arg[0],
> > +    PATH   = arg[0]:gsub('%.test%.lua', ''),
> > +    SUFFIX = package.cpath:match('?.(%a+);'),
> > +  }
> > +
> > +  local cmd = string.gsub('LUA_PATH="<PATH>/?.lua;$LUA_PATH" ' ..
> 
> Why do you use that way instead <CMakeLists.txt> configuration?

To make the tests using <selfrun> to be executed as a standalone script.

> 
> > +                          'LUA_CPATH="<PATH>/?.<SUFFIX>;$LUA_CPATH" ' ..
> > +                          'LD_LIBRARY_PATH=<PATH>:$LD_LIBRARY_PATH ' ..
> > +                          '<LUABIN> 2>&1 <SCRIPT>', '%<(%w+)>', vars)
> > +
> > +  for _, ch in pairs(checks) do
> > +    local testf = test[ch.test]
> > +    assert(testf, ("tap doesn't provide test.%s function"):format(ch.test))
> > +    local proc = io.popen((cmd .. (' %s'):rep(#ch.arg)):format(unpack(ch.arg)))
> > +    local res = proc:read('*all'):gsub('^%s+', ''):gsub('%s+$', '')
> 
> Are these spaces important?

Yes they are, if one uses exact match ('is') assertion.

> 
> > +    -- XXX: explicitly pass <test> as an argument to <testf>
> > +    -- to emulate test:is(...), test:like(...), etc.
> > +    testf(test, res, ch.res, ch.msg)
> > +  end
> > +
> > +  os.exit(test:check() and 0 or 1)
> > +end

<snipped>

> > -- 
> > 2.25.0
> > 
> 
> [1]: https://luaunit.readthedocs.io/en/luaunit_v3_2_1/
> [2]: https://fperrad.frama.io/lua-TestMore/
> [3]: https://metacpan.org/pod/Test::Deep
> [4]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/tap/#taptest-is
> [5]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/tap/
> [6]: https://perldoc.perl.org/Test::More#done_testing
> 
> -- 
> Best regards,
> Sergey Kaplun
> 

-- 
Best regards,
IM

  reply	other threads:[~2021-02-19 19:04 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-02 20:57 [Tarantool-patches] [PATCH luajit 0/5] Self-sufficient LuaJIT testing environment Igor Munkin via Tarantool-patches
2021-02-02 20:57 ` [Tarantool-patches] [PATCH luajit 1/5] build: preserve the original build system Igor Munkin via Tarantool-patches
2021-02-04 22:53   ` Timur Safin via Tarantool-patches
2021-02-08 15:56     ` Igor Munkin via Tarantool-patches
2021-02-09 11:38   ` Sergey Kaplun via Tarantool-patches
2021-02-09 12:47     ` Igor Munkin via Tarantool-patches
2021-02-09 14:45       ` Sergey Kaplun via Tarantool-patches
2021-02-09 15:28         ` Igor Munkin via Tarantool-patches
2021-02-10  9:35           ` Sergey Kaplun via Tarantool-patches
2021-02-02 20:57 ` [Tarantool-patches] [PATCH luajit 2/5] build: replace GNU Make with CMake Igor Munkin via Tarantool-patches
2021-02-04 22:53   ` Timur Safin via Tarantool-patches
2021-02-08 15:56     ` Igor Munkin via Tarantool-patches
2021-02-09 13:55       ` Timur Safin via Tarantool-patches
2021-02-09 15:09         ` Igor Munkin via Tarantool-patches
2021-02-11 19:23   ` Sergey Kaplun via Tarantool-patches
2021-02-16 15:28     ` Igor Munkin via Tarantool-patches
2021-02-18  9:56       ` Sergey Kaplun via Tarantool-patches
2021-02-20 19:18         ` Igor Munkin via Tarantool-patches
2021-02-27 10:48           ` Sergey Kaplun via Tarantool-patches
2021-02-28 18:18             ` Igor Munkin via Tarantool-patches
2021-02-13  3:47   ` Sergey Kaplun via Tarantool-patches
2021-02-16 15:32     ` Igor Munkin via Tarantool-patches
2021-02-02 20:57 ` [Tarantool-patches] [PATCH luajit 3/5] test: run LuaJIT tests via CMake Igor Munkin via Tarantool-patches
2021-02-08 15:05   ` Timur Safin via Tarantool-patches
2021-02-08 16:29     ` Igor Munkin via Tarantool-patches
2021-02-09  8:16       ` Timur Safin via Tarantool-patches
2021-02-09  8:43         ` Igor Munkin via Tarantool-patches
2021-02-09 13:59           ` Timur Safin via Tarantool-patches
2021-02-09 15:10             ` Igor Munkin via Tarantool-patches
2021-02-14 18:48   ` Sergey Kaplun via Tarantool-patches
2021-02-19 19:04     ` Igor Munkin via Tarantool-patches [this message]
2021-02-27 13:50       ` Sergey Kaplun via Tarantool-patches
2021-02-28 18:18         ` Igor Munkin via Tarantool-patches
2021-02-02 20:57 ` [Tarantool-patches] [PATCH luajit 4/5] test: fix warnings found with luacheck in misclib* Igor Munkin via Tarantool-patches
     [not found]   ` <012f01d6fe1a$a2aa6890$e7ff39b0$@tarantool.org>
2021-02-08 15:57     ` Igor Munkin via Tarantool-patches
     [not found]     ` <2c495492-50f4-acfd-ad66-2cb44abb5fa1@tarantool.org>
2021-02-08 15:40       ` Sergey Bronnikov via Tarantool-patches
2021-02-08 15:58       ` Igor Munkin via Tarantool-patches
2021-02-14 19:16   ` Sergey Kaplun via Tarantool-patches
2021-02-16 15:29     ` Igor Munkin via Tarantool-patches
2021-02-16 16:36       ` Sergey Kaplun via Tarantool-patches
2021-02-02 20:57 ` [Tarantool-patches] [PATCH luajit 5/5] test: run luacheck static analysis via CMake Igor Munkin via Tarantool-patches
2021-02-04 22:52   ` Timur Safin via Tarantool-patches
2021-02-08 15:57     ` Igor Munkin via Tarantool-patches
2021-02-14 19:32   ` Sergey Kaplun via Tarantool-patches
2021-02-19 19:14     ` Igor Munkin via Tarantool-patches
2021-02-28 22:04 ` [Tarantool-patches] [PATCH luajit 0/5] Self-sufficient LuaJIT testing environment Igor Munkin via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210219190430.GO5448@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=imun@tarantool.org \
    --cc=skaplun@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH luajit 3/5] test: run LuaJIT tests via CMake' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox