[Tarantool-patches] [PATCH v4 luajit 0/2] Enable running tests with Valgrind, add CI Valgrind testing workflow

[mandesero at gmail.com]

From: Maksim Tyushev <mandesero at gmail.com>

Changes in v4:
- Replaced the `VALGRIND_OPTIONS` variable with `VALGRIND_OPTS`, which
  is provided by Valgrind.
- Enabled running `tarantool-c-tests` under Valgrind:
  * Disabled the `tarantool-c-tests/gh-8594-sysprof-ffunc-crash.c_test` due
    to timeout.
- Renamed the variable for skipping tests to `LUAJIT_TEST_USE_VALGRIND`
  and setted it via `LUA_TEST_ENV_MORE`.
- Added the `--error-exitcode=1` flag to `VALGRIND_OPTS` in CI for stricter
  error handling.

Notes:

> From: Sergey Kaplun
> Also, I've found that tests from LuaJIT-tests suite reports tons of the
> assertions, but still the suite passes. This should be fixed:
>
> I suppose that we need to set `--error-exitcode` option as well.

<snipped>

> Same for PUC-Rio-Lua-5.1-tests, lua-Harness-test.
> ...

I noticed that Valgrind generates a significant number of reports, but it
seems these are merely warnings or informational messages. At the end, the
errors appear to be suppressed, as indicated in the output.

| 1/2 Test #1: LuaJIT-tests-deps ................   Passed    0.50 sec
| test 2
|   Start 2: test/LuaJIT-tests
|
| 2: Test command: /usr/bin/valgrind "--suppressions=/home/mandesero/myforks/luajit/src/lj.supp" "/home/mandesero/myforks/luajit/build/src/luajit" "-e" "dofile[[/home/mandesero/myforks/luajit/test/luajit-test-init.lua]]" "/home/mandesero/myforks/luajit/test/LuaJIT-tests/test.lua" "+slow" "+ffi" "+bit" "+jit"
|
| 2: --6813-- used_suppression:  62656 Optimized string compare /home/mandesero/myforks/luajit/src/lj.supp:38
| 2: ==6813==
| 2: ==6813== ERROR SUMMARY: 4707 errors from 27 contexts (suppressed: 62656 from 111)

Same for others tests.

> Also, what do you think about enabling option `--track-fds` for the CI
> (maybe in the separate patch set).

Enabling the `--track-fds` option for the CI sounds like a good idea.
However, I think it would be better to do this in a separate patch set.

Branch: https://github.com/tarantool/luajit/tree/mandesero/lj-3705-turn-off-strcmp-opt-in-debug

--
Best regards,
Maksim Tiushev


Maksim Tiushev (2):
  cmake: run tests with Valgrind
  ci: add Valgrind testing workflow

 .github/actions/setup-valgrind/README.md      | 12 +++
 .github/actions/setup-valgrind/action.yml     | 12 +++
 .github/workflows/valgrind-testing.yaml       | 99 +++++++++++++++++++
 CMakeLists.txt                                |  5 +
 test/CMakeLists.txt                           | 16 +++
 test/tarantool-c-tests/CMakeLists.txt         |  9 +-
 .../gh-8594-sysprof-ffunc-crash.test.c        |  7 ++
 .../gh-7745-oom-on-trace.test.lua             |  1 +
 .../lj-1034-tabov-error-frame.test.lua        |  1 +
 .../lj-512-profiler-hook-finalizers.test.lua  |  5 +-
 .../lj-726-profile-flush-close.test.lua       |  5 +-
 .../profilers/gh-5688-tool-cli-flag.test.lua  |  2 +
 ...4-add-proto-trace-sysprof-default.test.lua |  2 +
 .../profilers/misclib-sysprof-lapi.test.lua   |  2 +
 14 files changed, 175 insertions(+), 3 deletions(-)
 create mode 100644 .github/actions/setup-valgrind/README.md
 create mode 100644 .github/actions/setup-valgrind/action.yml
 create mode 100644 .github/workflows/valgrind-testing.yaml

-- 
2.34.1