[Tarantool-patches] [PATCH v2 luajit 0/3] Trace abort on FFI sandwich or mode change

[Igor Munkin <imun@tarantool.org>]

This series closes two issues related to the JIT machinery behaviour:
* "FFI sandwich"(*) detection is introduced. If sandwich is detected
  while trace recording the recording is aborted. The sandwich detected
  while mcode execution leads to the platform panic.
* luaJIT_setmode call is prohibited while mcode execution and leads to
  the platform panic.

(*) The following stack mix is called FFI sandwich.
    | Lua-FFI -> C routine -> Lua-C API -> Lua VM
    This sort of re-entrancy is explicitly not supported by LuaJIT
    compiler. For more info see [1].

Furthermore, this series also provides an aixillary module for LuaJIT
testing machinery.

[1]: https://github.com/tarantool/tarantool/issues/4427

@ChangeLog:
* "FFI sandwich"(*) detection is introduced. If sandwich is detected
  while trace recording the recording is aborted. The sandwich detected
  while mcode execution leads to the platform panic.
* luaJIT_setmode call is prohibited while mcode execution and leads to
  the platform panic.

(*) The following stack mix is called FFI sandwich.
    | Lua-FFI -> C routine -> Lua-C API -> Lua VM
    This sort of re-entrancy is explicitly not supported by LuaJIT
    compiler. For more info see gh-4427.

Branch: https://github.com/tarantool/luajit/tree/imun/ffi-sandwich

Changes in v2:
* Adjusted naming and commit message considering Sergos' comments.
* Reworked tests considering Vlad's comments.
* Fixed VM re-entry via lua_cpcall (many thanks to Vlad's preciseness).

Igor Munkin (3):
  test: add auxillary module for testing
  jit: abort trace recording and execution for C API
  jit: abort trace execution on JIT mode change

 src/lj_api.c                             | 44 ++++++++++++++----
 src/lj_dispatch.c                        |  6 +++
 src/lj_errmsg.h                          |  2 +
 test/gh-4427-ffi-sandwich.skipcond       |  7 +++
 test/gh-4427-ffi-sandwich.test.lua       | 49 ++++++++++++++++++++
 test/gh-4427-ffi-sandwich/CMakeLists.txt |  1 +
 test/gh-4427-ffi-sandwich/libsandwich.c  | 59 ++++++++++++++++++++++++
 test/lj-flush-on-trace.skipcond          |  7 +++
 test/lj-flush-on-trace.test.lua          | 48 +++++++++++++++++++
 test/lj-flush-on-trace/CMakeLists.txt    |  1 +
 test/lj-flush-on-trace/libflush.c        | 31 +++++++++++++
 test/suite.ini                           |  1 +
 test/utils.lua                           | 33 +++++++++++++
 13 files changed, 280 insertions(+), 9 deletions(-)
 create mode 100644 test/gh-4427-ffi-sandwich.skipcond
 create mode 100755 test/gh-4427-ffi-sandwich.test.lua
 create mode 100644 test/gh-4427-ffi-sandwich/CMakeLists.txt
 create mode 100644 test/gh-4427-ffi-sandwich/libsandwich.c
 create mode 100644 test/lj-flush-on-trace.skipcond
 create mode 100755 test/lj-flush-on-trace.test.lua
 create mode 100644 test/lj-flush-on-trace/CMakeLists.txt
 create mode 100644 test/lj-flush-on-trace/libflush.c
 create mode 100644 test/utils.lua

-- 
2.25.0