[Tarantool-patches] [PATCH] lua: lua_field_inspect_table without pushcfunction

[Sergey Kaplun]

On 22.05.20, Alexander Turenko wrote:
> On Mon, May 18, 2020 at 12:37:48PM +0300, Sergey Kaplun wrote:
> > Currently on encoding table we push cfunction (lua_field_try_serialize)
> > to lua stack with additional lightuserdata and table value and after
> > pcall that function to avoid a raise of error.
> 
> It is often used to catch 'not enough memory' error. Say, to avoid a
> leak if some resource should be freed before reporting an error. Or when
> you want to return an error instead of raising it.
> 
> Please, look whether it was added by intention and verify that
> everything still good after the patch.

Hi!

As far as I can tell from the code lua_field_try_serialize hasn't got any calls
of lua API that can raise error about OOM or so on, exept lua_tostring and
lua_pushvalue. But we have already used lua_pushvalue and lua_pushcfunction so
with this patch probability of raising error is smaller (because we need less
arguments to push).

And lua_tolstring calls only after checking type of lua
object so lj_gc_check branch is unreacheble.

504         if (!lua_isstring(L, -1)) {
505 +---  2 lines: diag_set(LuajitError, "invalid " LUAL_SERIALIZE " value");--
507         }
508         const char *type = lua_tostring(L, -1);

491 LUA_API const char *lua_tolstring(lua_State *L, int idx, size_t *len)
492 {
493   TValue *o = index2adr(L, idx);
494   GCstr *s;
495   if (LJ_LIKELY(tvisstr(o))) {
496     s = strV(o);
497   } else if (tvisnumber(o)) {
498     lj_gc_check(L);


If I understand correctly, we must avoid panic when called luaL_tofield. I was
able to come up with a similar example, how to show that the behavior has not
changed. But I am not 100% sure that this test covers all corner cases.
Do you have any idea of a more complete test?


| 13:00:04 jobs:0 burii at root:~/builds_workspace/tarantool/master$
| >>> ../no-ticket-lua-inspect-table-refactoring/src/tarantool 
| Tarantool 2.5.0-32-g7f20272ea
| type 'help' for interactive help
| tarantool> collectgarbage("stop")
|            local source = setmetatable({}, {__serialize = "map"})
|            print("start")
|            for i = 1, 340000 do
|                table.insert(source, (tostring(i)):rep(1024))
|            end
|            print("end")
|            local msgpack = require "msgpack"
|            return  pcall(msgpack.encode, source)
| start
| end
| ---
| - false
| - not enough memory
| ...
| 
| tarantool> 13:00:41 jobs:0 burii at root:~/builds_workspace/tarantool/master$
| >>> src/tarantool 
| Tarantool 2.5.0-1-gad13b6d57
| type 'help' for interactive help
| tarantool> collectgarbage("stop")
|            local source = {}
|            print("start")
|            for i = 1, 340000 do
|                table.insert(source, (tostring(i)):rep(1024))
|            end
|            print("end")
|            local msgpack = require "msgpack"
|            return  pcall(msgpack.encode, source)
| start
| end
| ---
| - false
| - not enough memory
| ...

-- 
Best regards,
Sergey Kaplun