[Tarantool-patches] [PATCH] serilaizer: check for recursive serialization

[Cyrill Gorcunov]

On Fri, Jul 10, 2020 at 03:01:09PM +0300, Roman Khabibov wrote:
>  luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
> @@ -490,6 +493,11 @@ static int
>  lua_field_try_serialize(struct lua_State *L, struct luaL_serializer *cfg,
>  			int idx, struct luaL_field *field)
>  {
> +	if (idx > SERIALIZER_CRITICAL_RECURSION_DEPTH) {
> +		diag_set(LuajitError, LUAL_SERIALIZE " generates too deep "
> +			 "recursion");
> +		return -1;
> +	}
>  	if (luaL_getmetafield(L, idx, LUAL_SERIALIZE) == 0)
>  		return 1;

Wait. The @idx stands for index in a table as far as I remember,
this just happen to hit when you're calling youself recursively
but @idx may be > SERIALIZER_CRITICAL_RECURSION_DEPTH for nonrecursive
calls as well.

Lets CC our Lua's master: Igor. I might be simply wrong.