[Tarantool-patches] [PATCH] serilaizer: check for recursive serialization

[Roman Khabibov]

Hi, Cyrill and Igor!

I tried to compare the addresses of the previous and the current iteration,
if they are equal, then throw "looks like recursion, bad function!”. But I
got swim tests failing. That is, they use some recursive serializers that do
not overflow the stack. Therefore, I settled on the idea of ​​introducing a 
recursion limit.

> On Jul 14, 2020, at 13:40, Cyrill Gorcunov <gorcunov at gmail.com> wrote:
> 
> On Tue, Jul 14, 2020 at 12:45:33PM +0300, Igor Munkin wrote:
>>> 
>>> Wait. The @idx stands for index in a table as far as I remember,
>>> this just happen to hit when you're calling youself recursively
>>> but @idx may be > SERIALIZER_CRITICAL_RECURSION_DEPTH for nonrecursive
>>> calls as well.
>> 
>> I guess you've just misread this part: @idx is just a guest stack slot
>> where Lua object being serialized is stored. If I get your concerns
>> right, you're talking about the following case:
> 
> Yes, you're right. Thanks, Igor! I don't have strong preference here
> on how to handle the case. Thus
> 
> Reviewed-by: Cyrill Gorcunov <gorcunov at gmail.com>