Re: [Tarantool-patches] [PATCH 1/1] json: use cord_ibuf for encoding and decoding

[Oleg Babin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

Hi! Thanks for your patch.


I see strange effect. After a patch following script:

```

for i = 1, 1e9 do pcall(json.encode, function() end) end

```

produces quite strange effects with memory. After some time

my system kills a process - also I see in htop that process consumes 
about 20% of memory.

In contrast before the patch process uses 0.1% of memory and doesn't 
have any oscillations

in "VIRT" and "RES" columns. Yes, it's a negative case but I believe 
such behaviour shouldn't be affected as well.


Valid json encoding looks to be worked ok.

One nit below.


On 23.05.2021 17:06, Vladislav Shpilevoy wrote:
> Lua json module used to have a global buffer for all encodings. It
> was reused by each next encode().
>
> This was not correct, because during encode() might happen a GC
> step, which might call encode() again and spoil the global buffer.
>
> The same problem was already fixed for the global static buffer in
> scope of #5632. Similarly to that time, the patch makes Lua json
> module use cord_ibuf to prevent "concurrent" usage of the buffer
> data. The global buffer is deleted.
>
> According to a few microbenchmarks it didn't affect the perf
> anyhow.
>
> Core part of the patch is strbuf changes. Firstly, its destruction
> is now optional, cord_ibuf can free itself on a next yield.
> Secondly, its reallocation algorithm is kept intact - ibuf is used
> as an allocator, not as the buffer itself. This is done so as not
> to be too intrusive in the third party module which might need an
> upgrade to the upstream in the future.
>
> Closes #6050
> ---
> diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c
> index 38e999870..85186d6d5 100644
> --- a/third_party/lua-cjson/lua_cjson.c
> +++ b/third_party/lua-cjson/lua_cjson.c
> @@ -51,8 +51,7 @@
>   #include "mp_extension_types.h" /* MP_DECIMAL, MP_UUID */
>   #include "tt_static.h"
>   #include "uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */
> -
> -#define DEFAULT_ENCODE_KEEP_BUFFER 1
> +#include "cord_buf.h"
>   
>   typedef enum {
>       T_OBJ_BEGIN,
> @@ -100,10 +99,6 @@ static struct luaL_serializer *luaL_json_default;
>   static json_token_type_t ch2token[256];
>   static char escape2char[256];  /* Decoding */
>   
> -/* encode_buf is only allocated and used when
> - * encode_keep_buffer is set */
> -static strbuf_t encode_buf;
> -
>   typedef struct {
>       const char *data;
>       const char *ptr;
> @@ -182,9 +177,6 @@ static int json_destroy_config(lua_State *l)
>   static void json_create_tokens()
>   {
>       int i;
> -#if DEFAULT_ENCODE_KEEP_BUFFER > 0
> -    strbuf_init(&encode_buf, 0);
> -#endif
>   
>       /* Decoding init */
>   
> @@ -444,7 +436,9 @@ static int json_encode(lua_State *l) {
>                     "expected 1 or 2 arguments");
>   
>       /* Reuse existing buffer. */
> -    strbuf_reset(&encode_buf);
> +    strbuf_t encode_buf;
> +    struct ibuf *ibuf = cord_ibuf_take();
> +    strbuf_create(&encode_buf, -1, ibuf);

Maybe it's better to use "0" here. I know it has the same effect but 
usually 0 is default value. But up to you.