From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 73DD47030C; Mon, 24 May 2021 16:05:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 73DD47030C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1621861544; bh=hpJpS20N7YoFZhqVarr95Xzr11rgx/NGqdS4Kl8vBZQ=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TPMA5klylpmbr9fyC7rNL+QSBgS1P9ELPwmRagfrB3e9ZKqFVVC9E9XbMaVADkKZj QDY6DJgLHvbUq4PvR6K6apHctXvpVGn3A1ihvdR9M44doXP8KP4hD9N2QlsNSkPX3X bnN8PMBtg1K7zqqVA1qNmsTD1wuf4H9nsxTQ+hNE= Received: from smtp38.i.mail.ru (smtp38.i.mail.ru [94.100.177.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 78A4A7030C for ; Mon, 24 May 2021 16:05:42 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 78A4A7030C Received: by smtp38.i.mail.ru with esmtpa (envelope-from ) id 1llAH7-0006NU-L2; Mon, 24 May 2021 16:05:42 +0300 To: Vladislav Shpilevoy , tarantool-patches@dev.tarantool.org, olegrok@tarantool.org References: <2fb4c066558879eea74acab2e20b8a1c8f85d86b.1621778740.git.v.shpilevoy@tarantool.org> Message-ID: <65ce8ac8-6af3-e822-9fb0-113b37f51b18@tarantool.org> Date: Mon, 24 May 2021 16:05:40 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: ru X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD91B019B01C53E51AFBEA66046F15EAF23C5E2F255C2D74AA700894C459B0CD1B99F262424BABD33DCBB39A5E955E41CDA2975760B6698B86B3EAE9E3EF2B3B1F1 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7C6068CE86C2B75F5EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006375D8840FA58F505298638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8B64DDD975B6AC1D576DA8E959C925277117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8BAA867293B0326636D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B6F7FD1A3A8AE6177F089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C9783163181B4C0F022A4177A8C5B82A5066A9 X-C1DE0DAB: 0D63561A33F958A5F9CAEEFD2174FF7097439336C92549B904220934AA799536D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA753C350047980234DB410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34F6CC1FD3366963360D6ED5D8E51EFF72D68B1217828931DBE45AEE25822EB7E8EAD3967AF1FDD10E1D7E09C32AA3244C55C0D760746B68DA04ED6D79F83592E897FE24653F78E668FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj+gjVyQcIK6LhU4/7EW3D/g== X-Mailru-Sender: 3B9A0136629DC9125D61937A2360A446DD998EEE617B1DB4AD4EFB424948970DDAD5C0D71094B1EE424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 1/1] json: use cord_ibuf for encoding and decoding X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 24.05.2021 16:01, Serge Petrenko via Tarantool-patches пишет: > > > 23.05.2021 17:06, Vladislav Shpilevoy пишет: >> Lua json module used to have a global buffer for all encodings. It >> was reused by each next encode(). >> >> This was not correct, because during encode() might happen a GC >> step, which might call encode() again and spoil the global buffer. >> >> The same problem was already fixed for the global static buffer in >> scope of #5632. Similarly to that time, the patch makes Lua json >> module use cord_ibuf to prevent "concurrent" usage of the buffer >> data. The global buffer is deleted. >> >> According to a few microbenchmarks it didn't affect the perf >> anyhow. >> >> Core part of the patch is strbuf changes. Firstly, its destruction >> is now optional, cord_ibuf can free itself on a next yield. >> Secondly, its reallocation algorithm is kept intact - ibuf is used >> as an allocator, not as the buffer itself. This is done so as not >> to be too intrusive in the third party module which might need an >> upgrade to the upstream in the future. >> >> Closes #6050 >> --- >> Branch: >> http://github.com/tarantool/tarantool/tree/gerold103/gh-6050-lua-json-static-buf-gc >> Issue: https://github.com/tarantool/tarantool/issues/6050 > > Hi! Thanks for the patch! > LGTM with one nit below. Sorry, I noticed this too late: luacheck has one warning on your branch: Checking test/app-tap/json.test.lua 1 warning 422 423 test/app-tap/json.test.lua:210:9: (W213) unused loop variable i 424 > > > ... > >> diff --git a/third_party/lua-cjson/strbuf.c >> b/third_party/lua-cjson/strbuf.c >> index f0f7f4b9a..22c1f2093 100644 >> --- a/third_party/lua-cjson/strbuf.c >> +++ b/third_party/lua-cjson/strbuf.c >> @@ -28,6 +28,7 @@ >>   #include >>     #include "strbuf.h" >> +#include "small/ibuf.h" >>     static void die(const char *fmt, ...) >>   { >> @@ -41,7 +42,7 @@ static void die(const char *fmt, ...) >>       exit(-1); >>   } >>   -void strbuf_init(strbuf_t *s, int len) >> +void strbuf_create(strbuf_t *s, int len, struct ibuf *ibuf) >>   { >>       int size; >>   @@ -50,37 +51,19 @@ void strbuf_init(strbuf_t *s, int len) >>       else >>           size = len + 1;         /* \0 terminator */ >>   -    s->buf = NULL; >> +    s->buf = ibuf_reserve(ibuf, size); >>       s->size = size; >>       s->length = 0; >>       s->increment = STRBUF_DEFAULT_INCREMENT; >> -    s->dynamic = 0; >>       s->reallocs = 0; >>       s->debug = 0; >> - >> -    s->buf = malloc(size); >> +    s->ibuf = ibuf; >>       if (!s->buf) >>           die("Out of memory"); >>         strbuf_ensure_null(s); >>   } >>   -strbuf_t *strbuf_new(int len) >> -{ >> -    strbuf_t *s; >> - >> -    s = malloc(sizeof(strbuf_t)); >> -    if (!s) >> -        die("Out of memory"); >> - >> -    strbuf_init(s, len); >> - >> -    /* Dynamic strbuf allocation / deallocation */ >> -    s->dynamic = 1; >> - >> -    return s; >> -} >> - >>   void strbuf_set_increment(strbuf_t *s, int increment) >>   { >>       /* Increment > 0:  Linear buffer growth rate >> @@ -99,36 +82,9 @@ static inline void debug_stats(strbuf_t *s) >>       } >>   } >>   -/* If strbuf_t has not been dynamically allocated, strbuf_free() can >> - * be called any number of times strbuf_init() */ >> -void strbuf_free(strbuf_t *s) >> -{ >> -    debug_stats(s); >> - >> -    if (s->buf) { >> -        free(s->buf); >> -        s->buf = NULL; >> -    } >> -    if (s->dynamic) >> -        free(s); >> -} >> - >> -char *strbuf_free_to_string(strbuf_t *s, int *len) >> +void strbuf_destroy(strbuf_t *s) >>   { >> -    char *buf; >> - >>       debug_stats(s); >> - >> -    strbuf_ensure_null(s); >> - >> -    buf = s->buf; >> -    if (len) >> -        *len = s->length; >> - >> -    if (s->dynamic) >> -        free(s); >> - >> -    return buf; >>   } >>     static int calculate_new_size(strbuf_t *s, int len) >> @@ -171,9 +127,21 @@ void strbuf_resize(strbuf_t *s, int len) >>           fprintf(stderr, "strbuf(%lx) resize: %d => %d\n", >>                   (long)s, s->size, newsize); >>       } >> - >>       s->size = newsize; > >  ^ Extraneous change > >> -    s->buf = realloc(s->buf, s->size); >> + >> +    struct ibuf *ibuf = s->ibuf; >> +    /* >> +     * Propagate the write position to enable memcpy() when realloc >> happens >> +     * inside of the ibuf. >> +     */ >> +    ibuf->wpos += s->length; >> +    s->buf = ibuf_reserve(ibuf, newsize - s->length) - s->length; >> +    /* >> +     * But then it is reverted because memcpy() of the old data is >> done, and the >> +     * write position + capacity are managed by strbuf itself. >> +     */ >> +    ibuf->wpos = s->buf; >> + >>       if (!s->buf) >>           die("Out of memory"); >>       s->reallocs++; > > ... > -- Serge Petrenko