From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 8BD257030C; Mon, 24 May 2021 16:01:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 8BD257030C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1621861288; bh=U/xewVOj2cknCRnW1BDMn5HL0cGrPohhy0QDZjVwLC4=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=zWpqxXesiq3vtvkvGohkcD4qThCrRSuJD6vyXXDTU+uD4ZiFHl6I5/udjJl4dzyQQ BE3dZrBeH/BwbsXjv+Js8jzqNHU5xs3SD6AC/wC7B/+LRFvXaDrM5SnAqZU4s/WVuu UAc0LA6KKGfK/9iHuZfFVWFuZwrQQ86wOSV2tCDU= Received: from smtp51.i.mail.ru (smtp51.i.mail.ru [94.100.177.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 796E17030C for ; Mon, 24 May 2021 16:01:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 796E17030C Received: by smtp51.i.mail.ru with esmtpa (envelope-from ) id 1llAD0-0004xL-Gv; Mon, 24 May 2021 16:01:27 +0300 To: Vladislav Shpilevoy , tarantool-patches@dev.tarantool.org, olegrok@tarantool.org References: <2fb4c066558879eea74acab2e20b8a1c8f85d86b.1621778740.git.v.shpilevoy@tarantool.org> Message-ID: Date: Mon, 24 May 2021 16:01:25 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <2fb4c066558879eea74acab2e20b8a1c8f85d86b.1621778740.git.v.shpilevoy@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD91B019B01C53E51AF6F63A46F26EE68FD6CF19A7203B0589400894C459B0CD1B9E3EC9C333EE594DB4C6A05EE0EBAE6E82975760B6698B86B8F3BE6377274225A X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B9FBA884A7C9B8BAEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637629A2DB6D7098C1B8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D85179B9E4232AE4301183E9C5CE8B687F117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8BAA867293B0326636D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B6821DB69C87B6DA73089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C97831DB398122FCA0F1EB1BBFD1EA3789D373 X-C1DE0DAB: 0D63561A33F958A5B537295CCBCC1022FC606411A31D12DC8054BFF02AE37FCFD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA753C350047980234DB410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D341ADA1A41A420E9B2F73DA6E2A8A7E4A8C7BDF23854F1714AD3B7C4E0CF8E3D0342DAE68E52BA8C031D7E09C32AA3244C2A5C3DC6068303984EC12447BCD529D260759606DA2E136AFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj+gjVyQcIK6JrGgVrMukGNQ== X-Mailru-Sender: 3B9A0136629DC9125D61937A2360A44680CEA14B50E9F53E58036ED10D07BC6EF5C4444D4FCF61CD424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 1/1] json: use cord_ibuf for encoding and decoding X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 23.05.2021 17:06, Vladislav Shpilevoy пишет: > Lua json module used to have a global buffer for all encodings. It > was reused by each next encode(). > > This was not correct, because during encode() might happen a GC > step, which might call encode() again and spoil the global buffer. > > The same problem was already fixed for the global static buffer in > scope of #5632. Similarly to that time, the patch makes Lua json > module use cord_ibuf to prevent "concurrent" usage of the buffer > data. The global buffer is deleted. > > According to a few microbenchmarks it didn't affect the perf > anyhow. > > Core part of the patch is strbuf changes. Firstly, its destruction > is now optional, cord_ibuf can free itself on a next yield. > Secondly, its reallocation algorithm is kept intact - ibuf is used > as an allocator, not as the buffer itself. This is done so as not > to be too intrusive in the third party module which might need an > upgrade to the upstream in the future. > > Closes #6050 > --- > Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-6050-lua-json-static-buf-gc > Issue: https://github.com/tarantool/tarantool/issues/6050 Hi! Thanks for the patch! LGTM with one nit below. ... > diff --git a/third_party/lua-cjson/strbuf.c b/third_party/lua-cjson/strbuf.c > index f0f7f4b9a..22c1f2093 100644 > --- a/third_party/lua-cjson/strbuf.c > +++ b/third_party/lua-cjson/strbuf.c > @@ -28,6 +28,7 @@ > #include > > #include "strbuf.h" > +#include "small/ibuf.h" > > static void die(const char *fmt, ...) > { > @@ -41,7 +42,7 @@ static void die(const char *fmt, ...) > exit(-1); > } > > -void strbuf_init(strbuf_t *s, int len) > +void strbuf_create(strbuf_t *s, int len, struct ibuf *ibuf) > { > int size; > > @@ -50,37 +51,19 @@ void strbuf_init(strbuf_t *s, int len) > else > size = len + 1; /* \0 terminator */ > > - s->buf = NULL; > + s->buf = ibuf_reserve(ibuf, size); > s->size = size; > s->length = 0; > s->increment = STRBUF_DEFAULT_INCREMENT; > - s->dynamic = 0; > s->reallocs = 0; > s->debug = 0; > - > - s->buf = malloc(size); > + s->ibuf = ibuf; > if (!s->buf) > die("Out of memory"); > > strbuf_ensure_null(s); > } > > -strbuf_t *strbuf_new(int len) > -{ > - strbuf_t *s; > - > - s = malloc(sizeof(strbuf_t)); > - if (!s) > - die("Out of memory"); > - > - strbuf_init(s, len); > - > - /* Dynamic strbuf allocation / deallocation */ > - s->dynamic = 1; > - > - return s; > -} > - > void strbuf_set_increment(strbuf_t *s, int increment) > { > /* Increment > 0: Linear buffer growth rate > @@ -99,36 +82,9 @@ static inline void debug_stats(strbuf_t *s) > } > } > > -/* If strbuf_t has not been dynamically allocated, strbuf_free() can > - * be called any number of times strbuf_init() */ > -void strbuf_free(strbuf_t *s) > -{ > - debug_stats(s); > - > - if (s->buf) { > - free(s->buf); > - s->buf = NULL; > - } > - if (s->dynamic) > - free(s); > -} > - > -char *strbuf_free_to_string(strbuf_t *s, int *len) > +void strbuf_destroy(strbuf_t *s) > { > - char *buf; > - > debug_stats(s); > - > - strbuf_ensure_null(s); > - > - buf = s->buf; > - if (len) > - *len = s->length; > - > - if (s->dynamic) > - free(s); > - > - return buf; > } > > static int calculate_new_size(strbuf_t *s, int len) > @@ -171,9 +127,21 @@ void strbuf_resize(strbuf_t *s, int len) > fprintf(stderr, "strbuf(%lx) resize: %d => %d\n", > (long)s, s->size, newsize); > } > - > s->size = newsize;  ^ Extraneous change > - s->buf = realloc(s->buf, s->size); > + > + struct ibuf *ibuf = s->ibuf; > + /* > + * Propagate the write position to enable memcpy() when realloc happens > + * inside of the ibuf. > + */ > + ibuf->wpos += s->length; > + s->buf = ibuf_reserve(ibuf, newsize - s->length) - s->length; > + /* > + * But then it is reverted because memcpy() of the old data is done, and the > + * write position + capacity are managed by strbuf itself. > + */ > + ibuf->wpos = s->buf; > + > if (!s->buf) > die("Out of memory"); > s->reallocs++; ... -- Serge Petrenko