From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 9F40B6EC40; Sat, 25 Sep 2021 14:06:08 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 9F40B6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1632567968; bh=VaSaqu6wfZ5x+vbe2xPe/e0mjNU1phGIvfxrcnUPLbA=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=yArn+w9kteYjMdQn1dc0g8jhMY9SlQNxnww+5gnkBJXBEuC/HWj6j1ah39qTzaSgI 61staqgh+Ix2YKeP/E+LhnJWvibWVvth8nmCdKNapkLnQl/Y8XtsOwYEDCArpwLVUF BSuEdjiaskNUeoqd+YtcEb+5ds6+xohuhgjElwlQ= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 0AD326EC40 for ; Sat, 25 Sep 2021 14:06:07 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0AD326EC40 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1mU5VO-000320-Bg; Sat, 25 Sep 2021 14:06:06 +0300 Date: Sat, 25 Sep 2021 14:06:05 +0300 To: v.shpilevoy@tarantool.org, tarantool-patches@dev.tarantool.org Message-ID: <20210925110605.GB290467@tarantool.org> References: <660d25eecfe9f32dc610f89da493622bba84f13e.1632220375.git.imeevma@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <660d25eecfe9f32dc610f89da493622bba84f13e.1632220375.git.imeevma@gmail.com> X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD96A58C36AA2E99649E150573C3DCDB299A56307BC7D0E2EC5182A05F538085040C3C44CDF48EED483E593DA4B156A560569725765A7D3CCB774818CD8981D6C12 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7AC4684DF4EC4B256EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AF78C68CB7D402738638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8923DD07EC8B882716F4A8374ED5F574A117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8B974A882099E279BDA471835C12D1D977C4224003CC8364762BB6847A3DEAEFB0F43C7A68FF6260569E8FC8737B5C2249EC8D19AE6D49635B68655334FD4449CB9ECD01F8117BC8BEAAAE862A0553A39223F8577A6DFFEA7CD1D040B6C1ECEA3F43847C11F186F3C59DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A5E6C2E41506DAA2CA7B186F86733FA8CA65019223CC37BBE5D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75BFC02AB3DF06BA5A410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340297C696F996E3840A3163C8D3F9446D58FC2539F8C983E5F7A565976299A3EB3462D8FD84A7B6131D7E09C32AA3244CF9531E203581DA1CF6C4B1B6FCFECD7D7C0C08F7987826B9729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojqHUEAOTxd4j70o1uHdv38A== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5DDCA45B9881DEB6E52F818543C7B38B3283D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v2 05/15] sql: introduce mem_append() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: Mergen Imeev Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thank you for the review! I added check for length of appended string. If it is 0, than we do not have to do anything. The result can be static or ephemeral now, but not sure that this should be an issue. On the other hand, we do not have to allocate memory for static or ephemeral MEMs, when zero-length string is appended to them. Diff and new patch below. On Tue, Sep 21, 2021 at 01:59:10PM +0300, Mergen Imeev via Tarantool-patches wrote: > This patch introduces function mem_append(). This function appends the > given string to the end of the STRING or VARBINARY contained in MEM. > In case MEM needs to increase the size of allocated memory, additional > memory is allocated in an attempt to reduce the total number of > allocations. The main use of this function is to add multiple strings to > a single MEM. > > Needed for #4145 > --- > src/box/sql/mem.c | 19 +++++++++++++++++++ > src/box/sql/mem.h | 8 ++++++++ > src/box/sql/vdbeaux.c | 9 ++------- > 3 files changed, 29 insertions(+), 7 deletions(-) > > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c > index cc0fd836b..a9011fc63 100644 > --- a/src/box/sql/mem.c > +++ b/src/box/sql/mem.c > @@ -1953,6 +1953,25 @@ mem_move(struct Mem *to, struct Mem *from) > from->zMalloc = NULL; > } > > +int > +mem_append(struct Mem *mem, const char *value, uint32_t len) > +{ > + assert((mem->type & (MEM_TYPE_BIN | MEM_TYPE_STR)) != 0); > + int new_size = mem->n + len; > + if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || > + mem->szMalloc < new_size) { > + /* > + * Force exponential buffer size growth to avoid having to call > + * this routine too often. > + */ > + if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) > + return -1; > + } > + memcpy(&mem->z[mem->n], value, len); > + mem->n = new_size; > + return 0; > +} > + > int > mem_concat(struct Mem *a, struct Mem *b, struct Mem *result) > { > diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h > index 0da45b8af..54a1e931b 100644 > --- a/src/box/sql/mem.h > +++ b/src/box/sql/mem.h > @@ -597,6 +597,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from); > void > mem_move(struct Mem *to, struct Mem *from); > > +/** > + * Append the given string to the end of the STRING or VARBINARY contained in > + * MEM. In case MEM needs to increase the size of allocated memory, additional > + * memory is allocated in an attempt to reduce the total number of allocations. > + */ > +int > +mem_append(struct Mem *mem, const char *value, uint32_t len); > + > /** > * Concatenate strings or binaries from the first and the second MEMs and write > * to the result MEM. In case the first MEM or the second MEM is NULL, the > diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c > index b7e44a386..6f13d2b03 100644 > --- a/src/box/sql/vdbeaux.c > +++ b/src/box/sql/vdbeaux.c > @@ -1300,14 +1300,9 @@ sqlVdbeList(Vdbe * p) > if (mem_copy_bin(pSub, bin, size) != 0) > return -1; > } else if (j == nSub) { > - struct Mem tmp; > - mem_create(&tmp); > - uint32_t size = sizeof(SubProgram *); > char *bin = (char *)&pOp->p4.pProgram; > - mem_set_bin_ephemeral(&tmp, bin, size); > - int rc = mem_concat(pSub, &tmp, pSub); > - mem_destroy(&tmp); > - if (rc != 0) > + uint32_t size = sizeof(SubProgram *); > + if (mem_append(pSub, bin, size) != 0) > return -1; > } > } > -- > 2.25.1 > Diff: diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index a9011fc63..f64cbe10a 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -1957,6 +1957,8 @@ int mem_append(struct Mem *mem, const char *value, uint32_t len) { assert((mem->type & (MEM_TYPE_BIN | MEM_TYPE_STR)) != 0); + if (len == 0) + return 0; int new_size = mem->n + len; if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || mem->szMalloc < new_size) { New patch: commit 5025bd6782f2199a120594371e042f99c1ad6c10 Author: Mergen Imeev Date: Mon Sep 6 13:49:47 2021 +0300 sql: introduce mem_append() This patch introduces function mem_append(). This function appends the given string to the end of the STRING or VARBINARY contained in MEM. In case MEM needs to increase the size of allocated memory, additional memory is allocated in an attempt to reduce the total number of allocations. The main use of this function is to add multiple strings to a single MEM. Needed for #4145 diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index cc0fd836b..f64cbe10a 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -1953,6 +1953,27 @@ mem_move(struct Mem *to, struct Mem *from) from->zMalloc = NULL; } +int +mem_append(struct Mem *mem, const char *value, uint32_t len) +{ + assert((mem->type & (MEM_TYPE_BIN | MEM_TYPE_STR)) != 0); + if (len == 0) + return 0; + int new_size = mem->n + len; + if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || + mem->szMalloc < new_size) { + /* + * Force exponential buffer size growth to avoid having to call + * this routine too often. + */ + if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) + return -1; + } + memcpy(&mem->z[mem->n], value, len); + mem->n = new_size; + return 0; +} + int mem_concat(struct Mem *a, struct Mem *b, struct Mem *result) { diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index 0da45b8af..54a1e931b 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -597,6 +597,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from); void mem_move(struct Mem *to, struct Mem *from); +/** + * Append the given string to the end of the STRING or VARBINARY contained in + * MEM. In case MEM needs to increase the size of allocated memory, additional + * memory is allocated in an attempt to reduce the total number of allocations. + */ +int +mem_append(struct Mem *mem, const char *value, uint32_t len); + /** * Concatenate strings or binaries from the first and the second MEMs and write * to the result MEM. In case the first MEM or the second MEM is NULL, the diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c index 4c2bd11ba..3015760e1 100644 --- a/src/box/sql/vdbeaux.c +++ b/src/box/sql/vdbeaux.c @@ -1294,14 +1294,9 @@ sqlVdbeList(Vdbe * p) if (mem_copy_bin(pSub, bin, size) != 0) return -1; } else if (j == nSub) { - struct Mem tmp; - mem_create(&tmp); - uint32_t size = sizeof(SubProgram *); char *bin = (char *)&pOp->p4.pProgram; - mem_set_bin_ephemeral(&tmp, bin, size); - int rc = mem_concat(pSub, &tmp, pSub); - mem_destroy(&tmp); - if (rc != 0) + uint32_t size = sizeof(SubProgram *); + if (mem_append(pSub, bin, size) != 0) return -1; } }