From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 7D2486ECE3; Tue, 19 Oct 2021 13:49:13 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7D2486ECE3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1634640553; bh=iq/XeXRgxncuBMHQ9ijRh2EqiTd9wp/LQi/7GoVKJ6g=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=K/Tf1Kd0zNyGLePvB8QiQRjMoOtwS9FFFeq5dRfCshAKylG5WITph7EAxr59dG00q EC5gUJcqPpCkeeqXdqzeMrWcJUysku/IWWLZIzZknTl3yYp3P2BXri9eQq9yIEZ2j8 cheHl8aNfXkBcIRiTL/j76+LYHJ+RrDVRMsRDbmM= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 5DB176ECE3 for ; Tue, 19 Oct 2021 13:49:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 5DB176ECE3 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1mcmgB-0006ed-I9; Tue, 19 Oct 2021 13:49:11 +0300 Date: Tue, 19 Oct 2021 13:49:10 +0300 To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org Message-ID: <20211019104910.GA160099@tarantool.org> References: <7447231b-f112-5228-ebaa-40a93f62aefa@tarantool.org> <20211005093250.GC55311@tarantool.org> <797b7cc5-800a-69a2-0d4d-51727d0a5607@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <797b7cc5-800a-69a2-0d4d-51727d0a5607@tarantool.org> X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9C7814344C8C501C8A875D1C8D969D20CAE380EF370D08BF9182A05F538085040051AC815B7AA007A6BB289BD3CF616E9840D077551572B635BA0E4D61D364B28 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE70C5E0F71D77D667BEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637B1AF1CB5CD5D34E98638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D824A14BA61193B1348FD38E238DEDF55F117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD182CC0D3CB04F14752D2E47CDBA5A96583BA9C0B312567BB231DD303D21008E29813377AFFFEAFD269A417C69337E82CC2E827F84554CEF50127C277FBC8AE2E8BA83251EDC214901ED5E8D9A59859A8B62CFFCC7B69C47339089D37D7C0E48F6C5571747095F342E88FB05168BE4CE3AF X-C1DE0DAB: 0D63561A33F958A580F9B1C4ECF8C7C2A95F223AD9594F71556F59B07AECBE4DD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7557E988E9157162368E8E86DC7131B365E7726E8460B7C23C X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34524B2D44961BF9C674AD32B2CC08E54CD8750859B53188694D191B01DCA99D3CA6EEE727CB6387941D7E09C32AA3244C7816F3BF48F8B2918F22E9936282ADDD5595C85A795C7BAE729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojSRIpe8siFhVEHXRqOuQQzw== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5D4863F17F0738C257B7852C0A026AEA1483D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v4 06/16] sql: introduce mem_append() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: Mergen Imeev Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thank you for the review! My answer, diff and new patch below. On Mon, Oct 11, 2021 at 11:50:08PM +0200, Vladislav Shpilevoy wrote: > Hi! Thanks for the fixes! > > >>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c > >>> index 079083fa1..b8ead8f41 100644 > >>> --- a/src/box/sql/mem.c > >>> +++ b/src/box/sql/mem.c > >>> @@ -1925,6 +1925,27 @@ mem_move(struct Mem *to, struct Mem *from) > >>> from->zMalloc = NULL; > >>> } > >>> > >>> +int > >>> +mem_append(struct Mem *mem, const char *value, uint32_t len) > >>> +{ > >>> + assert((mem->type & (MEM_TYPE_BIN | MEM_TYPE_STR)) != 0); > >>> + if (len == 0) > >>> + return 0; > >>> + int new_size = mem->n + len; > >>> + if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || > >>> + mem->szMalloc < new_size) { > >>> + /* > >>> + * Force exponential buffer size growth to avoid having to call > >>> + * this routine too often. > >>> + */ > >>> + if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) > >> > >> Looks like you could call sqlVdbeMemGrow() without all these checks > >> above. The grow function does them already. > >> > > True, fixed. There is a case where the behavior will be different - I > > mean the case where new_size < szMalloc and new_size + n > szMalloc, > > but that doesn't seem to be a very common case. > > Wait, this does not look correct either. I thought sqlVdbeMemGrow() > does the exponential grow. But if it does not, we need to either patch > it, or keep some checks above. The 'standard' way is to grow only when > necessary, but at least x2. How this new way works I can't tell. It > might be worse. You will always have unused free space. Lets either > stick to the old way or patch the grow function. I think it's better to use the old way, because in all other cases we only need to reallocate the memory once, and it would be in vain to double it every time. Diff: diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index 343a730b3..de4c41ccf 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -1934,12 +1934,15 @@ mem_append(struct Mem *mem, const char *value, uint32_t len) if (len == 0) return 0; int new_size = mem->n + len; - /* - * Force exponential buffer size growth to avoid having to call this - * routine too often. - */ - if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) - return -1; + if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || + mem->szMalloc < new_size) { + /* + * Force exponential buffer size growth to avoid having to call + * this routine too often. + */ + if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) + return -1; + } memcpy(&mem->z[mem->n], value, len); mem->n = new_size; return 0; New patch: commit 7cb6f95fcf4a85b36e8c4a3f41cbcc5e8741e417 Author: Mergen Imeev Date: Mon Sep 6 13:49:47 2021 +0300 sql: introduce mem_append() This patch introduces the mem_append() function. This function appends the specified string to the end of the STRING or VARBINARY contained in MEM. In case MEM needs to increase the size of allocated memory, extra memory is allocated in an attempt to reduce the total number of allocations. Needed for #4145 diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index f1b61dec6..de4c41ccf 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -1927,6 +1927,27 @@ mem_move(struct Mem *to, struct Mem *from) from->zMalloc = NULL; } +int +mem_append(struct Mem *mem, const char *value, uint32_t len) +{ + assert((mem->type & (MEM_TYPE_BIN | MEM_TYPE_STR)) != 0); + if (len == 0) + return 0; + int new_size = mem->n + len; + if (((mem->flags & (MEM_Static | MEM_Dyn | MEM_Ephem)) != 0) || + mem->szMalloc < new_size) { + /* + * Force exponential buffer size growth to avoid having to call + * this routine too often. + */ + if (sqlVdbeMemGrow(mem, new_size + mem->n, 1) != 0) + return -1; + } + memcpy(&mem->z[mem->n], value, len); + mem->n = new_size; + return 0; +} + int mem_concat(const struct Mem *a, const struct Mem *b, struct Mem *result) { diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index 1574da22d..b567aea50 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -582,6 +582,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from); void mem_move(struct Mem *to, struct Mem *from); +/** + * Append the given string to the end of the STRING or VARBINARY contained in + * MEM. In case MEM needs to increase the size of allocated memory, additional + * memory is allocated in an attempt to reduce the total number of allocations. + */ +int +mem_append(struct Mem *mem, const char *value, uint32_t len); + /** * Concatenate strings or binaries from the first and the second MEMs and write * to the result MEM. In case the first MEM or the second MEM is NULL, the diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c index 4c2bd11ba..3015760e1 100644 --- a/src/box/sql/vdbeaux.c +++ b/src/box/sql/vdbeaux.c @@ -1294,14 +1294,9 @@ sqlVdbeList(Vdbe * p) if (mem_copy_bin(pSub, bin, size) != 0) return -1; } else if (j == nSub) { - struct Mem tmp; - mem_create(&tmp); - uint32_t size = sizeof(SubProgram *); char *bin = (char *)&pOp->p4.pProgram; - mem_set_bin_ephemeral(&tmp, bin, size); - int rc = mem_concat(pSub, &tmp, pSub); - mem_destroy(&tmp); - if (rc != 0) + uint32_t size = sizeof(SubProgram *); + if (mem_append(pSub, bin, size) != 0) return -1; } }