From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id D56846ECE3; Wed, 24 Nov 2021 10:24:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D56846ECE3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1637738696; bh=9prGnHOB+A5diI0sEDmREPA6pL4tBLsw/e2/QCdqGUQ=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Jt7TASggPoUrM10u1DHRX0yZ3httxTqs6XsuMI6aAxF6Q9Jxb+HXQvKHr2fY1MrOW CQfy0KMkUCr2dObBN/eMpsk3diSyn1pT19EJoZ5zV4B3bB22c3k2DLw7xQoPRxuQsg XeFoxEL5iJgn5TG17ASCyFNv9xU0yBYM78qcVfw0= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 5108A6ECE3 for ; Wed, 24 Nov 2021 10:24:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 5108A6ECE3 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1mpmeD-0006Et-Gi; Wed, 24 Nov 2021 10:24:53 +0300 Date: Wed, 24 Nov 2021 10:24:52 +0300 To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org Message-ID: <20211124072452.GA41102@tarantool.org> References: <5c49fd526a5fcf59c4d8b1d222016dc3c08f35c7.1637159909.git.imeevma@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-4EC0790: 10 X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD9FE0487E50246814697FE70F94637C0E3A7B9BB53A0E4B962182A05F538085040528F47F15A2B70C7B73CCBB684E5A2D6FFAD85E3D04ACC33B4D9806F0D95E51C X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7AEADB70076298E75EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063727599E9BE65898D58638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8F20DDB36DD7D7EF37A850C37E16024CA117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8B3A703B70628EAD7BA471835C12D1D977C4224003CC8364762BB6847A3DEAEFB02EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE7E1BCFB2C0BE3F189731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: 0D63561A33F958A5CCCB8F6DEA470763B53CD232D4D1C63EA123D441FC4245FBD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA759F66ED85EB5F25FD410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3453741B480A6503C3D0C360F72ABFD57B1F471255418570C174223727028ACE819BBBA4175E65299D1D7E09C32AA3244C99E5D1BE074A50F441C3F5998663035C39C99C45E8D137E9729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojFw1tprMwPO2nVkpBZ6CdwQ== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5DA1F7084E9C49F008C5D805EF0BBEA60B83D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v1 2/3] sql: refactor sql_vdbe_mem_encode_tuple() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: Mergen Imeev Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thank you for the review! My answer, diff and new patch below. Also, I fixed one comment about region_truncate() from the letters about MAP. On Sat, Nov 20, 2021 at 12:35:46AM +0100, Vladislav Shpilevoy wrote: > Hi! Thanks for the patch! > > > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c > > index 356b2c7be..2ba4135b0 100644 > > --- a/src/box/sql/mem.c > > +++ b/src/box/sql/mem.c > > @@ -3013,31 +3013,31 @@ mem_to_mpstream(const struct Mem *var, struct mpstream *stream) > > } > > > > char * > > -sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count, > > - uint32_t *tuple_size, struct region *region) > > +mem_encode_array(const struct Mem *mems, uint32_t count, uint32_t *size, > > + struct region *region) > > { > > size_t used = region_used(region); > > bool is_error = false; > > struct mpstream stream; > > mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb, > > set_encode_error, &is_error); > > - mpstream_encode_array(&stream, field_count); > > - for (struct Mem *field = fields; field < fields + field_count; field++) > > - mem_to_mpstream(field, &stream); > > + mpstream_encode_array(&stream, count); > > + for (const struct Mem *mem = mems; mem < mems + count; mem++) > > + mem_to_mpstream(mem, &stream); > > mpstream_flush(&stream); > > if (is_error) { > > If some memory was allocated/reserved before mpstream_flush() failed (inside > of mem_to_mpstream()), then the region might leak. I would recommend to add a > truncate here. > Thank you. Fixed. > > diag_set(OutOfMemory, stream.pos - stream.buf, > > "mpstream_flush", "stream"); > > return NULL; > > } Diff: diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index 71a10c9b3..32b8825bc 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -3054,6 +3054,7 @@ mem_encode_array(const struct Mem *mems, uint32_t count, uint32_t *size, mem_to_mpstream(mem, &stream); mpstream_flush(&stream); if (is_error) { + region_truncate(region, used); diag_set(OutOfMemory, stream.pos - stream.buf, "mpstream_flush", "stream"); return NULL; @@ -3061,6 +3062,7 @@ mem_encode_array(const struct Mem *mems, uint32_t count, uint32_t *size, *size = region_used(region) - used; char *array = region_join(region, *size); if (array == NULL) { + region_truncate(region, used); diag_set(OutOfMemory, *size, "region_join", "array"); return NULL; } New patch: commit 14ddc911967143075a373a24d318b36120144267 Author: Mergen Imeev Date: Tue Nov 16 09:54:09 2021 +0300 sql: refactor sql_vdbe_mem_encode_tuple() This function is used to create ARRAY value from array of MEMs. Since ARRAY was added to SQL, this function needs to be refactored. Part of #4762 diff --git a/src/box/sql.c b/src/box/sql.c index d15159d6e..2a78a96d5 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -211,7 +211,7 @@ sql_cursor_seek(struct BtCursor *cur, struct Mem *mems, uint32_t len, int *res) struct region *region = &fiber()->gc; size_t used = region_used(region); uint32_t size; - const char *tuple = sql_vdbe_mem_encode_tuple(mems, len, &size, region); + const char *tuple = mem_encode_array(mems, len, &size, region); if (tuple == NULL) return -1; if (key_alloc(cur, size) != 0) diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index 303c3a1a3..32b8825bc 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -3041,31 +3041,33 @@ mem_to_mpstream(const struct Mem *var, struct mpstream *stream) } char * -sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count, - uint32_t *tuple_size, struct region *region) +mem_encode_array(const struct Mem *mems, uint32_t count, uint32_t *size, + struct region *region) { size_t used = region_used(region); bool is_error = false; struct mpstream stream; mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb, set_encode_error, &is_error); - mpstream_encode_array(&stream, field_count); - for (struct Mem *field = fields; field < fields + field_count; field++) - mem_to_mpstream(field, &stream); + mpstream_encode_array(&stream, count); + for (const struct Mem *mem = mems; mem < mems + count; mem++) + mem_to_mpstream(mem, &stream); mpstream_flush(&stream); if (is_error) { + region_truncate(region, used); diag_set(OutOfMemory, stream.pos - stream.buf, "mpstream_flush", "stream"); return NULL; } - *tuple_size = region_used(region) - used; - char *tuple = region_join(region, *tuple_size); - if (tuple == NULL) { - diag_set(OutOfMemory, *tuple_size, "region_join", "tuple"); + *size = region_used(region) - used; + char *array = region_join(region, *size); + if (array == NULL) { + region_truncate(region, used); + diag_set(OutOfMemory, *size, "region_join", "array"); return NULL; } - mp_tuple_assert(tuple, tuple + *tuple_size); - return tuple; + mp_tuple_assert(array, array + *size); + return array; } /** diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index ea9ef709d..7f5ecf954 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -862,15 +862,15 @@ void mem_to_mpstream(const struct Mem *var, struct mpstream *stream); /** - * Perform encoding field_count Vdbe memory fields on region as - * msgpack array. - * @param fields The first Vdbe memory field to encode. - * @param field_count Count of fields to encode. - * @param[out] tuple_size Size of encoded tuple. + * Encode array of MEMs as msgpack array on region. + * + * @param mems array of MEMs to encode. + * @param count number of elements in the array. + * @param[out] size Size of encoded msgpack array. * @param region Region to use. * @retval NULL on error, diag message is set. - * @retval Pointer to valid tuple on success. + * @retval Pointer to valid msgpack array on success. */ char * -sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count, - uint32_t *tuple_size, struct region *region); +mem_encode_array(const struct Mem *mems, uint32_t count, uint32_t *size, + struct region *region); diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 3892cc102..2e6893f1a 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -2036,8 +2036,7 @@ case OP_MakeRecord: { struct region *region = &fiber()->gc; size_t used = region_used(region); uint32_t tuple_size; - char *tuple = - sql_vdbe_mem_encode_tuple(pData0, nField, &tuple_size, region); + char *tuple = mem_encode_array(pData0, nField, &tuple_size, region); if (tuple == NULL) goto abort_due_to_error; if ((int64_t)tuple_size > db->aLimit[SQL_LIMIT_LENGTH]) diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index 3894bb943..4ce5feeae 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -212,8 +212,8 @@ sql_stmt_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size, struct region *region) { struct Vdbe *vdbe = (struct Vdbe *)stmt; - return sql_vdbe_mem_encode_tuple(vdbe->pResultSet, vdbe->nResColumn, - tuple_size, region); + return mem_encode_array(vdbe->pResultSet, vdbe->nResColumn, tuple_size, + region); } /*