From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 042A66EC5F; Tue, 13 Apr 2021 19:57:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 042A66EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618333077; bh=C9FvzQ9UjniLliWTk8Nyt/0zk9RVVR6baL+UO/Iv/nE=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=UYlQKZNz1GA/VDgVV3SnkJ4uNj57qQrrq4ocI7DjifQmpSi5L4lXP6dLMG5fL4H4X LvFFcoge6/9aIkCDPbF8e0fp497m+5RPr6w5GhDryuOdv/N8OWx3u1fPn1v4DVhaie bnQRfZHxAjWeYuX22K13CZm4uMFTb+aPkO+NrMLI= Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id A1A7F6EC5F for ; Tue, 13 Apr 2021 19:57:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org A1A7F6EC5F Received: by smtpng2.m.smailru.net with esmtpa (envelope-from ) id 1lWMMM-0005L8-Dd; Tue, 13 Apr 2021 19:57:54 +0300 Date: Tue, 13 Apr 2021 19:57:53 +0300 To: Vladislav Shpilevoy Message-ID: <20210413165753.GA185709@tarantool.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E74806859AC5FE18436AEED970E897805ADA4182A05F5380850407A58F7134E47530595EFA9779D3AE4A40CA2E1B429B4DB901AE5828DFC4E145E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B4D64F1449264784EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637DD7A7F9003AF293F8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2456FE21E4F5D3CC0369F4146DCD94C1E79373A8315E4CF65D2E47CDBA5A96583C09775C1D3CA48CFED8438A78DFE0A9E117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE7ABB305BD10C6E5099FA2833FD35BB23DF004C906525384302BEBFE083D3B9BA73A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E7354E672349037D5FA5C4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A57EDEC81AA6F5A6F4C556A4C08669FB98796C5D2E3B636D37D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A533D6DF7731C66485A1C081321CE835604BE2F1270509060A8D21DA841FCECC680E47760E05D5EC1D7E09C32AA3244CE780CE5CFB11548994692416F7A009E2D9ADFF0C0BDB8D1FFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojnA7/qPBUIXGyZsimD8xreA== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822B5CFDDDACF53BA743611B7A267AB39DA83D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: Mergen Imeev Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thank you for the review! My answers, diff and new patch below. On Sun, Apr 11, 2021 at 08:11:53PM +0200, Vladislav Shpilevoy wrote: > Good job on the patch! > > See 2 comments below. > > On 09.04.2021 19:57, Mergen Imeev via Tarantool-patches wrote: > > This patch introduces mem_concat(). Function mem_concat() concatenates > > values from two MEMs in case these values are strings or binaries and > > writes the result to the third MEM. > > > > Part of #5818 > > --- > > src/box/sql/mem.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ > > src/box/sql/mem.h | 8 ++++++ > > src/box/sql/vdbe.c | 50 ++---------------------------------- > > 3 files changed, 74 insertions(+), 48 deletions(-) > > > > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c > > index b417c1007..2d76ef88d 100644 > > --- a/src/box/sql/mem.c > > +++ b/src/box/sql/mem.c > > @@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from) > > return 0; > > } > > > > +static bool > > +is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result, > > + enum field_type type) > > 1. Functions called 'is_*' never should change anything. > Fixed. Renamed to check_result_null(). > Another question is why do you even need it? It is used in a single place, > where it could be just inlined. And is not used in a place, where it could > be applied. > I added it here since it was the first commit, which used it. This functions will be used in all arithmetic and bitwise operations with two operands. > > +{ > > + mem_clear(result); > > + result->field_type = type; > > + return (((a->flags | b->flags) & MEM_Null) != 0); > > +} > > + > > +int > > +mem_concat(struct Mem *a, struct Mem *b, struct Mem *result) > > +{ > > + assert(result != b); > > + if (a != result) { > > + if (is_result_null(a, b, result, FIELD_TYPE_STRING)) > > + return 0; > > + } else { > > + if (((a->flags | b->flags) & MEM_Null) != 0) { > > + mem_clear(a); > > + result->field_type = FIELD_TYPE_STRING; > > + return 0; > > + } > > + } > > + > > + /* Concatenation operation can be applied only to strings and blobs. */ > > + if ((b->flags & (MEM_Str | MEM_Blob)) == 0) { > > + diag_set(ClientError, ER_INCONSISTENT_TYPES, > > + "text or varbinary", mem_type_to_str(b)); > > + return -1; > > + } > > + if ((a->flags & (MEM_Str | MEM_Blob)) == 0) { > > + diag_set(ClientError, ER_INCONSISTENT_TYPES, > > + "text or varbinary", mem_type_to_str(a)); > > + return -1; > > + } > > + > > + /* Moreover, both operands must be of the same type. */ > > + if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) { > > + diag_set(ClientError, ER_INCONSISTENT_TYPES, > > + mem_type_to_str(a), mem_type_to_str(b)); > > + return -1; > > + } > > + > > + if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0) > > + return -1; > > + > > + uint32_t size = a->n + b->n; > > + if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) { > > + diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big"); > > + return -1; > > + } > > + if (sqlVdbeMemGrow(result, size, result == a) != 0) > > + return -1; > > + > > + result->flags = a->flags & (MEM_Str | MEM_Blob); > > 2. Why isn't result cleared? What if it was an Agg, or Frame? > I see before your patch they called vdbe_prepare_null_out(), which > cleared the mem. > In case result != left result is cleared in check_result_null(). In the other case it is cleared if one of operands is NULL. If it is not cleared than result == left, which means that if it is not varbinary of string, the error will be returned. There shouldn't be any problem during destruction, I think. Also, if we clear result when result == left we will get NULL as result of concatenation no matter what left and right were. I do not know why result was cleared previously, since it may lead to result described above. > > + if ((result->flags & MEM_Blob) != 0) > > + result->field_type = FIELD_TYPE_VARBINARY; > > + if (result != a) > > + memcpy(result->z, a->z, a->n); > > + memcpy(&result->z[a->n], b->z, b->n); > > + result->n = size; > > + return 0; > > +} Diff: diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index 67c8d90ee..f3bb62369 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -181,8 +181,8 @@ mem_move(struct Mem *to, struct Mem *from) } static bool -is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result, - enum field_type type) +check_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result, + enum field_type type) { mem_clear(result); result->field_type = type; @@ -194,7 +194,7 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result) { assert(result != b); if (a != result) { - if (is_result_null(a, b, result, FIELD_TYPE_STRING)) + if (check_result_null(a, b, result, FIELD_TYPE_STRING)) return 0; } else { if (((a->flags | b->flags) & MEM_Null) != 0) { New patch: commit e95f6aea6698865e5c864fa8ae1b5393c4faf553 Author: Mergen Imeev Date: Sat Mar 13 21:30:54 2021 +0300 sql: introduce mem_concat() This patch introduces mem_concat(). Function mem_concat() concatenates values from two MEMs in case these values are strings or binaries and writes the result to the third MEM. Part of #5818 diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index a7caac014..f3bb62369 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -180,6 +180,70 @@ mem_move(struct Mem *to, struct Mem *from) from->zMalloc = NULL; } +static bool +check_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result, + enum field_type type) +{ + mem_clear(result); + result->field_type = type; + return (((a->flags | b->flags) & MEM_Null) != 0); +} + +int +mem_concat(struct Mem *a, struct Mem *b, struct Mem *result) +{ + assert(result != b); + if (a != result) { + if (check_result_null(a, b, result, FIELD_TYPE_STRING)) + return 0; + } else { + if (((a->flags | b->flags) & MEM_Null) != 0) { + mem_clear(a); + result->field_type = FIELD_TYPE_STRING; + return 0; + } + } + + /* Concatenation operation can be applied only to strings and blobs. */ + if ((b->flags & (MEM_Str | MEM_Blob)) == 0) { + diag_set(ClientError, ER_INCONSISTENT_TYPES, + "text or varbinary", mem_type_to_str(b)); + return -1; + } + if ((a->flags & (MEM_Str | MEM_Blob)) == 0) { + diag_set(ClientError, ER_INCONSISTENT_TYPES, + "text or varbinary", mem_type_to_str(a)); + return -1; + } + + /* Moreover, both operands must be of the same type. */ + if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) { + diag_set(ClientError, ER_INCONSISTENT_TYPES, + mem_type_to_str(a), mem_type_to_str(b)); + return -1; + } + + if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0) + return -1; + + uint32_t size = a->n + b->n; + if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) { + diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big"); + return -1; + } + if (sqlVdbeMemGrow(result, size, result == a) != 0) + return -1; + + result->flags = a->flags & (MEM_Str | MEM_Blob); + if ((result->flags & MEM_Blob) != 0) + result->field_type = FIELD_TYPE_VARBINARY; + if (result != a) + memcpy(result->z, a->z, a->n); + memcpy(&result->z[a->n], b->z, b->n); + result->n = size; + return 0; +} + static inline bool mem_has_msgpack_subtype(struct Mem *mem) { diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index 37f6ac5df..d17ed0593 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -322,6 +322,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from); void mem_move(struct Mem *to, struct Mem *from); +/** + * Concatenate strings or binaries from the first and the second MEMs and write + * to the result MEM. In case the first MEM or the second MEM is NULL, the + * result MEM is set to NULL even if the result MEM is actually the first MEM. + */ +int +mem_concat(struct Mem *left, struct Mem *right, struct Mem *result); + /** * Simple type to str convertor. It is used to simplify * error reporting. diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index e73ed173d..c017c5091 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -1060,57 +1060,11 @@ case OP_ResultRow: { * types (i.e. TEXT and BLOB). */ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */ - i64 nByte; - pIn1 = &aMem[pOp->p1]; pIn2 = &aMem[pOp->p2]; - pOut = vdbe_prepare_null_out(p, pOp->p3); - assert(pIn1!=pOut); - if (mem_is_any_null(pIn1, pIn2)) { - /* Force NULL be of type STRING. */ - pOut->field_type = FIELD_TYPE_STRING; - break; - } - /* - * Concatenation operation can be applied only to - * strings and blobs. - */ - if (!mem_is_bytes(pIn1) || !mem_is_bytes(pIn2)) { - char *inconsistent_type = !mem_is_bytes(pIn1) ? - mem_type_to_str(pIn1) : - mem_type_to_str(pIn2); - diag_set(ClientError, ER_INCONSISTENT_TYPES, - "text or varbinary", inconsistent_type); - goto abort_due_to_error; - } - - /* Moreover, both operands must be of the same type. */ - if (!mem_is_same_type(pIn1, pIn2)) { - diag_set(ClientError, ER_INCONSISTENT_TYPES, - mem_type_to_str(pIn2), mem_type_to_str(pIn1)); - goto abort_due_to_error; - } - if (ExpandBlob(pIn1) != 0 || ExpandBlob(pIn2) != 0) + pOut = &aMem[pOp->p3]; + if (mem_concat(pIn2, pIn1, pOut) != 0) goto abort_due_to_error; - nByte = pIn1->n + pIn2->n; - if (nByte>db->aLimit[SQL_LIMIT_LENGTH]) { - goto too_big; - } - if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) { - goto no_mem; - } - if (mem_is_str(pIn1)) - MemSetTypeFlag(pOut, MEM_Str); - else - MemSetTypeFlag(pOut, MEM_Blob); - if (pOut!=pIn2) { - memcpy(pOut->z, pIn2->z, pIn2->n); - } - memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n); - pOut->z[nByte]=0; - pOut->z[nByte+1] = 0; - pOut->flags |= MEM_Term; - pOut->n = (int)nByte; UPDATE_MAX_BLOBSIZE(pOut); break; }