From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E78E36BD03; Sat, 10 Apr 2021 00:10:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E78E36BD03 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618002651; bh=fCzq9+eabvGS2P975innFeBhPnC2yK0I3IhmlR1nSjY=; h=To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Y1gF3akzsfdl6OPDAUbLbBiZ6bnIgag2wGlMdBkX7gs+80w+PPfQMUONH+KVdcSu5 3Sycu6tUH/dmaMyYE8YH76PGZekMjjB/jXah379yAc/C9jPVamyxKzrGRjFbVKITvZ mK73WrOUMs8QJ+S7nDfJyY1yu3HyOnigQWg+evEg= Received: from smtpng1.m.smailru.net (smtpng1.m.smailru.net [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id D09276EC63 for ; Sat, 10 Apr 2021 00:08:29 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org D09276EC63 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1lUyMf-0003uf-1w; Sat, 10 Apr 2021 00:08:29 +0300 To: v.shpilevoy@tarantool.org, tsafin@tarantool.org Cc: tarantool-patches@dev.tarantool.org Date: Sat, 10 Apr 2021 00:08:28 +0300 Message-Id: <326fac6c97ae260b32be9987e48e3381c600ba9c.1618000037.git.imeevma@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: EEAE043A70213CC8 X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E74806859AC5FE18436AEED970E897805ADA4182A05F538085040BA4FFD5F5E13EC3DDCC35A8390C6AB922D4E2CB8F93A5A27667AE3EE7D235081 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B7733D0215A2F71AEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637DD7A7F9003AF293F8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B281477D3A8D9A0C8889F1025CE78CC6D9AD368CDB2FD0D9CCD2E47CDBA5A96583C09775C1D3CA48CFCA5A41EBD8A3A0199FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C3CA5A41EBD8A3A0199FA2833FD35BB23DF004C906525384302BEBFE083D3B9BA73A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735E4A630A5B664A4FFC4224003CC83647689D4C264860C145E X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2AD77751E876CB595E8F7B195E1C97831E46709FC235F5DF638E9427ECC4B679E X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975CD0035DD76F8A8A4F848298EA73FEAD8DD78886C24B1FAA8F9C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF0417BEADF48D1460699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3472E5ECC12A9739C18703D12CA42646814EA0A3E295C40CC0C60C15C753D399A624384E96583EBCD71D7E09C32AA3244C1E9DE515B4809F3EE313F0029D746956E8FBBEFAE1C4874CFACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojqcJA+pXcDun2XVjcUpSKJQ== X-Mailru-Sender: 689FA8AB762F73936BC43F508A063822ED4CCF223B67858B23B109BD6391F93783D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E33667EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH v5 51/52] sql: introduce mem_get_bytes_len() X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: imeevma@tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thank you for the review! My answer and new patch below. Also, I see that I did a lot of misspelling and other errors in my answers and I am very sorry about this. I will try to add more info to my answers tomorrow. On 30.03.2021 02:09, Vladislav Shpilevoy wrote: > Thanks for the patch! > >> diff --git a/src/box/sql/func.c b/src/box/sql/func.c >> index 199f3abef..6687fca3c 100644 >> --- a/src/box/sql/func.c >> +++ b/src/box/sql/func.c >> @@ -967,7 +968,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv) \ >> return; \ >> } \ >> z2 = mem_get_str(argv[0]); \ >> - n = sql_value_bytes(argv[0]); \ >> + n = mem_get_length(argv[0]); \ > > All these \ were aligned before your patchset. Please, keep it straight. > Fixed. >> /* \ >> * Verify that the call to _bytes() \ >> * does not invalidate the _text() pointer. \ New patch: commit 326fac6c97ae260b32be9987e48e3381c600ba9c Author: Mergen Imeev Date: Mon Mar 22 12:27:33 2021 +0300 sql: introduce mem_get_bytes_len() This patch introduces mem_get_bytes_len(). This function is used to receive length of string or binary value of MEM. If MEM is not of STRING or VARBINARy type this function returns -1. Part of #5818 diff --git a/src/box/sql/func.c b/src/box/sql/func.c index 2896a5c31..746bda0f4 100644 --- a/src/box/sql/func.c +++ b/src/box/sql/func.c @@ -65,6 +65,14 @@ mem_as_bin(struct Mem *mem) return s; } +static int +mem_get_length(struct Mem *mem) +{ + uint32_t len; + mem_get_bytes_len(mem, &len); + return len; +} + /* * Return the collating function associated with a function. */ @@ -187,14 +195,15 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv) case MP_UINT: case MP_BOOL: case MP_DOUBLE:{ - sql_result_uint(context, sql_value_bytes(argv[0])); + mem_as_bin(argv[0]); + sql_result_uint(context, mem_get_length(argv[0])); break; } case MP_STR:{ const unsigned char *z = mem_as_ustr(argv[0]); if (z == 0) return; - len = sql_utf8_char_count(z, sql_value_bytes(argv[0])); + len = sql_utf8_char_count(z, mem_get_length(argv[0])); sql_result_uint(context, len); break; } @@ -310,8 +319,8 @@ position_func(struct sql_context *context, int argc, struct Mem **argv) return; } - int n_needle_bytes = sql_value_bytes(needle); - int n_haystack_bytes = sql_value_bytes(haystack); + int n_needle_bytes = mem_get_length(needle); + int n_haystack_bytes = mem_get_length(haystack); int position = 1; if (n_needle_bytes > 0) { const unsigned char *haystack_str; @@ -457,18 +466,18 @@ substrFunc(sql_context * context, int argc, sql_value ** argv) bool unused; mem_get_int(argv[1], &p1, &unused); if (p0type == MP_BIN) { - len = sql_value_bytes(argv[0]); z = mem_as_bin(argv[0]); + len = mem_get_length(argv[0]); if (z == 0) return; - assert(len == sql_value_bytes(argv[0])); + assert(len == mem_get_length(argv[0])); } else { z = mem_as_ustr(argv[0]); if (z == 0) return; len = 0; if (p1 < 0) - len = sql_utf8_char_count(z, sql_value_bytes(argv[0])); + len = sql_utf8_char_count(z, mem_get_length(argv[0])); } if (argc == 3) { mem_get_int(argv[2], &p2, &unused); @@ -507,7 +516,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv) * used because '\0' is not supposed to be * end-of-string symbol. */ - int byte_size = sql_value_bytes(argv[0]); + int byte_size = mem_get_length(argv[0]); int n_chars = sql_utf8_char_count(z, byte_size); int cnt = 0; int i = 0; @@ -627,7 +636,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv) \ return; \ } \ z2 = mem_as_str0(argv[0]); \ - n = sql_value_bytes(argv[0]); \ + n = mem_get_length(argv[0]); \ /* \ * Verify that the call to _bytes() \ * does not invalidate the _text() pointer. \ @@ -965,15 +974,15 @@ likeFunc(sql_context *context, int argc, sql_value **argv) } const char *zB = mem_as_str0(argv[0]); const char *zA = mem_as_str0(argv[1]); - const char *zB_end = zB + sql_value_bytes(argv[0]); - const char *zA_end = zA + sql_value_bytes(argv[1]); + const char *zB_end = zB + mem_get_length(argv[0]); + const char *zA_end = zA + mem_get_length(argv[1]); /* * Limit the length of the LIKE pattern to avoid problems * of deep recursion and N*N behavior in * sql_utf8_pattern_compare(). */ - nPat = sql_value_bytes(argv[0]); + nPat = mem_get_length(argv[0]); testcase(nPat == db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH]); testcase(nPat == db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH] + 1); if (nPat > db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH]) { @@ -994,7 +1003,7 @@ likeFunc(sql_context *context, int argc, sql_value **argv) const unsigned char *zEsc = mem_as_ustr(argv[2]); if (zEsc == 0) return; - if (sql_utf8_char_count(zEsc, sql_value_bytes(argv[2])) != 1) { + if (sql_utf8_char_count(zEsc, mem_get_length(argv[2])) != 1) { diag_set(ClientError, ER_SQL_EXECUTE, "ESCAPE "\ "expression must be a single character"); context->is_aborted = true; @@ -1094,7 +1103,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv) case MP_MAP: { char *zText = 0; char const *zBlob = mem_as_bin(argv[0]); - int nBlob = sql_value_bytes(argv[0]); + int nBlob = mem_get_length(argv[0]); assert(zBlob == mem_as_bin(argv[0])); /* No encoding change */ zText = (char *)contextMalloc(context, @@ -1230,7 +1239,7 @@ hexFunc(sql_context * context, int argc, sql_value ** argv) assert(argc == 1); UNUSED_PARAMETER(argc); pBlob = mem_as_bin(argv[0]); - n = sql_value_bytes(argv[0]); + n = mem_get_length(argv[0]); assert(pBlob == mem_as_bin(argv[0])); /* No encoding change */ z = zHex = contextMalloc(context, ((i64) n) * 2 + 1); if (zHex) { @@ -1289,7 +1298,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv) zStr = mem_as_ustr(argv[0]); if (zStr == 0) return; - nStr = sql_value_bytes(argv[0]); + nStr = mem_get_length(argv[0]); assert(zStr == mem_as_ustr(argv[0])); /* No encoding change */ zPattern = mem_as_ustr(argv[1]); if (zPattern == 0) { @@ -1297,7 +1306,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv) || sql_context_db_handle(context)->mallocFailed); return; } - nPattern = sql_value_bytes(argv[1]); + nPattern = mem_get_length(argv[1]); if (nPattern == 0) { assert(!mem_is_null(argv[1])); sql_result_value(context, argv[0]); @@ -1307,7 +1316,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv) zRep = mem_as_ustr(argv[2]); if (zRep == 0) return; - nRep = sql_value_bytes(argv[2]); + nRep = mem_get_length(argv[2]); assert(zRep == mem_as_ustr(argv[2])); nOut = nStr + 1; assert(nOut < SQL_MAX_LENGTH); @@ -1469,7 +1478,7 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg) default_trim = (const unsigned char *) "\0"; else default_trim = (const unsigned char *) " "; - int input_str_sz = sql_value_bytes(arg); + int input_str_sz = mem_get_length(arg); const unsigned char *input_str = mem_as_ustr(arg); uint8_t trim_char_len[1] = { 1 }; trim_procedure(context, TRIM_BOTH, default_trim, trim_char_len, 1, @@ -1495,7 +1504,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1, if ((input_str = mem_as_ustr(arg2)) == NULL) return; - int input_str_sz = sql_value_bytes(arg2); + int input_str_sz = mem_get_length(arg2); if (sql_value_type(arg1) == MP_INT || sql_value_type(arg1) == MP_UINT) { uint8_t len_one = 1; uint64_t n; @@ -1503,7 +1512,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1, trim_procedure(context, n, (const unsigned char *) " ", &len_one, 1, input_str, input_str_sz); } else if ((trim_set = mem_as_ustr(arg1)) != NULL) { - int trim_set_sz = sql_value_bytes(arg1); + int trim_set_sz = mem_get_length(arg1); uint8_t *char_len; int char_cnt = trim_prepare_char_len(context, trim_set, trim_set_sz, &char_len); @@ -1532,8 +1541,8 @@ trim_func_three_args(struct sql_context *context, sql_value *arg1, (trim_set = mem_as_ustr(arg2)) == NULL) return; - int trim_set_sz = sql_value_bytes(arg2); - int input_str_sz = sql_value_bytes(arg3); + int trim_set_sz = mem_get_length(arg2); + int input_str_sz = mem_get_length(arg3); uint8_t *char_len; int char_cnt = trim_prepare_char_len(context, trim_set, trim_set_sz, &char_len); @@ -1853,7 +1862,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv) if (!firstTerm) { if (argc == 2) { zSep = mem_as_str0(argv[1]); - nSep = sql_value_bytes(argv[1]); + nSep = mem_get_length(argv[1]); } else { zSep = ","; nSep = 1; @@ -1862,7 +1871,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv) sqlStrAccumAppend(pAccum, zSep, nSep); } zVal = mem_as_str0(argv[0]); - nVal = sql_value_bytes(argv[0]); + nVal = mem_get_length(argv[0]); if (zVal) sqlStrAccumAppend(pAccum, zVal, nVal); } diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index 2e69f4a80..3e9544cae 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -1345,6 +1345,18 @@ mem_get_bin(const struct Mem *mem, const char **s) return 0; } +int +mem_get_bytes_len(const struct Mem *mem, uint32_t *len) +{ + if ((mem->flags & (MEM_Str | MEM_Blob)) == 0) + return -1; + if ((mem->flags & MEM_Blob) !=0 && (mem->flags & MEM_Zero) != 0) + *len = mem->n + mem->u.nZero; + else + *len = mem->n; + return 0; +} + int mem_copy(struct Mem *to, const struct Mem *from) { @@ -2116,41 +2128,6 @@ sql_value_type(sql_value *pVal) return mem_mp_type(mem); } -/* - * The sqlValueBytes() routine returns the number of bytes in the - * sql_value object assuming that it uses the encoding "enc". - * The valueBytes() routine is a helper function. - */ -static SQL_NOINLINE int -valueBytes(sql_value * pVal) -{ - if (mem_to_str(pVal) != 0) - return 0; - return pVal->n; -} - -int -sqlValueBytes(sql_value * pVal) -{ - Mem *p = (Mem *) pVal; - assert((p->flags & MEM_Null) == 0 - || (p->flags & (MEM_Str | MEM_Blob)) == 0); - if ((p->flags & MEM_Str) != 0) { - return p->n; - } - if ((p->flags & MEM_Blob) != 0) { - if (p->flags & MEM_Zero) { - return p->n + p->u.nZero; - } else { - return p->n; - } - } - if (p->flags & MEM_Null) - return 0; - return valueBytes(pVal); -} - - #ifdef SQL_DEBUG /* * Check invariants on a Mem object. @@ -2457,53 +2434,6 @@ releaseMemArray(Mem * p, int N) } } -int -sql_value_bytes(sql_value * pVal) -{ - return sqlValueBytes(pVal); -} - -/* - * Return a pointer to static memory containing an SQL NULL value. - */ -const Mem * -columnNullValue(void) -{ - /* Even though the Mem structure contains an element - * of type i64, on certain architectures (x86) with certain compiler - * switches (-Os), gcc may align this Mem object on a 4-byte boundary - * instead of an 8-byte one. This all works fine, except that when - * running with SQL_DEBUG defined the sql code sometimes assert()s - * that a Mem structure is located on an 8-byte boundary. To prevent - * these assert()s from failing, when building with SQL_DEBUG defined - * using gcc, we force nullMem to be 8-byte aligned using the magical - * __attribute__((aligned(8))) macro. - */ - static const Mem nullMem -#if defined(SQL_DEBUG) && defined(__GNUC__) - __attribute__ ((aligned(8))) -#endif - = { - /* .u = */ { - 0}, - /* .flags = */ (u16) MEM_Null, - /* .eSubtype = */ (u8) 0, - /* .field_type = */ field_type_MAX, - /* .n = */ (int)0, - /* .z = */ (char *)0, - /* .zMalloc = */ (char *)0, - /* .szMalloc = */ (int)0, - /* .uTemp = */ (u32) 0, - /* .db = */ (sql *) 0, - /* .xDel = */ (void (*)(void *))0, -#ifdef SQL_DEBUG - /* .pScopyFrom = */ (Mem *) 0, - /* .pFiller = */ (void *)0, -#endif - }; - return &nullMem; -} - /* * Return true if the Mem object contains a TEXT or BLOB that is * too large - whose size exceeds SQL_MAX_LENGTH. diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index e2d0f343e..9c6d18e1f 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -598,6 +598,13 @@ mem_as_str0(struct Mem *mem) int mem_get_bin(const struct Mem *mem, const char **s); +/** + * Return length of value for MEM of STRING or VARBINARY type. Original MEM is + * not changed. + */ +int +mem_get_bytes_len(const struct Mem *mem, uint32_t *len); + /** * Simple type to str convertor. It is used to simplify * error reporting. @@ -616,8 +623,6 @@ mem_mp_type(struct Mem *mem); enum mp_type sql_value_type(struct Mem *); -int sqlValueBytes(struct Mem *); - #ifdef SQL_DEBUG int sqlVdbeCheckMemInvariants(struct Mem *); void sqlVdbeMemPrettyPrint(Mem * pMem, char *zBuf); @@ -647,18 +652,8 @@ struct Mem *sqlValueNew(struct sql *); void releaseMemArray(Mem * p, int N); -/** Getters. */ - -int -sql_value_bytes(struct Mem *); - #define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))]) -const Mem * -columnNullValue(void); - -/** Checkers. */ - int sqlVdbeMemTooBig(Mem *); int sqlMemCompare(const Mem *, const Mem *, const struct coll *); diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index 8a7a87b49..8feb112f5 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -433,9 +433,6 @@ sql_stmt_compile(const char *sql, int bytes_count, struct Vdbe *re_prepared, int sql_step(sql_stmt *); -int -sql_column_bytes(sql_stmt *, int iCol); - int sql_column_bytes16(sql_stmt *, int iCol); diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index 19b70a55b..b0334f3ed 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -442,41 +442,6 @@ sql_data_count(sql_stmt * pStmt) return pVm->nResColumn; } -/* - * Check to see if column iCol of the given statement is valid. If - * it is, return a pointer to the Mem for the value of that column. - * If iCol is not valid, return a pointer to a Mem which has a value - * of NULL. - */ -static Mem * -columnMem(sql_stmt * pStmt, int i) -{ - Vdbe *pVm; - Mem *pOut; - - pVm = (Vdbe *) pStmt; - if (pVm == 0) - return (Mem *) columnNullValue(); - assert(pVm->db); - if (pVm->pResultSet != 0 && i < pVm->nResColumn && i >= 0) { - pOut = &pVm->pResultSet[i]; - } else { - pOut = (Mem *) columnNullValue(); - } - return pOut; -} - -/**************************** sql_column_ ****************************** - * The following routines are used to access elements of the current row - * in the result set. - */ - -int -sql_column_bytes(sql_stmt * pStmt, int i) -{ - return sql_value_bytes(columnMem(pStmt, i)); -} - char * sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size, struct region *region)