* [tarantool-patches] [PATCH 1/6] sql: refactor sql_atoi64()
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
2019-06-07 15:37 ` [tarantool-patches] [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints Nikita Pettik
` (5 subsequent siblings)
6 siblings, 1 reply; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
We are going to allow using unsigned values in SQL and extend range of
INTEGER type. Hence, we should be able to parse and operate on integers
in range of [2^63, 2^64 - 1]. Current mechanism which involves
sql_atoi64() function doesn't allow this.
Let's refactor this function: firstly, get rid of manual parsing and use
strtoll() and strtoull() functions from standard library. Then, let's
return sign of parsed literal. In case of success now function returns 0,
-1 otherwise.
This patch also inlines sql_dec_or_hex_to_i64() to place of its only
usage: it makes code cleaner and more straightforward.
Needed for #3810
Needed for #4015
---
src/box/errcode.h | 2 +-
src/box/sql/expr.c | 65 ++++++++++++++++-------
src/box/sql/sqlInt.h | 47 ++++-------------
src/box/sql/sqlLimit.h | 9 ++++
src/box/sql/util.c | 140 +++++++++++--------------------------------------
src/box/sql/vdbe.c | 6 ++-
src/box/sql/vdbemem.c | 14 +++--
7 files changed, 110 insertions(+), 173 deletions(-)
diff --git a/src/box/errcode.h b/src/box/errcode.h
index 9c15f3322..f4aba2f54 100644
--- a/src/box/errcode.h
+++ b/src/box/errcode.h
@@ -242,7 +242,7 @@ struct errcode_record {
/*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
/*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
/*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
- /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range %lld - %lld") \
+ /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %lld") \
/*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \
/*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \
/*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index ba7eea59d..158631416 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -1196,7 +1196,8 @@ sqlExprAssignVarNumber(Parse * pParse, Expr * pExpr, u32 n)
* variable number
*/
int64_t i;
- bool is_ok = 0 == sql_atoi64(&z[1], &i, n - 1);
+ bool is_neg;
+ bool is_ok = 0 == sql_atoi64(&z[1], &i, &is_neg, n - 1);
x = (ynVar) i;
testcase(i == 0);
testcase(i == 1);
@@ -3325,29 +3326,53 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
if (is_neg)
i = -i;
sqlVdbeAddOp2(v, OP_Integer, i, mem);
+ return;
+ }
+ int64_t value;
+ const char *z = expr->u.zToken;
+ assert(z != NULL);
+ const char *sign = is_neg ? "-" : "";
+ if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
+ uint64_t u = 0;
+ int i, k;
+ for (i = 2; z[i] == '0'; i++);
+ for (k = i; sqlIsxdigit(z[k]); k++)
+ u = u * 16 + sqlHexToInt(z[k]);
+ memcpy(&value, &u, 8);
+ if (z[k] != 0 || k - i > 16) {
+ diag_set(ClientError, ER_HEX_LITERAL_MAX, sign, z,
+ strlen(z) - 2, 16);
+ parse->is_aborted = true;
+ return;
+ }
} else {
- int64_t value;
- const char *z = expr->u.zToken;
- assert(z != NULL);
- int c = sql_dec_or_hex_to_i64(z, &value);
- if (c == 1 || (c == 2 && !is_neg) ||
- (is_neg && value == SMALLEST_INT64)) {
- const char *sign = is_neg ? "-" : "";
- if (sql_strnicmp(z, "0x", 2) == 0) {
- diag_set(ClientError, ER_HEX_LITERAL_MAX, sign,
- z, strlen(z) - 2, 16);
- } else {
- diag_set(ClientError, ER_INT_LITERAL_MAX, sign,
- z, INT64_MIN, INT64_MAX);
- }
+ size_t len = strlen(z);
+ if (is_neg) {
+ /*
+ * UINT64_MAX is constant, so each integer
+ * literal certainly contains less digits.
+ * It makes no sense to continue processing
+ * string if it contains more characters.
+ */
+ if (len > UINT64_MAX_DIGIT_COUNT)
+ goto int_overflow;
+ char *neg_z = tt_static_buf();
+ neg_z[0] = '-';
+ memcpy(neg_z + 1, z, len++);
+ neg_z[len]= '\0';
+ z = neg_z;
+ }
+ bool unused;
+ int rc = sql_atoi64(z, &value, &unused, len);
+ if (rc != 0) {
+int_overflow:
+ diag_set(ClientError, ER_INT_LITERAL_MAX, z, INT64_MIN,
+ INT64_MAX);
parse->is_aborted = true;
- } else {
- if (is_neg)
- value = c == 2 ? SMALLEST_INT64 : -value;
- sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0,
- (u8 *)&value, P4_INT64);
+ return;
}
}
+ sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0, (u8 *)&value, P4_INT64);
}
/*
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 997a46535..da17e24ed 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -4405,52 +4405,25 @@ field_type_sequence_dup(struct Parse *parse, enum field_type *types,
uint32_t len);
/**
- * Convert z to a 64-bit signed integer. z must be decimal. This
- * routine does *not* accept hexadecimal notation.
- *
- * If the z value is representable as a 64-bit twos-complement
- * integer, then write that value into *val and return 0.
- *
- * If z is exactly 9223372036854775808, return 2. This special
- * case is broken out because while 9223372036854775808 cannot be
- * a signed 64-bit integer, its negative -9223372036854775808 can
- * be.
- *
- * If z is too big for a 64-bit integer and is not
- * 9223372036854775808 or if z contains any non-numeric text,
- * then return 1.
+ * Convert z to a 64-bit signed or unsigned integer.
+ * z must be decimal. This routine does *not* accept
+ * hexadecimal notation. Under the hood it calls
+ * strtoll or strtoull depending on presence of '-' char.
*
* length is the number of bytes in the string (bytes, not
* characters). The string is not necessarily zero-terminated.
- * The encoding is given by enc.
*
* @param z String being parsed.
* @param[out] val Output integer value.
+ * @param[out] is_neg Sign of the result.
* @param length String length in bytes.
- * @retval
- * 0 Successful transformation. Fits in a 64-bit signed
- * integer.
- * 1 Integer too large for a 64-bit signed integer or is
- * malformed
- * 2 Special case of 9223372036854775808
- */
-int
-sql_atoi64(const char *z, int64_t *val, int length);
-
-/**
- * Transform a UTF-8 integer literal, in either decimal or
- * hexadecimal, into a 64-bit signed integer. This routine
- * accepts hexadecimal literals, whereas sql_atoi64() does not.
- *
- * @param z Literal being parsed.
- * @param[out] val Parsed value.
- * @retval
- * 0 Successful transformation. Fits in a 64-bit signed integer.
- * 1 Integer too large for a 64-bit signed integer or is malformed
- * 2 Special case of 9223372036854775808
+ * @retval 0 Successful transformation. Fits in a 64-bit signed
+ * or unsigned integer.
+ * @retval -1 Error during parsing: it contains non-digit
+ * characters or it doesn't fit into 64-bit int.
*/
int
-sql_dec_or_hex_to_i64(const char *z, int64_t *val);
+sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length);
void sqlErrorWithMsg(sql *, int, const char *, ...);
void sqlError(sql *, int);
diff --git a/src/box/sql/sqlLimit.h b/src/box/sql/sqlLimit.h
index 53dbe1505..1c3038ed3 100644
--- a/src/box/sql/sqlLimit.h
+++ b/src/box/sql/sqlLimit.h
@@ -44,6 +44,15 @@ enum {
SQL_BIND_PARAMETER_MAX = 65000,
};
+enum {
+ /**
+ * Each unsigned (ergo signed) contains less than
+ * 21 characters. It allows us to prevent from
+ * parsing oversized literals and raise an error.
+ */
+ UINT64_MAX_DIGIT_COUNT = 21
+};
+
/*
* The maximum length of a TEXT or BLOB in bytes. This also
* limits the size of a row in a table or index.
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index d5c93f8aa..2e7c298e1 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -593,120 +593,42 @@ sqlAtoF(const char *z, double *pResult, int length)
return z == zEnd && nDigits > 0 && eValid && nonNum == 0;
}
-/*
- * Compare the 19-character string zNum against the text representation
- * value 2^63: 9223372036854775808. Return negative, zero, or positive
- * if zNum is less than, equal to, or greater than the string.
- * Note that zNum must contain exactly 19 characters.
- *
- * Unlike memcmp() this routine is guaranteed to return the difference
- * in the values of the last digit if the only difference is in the
- * last digit. So, for example,
- *
- * compare2pow63("9223372036854775800", 1)
- *
- * will return -8.
- */
-static int
-compare2pow63(const char *zNum, int incr)
-{
- int c = 0;
- int i;
- /* 012345678901234567 */
- const char *pow63 = "922337203685477580";
- for (i = 0; c == 0 && i < 18; i++) {
- c = (zNum[i * incr] - pow63[i]) * 10;
- }
- if (c == 0) {
- c = zNum[18 * incr] - '8';
- testcase(c == (-1));
- testcase(c == 0);
- testcase(c == (+1));
- }
- return c;
-}
-
int
-sql_atoi64(const char *z, int64_t *val, int length)
+sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
{
- int incr = 1;
- u64 u = 0;
- int neg = 0; /* assume positive */
- int i;
- int c = 0;
- int nonNum = 0; /* True if input contains UTF16 with high byte non-zero */
- const char *zStart;
- const char *zEnd = z + length;
- incr = 1;
- while (z < zEnd && sqlIsspace(*z))
- z += incr;
- if (z < zEnd) {
- if (*z == '-') {
- neg = 1;
- z += incr;
- } else if (*z == '+') {
- z += incr;
- }
- }
- zStart = z;
- /* Skip leading zeros. */
- while (z < zEnd && z[0] == '0') {
- z += incr;
- }
- for (i = 0; &z[i] < zEnd && (c = z[i]) >= '0' && c <= '9';
- i += incr) {
- u = u * 10 + c - '0';
- }
- if (u > LARGEST_INT64) {
- *val = neg ? SMALLEST_INT64 : LARGEST_INT64;
- } else if (neg) {
- *val = -(i64) u;
+ *is_neg = false;
+ const char *str_end = z + length;
+ for (; z < str_end && sqlIsspace(*z); z++)
+ /* invalid format */
+ if (z >= str_end)
+ return -1;
+ if (*z == '-')
+ *is_neg = true;
+
+ char* end = NULL;
+ errno = 0;
+ if (*is_neg) {
+ *val = strtoll(z, &end, 10);
} else {
- *val = (i64) u;
+ uint64_t u_val = strtoull(z, &end, 10);
+ if (u_val > INT64_MAX)
+ return -1;
+ *val = u_val;
}
- if (&z[i] < zEnd || (i == 0 && zStart == z) || i > 19 * incr ||
- nonNum) {
- /* zNum is empty or contains non-numeric text or is longer
- * than 19 digits (thus guaranteeing that it is too large)
- */
- return 1;
- } else if (i < 19 * incr) {
- /* Less than 19 digits, so we know that it fits in 64 bits */
- assert(u <= LARGEST_INT64);
- return 0;
- } else {
- /* zNum is a 19-digit numbers. Compare it against 9223372036854775808. */
- c = compare2pow63(z, incr);
- if (c < 0) {
- /* zNum is less than 9223372036854775808 so it fits */
- assert(u <= LARGEST_INT64);
- return 0;
- } else if (c > 0) {
- /* zNum is greater than 9223372036854775808 so it overflows */
- return 1;
- } else {
- /* zNum is exactly 9223372036854775808. Fits if negative. The
- * special case 2 overflow if positive
- */
- assert(u - 1 == LARGEST_INT64);
- return neg ? 0 : 2;
- }
- }
-}
+ /* No digits were found, e.g. an empty string. */
+ if (end == z)
+ return -1;
+ /*
+ * String has been parsed, but there's
+ * additional character(s) remained.
+ */
+ if (*end != '\0')
+ return -1;
+ /* Overflow and underflow errors. */
+ if (errno != 0)
+ return -1;
-int
-sql_dec_or_hex_to_i64(const char *z, int64_t *val)
-{
- if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
- uint64_t u = 0;
- int i, k;
- for (i = 2; z[i] == '0'; i++);
- for (k = i; sqlIsxdigit(z[k]); k++)
- u = u * 16 + sqlHexToInt(z[k]);
- memcpy(val, &u, 8);
- return (z[k] == 0 && k - i <= 16) ? 0 : 1;
- }
- return sql_atoi64(z, val, sqlStrlen30(z));
+ return 0;
}
/*
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index d083d3709..6ecdb26fc 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -279,7 +279,8 @@ mem_apply_numeric_type(struct Mem *record)
if ((record->flags & (MEM_Str | MEM_Int | MEM_Real)) != MEM_Str)
return -1;
int64_t integer_value;
- if (sql_atoi64(record->z, &integer_value, record->n) == 0) {
+ bool is_neg;
+ if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
record->u.i = integer_value;
MemSetTypeFlag(record, MEM_Int);
return 0;
@@ -384,7 +385,8 @@ static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
return 0;
- if (sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, pMem->n)==SQL_OK)
+ bool is_neg;
+ if (sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg, pMem->n)==SQL_OK)
return MEM_Int;
return MEM_Real;
}
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 08b649926..3a361d066 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -466,7 +466,8 @@ sqlVdbeIntValue(Mem * pMem, int64_t *i)
return doubleToInt64(pMem->u.r, i);
} else if (flags & (MEM_Str)) {
assert(pMem->z || pMem->n == 0);
- if (sql_atoi64(pMem->z, (int64_t *)i, pMem->n) == 0)
+ bool is_neg;
+ if (sql_atoi64(pMem->z, (int64_t *)i, &is_neg, pMem->n) == 0)
return 0;
}
return -1;
@@ -586,7 +587,9 @@ sqlVdbeMemNumerify(Mem * pMem)
{
if ((pMem->flags & (MEM_Int | MEM_Real | MEM_Null)) == 0) {
assert((pMem->flags & (MEM_Blob | MEM_Str)) != 0);
- if (0 == sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, pMem->n)) {
+ bool is_neg;
+ if (0 == sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg,
+ pMem->n)) {
MemSetTypeFlag(pMem, MEM_Int);
} else {
double v;
@@ -651,7 +654,9 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
if (pMem->flags & MEM_Null)
return SQL_OK;
if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, pMem->n) == 0) {
+ bool is_neg;
+ if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
+ pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
pMem->u.r = pMem->u.i;
return 0;
@@ -682,7 +687,8 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return -1;
case FIELD_TYPE_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i,
+ bool is_neg;
+ if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
pMem->n) != 0)
return -1;
MemSetTypeFlag(pMem, MEM_Int);
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 1/6] sql: refactor sql_atoi64()
2019-06-07 15:37 ` [tarantool-patches] [PATCH 1/6] sql: refactor sql_atoi64() Nikita Pettik
@ 2019-06-11 21:11 ` Vladislav Shpilevoy
2019-07-01 14:20 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-06-11 21:11 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Hi! Thanks for the patchset! See 8 comments below,
review fixes at the end of the email, and on the branch
in a separate commit.
On 07/06/2019 18:37, Nikita Pettik wrote:
> We are going to allow using unsigned values in SQL and extend range of
> INTEGER type. Hence, we should be able to parse and operate on integers
> in range of [2^63, 2^64 - 1]. Current mechanism which involves
> sql_atoi64() function doesn't allow this.
>
> Let's refactor this function: firstly, get rid of manual parsing and use
> strtoll() and strtoull() functions from standard library. Then, let's
> return sign of parsed literal.> In case of success now function returns 0,
> -1 otherwise.
>
> This patch also inlines sql_dec_or_hex_to_i64() to place of its only
> usage: it makes code cleaner and more straightforward.
>
> Needed for #3810
> Needed for #4015
> ---
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index ba7eea59d..158631416 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c> @@ -3325,29 +3326,53 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
> if (is_neg)
> i = -i;
> sqlVdbeAddOp2(v, OP_Integer, i, mem);
> + return;
> + }
> + int64_t value;
> + const char *z = expr->u.zToken;
> + assert(z != NULL);
> + const char *sign = is_neg ? "-" : "";
> + if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
> + uint64_t u = 0;
> + int i, k;
> + for (i = 2; z[i] == '0'; i++);
> + for (k = i; sqlIsxdigit(z[k]); k++)
> + u = u * 16 + sqlHexToInt(z[k]);
1. Why not to use strtoll and strtol? They accept 'base' argument.
Set it to 16, and they will do the same, but probably faster.
> + memcpy(&value, &u, 8);
> + if (z[k] != 0 || k - i > 16) {
2. You treat tail spaces as an error. Not sure if it is right.
> + diag_set(ClientError, ER_HEX_LITERAL_MAX, sign, z,
> + strlen(z) - 2, 16);
> + parse->is_aborted = true;
> + return;
> + }
> } else {
> - int64_t value;
> - const char *z = expr->u.zToken;
> - assert(z != NULL);
> - int c = sql_dec_or_hex_to_i64(z, &value);
> - if (c == 1 || (c == 2 && !is_neg) ||
> - (is_neg && value == SMALLEST_INT64)) {
> - const char *sign = is_neg ? "-" : "";
> - if (sql_strnicmp(z, "0x", 2) == 0) {
> - diag_set(ClientError, ER_HEX_LITERAL_MAX, sign,
> - z, strlen(z) - 2, 16);
> - } else {
> - diag_set(ClientError, ER_INT_LITERAL_MAX, sign,
> - z, INT64_MIN, INT64_MAX);
> - }
> + size_t len = strlen(z);
> + if (is_neg) {
> + /*
> + * UINT64_MAX is constant, so each integer
> + * literal certainly contains less digits.
> + * It makes no sense to continue processing
> + * string if it contains more characters.
> + */
> + if (len > UINT64_MAX_DIGIT_COUNT)
> + goto int_overflow;
> + char *neg_z = tt_static_buf();
> + neg_z[0] = '-';
> + memcpy(neg_z + 1, z, len++);
> + neg_z[len]= '\0';
> + z = neg_z;
> + }
3. This looks like a crutch. Why do you need strlen() and number of
digits here? Firstly, sql_atoi64 already returns an error in case of
too long integer. Secondly, if you wanted to add '-', it could be done
much faster - make 'value = -value;' in case of 'neg' with a preliminary
check that the value is not too big. Will it work?
> + bool unused;
> + int rc = sql_atoi64(z, &value, &unused, len);
> + if (rc != 0) {
> +int_overflow:
> + diag_set(ClientError, ER_INT_LITERAL_MAX, z, INT64_MIN,
> + INT64_MAX);
> parse->is_aborted = true;
> - } else {
> - if (is_neg)
> - value = c == 2 ? SMALLEST_INT64 : -value;
> - sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0,
> - (u8 *)&value, P4_INT64);
> + return;
> }
> }
> + sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0, (u8 *)&value, P4_INT64);
> }> diff --git a/src/box/sql/util.c b/src/box/sql/util.c
> index d5c93f8aa..2e7c298e1 100644
> --- a/src/box/sql/util.c
> +++ b/src/box/sql/util.c
> @@ -593,120 +593,42 @@ sqlAtoF(const char *z, double *pResult, int length)
> int
> -sql_atoi64(const char *z, int64_t *val, int length)
> +sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
> {
> - int incr = 1;
> - u64 u = 0;
> - int neg = 0; /* assume positive */
> - int i;
> - int c = 0;
> - int nonNum = 0; /* True if input contains UTF16 with high byte non-zero */
> - const char *zStart;
> - const char *zEnd = z + length;
> - incr = 1;
> - while (z < zEnd && sqlIsspace(*z))
> - z += incr;
> - if (z < zEnd) {
> - if (*z == '-') {
> - neg = 1;
> - z += incr;
> - } else if (*z == '+') {
> - z += incr;
> - }
> - }
> - zStart = z;
> - /* Skip leading zeros. */
> - while (z < zEnd && z[0] == '0') {
> - z += incr;
> - }
> - for (i = 0; &z[i] < zEnd && (c = z[i]) >= '0' && c <= '9';
> - i += incr) {
> - u = u * 10 + c - '0';
> - }
> - if (u > LARGEST_INT64) {
> - *val = neg ? SMALLEST_INT64 : LARGEST_INT64;
> - } else if (neg) {
> - *val = -(i64) u;
> + *is_neg = false;
> + const char *str_end = z + length;
> + for (; z < str_end && sqlIsspace(*z); z++)
> + /* invalid format */
4. Usually we start sentences from capital letters and finish
with dots.
> + if (z >= str_end)
> + return -1;
> + if (*z == '-')
> + *is_neg = true;
> +
> + char* end = NULL;
> + errno = 0;
> + if (*is_neg) {
5. I don't like this double check. 4 lines above you already
know if the value is negative, but you double check it here.
> + *val = strtoll(z, &end, 10);
> } else {
> - *val = (i64) u;
> + uint64_t u_val = strtoull(z, &end, 10);
> + if (u_val > INT64_MAX)
> + return -1;
> + *val = u_val;
> }
> - if (&z[i] < zEnd || (i == 0 && zStart == z) || i > 19 * incr ||
> - nonNum) {
> - /* zNum is empty or contains non-numeric text or is longer
> - * than 19 digits (thus guaranteeing that it is too large)
> - */
> - return 1;
> - } else if (i < 19 * incr) {
> - /* Less than 19 digits, so we know that it fits in 64 bits */
> - assert(u <= LARGEST_INT64);
> - return 0;
> - } else {
> - /* zNum is a 19-digit numbers. Compare it against 9223372036854775808. */
> - c = compare2pow63(z, incr);
> - if (c < 0) {
> - /* zNum is less than 9223372036854775808 so it fits */
> - assert(u <= LARGEST_INT64);
> - return 0;
> - } else if (c > 0) {
> - /* zNum is greater than 9223372036854775808 so it overflows */
> - return 1;
> - } else {
> - /* zNum is exactly 9223372036854775808. Fits if negative. The
> - * special case 2 overflow if positive
> - */
> - assert(u - 1 == LARGEST_INT64);
> - return neg ? 0 : 2;
> - }
> - }
> -}
> + /* No digits were found, e.g. an empty string. */
> + if (end == z)
> + return -1;
6. You have already checked emptiness above with
this condition: 'if (z >= str_end)'.
> + /*
> + * String has been parsed, but there's
> + * additional character(s) remained.
> + */
> + if (*end != '\0')
> + return -1;
7. What if other symbols are spaces?
> + /* Overflow and underflow errors. */
> + if (errno != 0)
> + return -1;
8. You didn't remove 'sql_atoi64() == SQL_OK' in some
places.
Consider my review fixes here and on the branch in a
separate commit. These changes and the comments are arguable,
probably I am wrong somewhere.
=============================================================
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 158631416..0df6c16d1 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3363,8 +3363,7 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
z = neg_z;
}
bool unused;
- int rc = sql_atoi64(z, &value, &unused, len);
- if (rc != 0) {
+ if (sql_atoi64(z, &value, &unused, len) != 0) {
int_overflow:
diag_set(ClientError, ER_INT_LITERAL_MAX, z, INT64_MIN,
INT64_MAX);
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 2e7c298e1..de469ec36 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -38,6 +38,7 @@
*/
#include "sqlInt.h"
#include <stdarg.h>
+#include <ctype.h>
#if HAVE_ISNAN || SQL_HAVE_ISNAN
#include <math.h>
#endif
@@ -596,38 +597,29 @@ sqlAtoF(const char *z, double *pResult, int length)
int
sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
{
- *is_neg = false;
const char *str_end = z + length;
- for (; z < str_end && sqlIsspace(*z); z++)
- /* invalid format */
+ for (; z < str_end && isspace(*z); z++)
if (z >= str_end)
return -1;
- if (*z == '-')
- *is_neg = true;
-
- char* end = NULL;
+ char *end = NULL;
errno = 0;
- if (*is_neg) {
+ if (*z == '-') {
+ *is_neg = true;
*val = strtoll(z, &end, 10);
} else {
+ *is_neg = false;
uint64_t u_val = strtoull(z, &end, 10);
if (u_val > INT64_MAX)
return -1;
*val = u_val;
}
- /* No digits were found, e.g. an empty string. */
- if (end == z)
- return -1;
- /*
- * String has been parsed, but there's
- * additional character(s) remained.
- */
- if (*end != '\0')
- return -1;
/* Overflow and underflow errors. */
if (errno != 0)
return -1;
-
+ for (; *end != 0; ++end) {
+ if (! isspace(*end))
+ return -1;
+ }
return 0;
}
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6ecdb26fc..11a92372b 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -386,7 +386,7 @@ static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
return 0;
bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg, pMem->n)==SQL_OK)
+ if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg, pMem->n) == 0)
return MEM_Int;
return MEM_Real;
}
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 3a361d066..503f199f1 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -467,7 +467,7 @@ sqlVdbeIntValue(Mem * pMem, int64_t *i)
} else if (flags & (MEM_Str)) {
assert(pMem->z || pMem->n == 0);
bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *)i, &is_neg, pMem->n) == 0)
+ if (sql_atoi64(pMem->z, i, &is_neg, pMem->n) == 0)
return 0;
}
return -1;
@@ -588,8 +588,8 @@ sqlVdbeMemNumerify(Mem * pMem)
if ((pMem->flags & (MEM_Int | MEM_Real | MEM_Null)) == 0) {
assert((pMem->flags & (MEM_Blob | MEM_Str)) != 0);
bool is_neg;
- if (0 == sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg,
- pMem->n)) {
+ if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg,
+ pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Int);
} else {
double v;
@@ -655,8 +655,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return SQL_OK;
if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
- pMem->n) == 0) {
+ if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg, pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
pMem->u.r = pMem->u.i;
return 0;
@@ -688,7 +687,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
case FIELD_TYPE_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
+ if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg,
pMem->n) != 0)
return -1;
MemSetTypeFlag(pMem, MEM_Int);
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 1/6] sql: refactor sql_atoi64()
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-01 14:20 ` n.pettik
2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-01 14:20 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index ba7eea59d..158631416 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c> @@ -3325,29 +3326,53 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
>> if (is_neg)
>> i = -i;
>> sqlVdbeAddOp2(v, OP_Integer, i, mem);
>> + return;
>> + }
>> + int64_t value;
>> + const char *z = expr->u.zToken;
>> + assert(z != NULL);
>> + const char *sign = is_neg ? "-" : "";
>> + if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
>> + uint64_t u = 0;
>> + int i, k;
>> + for (i = 2; z[i] == '0'; i++);
>> + for (k = i; sqlIsxdigit(z[k]); k++)
>> + u = u * 16 + sqlHexToInt(z[k]);
>
> 1. Why not to use strtoll and strtol? They accept 'base' argument.
> Set it to 16, and they will do the same, but probably faster.
NP:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 158631416..182950e9f 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3333,13 +3333,16 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
assert(z != NULL);
const char *sign = is_neg ? "-" : "";
if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
- uint64_t u = 0;
- int i, k;
- for (i = 2; z[i] == '0'; i++);
- for (k = i; sqlIsxdigit(z[k]); k++)
- u = u * 16 + sqlHexToInt(z[k]);
- memcpy(&value, &u, 8);
- if (z[k] != 0 || k - i > 16) {
+ errno = 0;
+ char *end = NULL;
+ if (is_neg) {
+ value = strtoll(z, &end, 16);
+ } else {
+ value = strtoull(z, &end, 16);
+ if (value > INT64_MAX)
+ goto int_overflow;
+ }
+ if (errno != 0) {
diag_set(ClientError, ER_HEX_LITERAL_MAX, sign, z,
strlen(z) - 2, 16);
parse->is_aborted = true;
>
>> + memcpy(&value, &u, 8);
>> + if (z[k] != 0 || k - i > 16) {
>
> 2. You treat tail spaces as an error. Not sure if it is right.
This code has been removed (see fix above).
>
>> + diag_set(ClientError, ER_HEX_LITERAL_MAX, sign, z,
>> + strlen(z) - 2, 16);
>> + parse->is_aborted = true;
>> + return;
>> + }
>> } else {
>> - int64_t value;
>> - const char *z = expr->u.zToken;
>> - assert(z != NULL);
>> - int c = sql_dec_or_hex_to_i64(z, &value);
>> - if (c == 1 || (c == 2 && !is_neg) ||
>> - (is_neg && value == SMALLEST_INT64)) {
>> - const char *sign = is_neg ? "-" : "";
>> - if (sql_strnicmp(z, "0x", 2) == 0) {
>> - diag_set(ClientError, ER_HEX_LITERAL_MAX, sign,
>> - z, strlen(z) - 2, 16);
>> - } else {
>> - diag_set(ClientError, ER_INT_LITERAL_MAX, sign,
>> - z, INT64_MIN, INT64_MAX);
>> - }
>> + size_t len = strlen(z);
>> + if (is_neg) {
>> + /*
>> + * UINT64_MAX is constant, so each integer
>> + * literal certainly contains less digits.
>> + * It makes no sense to continue processing
>> + * string if it contains more characters.
>> + */
>> + if (len > UINT64_MAX_DIGIT_COUNT)
>> + goto int_overflow;
>> + char *neg_z = tt_static_buf();
>> + neg_z[0] = '-';
>> + memcpy(neg_z + 1, z, len++);
>> + neg_z[len]= '\0';
>> + z = neg_z;
>> + }
>
> 3. This looks like a crutch. Why do you need strlen() and number of
> digits here? Firstly, sql_atoi64 already returns an error in case of
> too long integer. Secondly, if you wanted to add '-', it could be done
> much faster - make 'value = -value;' in case of 'neg' with a preliminary
> check that the value is not too big. Will it work?
It will, but we have to allow parsing in sql_atoi() positive values up to
INT64_MAX + 1. So, several tests fail with this change (sql-tap/func.test.lua and
sql/integer-overflow.test.lua). It is not critical, since in the next patch we are
going to allow using integers up to UINT64_MAX. Hence, I don’t think this
deserves any workaround, so I simply fixed tests. Diff:
--- a/src/box/errcode.h
+++ b/src/box/errcode.h
@@ -242,7 +242,7 @@ struct errcode_record {
/*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
/*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
/*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
- /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %lld") \
+ /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range %lld - %lld") \
/*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \
/*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \
/*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 158631416..4bb5ac180 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3347,31 +3350,19 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
}
} else {
size_t len = strlen(z);
- if (is_neg) {
- /*
- * UINT64_MAX is constant, so each integer
- * literal certainly contains less digits.
- * It makes no sense to continue processing
- * string if it contains more characters.
- */
- if (len > UINT64_MAX_DIGIT_COUNT)
- goto int_overflow;
- char *neg_z = tt_static_buf();
- neg_z[0] = '-';
- memcpy(neg_z + 1, z, len++);
- neg_z[len]= '\0';
- z = neg_z;
- }
bool unused;
- int rc = sql_atoi64(z, &value, &unused, len);
- if (rc != 0) {
+ if (sql_atoi64(z, &value, &unused, len) != 0 ||
+ (is_neg && (uint64_t) value > (uint64_t) INT64_MAX + 1) ||
+ (!is_neg && (uint64_t) value > INT64_MAX)) {
int_overflow:
- diag_set(ClientError, ER_INT_LITERAL_MAX, z, INT64_MIN,
- INT64_MAX);
+ diag_set(ClientError, ER_INT_LITERAL_MAX, sign, z,
+ INT64_MIN, INT64_MAX);
parse->is_aborted = true;
return;
}
}
+ if (is_neg)
+ value = -value;
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0, (u8 *)&value, P4_INT64);
}
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 2e7c298e1..a2340f001 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -611,7 +611,7 @@ sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
*val = strtoll(z, &end, 10);
} else {
uint64_t u_val = strtoull(z, &end, 10);
- if (u_val > INT64_MAX)
+ if (u_val > (uint64_t) INT64_MAX + 1)
return -1;
*val = u_val;
}
diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
index 09b1cf967..6cb14f3a3 100755
--- a/test/sql-tap/func.test.lua
+++ b/test/sql-tap/func.test.lua
@@ -912,25 +912,25 @@ test:do_execsql_test(
-- </func-8.6>
})
-test:do_execsql_test(
+test:do_catchsql_test(
"func-8.7",
[[
SELECT typeof(sum(x)) FROM (SELECT '9223372036' || '854775808' AS x
UNION ALL SELECT -9223372036854775807)
]], {
-- <func-8.7>
- "number"
+ 1, "Failed to execute SQL statement: integer overflow"
-- </func-8.7>
})
-test:do_execsql_test(
+test:do_catchsql_test(
"func-8.8",
[[
SELECT sum(x)>0.0 FROM (SELECT '9223372036' || '854775808' AS x
UNION ALL SELECT -9223372036850000000)
]], {
-- <func-8.8>
- 1
+ 1, "Failed to execute SQL statement: integer overflow"
-- </func-8.8>
})
diff --git a/test/sql/integer-overflow.result b/test/sql/integer-overflow.result
index d6ca66ec9..a4944d1a0 100644
--- a/test/sql/integer-overflow.result
+++ b/test/sql/integer-overflow.result
@@ -48,7 +48,11 @@ box.execute('SELECT 9223372036854775808 - 1;')
--
box.execute('SELECT CAST(\'9223372036854775808\' AS INTEGER);')
---
-- error: 'Type mismatch: can not convert 9223372036854775808 to integer'
+- metadata:
+ - name: CAST('9223372036854775808' AS INTEGER)
+ type: integer
+ rows:
+ - [-9223372036854775808]
...
-- Due to inexact represantation of large integers in terms of
-- floating point numbers, numerics with value < INT64_MAX
>> }> diff --git a/src/box/sql/util.c b/src/box/sql/util.c
>> index d5c93f8aa..2e7c298e1 100644
>> --- a/src/box/sql/util.c
>> +++ b/src/box/sql/util.c
>> @@ -593,120 +593,42 @@ sqlAtoF(const char *z, double *pResult, int length)
>> int
>> -sql_atoi64(const char *z, int64_t *val, int length)
>> +sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
>> {
>> - int incr = 1;
>> - u64 u = 0;
>> - int neg = 0; /* assume positive */
>> - int i;
>> - int c = 0;
>> - int nonNum = 0; /* True if input contains UTF16 with high byte non-zero */
>> - const char *zStart;
>> - const char *zEnd = z + length;
>> - incr = 1;
>> - while (z < zEnd && sqlIsspace(*z))
>> - z += incr;
>> - if (z < zEnd) {
>> - if (*z == '-') {
>> - neg = 1;
>> - z += incr;
>> - } else if (*z == '+') {
>> - z += incr;
>> - }
>> - }
>> - zStart = z;
>> - /* Skip leading zeros. */
>> - while (z < zEnd && z[0] == '0') {
>> - z += incr;
>> - }
>> - for (i = 0; &z[i] < zEnd && (c = z[i]) >= '0' && c <= '9';
>> - i += incr) {
>> - u = u * 10 + c - '0';
>> - }
>> - if (u > LARGEST_INT64) {
>> - *val = neg ? SMALLEST_INT64 : LARGEST_INT64;
>> - } else if (neg) {
>> - *val = -(i64) u;
>> + *is_neg = false;
>> + const char *str_end = z + length;
>> + for (; z < str_end && sqlIsspace(*z); z++)
>> + /* invalid format */
>
> 4. Usually we start sentences from capital letters and finish
> with dots.
Refactored a bit whole function (almost applied your diff):
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 2e7c298e1..826e4a673 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -38,6 +38,7 @@
*/
#include "sqlInt.h"
#include <stdarg.h>
+#include <ctype.h>
#if HAVE_ISNAN || SQL_HAVE_ISNAN
#include <math.h>
#endif
@@ -598,26 +599,24 @@ sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
{
*is_neg = false;
const char *str_end = z + length;
- for (; z < str_end && sqlIsspace(*z); z++)
- /* invalid format */
+ for (; z < str_end && isspace(*z); z++);
if (z >= str_end)
return -1;
if (*z == '-')
*is_neg = true;
- char* end = NULL;
+ char *end = NULL;
errno = 0;
- if (*is_neg) {
+ if (*z == '-') {
+ *is_neg = true;
*val = strtoll(z, &end, 10);
} else {
+ *is_neg = false;
uint64_t u_val = strtoull(z, &end, 10);
- if (u_val > INT64_MAX)
+ if (u_val > (uint64_t) INT64_MAX + 1)
return -1;
*val = u_val;
}
- /* No digits were found, e.g. an empty string. */
- if (end == z)
- return -1;
>> + if (z >= str_end)
>> + return -1;
>> + if (*z == '-')
>> + *is_neg = true;
>> +
>> + char* end = NULL;
>> + errno = 0;
>> + if (*is_neg) {
>
> 5. I don't like this double check. 4 lines above you already
> know if the value is negative, but you double check it here.
Ok, see diff above.
>> + *val = strtoll(z, &end, 10);
>> } else {
>> - *val = (i64) u;
>> + uint64_t u_val = strtoull(z, &end, 10);
>> + if (u_val > INT64_MAX)
>> + return -1;
>> + *val = u_val;
>> }
>> - if (&z[i] < zEnd || (i == 0 && zStart == z) || i > 19 * incr ||
>> - nonNum) {
>> - /* zNum is empty or contains non-numeric text or is longer
>> - * than 19 digits (thus guaranteeing that it is too large)
>> - */
>> - return 1;
>> - } else if (i < 19 * incr) {
>> - /* Less than 19 digits, so we know that it fits in 64 bits */
>> - assert(u <= LARGEST_INT64);
>> - return 0;
>> - } else {
>> - /* zNum is a 19-digit numbers. Compare it against 9223372036854775808. */
>> - c = compare2pow63(z, incr);
>> - if (c < 0) {
>> - /* zNum is less than 9223372036854775808 so it fits */
>> - assert(u <= LARGEST_INT64);
>> - return 0;
>> - } else if (c > 0) {
>> - /* zNum is greater than 9223372036854775808 so it overflows */
>> - return 1;
>> - } else {
>> - /* zNum is exactly 9223372036854775808. Fits if negative. The
>> - * special case 2 overflow if positive
>> - */
>> - assert(u - 1 == LARGEST_INT64);
>> - return neg ? 0 : 2;
>> - }
>> - }
>> -}
>> + /* No digits were found, e.g. an empty string. */
>> + if (end == z)
>> + return -1;
>
> 6. You have already checked emptiness above with
> this condition: 'if (z >= str_end)’.
Indeed, see diff above.
>> + /*
>> + * String has been parsed, but there's
>> + * additional character(s) remained.
>> + */
>> + if (*end != '\0')
>> + return -1;
>
> 7. What if other symbols are spaces?
So, you are speaking about situation like:
SELECT CAST(“123 “ AS INTEGER);
According to ANSI cast is allowed in this case:
…
8) If TD is exact numeric, then
…
b) If SD is character string, then SV is replaced by SV with any leading or trailing s removed.
Hence, you are right, we should skip trailing space as well.
Applied your diff:
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 826e4a673..e73fe3f87 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -618,21 +618,18 @@ sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
*val = u_val;
}
- /*
- * String has been parsed, but there's
- * additional character(s) remained.
- */
- if (*end != '\0')
- return -1;
/* Overflow and underflow errors. */
if (errno != 0)
return -1;
+ for (; *end != 0; ++end) {
+ if (! isspace(*end))
+ return -1;
+ }
return 0;
}
>> + /* Overflow and underflow errors. */
>> + if (errno != 0)
>> + return -1;
>
> 8. You didn't remove 'sql_atoi64() == SQL_OK' in some
> places.
Thx, applied this part of your diff.
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 6ecdb26fc..11a92372b 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -386,7 +386,7 @@ static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
> if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
> return 0;
> bool is_neg;
> - if (sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg, pMem->n)==SQL_OK)
> + if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg, pMem->n) == 0)
This cast is required.. Otherwise on linux compilation error is raised:
/tarantool/src/box/sql/vdbe.c:387:26: error: passing argument 2 of ‘sql_atoi64’ from incompatible pointer type [-Werror=incompatible-pointer-types]
if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg, pMem->n) == 0)
^
In file included from /tarantool/src/box/sql/vdbe.c:47:0:
/tarantool/src/box/sql/sqlInt.h:4435:1: note: expected ‘int64_t * {aka long int *}’ but argument is of type ‘i64 * {aka long long int *}’
sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length);
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 1/6] sql: refactor sql_atoi64()
2019-07-01 14:20 ` n.pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
2019-07-05 16:32 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi! Thanks for the fixes!
Consider new ones below, and on the branch
in a separate commit.
=====================================================
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 0df050e7a..bf347b401 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3292,11 +3292,10 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
const char *sign = is_neg ? "-" : "";
if (z[0] == '0' && (z[1] == 'x' || z[1] == 'X')) {
errno = 0;
- char *end = NULL;
if (is_neg) {
- value = strtoll(z, &end, 16);
+ value = strtoll(z, NULL, 16);
} else {
- value = strtoull(z, &end, 16);
+ value = strtoull(z, NULL, 16);
if (value > INT64_MAX)
goto int_overflow;
}
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
2019-06-07 15:37 ` [tarantool-patches] [PATCH 1/6] sql: refactor sql_atoi64() Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
2019-06-07 15:37 ` [tarantool-patches] [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints Nikita Pettik
` (4 subsequent siblings)
6 siblings, 1 reply; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
As it was stated in the previous commit message, we are going to support
operations on unsigned values. Since unsigned and signed integers have
different memory representations, to provide correct results of
arithmetic operations we should be able to tell whether value is signed
or not.
This patch introduces new type of value placed in VDBE memory cell -
MEM_UInt. This flag means that value is integer and greater than zero,
hence can be fitted in range [0, 2^64 - 1]. Such approach would make
further replacing MEM_* flags with MP_ format types quite easy: during
decoding and encoding msgpack we assume that negative integers have
MP_INT type and positive - MP_UINT. We also add and refactor several
auxiliary helpers to operate on integers. Note that current changes
don't add ability to operate on unsigned integers - it is still
unavailable.
Needed for #3810
Needed for #4015
---
src/box/execute.c | 19 ++--
src/box/lua/lua_sql.c | 3 +
src/box/sql/func.c | 14 ++-
src/box/sql/vdbe.c | 199 +++++++++++++++++++++++------------------
src/box/sql/vdbeInt.h | 17 +++-
src/box/sql/vdbeapi.c | 13 +--
src/box/sql/vdbeaux.c | 46 +++++++---
src/box/sql/vdbemem.c | 131 ++++++++++++++-------------
test/sql-tap/position.test.lua | 2 +-
test/sql/types.result | 12 +--
10 files changed, 265 insertions(+), 191 deletions(-)
diff --git a/src/box/execute.c b/src/box/execute.c
index a3d4a92b8..f5aead391 100644
--- a/src/box/execute.c
+++ b/src/box/execute.c
@@ -138,17 +138,20 @@ sql_column_to_messagepack(struct sql_stmt *stmt, int i,
switch (type) {
case MP_INT: {
int64_t n = sql_column_int64(stmt, i);
- if (n >= 0)
- size = mp_sizeof_uint(n);
- else
- size = mp_sizeof_int(n);
+ size = mp_sizeof_int(n);
char *pos = (char *) region_alloc(region, size);
if (pos == NULL)
goto oom;
- if (n >= 0)
- mp_encode_uint(pos, n);
- else
- mp_encode_int(pos, n);
+ mp_encode_int(pos, n);
+ break;
+ }
+ case MP_UINT: {
+ uint64_t n = sql_column_int64(stmt, i);
+ size = mp_sizeof_uint(n);
+ char *pos = (char *) region_alloc(region, size);
+ if (pos == NULL)
+ goto oom;
+ mp_encode_uint(pos, n);
break;
}
case MP_DOUBLE: {
diff --git a/src/box/lua/lua_sql.c b/src/box/lua/lua_sql.c
index 36b75ff08..59ea260bf 100644
--- a/src/box/lua/lua_sql.c
+++ b/src/box/lua/lua_sql.c
@@ -60,6 +60,9 @@ lua_sql_call(sql_context *pCtx, int nVal, sql_value **apVal) {
case MP_INT:
luaL_pushint64(L, sql_value_int64(param));
break;
+ case MP_UINT:
+ luaL_pushuint64(L, sql_value_int64(param));
+ break;
case MP_DOUBLE:
lua_pushnumber(L, sql_value_double(param));
break;
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index bb7405e68..f4c1cbcca 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -111,6 +111,7 @@ typeofFunc(sql_context * context, int NotUsed, sql_value ** argv)
UNUSED_PARAMETER(NotUsed);
switch (sql_value_type(argv[0])) {
case MP_INT:
+ case MP_UINT:
z = "integer";
break;
case MP_STR:
@@ -145,6 +146,7 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv)
switch (sql_value_type(argv[0])) {
case MP_BIN:
case MP_INT:
+ case MP_UINT:
case MP_DOUBLE:{
sql_result_int(context,
sql_value_bytes(argv[0]));
@@ -177,6 +179,7 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
assert(argc == 1);
UNUSED_PARAMETER(argc);
switch (sql_value_type(argv[0])) {
+ case MP_UINT:
case MP_INT:{
i64 iVal = sql_value_int64(argv[0]);
if (iVal < 0) {
@@ -1041,6 +1044,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
SQL_TRANSIENT);
break;
}
+ case MP_UINT:
case MP_INT:{
sql_result_value(context, argv[0]);
break;
@@ -1442,7 +1446,8 @@ trim_func_two_args(struct sql_context *context, int argc, sql_value **argv)
return;
int input_str_sz = sql_value_bytes(argv[1]);
- if (sql_value_type(argv[0]) == MP_INT) {
+ if (sql_value_type(argv[0]) == MP_INT ||
+ sql_value_type(argv[0]) == MP_UINT) {
uint8_t len_one = 1;
trim_procedure(context, sql_value_int(argv[0]),
(const unsigned char *) " ", &len_one, 1,
@@ -1473,7 +1478,8 @@ trim_func_three_args(struct sql_context *context, int argc, sql_value **argv)
assert(argc == 3);
(void) argc;
- assert(sql_value_type(argv[0]) == MP_INT);
+ assert(sql_value_type(argv[0]) == MP_INT ||
+ sql_value_type(argv[0]) == MP_UINT);
const unsigned char *input_str, *trim_set;
if ((input_str = sql_value_text(argv[2])) == NULL ||
(trim_set = sql_value_text(argv[1])) == NULL)
@@ -1601,7 +1607,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
int type = sql_value_type(argv[0]);
if (type == MP_NIL || p == NULL)
return;
- if (type != MP_DOUBLE && type != MP_INT) {
+ if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
if (mem_apply_numeric_type(argv[0]) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(argv[0]), "number");
@@ -1612,7 +1618,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
type = sql_value_type(argv[0]);
}
p->cnt++;
- if (type == MP_INT) {
+ if (type == MP_INT || type == MP_UINT) {
int64_t v = sql_value_int64(argv[0]);
p->rSum += v;
if ((p->approx | p->overflow) == 0 &&
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6ecdb26fc..d141397a0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -276,13 +276,12 @@ allocateCursor(
int
mem_apply_numeric_type(struct Mem *record)
{
- if ((record->flags & (MEM_Str | MEM_Int | MEM_Real)) != MEM_Str)
+ if ((record->flags & (MEM_Str | MEM_Int | MEM_Real | MEM_UInt)) != MEM_Str)
return -1;
int64_t integer_value;
bool is_neg;
if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
- record->u.i = integer_value;
- MemSetTypeFlag(record, MEM_Int);
+ mem_set_int(record, integer_value, is_neg);
return 0;
}
double float_value;
@@ -325,12 +324,12 @@ mem_apply_type(struct Mem *record, enum field_type type)
case FIELD_TYPE_UNSIGNED:
if ((record->flags & MEM_Int) == MEM_Int)
return 0;
+ if ((record->flags & MEM_UInt) == MEM_UInt)
+ return 0;
if ((record->flags & MEM_Real) == MEM_Real) {
int64_t i = (int64_t) record->u.r;
- if (i == record->u.r) {
- record->u.i = i;
- MemSetTypeFlag(record, MEM_Int);
- }
+ if (i == record->u.r)
+ mem_set_int(record, i, i < 0);
return 0;
}
return sqlVdbeMemIntegerify(record, false);
@@ -339,7 +338,7 @@ mem_apply_type(struct Mem *record, enum field_type type)
return 0;
return -1;
case FIELD_TYPE_NUMBER:
- if ((record->flags & (MEM_Real | MEM_Int)) != 0)
+ if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
return 0;
return sqlVdbeMemRealify(record);
case FIELD_TYPE_STRING:
@@ -349,10 +348,10 @@ mem_apply_type(struct Mem *record, enum field_type type)
* NULL do not get converted).
*/
if ((record->flags & MEM_Str) == 0) {
- if ((record->flags & (MEM_Real | MEM_Int)))
+ if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)))
sqlVdbeMemStringify(record, 1);
}
- record->flags &= ~(MEM_Real | MEM_Int);
+ record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
return 0;
case FIELD_TYPE_SCALAR:
return 0;
@@ -381,13 +380,13 @@ sql_value_apply_type(
*/
static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
{
- assert((pMem->flags & (MEM_Int|MEM_Real))==0);
+ assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real))==0);
assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
return 0;
bool is_neg;
if (sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg, pMem->n)==SQL_OK)
- return MEM_Int;
+ return is_neg ? MEM_Int : MEM_UInt;
return MEM_Real;
}
@@ -400,8 +399,8 @@ static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
*/
static u16 numericType(Mem *pMem)
{
- if (pMem->flags & (MEM_Int|MEM_Real)) {
- return pMem->flags & (MEM_Int|MEM_Real);
+ if (pMem->flags & (MEM_Int| MEM_UInt | MEM_Real)) {
+ return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
}
if (pMem->flags & (MEM_Str|MEM_Blob)) {
return computeNumericType(pMem);
@@ -506,6 +505,8 @@ memTracePrint(Mem *p)
printf(" si:%lld", p->u.i);
} else if (p->flags & MEM_Int) {
printf(" i:%lld", p->u.i);
+ } else if (p->flags & MEM_UInt) {
+ printf(" u:%"PRIu64"", p->u.u);
} else if (p->flags & MEM_Real) {
printf(" r:%g", p->u.r);
} else if (p->flags & MEM_Bool) {
@@ -604,6 +605,8 @@ mem_type_to_str(const struct Mem *p)
return "TEXT";
case MEM_Int:
return "INTEGER";
+ case MEM_UInt:
+ return "UNSIGNED";
case MEM_Real:
return "REAL";
case MEM_Blob:
@@ -858,8 +861,9 @@ case OP_Gosub: { /* jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
memAboutToChange(p, pIn1);
- pIn1->flags = MEM_Int;
- pIn1->u.i = (int)(pOp-aOp);
+ pIn1->flags = MEM_UInt;
+ assert((pOp-aOp) >= 0);
+ pIn1->u.u = (int)(pOp-aOp);
REGISTER_TRACE(pOp->p1, pIn1);
/* Most jump operations do a goto to this spot in order to update
@@ -877,8 +881,8 @@ case OP_Gosub: { /* jump */
*/
case OP_Return: { /* in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags==MEM_Int);
- pOp = &aOp[pIn1->u.i];
+ assert(pIn1->flags==MEM_UInt);
+ pOp = &aOp[pIn1->u.u];
pIn1->flags = MEM_Undefined;
break;
}
@@ -900,8 +904,8 @@ case OP_InitCoroutine: { /* jump */
assert(pOp->p3>=0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
assert(!VdbeMemDynamic(pOut));
- pOut->u.i = pOp->p3 - 1;
- pOut->flags = MEM_Int;
+ pOut->u.u = pOp->p3 - 1;
+ pOut->flags = MEM_UInt;
if (pOp->p2) goto jump_to_p2;
break;
}
@@ -917,9 +921,9 @@ case OP_InitCoroutine: { /* jump */
case OP_EndCoroutine: { /* in1 */
VdbeOp *pCaller;
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags==MEM_Int);
- assert(pIn1->u.i>=0 && pIn1->u.i<p->nOp);
- pCaller = &aOp[pIn1->u.i];
+ assert(pIn1->flags == MEM_UInt);
+ assert(pIn1->u.u < (uint64_t) p->nOp);
+ pCaller = &aOp[pIn1->u.u];
assert(pCaller->opcode==OP_Yield);
assert(pCaller->p2>=0 && pCaller->p2<p->nOp);
pOp = &aOp[pCaller->p2 - 1];
@@ -944,9 +948,9 @@ case OP_Yield: { /* in1, jump */
int pcDest;
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
- pIn1->flags = MEM_Int;
- pcDest = (int)pIn1->u.i;
- pIn1->u.i = (int)(pOp - aOp);
+ pIn1->flags = MEM_UInt;
+ pcDest = (int)pIn1->u.u;
+ pIn1->u.u = (int)(pOp - aOp);
REGISTER_TRACE(pOp->p1, pIn1);
pOp = &aOp[pcDest];
break;
@@ -1074,7 +1078,12 @@ case OP_Halt: {
*/
case OP_Integer: { /* out2 */
pOut = out2Prerelease(p, pOp);
- pOut->u.i = pOp->p1;
+ if (pOp->p1 < 0) {
+ pOut->u.i = pOp->p1;
+ } else {
+ pOut->u.u = pOp->p1;
+ MemSetTypeFlag(pOut, MEM_UInt);
+ }
break;
}
@@ -1099,7 +1108,7 @@ case OP_Bool: { /* out2 */
case OP_Int64: { /* out2 */
pOut = out2Prerelease(p, pOp);
assert(pOp->p4.pI64!=0);
- pOut->u.i = *pOp->p4.pI64;
+ mem_set_int(pOut, *pOp->p4.pI64, *pOp->p4.pI64 < 0);
break;
}
@@ -1187,8 +1196,7 @@ case OP_NextAutoincValue: {
}
pOut = out2Prerelease(p, pOp);
- pOut->flags = MEM_Int;
- pOut->u.i = value;
+ mem_set_int(pOut, value, value < 0);
break;
}
@@ -1384,9 +1392,9 @@ case OP_SCopy: { /* out2 */
*/
case OP_IntCopy: { /* out2 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & MEM_Int)!=0);
+ assert((pIn1->flags & (MEM_Int | MEM_UInt))!=0);
pOut = &aMem[pOp->p2];
- sqlVdbeMemSetInt64(pOut, pIn1->u.i);
+ mem_set_int(pOut, pIn1->u.i, pIn1->flags & MEM_Int);
break;
}
@@ -1609,7 +1617,8 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
pOut = &aMem[pOp->p3];
flags = pIn1->flags | pIn2->flags;
if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
- if ((type1 & type2 & MEM_Int)!=0) {
+ if ((type1 & (MEM_Int | MEM_UInt)) !=0 &&
+ (type2 & (MEM_Int | MEM_UInt)) !=0) {
iA = pIn1->u.i;
iB = pIn2->u.i;
bIntint = 1;
@@ -1632,8 +1641,7 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
break;
}
}
- pOut->u.i = iB;
- MemSetTypeFlag(pOut, MEM_Int);
+ mem_set_int(pOut, iB, iB < 0);
} else {
bIntint = 0;
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
@@ -1874,13 +1882,14 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
sqlVdbeMemSetNull(pOut);
break;
}
- if (sqlVdbeIntValue(pIn2, (int64_t *) &iA) != 0) {
+ bool unused;
+ if (sqlVdbeIntValue(pIn2, &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn2), "integer");
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
- if (sqlVdbeIntValue(pIn1, (int64_t *) &iB) != 0) {
+ if (sqlVdbeIntValue(pIn1, &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn1), "integer");
rc = SQL_TARANTOOL_ERROR;
@@ -1915,8 +1924,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
memcpy(&iA, &uA, sizeof(iA));
}
}
- pOut->u.i = iA;
- MemSetTypeFlag(pOut, MEM_Int);
+ mem_set_int(pOut, iA, iA < 0);
break;
}
@@ -1924,15 +1932,14 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
* Synopsis: r[P1]=r[P1]+P2
*
* Add the constant P2 to the value in register P1.
- * The result is always an integer.
- *
- * To force any register to be an integer, just add 0.
+ * Content of register P1 and value P2 are assumed to be
+ * unsigned.
*/
case OP_AddImm: { /* in1 */
pIn1 = &aMem[pOp->p1];
memAboutToChange(p, pIn1);
- assert((pIn1->flags & MEM_Int) != 0);
- pIn1->u.i += pOp->p2;
+ assert((pIn1->flags & MEM_UInt) != 0 && pOp->p2 >= 0);
+ pIn1->u.u += pOp->p2;
break;
}
@@ -1945,10 +1952,9 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if ((pIn1->flags & MEM_Int)==0) {
+ if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- VdbeBranchTaken((pIn1->flags&MEM_Int)==0, 2);
- if ((pIn1->flags & MEM_Int)==0) {
+ if ((pIn1->flags & (MEM_Int | MEM_UInt))== 0) {
if (pOp->p2==0) {
rc = SQL_MISMATCH;
goto abort_due_to_error;
@@ -1957,7 +1963,6 @@ case OP_MustBeInt: { /* jump, in1 */
}
}
}
- MemSetTypeFlag(pIn1, MEM_Int);
break;
}
@@ -1972,7 +1977,7 @@ case OP_MustBeInt: { /* jump, in1 */
*/
case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
- if (pIn1->flags & MEM_Int) {
+ if ((pIn1->flags & (MEM_Int | MEM_UInt)) != 0) {
sqlVdbeMemRealify(pIn1);
}
break;
@@ -2166,12 +2171,12 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (sql_type_is_numeric(type)) {
if ((flags1 | flags3)&MEM_Str) {
- if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((flags1 & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
mem_apply_numeric_type(pIn1);
testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
flags3 = pIn3->flags;
}
- if ((flags3 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((flags3 & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
if (mem_apply_numeric_type(pIn3) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
@@ -2186,14 +2191,14 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
/* Handle the common case of integer comparison here, as an
* optimization, to avoid a call to sqlMemCompare()
*/
- if ((pIn1->flags & pIn3->flags & MEM_Int)!=0) {
+ if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt))!=0) {
if (pIn3->u.i > pIn1->u.i) { res = +1; goto compare_op; }
if (pIn3->u.i < pIn1->u.i) { res = -1; goto compare_op; }
res = 0;
goto compare_op;
}
} else if (type == FIELD_TYPE_STRING) {
- if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_Real))!=0) {
+ if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_UInt|MEM_Real))!=0) {
testcase( pIn1->flags & MEM_Int);
testcase( pIn1->flags & MEM_Real);
sqlVdbeMemStringify(pIn1, 1);
@@ -2201,7 +2206,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
assert(pIn1!=pIn3);
}
- if ((flags3 & MEM_Str)==0 && (flags3 & (MEM_Int|MEM_Real))!=0) {
+ if ((flags3 & MEM_Str)==0 && (flags3 & (MEM_Int|MEM_UInt|MEM_Real))!=0) {
testcase( pIn3->flags & MEM_Int);
testcase( pIn3->flags & MEM_Real);
sqlVdbeMemStringify(pIn3, 1);
@@ -2485,14 +2490,14 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
sqlVdbeMemSetNull(pOut);
if ((pIn1->flags & MEM_Null)==0) {
int64_t i;
- if (sqlVdbeIntValue(pIn1, &i) != 0) {
+ bool is_neg;
+ if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn1), "integer");
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
- pOut->flags = MEM_Int;
- pOut->u.i = ~i;
+ mem_set_int(pOut, ~i, ~i < 0);
}
break;
}
@@ -2750,7 +2755,7 @@ case OP_Column: {
pDest->flags = MEM_Blob|MEM_Ephem|MEM_Subtype;
pDest->subtype = SQL_SUBTYPE_MSGPACK;
}
- if ((pDest->flags & MEM_Int) != 0 &&
+ if ((pDest->flags & (MEM_Int|MEM_UInt)) != 0 &&
pC->eCurType == CURTYPE_TARANTOOL) {
enum field_type f = FIELD_TYPE_ANY;
/*
@@ -2954,7 +2959,9 @@ case OP_Count: { /* out2 */
}
if (rc) goto abort_due_to_error;
pOut = out2Prerelease(p, pOp);
- pOut->u.i = nEntry;
+ assert(nEntry >= 0);
+ pOut->u.u = nEntry;
+ pOut->flags = MEM_UInt;
break;
}
@@ -3499,6 +3506,7 @@ case OP_SeekGT: { /* jump, in3 */
#endif
iKey = 0;
reg_ipk = pOp->p5;
+ bool is_neg = false;
if (reg_ipk > 0) {
@@ -3507,12 +3515,16 @@ case OP_SeekGT: { /* jump, in3 */
* the seek, so convert it.
*/
pIn3 = &aMem[reg_ipk];
- if ((pIn3->flags & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((pIn3->flags & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
mem_apply_numeric_type(pIn3);
}
int64_t i;
if ((pIn3->flags & MEM_Int) == MEM_Int) {
i = pIn3->u.i;
+ is_neg = true;
+ } else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
+ i = pIn3->u.u;
+ is_neg = false;
} else if ((pIn3->flags & MEM_Real) == MEM_Real) {
if (pIn3->u.r > INT64_MAX)
i = INT64_MAX;
@@ -3520,6 +3532,7 @@ case OP_SeekGT: { /* jump, in3 */
i = INT64_MIN;
else
i = pIn3->u.r;
+ is_neg = i < 0;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn3), "integer");
@@ -3531,7 +3544,7 @@ case OP_SeekGT: { /* jump, in3 */
/* If the P3 value could not be converted into an integer without
* loss of information, then special processing is required...
*/
- if ((pIn3->flags & MEM_Int)==0) {
+ if ((pIn3->flags & (MEM_Int | MEM_UInt)) == 0) {
if ((pIn3->flags & MEM_Real)==0) {
/* If the P3 value cannot be converted into any kind of a number,
* then the seek is not possible, so jump to P2
@@ -3587,10 +3600,8 @@ case OP_SeekGT: { /* jump, in3 */
r.key_def = pC->key_def;
r.nField = (u16)nField;
- if (reg_ipk > 0) {
- aMem[reg_ipk].u.i = iKey;
- aMem[reg_ipk].flags = MEM_Int;
- }
+ if (reg_ipk > 0)
+ mem_set_int(&aMem[reg_ipk], iKey, is_neg);
r.default_rc = ((1 & (oc - OP_SeekLT)) ? -1 : +1);
assert(oc!=OP_SeekGT || r.default_rc==-1);
@@ -3801,7 +3812,10 @@ case OP_Sequence: { /* out2 */
assert(pOp->p1>=0 && pOp->p1<p->nCursor);
assert(p->apCsr[pOp->p1]!=0);
pOut = out2Prerelease(p, pOp);
- pOut->u.i = p->apCsr[pOp->p1]->seqCount++;
+ int64_t seq_val = p->apCsr[pOp->p1]->seqCount++;
+ assert(seq_val >= 0);
+ pOut->u.u = seq_val;
+ pOut->flags = MEM_UInt;
break;
}
@@ -3814,10 +3828,10 @@ case OP_Sequence: { /* out2 */
*/
case OP_NextSequenceId: {
pOut = &aMem[pOp->p2];
- tarantoolSqlNextSeqId((uint64_t *) &pOut->u.i);
-
- pOut->u.i += 1;
- pOut->flags = MEM_Int;
+ int64_t id;
+ tarantoolSqlNextSeqId((uint64_t *) &id);
+ id++;
+ mem_set_int(pOut, id, id < 0);
break;
}
@@ -3850,8 +3864,8 @@ case OP_NextIdEphemeral: {
goto abort_due_to_error;
}
pOut = &aMem[pOp->p2];
- pOut->u.i = rowid;
- pOut->flags = MEM_Int;
+ pOut->u.u = rowid;
+ pOut->flags = MEM_UInt;
break;
}
@@ -3883,12 +3897,11 @@ case OP_FCopy: { /* out2 */
/* Flag is set and register is NULL -> do nothing */
} else {
assert(memIsValid(pIn1));
- assert(pIn1->flags & MEM_Int);
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
pOut = &aMem[pOp->p2];
- MemSetTypeFlag(pOut, MEM_Int);
-
- pOut->u.i = pIn1->u.i;
+ mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_UInt ?
+ false : true);
}
break;
}
@@ -5079,10 +5092,17 @@ case OP_FkIfZero: { /* jump */
*/
case OP_IfPos: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags&MEM_Int);
- VdbeBranchTaken( pIn1->u.i>0, 2);
- if (pIn1->u.i>0) {
- pIn1->u.i -= pOp->p3;
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
+ assert(pOp->p3 >= 0);
+ uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
+ /*
+ * To not bother setting integer flag in case
+ * result of subtraction is negative, just
+ * use saturated arithmetic.
+ */
+ res &= -(res <= pIn1->u.u);
+ pIn1->u.u = res;
goto jump_to_p2;
}
break;
@@ -5111,10 +5131,10 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
pOut = out2Prerelease(p, pOp);
- assert(pIn1->flags & MEM_Int);
- assert(pIn3->flags & MEM_Int);
+ assert(pIn1->flags & (MEM_Int | MEM_UInt));
+ assert(pIn3->flags & (MEM_Int | MEM_UInt));
x = pIn1->u.i;
- if (x<=0 || sqlAddInt64(&x, pIn3->u.i>0?pIn3->u.i:0)) {
+ if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
/* If the LIMIT is less than or equal to zero, loop forever. This
* is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
* also loop forever. This is undocumented. In fact, one could argue
@@ -5124,8 +5144,9 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
* looping forever is a reasonable approximation.
*/
pOut->u.i = -1;
+ pOut->flags = MEM_Int;
} else {
- pOut->u.i = x;
+ mem_set_int(pOut, x, x < 0);
}
break;
}
@@ -5140,7 +5161,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags&MEM_Int);
+ assert(pIn1->flags&(MEM_Int|MEM_UInt));
VdbeBranchTaken(pIn1->u.i<0, 2);
if (pIn1->u.i) {
if (pIn1->u.i>0) pIn1->u.i--;
@@ -5157,7 +5178,7 @@ case OP_IfNotZero: { /* jump, in1 */
*/
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags&MEM_Int);
+ assert((pIn1->flags&(MEM_Int|MEM_UInt)) != 0);
if (pIn1->u.i>SMALLEST_INT64) pIn1->u.i--;
VdbeBranchTaken(pIn1->u.i==0, 2);
if (pIn1->u.i==0) goto jump_to_p2;
@@ -5396,12 +5417,12 @@ case OP_Init: { /* jump */
case OP_IncMaxid: {
assert(pOp->p1 > 0);
pOut = &aMem[pOp->p1];
-
- rc = tarantoolsqlIncrementMaxid((uint64_t*) &pOut->u.i);
+ int64_t id;
+ rc = tarantoolsqlIncrementMaxid((uint64_t*) &id);
if (rc!=SQL_OK) {
goto abort_due_to_error;
}
- pOut->flags = MEM_Int;
+ mem_set_int(pOut, id, id < 0);
break;
}
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index a3100e513..abed46486 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -183,6 +183,7 @@ struct Mem {
union MemValue {
double r; /* Real value used when MEM_Real is set in flags */
i64 i; /* Integer value used when MEM_Int is set in flags */
+ uint64_t u; /* Unsigned integer used when MEM_UInt is set. */
bool b; /* Boolean value used when MEM_Bool is set in flags */
int nZero; /* Used when bit MEM_Zero is set in flags */
void *p; /* Generic pointer */
@@ -230,7 +231,7 @@ struct Mem {
#define MEM_Real 0x0008 /* Value is a real number */
#define MEM_Blob 0x0010 /* Value is a BLOB */
#define MEM_Bool 0x0020 /* Value is a bool */
-#define MEM_Ptr 0x0040 /* Value is a generic pointer */
+#define MEM_UInt 0x0040 /* Value is a unsigned integer */
#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
#define MEM_Undefined 0x0100 /* Value is undefined */
#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
@@ -248,6 +249,7 @@ struct Mem {
#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
+#define MEM_Ptr 0x20000 /* Value is a generic pointer */
#ifdef SQL_OMIT_INCRBLOB
#undef MEM_Zero
#define MEM_Zero 0x0000
@@ -259,9 +261,14 @@ struct Mem {
* auxiliary flags.
*/
enum {
- MEM_PURE_TYPE_MASK = 0x3f
+ MEM_PURE_TYPE_MASK = 0x7f
};
+static_assert((int) MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int |
+ MEM_Real | MEM_Blob | MEM_Bool |
+ MEM_UInt),
+ "value of type mask must consist of corresponding to memory type bits");
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -499,13 +506,17 @@ void sqlVdbeMemSetInt64(Mem *, i64);
void
mem_set_bool(struct Mem *mem, bool value);
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
+
void sqlVdbeMemSetDouble(Mem *, double);
void sqlVdbeMemInit(Mem *, sql *, u32);
void sqlVdbeMemSetNull(Mem *);
void sqlVdbeMemSetZeroBlob(Mem *, int);
int sqlVdbeMemMakeWriteable(Mem *);
int sqlVdbeMemStringify(Mem *, u8);
-int sqlVdbeIntValue(Mem *, int64_t *);
+int sqlVdbeIntValue(Mem *, int64_t *, bool *is_neg);
+
int sqlVdbeMemIntegerify(Mem *, bool is_forced);
int sqlVdbeRealValue(Mem *, double *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index e480ae720..393782c23 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -225,7 +225,8 @@ int
sql_value_int(sql_value * pVal)
{
int64_t i = 0;
- sqlVdbeIntValue((Mem *) pVal, &i);
+ bool is_neg;
+ sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
return (int)i;
}
@@ -233,7 +234,8 @@ sql_int64
sql_value_int64(sql_value * pVal)
{
int64_t i = 0;
- sqlVdbeIntValue((Mem *) pVal, &i);
+ bool is_neg;
+ sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
return i;
}
@@ -258,6 +260,7 @@ sql_value_type(sql_value *pVal)
{
switch (pVal->flags & MEM_PURE_TYPE_MASK) {
case MEM_Int: return MP_INT;
+ case MEM_UInt: return MP_UINT;
case MEM_Real: return MP_DOUBLE;
case MEM_Str: return MP_STR;
case MEM_Blob: return MP_BIN;
@@ -385,7 +388,7 @@ sql_result_error(sql_context * pCtx, const char *z, int n)
void
sql_result_int(sql_context * pCtx, int iVal)
{
- sqlVdbeMemSetInt64(pCtx->pOut, (i64) iVal);
+ mem_set_int(pCtx->pOut, (i64) iVal, iVal < 0);
}
void
@@ -397,7 +400,7 @@ sql_result_bool(struct sql_context *ctx, bool value)
void
sql_result_int64(sql_context * pCtx, i64 iVal)
{
- sqlVdbeMemSetInt64(pCtx->pOut, iVal);
+ mem_set_int(pCtx->pOut, iVal, iVal < 0);
}
void
@@ -1376,7 +1379,7 @@ sql_bind_int64(sql_stmt * pStmt, int i, sql_int64 iValue)
rc = vdbeUnbind(p, i);
if (rc == SQL_OK) {
rc = sql_bind_type(p, i, "INTEGER");
- sqlVdbeMemSetInt64(&p->aVar[i - 1], iValue);
+ mem_set_int(&p->aVar[i - 1], iValue, iValue < 0);
}
return rc;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 25d4cd759..5a71e1801 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1369,6 +1369,8 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
zP4 = pMem->z;
} else if (pMem->flags & MEM_Int) {
sqlXPrintf(&x, "%lld", pMem->u.i);
+ } else if (pMem->flags & MEM_UInt) {
+ sqlXPrintf(&x, "%llu", pMem->u.u);
} else if (pMem->flags & MEM_Real) {
sqlXPrintf(&x, "%.16g", pMem->u.r);
} else if (pMem->flags & MEM_Null) {
@@ -1639,6 +1641,8 @@ sqlVdbeList(Vdbe * p)
if (p->explain == 1) {
pMem->flags = MEM_Int;
pMem->u.i = i; /* Program counter */
+ mem_set_int(pMem, i, i < 0);
+
pMem++;
pMem->flags = MEM_Static | MEM_Str | MEM_Term;
@@ -1672,16 +1676,13 @@ sqlVdbeList(Vdbe * p)
}
}
- pMem->flags = MEM_Int;
- pMem->u.i = pOp->p1; /* P1 */
+ mem_set_int(pMem, pOp->p1, pOp->p1 < 0);
pMem++;
- pMem->flags = MEM_Int;
- pMem->u.i = pOp->p2; /* P2 */
+ mem_set_int(pMem, pOp->p2, pOp->p2 < 0);
pMem++;
- pMem->flags = MEM_Int;
- pMem->u.i = pOp->p3; /* P3 */
+ mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
pMem++;
if (sqlVdbeMemClearAndResize(pMem, 256)) {
@@ -3372,7 +3373,7 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
/* At least one of the two values is a number
*/
- if (combined_flags & (MEM_Int | MEM_Real)) {
+ if (combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) {
if ((f1 & f2 & MEM_Int) != 0) {
if (pMem1->u.i < pMem2->u.i)
return -1;
@@ -3380,6 +3381,13 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
return +1;
return 0;
}
+ if ((f1 & f2 & MEM_UInt) != 0) {
+ if (pMem1->u.u < pMem2->u.u)
+ return -1;
+ if (pMem1->u.u > pMem2->u.u)
+ return +1;
+ return 0;
+ }
if ((f1 & f2 & MEM_Real) != 0) {
if (pMem1->u.r < pMem2->u.r)
return -1;
@@ -3395,10 +3403,23 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
return -1;
}
}
+ if ((f1 & MEM_UInt) != 0) {
+ if ((f2 & MEM_Real) != 0) {
+ return sqlIntFloatCompare(pMem1->u.i,
+ pMem2->u.r);
+ } else if ((f2 & MEM_Int) != 0) {
+ return +1;
+ } else {
+ return -1;
+ }
+ }
if ((f1 & MEM_Real) != 0) {
if ((f2 & MEM_Int) != 0) {
return -sqlIntFloatCompare(pMem2->u.i,
pMem1->u.r);
+ } else if ((f2 & MEM_UInt) != 0) {
+ return -sqlIntFloatCompare(pMem2->u.u,
+ pMem1->u.r);
} else {
return -1;
}
@@ -3563,13 +3584,14 @@ sqlVdbeCompareMsgpack(const char **key1,
mem1.u.r = (double)v;
goto do_float;
}
- mem1.u.i = v;
+ mem1.u.u = v;
goto do_int;
}
case MP_INT:{
mem1.u.i = mp_decode_int(&aKey1);
do_int:
- if (pKey2->flags & MEM_Int) {
+ if ((pKey2->flags & MEM_Int) ||
+ (pKey2->flags & MEM_UInt)) {
if (mem1.u.i < pKey2->u.i) {
rc = -1;
} else if (mem1.u.i > pKey2->u.i) {
@@ -3590,7 +3612,7 @@ sqlVdbeCompareMsgpack(const char **key1,
case MP_DOUBLE:{
mem1.u.r = mp_decode_double(&aKey1);
do_float:
- if (pKey2->flags & MEM_Int) {
+ if (pKey2->flags & MEM_Int || pKey2->flags & MEM_UInt) {
rc = -sqlIntFloatCompare(pKey2->u.i,
mem1.u.r);
} else if (pKey2->flags & MEM_Real) {
@@ -3718,8 +3740,8 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
"integer is overflowed");
return -1;
}
- mem->u.i = v;
- mem->flags = MEM_Int;
+ mem->u.u = v;
+ mem->flags = MEM_UInt;
break;
}
case MP_INT: {
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 3a361d066..25119ff16 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -67,6 +67,8 @@ sqlVdbeCheckMemInvariants(Mem * p)
/* Cannot be both MEM_Int and MEM_Real at the same time */
assert((p->flags & (MEM_Int | MEM_Real)) != (MEM_Int | MEM_Real));
+ /* Can't be both UInt and Int at the same time. */
+ assert((p->flags & (MEM_Int | MEM_UInt)) != (MEM_Int | MEM_UInt));
/* The szMalloc field holds the correct memory allocation size */
assert(p->szMalloc == 0
@@ -289,7 +291,7 @@ sqlVdbeMemStringify(Mem * pMem, u8 bForce)
return SQL_OK;
assert(!(fg & MEM_Zero));
- assert(fg & (MEM_Int | MEM_Real | MEM_Bool));
+ assert(fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool));
assert(EIGHT_BYTE_ALIGNMENT(pMem));
if (sqlVdbeMemClearAndResize(pMem, nByte)) {
@@ -297,6 +299,8 @@ sqlVdbeMemStringify(Mem * pMem, u8 bForce)
}
if (fg & MEM_Int) {
sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
+ } else if ((fg & MEM_UInt) != 0) {
+ sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
} else if ((fg & MEM_Bool) != 0) {
sql_snprintf(nByte, pMem->z, "%s", pMem->u.b ? "true" : "false");
} else {
@@ -306,7 +310,7 @@ sqlVdbeMemStringify(Mem * pMem, u8 bForce)
pMem->n = sqlStrlen30(pMem->z);
pMem->flags |= MEM_Str | MEM_Term;
if (bForce)
- pMem->flags &= ~(MEM_Int | MEM_Real);
+ pMem->flags &= ~(MEM_Int | MEM_UInt | MEM_Real);
return SQL_OK;
}
@@ -418,7 +422,7 @@ sqlVdbeMemRelease(Mem * p)
* return the closest available 64-bit signed integer.
*/
static int
-doubleToInt64(double r, int64_t *i)
+doubleToInt64(double r, int64_t *i, bool *is_neg)
{
/*
* Many compilers we encounter do not define constants for the
@@ -429,7 +433,7 @@ doubleToInt64(double r, int64_t *i)
*/
static const int64_t maxInt = LARGEST_INT64;
static const int64_t minInt = SMALLEST_INT64;
-
+ *is_neg = r < 0.f;
if (r <= (double)minInt) {
*i = minInt;
return -1;
@@ -454,20 +458,24 @@ doubleToInt64(double r, int64_t *i)
* If pMem represents a string value, its encoding might be changed.
*/
int
-sqlVdbeIntValue(Mem * pMem, int64_t *i)
+sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
{
int flags;
assert(EIGHT_BYTE_ALIGNMENT(pMem));
flags = pMem->flags;
if (flags & MEM_Int) {
*i = pMem->u.i;
+ *is_neg = true;
+ return 0;
+ } else if (flags & MEM_UInt) {
+ *i = pMem->u.u;
+ *is_neg = false;
return 0;
} else if (flags & MEM_Real) {
- return doubleToInt64(pMem->u.r, i);
+ return doubleToInt64(pMem->u.r, i, is_neg);
} else if (flags & (MEM_Str)) {
assert(pMem->z || pMem->n == 0);
- bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *)i, &is_neg, pMem->n) == 0)
+ if (sql_atoi64(pMem->z, (int64_t *)i, is_neg, pMem->n) == 0)
return 0;
}
return -1;
@@ -489,7 +497,11 @@ sqlVdbeRealValue(Mem * pMem, double *v)
} else if (pMem->flags & MEM_Int) {
*v = (double)pMem->u.i;
return 0;
- } else if (pMem->flags & MEM_Str) {
+ } else if (pMem->flags & MEM_UInt) {
+ *v = (double)pMem->u.u;
+ return 0;
+ }
+ else if (pMem->flags & MEM_Str) {
if (sqlAtoF(pMem->z, v, pMem->n))
return 0;
}
@@ -518,10 +530,9 @@ mem_apply_integer_type(Mem *pMem)
assert(pMem->flags & MEM_Real);
assert(EIGHT_BYTE_ALIGNMENT(pMem));
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
- pMem->u.i = ix;
- MemSetTypeFlag(pMem, MEM_Int);
- }
+ bool is_neg;
+ if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix, &is_neg)) == 0)
+ mem_set_int(pMem, ix, is_neg);
return rc;
}
@@ -534,15 +545,14 @@ sqlVdbeMemIntegerify(Mem * pMem, bool is_forced)
assert(EIGHT_BYTE_ALIGNMENT(pMem));
int64_t i;
- if (sqlVdbeIntValue(pMem, &i) == 0) {
- pMem->u.i = i;
- MemSetTypeFlag(pMem, MEM_Int);
+ bool is_neg;
+ if (sqlVdbeIntValue(pMem, &i, &is_neg) == 0) {
+ mem_set_int(pMem, i, is_neg);
return 0;
} else if ((pMem->flags & MEM_Real) != 0 && is_forced) {
if (pMem->u.r >= INT64_MAX || pMem->u.r < INT64_MIN)
return -1;
- pMem->u.i = (int64_t) pMem->u.r;
- MemSetTypeFlag(pMem, MEM_Int);
+ mem_set_int(pMem, pMem->u.r, pMem->u.r <= -1);
return 0;
}
@@ -551,9 +561,7 @@ sqlVdbeMemIntegerify(Mem * pMem, bool is_forced)
return -1;
if (d >= INT64_MAX || d < INT64_MIN)
return -1;
-
- pMem->u.i = (int64_t) d;
- MemSetTypeFlag(pMem, MEM_Int);
+ mem_set_int(pMem, d, d <= -1);
return 0;
}
@@ -585,12 +593,13 @@ sqlVdbeMemRealify(Mem * pMem)
int
sqlVdbeMemNumerify(Mem * pMem)
{
- if ((pMem->flags & (MEM_Int | MEM_Real | MEM_Null)) == 0) {
+ if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) == 0) {
assert((pMem->flags & (MEM_Blob | MEM_Str)) != 0);
bool is_neg;
if (0 == sql_atoi64(pMem->z, (int64_t *)&pMem->u.i, &is_neg,
pMem->n)) {
- MemSetTypeFlag(pMem, MEM_Int);
+ int flag = is_neg ? MEM_Int : MEM_UInt;
+ MemSetTypeFlag(pMem, flag);
} else {
double v;
if (sqlVdbeRealValue(pMem, &v))
@@ -600,7 +609,7 @@ sqlVdbeMemNumerify(Mem * pMem)
mem_apply_integer_type(pMem);
}
}
- assert((pMem->flags & (MEM_Int | MEM_Real | MEM_Null)) != 0);
+ assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0);
pMem->flags &= ~(MEM_Str | MEM_Blob | MEM_Zero);
return SQL_OK;
}
@@ -671,6 +680,10 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
mem_set_bool(pMem, pMem->u.i);
return 0;
}
+ if ((pMem->flags & MEM_UInt) != 0) {
+ mem_set_bool(pMem, pMem->u.u);
+ return 0;
+ }
if ((pMem->flags & MEM_Real) != 0) {
mem_set_bool(pMem, pMem->u.r);
return 0;
@@ -688,15 +701,16 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
case FIELD_TYPE_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
+ int64_t val;
+ if (sql_atoi64(pMem->z, (int64_t *) &val, &is_neg,
pMem->n) != 0)
return -1;
- MemSetTypeFlag(pMem, MEM_Int);
+ mem_set_int(pMem, val, is_neg);
return 0;
}
if ((pMem->flags & MEM_Bool) != 0) {
- pMem->u.i = pMem->u.b;
- MemSetTypeFlag(pMem, MEM_Int);
+ pMem->u.u = pMem->u.b;
+ MemSetTypeFlag(pMem, MEM_UInt);
return 0;
}
return sqlVdbeMemIntegerify(pMem, true);
@@ -714,7 +728,8 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
sql_value_apply_type(pMem, FIELD_TYPE_STRING);
assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
- pMem->flags &= ~(MEM_Int | MEM_Real | MEM_Blob | MEM_Zero);
+ pMem->flags &=
+ ~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
return SQL_OK;
}
}
@@ -777,42 +792,29 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
pMem->z = 0;
}
-/*
- * The pMem is known to contain content that needs to be destroyed prior
- * to a value change. So invoke the destructor, then set the value to
- * a 64-bit integer.
- */
-static SQL_NOINLINE void
-vdbeReleaseAndSetInt64(Mem * pMem, i64 val)
+void
+mem_set_bool(struct Mem *mem, bool value)
{
- sqlVdbeMemSetNull(pMem);
- pMem->u.i = val;
- pMem->flags = MEM_Int;
+ sqlVdbeMemSetNull(mem);
+ mem->u.b = value;
+ mem->flags = MEM_Bool;
}
-/*
- * Delete any previous value and set the value stored in *pMem to val,
- * manifest type INTEGER.
- */
void
-sqlVdbeMemSetInt64(Mem * pMem, i64 val)
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
{
- if (VdbeMemDynamic(pMem)) {
- vdbeReleaseAndSetInt64(pMem, val);
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ if (is_neg) {
+ assert(value < 0);
+ mem->u.i = value;
+ MemSetTypeFlag(mem, MEM_Int);
} else {
- pMem->u.i = val;
- pMem->flags = MEM_Int;
+ mem->u.u = value;
+ MemSetTypeFlag(mem, MEM_UInt);
}
}
-void
-mem_set_bool(struct Mem *mem, bool value)
-{
- sqlVdbeMemSetNull(mem);
- mem->u.b = value;
- mem->flags = MEM_Bool;
-}
-
/*
* Delete any previous value and set the value stored in *pMem to val,
* manifest type REAL.
@@ -1377,8 +1379,8 @@ valueFromExpr(sql * db, /* The database connection */
if (pVal == 0)
goto no_mem;
if (ExprHasProperty(pExpr, EP_IntValue)) {
- sqlVdbeMemSetInt64(pVal,
- (i64) pExpr->u.iValue * negInt);
+ mem_set_int(pVal, (i64) pExpr->u.iValue * negInt,
+ pExpr->u.iValue * negInt < 0);
} else {
zVal =
sqlMPrintf(db, "%s%s", zNeg, pExpr->u.zToken);
@@ -1797,16 +1799,19 @@ mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
* pass to INT iterator.
*/
i = var->u.r;
- if (i == var->u.r)
+ if (i == var->u.r && i < 0)
goto encode_int;
+ if (i == var->u.r && i >= 0)
+ goto encode_uint;
mpstream_encode_double(stream, var->u.r);
} else if (var->flags & MEM_Int) {
i = var->u.i;
encode_int:
- if (var->u.i >= 0)
- mpstream_encode_uint(stream, i);
- else
- mpstream_encode_int(stream, i);
+ mpstream_encode_int(stream, i);
+ } else if (var->flags & MEM_UInt) {
+ i = var->u.u;
+encode_uint:
+ mpstream_encode_uint(stream, i);
} else if (var->flags & MEM_Str) {
mpstream_encode_strn(stream, var->z, var->n);
} else if (var->flags & MEM_Bool) {
diff --git a/test/sql-tap/position.test.lua b/test/sql-tap/position.test.lua
index 8c46d7b9e..40b8a943b 100755
--- a/test/sql-tap/position.test.lua
+++ b/test/sql-tap/position.test.lua
@@ -228,7 +228,7 @@ test:do_test(
return test:catchsql "SELECT position(34, 12345);"
end, {
-- <position-1.23>
- 1, "Inconsistent types: expected TEXT or BLOB got INTEGER"
+ 1, "Inconsistent types: expected TEXT or BLOB got UNSIGNED"
-- </position-1.23>
})
diff --git a/test/sql/types.result b/test/sql/types.result
index a53d6f7ce..4670fd38a 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -152,7 +152,7 @@ sp:drop()
--
box.execute("SELECT 'abc' || 1;")
---
-- error: 'Inconsistent types: expected TEXT or BLOB got INTEGER'
+- error: 'Inconsistent types: expected TEXT or BLOB got UNSIGNED'
...
box.execute("SELECT 'abc' || 1.123;")
---
@@ -160,7 +160,7 @@ box.execute("SELECT 'abc' || 1.123;")
...
box.execute("SELECT 1 || 'abc';")
---
-- error: 'Inconsistent types: expected TEXT or BLOB got INTEGER'
+- error: 'Inconsistent types: expected TEXT or BLOB got UNSIGNED'
...
box.execute("SELECT 1.123 || 'abc';")
---
@@ -168,7 +168,7 @@ box.execute("SELECT 1.123 || 'abc';")
...
box.execute("SELECt 'a' || 'b' || 1;")
---
-- error: 'Inconsistent types: expected TEXT or BLOB got INTEGER'
+- error: 'Inconsistent types: expected TEXT or BLOB got UNSIGNED'
...
-- What is more, they must be of the same type.
--
@@ -230,11 +230,11 @@ box.execute("INSERT INTO t1 VALUES (1);")
...
box.execute("SELECT * FROM t1 WHERE s LIKE 'int';")
---
-- error: 'Inconsistent types: expected TEXT got INTEGER'
+- error: 'Inconsistent types: expected TEXT got UNSIGNED'
...
box.execute("SELECT * FROM t1 WHERE 'int' LIKE 4;")
---
-- error: 'Inconsistent types: expected TEXT got INTEGER'
+- error: 'Inconsistent types: expected TEXT got UNSIGNED'
...
box.execute("SELECT NULL LIKE s FROM t1;")
---
@@ -355,7 +355,7 @@ box.execute("SELECT unknown = true;")
...
box.execute("SELECT 1 = true;")
---
-- error: 'Type mismatch: can not convert INTEGER to boolean'
+- error: 'Type mismatch: can not convert UNSIGNED to boolean'
...
box.execute("SELECT 'abc' = true;")
---
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints
2019-06-07 15:37 ` [tarantool-patches] [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints Nikita Pettik
@ 2019-06-11 21:11 ` Vladislav Shpilevoy
2019-07-01 14:21 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-06-11 21:11 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! It is a masterpiece!
Excellent! Almost the best patch ever! Buuut ...
see 25 tiny minor comments below, review fixes at
the end of the email, and on the branch in a separate
commit.
On 07/06/2019 18:37, Nikita Pettik wrote:
> As it was stated in the previous commit message, we are going to support
> operations on unsigned values. Since unsigned and signed integers have
> different memory representations, to provide correct results of
> arithmetic operations we should be able to tell whether value is signed
> or not.
> This patch introduces new type of value placed in VDBE memory cell -
> MEM_UInt. This flag means that value is integer and greater than zero,
> hence can be fitted in range [0, 2^64 - 1]. Such approach would make
> further replacing MEM_* flags with MP_ format types quite easy: during
> decoding and encoding msgpack we assume that negative integers have
> MP_INT type and positive - MP_UINT. We also add and refactor several
> auxiliary helpers to operate on integers. Note that current changes
> don't add ability to operate on unsigned integers - it is still
> unavailable.
>
> Needed for #3810
> Needed for #4015
> ---
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index bb7405e68..f4c1cbcca 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -111,6 +111,7 @@ typeofFunc(sql_context * context, int NotUsed, sql_value ** argv)
> UNUSED_PARAMETER(NotUsed);
> switch (sql_value_type(argv[0])) {
> case MP_INT:
> + case MP_UINT:
1. Why on MP_UINT do you return 'integer' instead of
'unsigned'?
> z = "integer";
> break;
> case MP_STR:
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 6ecdb26fc..d141397a0 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -900,8 +904,8 @@ case OP_InitCoroutine: { /* jump */
> assert(pOp->p3>=0 && pOp->p3<p->nOp);
> pOut = &aMem[pOp->p1];
> assert(!VdbeMemDynamic(pOut));
> - pOut->u.i = pOp->p3 - 1;
> - pOut->flags = MEM_Int;
> + pOut->u.u = pOp->p3 - 1;
2. Why are you sure, that p3 >= 1? According to
the assertion above it is >= 0, but we don't know
anything about p3 >= 1.
> + pOut->flags = MEM_UInt;
> if (pOp->p2) goto jump_to_p2;
> break;
> }
> @@ -1099,7 +1108,7 @@ case OP_Bool: { /* out2 */
> case OP_Int64: { /* out2 */
> pOut = out2Prerelease(p, pOp);
> assert(pOp->p4.pI64!=0);
> - pOut->u.i = *pOp->p4.pI64;
> + mem_set_int(pOut, *pOp->p4.pI64, *pOp->p4.pI64 < 0);
3. Sorry, but such places are hell. You pass separately a value
and a flag if it is negative. Please, introduce separate functions
/* Set a value, check sign inside. */
mem_set_i64(int64_t value);
/* Set a non-negative value. */
mem_set_u64(uint64_t value);
/* Set a big integer with sign passed separately. */
mem_set_int(uint64_t value, bool is_negative);
In most places you will use mem_set_i64 and mem_set_u64. It
will look shorter and somewhere work faster.
> break;
> }
>
> @@ -1609,7 +1617,8 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
> pOut = &aMem[pOp->p3];
> flags = pIn1->flags | pIn2->flags;
> if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
> - if ((type1 & type2 & MEM_Int)!=0) {
> + if ((type1 & (MEM_Int | MEM_UInt)) !=0 &&
> + (type2 & (MEM_Int | MEM_UInt)) !=0) {
> iA = pIn1->u.i;
> iB = pIn2->u.i;
4. How can you access pIn1/2->u.i if the types can be MEM_UInt?
> bIntint = 1;
> @@ -2485,14 +2490,14 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
> sqlVdbeMemSetNull(pOut);
> if ((pIn1->flags & MEM_Null)==0) {
> int64_t i;
> - if (sqlVdbeIntValue(pIn1, &i) != 0) {
> + bool is_neg;
> + if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
> diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> sql_value_text(pIn1), "integer");
> rc = SQL_TARANTOOL_ERROR;
> goto abort_due_to_error;
> }
> - pOut->flags = MEM_Int;
> - pOut->u.i = ~i;
> + mem_set_int(pOut, ~i, ~i < 0);
5. Why do you compare 'i' with 0? What if it was a big
unsigned value? You have 'is_neg' flag for this, don't you?
> }
> break;
> }
> @@ -3507,12 +3515,16 @@ case OP_SeekGT: { /* jump, in3 */
> * the seek, so convert it.
> */
> pIn3 = &aMem[reg_ipk];
> - if ((pIn3->flags & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
> + if ((pIn3->flags & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
6. Please, remind me how is it possible, that a value is at the
same time string and number? Why not to check for flags & MEM_Str != 0
only?
> mem_apply_numeric_type(pIn3);
> }
> int64_t i;
> if ((pIn3->flags & MEM_Int) == MEM_Int) {
> i = pIn3->u.i;
> + is_neg = true;
> + } else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
> + i = pIn3->u.u;
> + is_neg = false;
> } else if ((pIn3->flags & MEM_Real) == MEM_Real) {
> if (pIn3->u.r > INT64_MAX)
> i = INT64_MAX;
> @@ -3814,10 +3828,10 @@ case OP_Sequence: { /* out2 */
> */
> case OP_NextSequenceId: {
> pOut = &aMem[pOp->p2];
> - tarantoolSqlNextSeqId((uint64_t *) &pOut->u.i);
> -
> - pOut->u.i += 1;
> - pOut->flags = MEM_Int;
> + int64_t id;
> + tarantoolSqlNextSeqId((uint64_t *) &id);
> + id++;
> + mem_set_int(pOut, id, id < 0);
7. tarantoolSqlNextSeqId returns _sequence.id field. The
field has unsigned type. How is it possible, that it is < 0?
> break;
> }
>
> @@ -5079,10 +5092,17 @@ case OP_FkIfZero: { /* jump */
> */
> case OP_IfPos: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - assert(pIn1->flags&MEM_Int);
> - VdbeBranchTaken( pIn1->u.i>0, 2);
> - if (pIn1->u.i>0) {
> - pIn1->u.i -= pOp->p3;
> + assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
> + if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
> + assert(pOp->p3 >= 0);
> + uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
> + /*
> + * To not bother setting integer flag in case
> + * result of subtraction is negative, just
> + * use saturated arithmetic.
> + */
> + res &= -(res <= pIn1->u.u);
8. I do not understand. Why do you compare res and pIn1->u.u?
You compare 'u.u' and 'u.u - p3'. Obvioiusly, the latter is
smaller. Probably, this is because you rely on 'res' being
overflowed will be bigger than 'u.u'?
In such a case, I think it would be cleaner to compare 'u.u' and
'p3'.
res &= -(u.u > p3);
Additionally, now this opcode's comment is wrong - P1 is not equal
to P1 - P3 from this moment.
> + pIn1->u.u = res;
> goto jump_to_p2;
> }
> break;
> @@ -5111,10 +5131,10 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
> pIn1 = &aMem[pOp->p1];
> pIn3 = &aMem[pOp->p3];
> pOut = out2Prerelease(p, pOp);
> - assert(pIn1->flags & MEM_Int);
> - assert(pIn3->flags & MEM_Int);
> + assert(pIn1->flags & (MEM_Int | MEM_UInt));
> + assert(pIn3->flags & (MEM_Int | MEM_UInt));
> x = pIn1->u.i;
> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i>0?pIn3->u.i:0)) {
> + if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
9. Why do you touch pIn3.u.i without checking its type? In case it
is a big unsigned, you will get 'u.i < 0'. Additionally, as I understand,
now no one should treat 'u.i' as an unsigned. It is supposed to be
negative always.
> /* If the LIMIT is less than or equal to zero, loop forever. This
> * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
> * also loop forever. This is undocumented. In fact, one could argue
> @@ -5124,8 +5144,9 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
> * looping forever is a reasonable approximation.
> */
> pOut->u.i = -1;
> + pOut->flags = MEM_Int;
> } else {
> - pOut->u.i = x;
> + mem_set_int(pOut, x, x < 0);
10. Here 'x' is always >= 0.
> }
> break;
> }
> @@ -5140,7 +5161,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
> */
> case OP_IfNotZero: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - assert(pIn1->flags&MEM_Int);
> + assert(pIn1->flags&(MEM_Int|MEM_UInt));
> VdbeBranchTaken(pIn1->u.i<0, 2);
> if (pIn1->u.i) {
> if (pIn1->u.i>0) pIn1->u.i--;
11. You can't treat 'u.i' as a positive value without
checking the flags.
> @@ -5157,7 +5178,7 @@ case OP_IfNotZero: { /* jump, in1 */
> */
> case OP_DecrJumpZero: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - assert(pIn1->flags&MEM_Int);
> + assert((pIn1->flags&(MEM_Int|MEM_UInt)) != 0);
> if (pIn1->u.i>SMALLEST_INT64) pIn1->u.i--;
12. The same.
> VdbeBranchTaken(pIn1->u.i==0, 2);
> if (pIn1->u.i==0) goto jump_to_p2;
> @@ -5396,12 +5417,12 @@ case OP_Init: { /* jump */
> case OP_IncMaxid: {
> assert(pOp->p1 > 0);
> pOut = &aMem[pOp->p1];
> -
> - rc = tarantoolsqlIncrementMaxid((uint64_t*) &pOut->u.i);
> + int64_t id;
> + rc = tarantoolsqlIncrementMaxid((uint64_t*) &id);
13. Id is always non-negative.
> if (rc!=SQL_OK) {
> goto abort_due_to_error;
> }
> - pOut->flags = MEM_Int;
> + mem_set_int(pOut, id, id < 0);
> break;
> }
>
> diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
> index a3100e513..abed46486 100644
> --- a/src/box/sql/vdbeInt.h
> +++ b/src/box/sql/vdbeInt.h
> @@ -259,9 +261,14 @@ struct Mem {
> * auxiliary flags.
> */
> enum {
> - MEM_PURE_TYPE_MASK = 0x3f
> + MEM_PURE_TYPE_MASK = 0x7f
> };
>
> +static_assert((int) MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int |
14. Why do you need the cast to 'int'?
> + MEM_Real | MEM_Blob | MEM_Bool |
> + MEM_UInt),
> + "value of type mask must consist of corresponding to memory type bits");
15. Out of 80.
> +
> /**
> * Simple type to str convertor. It is used to simplify
> * error reporting.
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index e480ae720..393782c23 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -233,7 +234,8 @@ sql_int64
> sql_value_int64(sql_value * pVal)
> {
> int64_t i = 0;
> - sqlVdbeIntValue((Mem *) pVal, &i);
> + bool is_neg;
> + sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
> return i;
16. The result integer is invalid in case it was a
big unsigned value.
> }
>
> @@ -385,7 +388,7 @@ sql_result_error(sql_context * pCtx, const char *z, int n)
> void
> sql_result_int(sql_context * pCtx, int iVal)
> {
> - sqlVdbeMemSetInt64(pCtx->pOut, (i64) iVal);
> + mem_set_int(pCtx->pOut, (i64) iVal, iVal < 0);
17. int to int64 cast works implicitly. You don't need an
explicit cast.
> }
>
> void
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 25d4cd759..5a71e1801 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1639,6 +1641,8 @@ sqlVdbeList(Vdbe * p)
> if (p->explain == 1) {
> pMem->flags = MEM_Int;
> pMem->u.i = i; /* Program counter */
> + mem_set_int(pMem, i, i < 0);
> +
> pMem++;
18. Stray empty line + you do mem_set_int() twice - once you
call the function, and second you set flag and value manually.
Besides, program counter is always > 0.
>
> pMem->flags = MEM_Static | MEM_Str | MEM_Term;
> @@ -3395,10 +3403,23 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
> return -1;
> }
> }
> + if ((f1 & MEM_UInt) != 0) {
> + if ((f2 & MEM_Real) != 0) {
> + return sqlIntFloatCompare(pMem1->u.i,
> + pMem2->u.r);
> + } else if ((f2 & MEM_Int) != 0) {
> + return +1;
> + } else {
> + return -1;
> + }
> + }
> if ((f1 & MEM_Real) != 0) {
> if ((f2 & MEM_Int) != 0) {
> return -sqlIntFloatCompare(pMem2->u.i,
> pMem1->u.r);
> + } else if ((f2 & MEM_UInt) != 0) {
> + return -sqlIntFloatCompare(pMem2->u.u,
19. You treat 'u.u' here as int64_t. Why?
> + pMem1->u.r);
> } else {
> return -1;
> }
> @@ -3563,13 +3584,14 @@ sqlVdbeCompareMsgpack(const char **key1,
> mem1.u.r = (double)v;
> goto do_float;
> }
> - mem1.u.i = v;
> + mem1.u.u = v;
> goto do_int;
> }
> case MP_INT:{
> mem1.u.i = mp_decode_int(&aKey1);
> do_int:
> - if (pKey2->flags & MEM_Int) {
> + if ((pKey2->flags & MEM_Int) ||
> + (pKey2->flags & MEM_UInt)) {
> if (mem1.u.i < pKey2->u.i) {
20. How can you touch pKey2->u.i, if you don't know
if it is MEM_Int or MEM_UInt? The only thing you know is
that it is an integer. You don't know which.
> rc = -1;
> } else if (mem1.u.i > pKey2->u.i) {
> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
> index 3a361d066..25119ff16 100644
> --- a/src/box/sql/vdbemem.c
> +++ b/src/box/sql/vdbemem.c
> @@ -418,7 +422,7 @@ sqlVdbeMemRelease(Mem * p)
> * return the closest available 64-bit signed integer.
> */
> static int
> -doubleToInt64(double r, int64_t *i)
> +doubleToInt64(double r, int64_t *i, bool *is_neg)
21. I understand 'is_neg' in the most of other places,
but why here? In a caller function you can always check
'r < 0' if necessary.
> {
> /*
> * Many compilers we encounter do not define constants for the
> @@ -454,20 +458,24 @@ doubleToInt64(double r, int64_t *i)
> * If pMem represents a string value, its encoding might be changed.
> */
> int
> -sqlVdbeIntValue(Mem * pMem, int64_t *i)
> +sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
> {
> int flags;
> assert(EIGHT_BYTE_ALIGNMENT(pMem));
> flags = pMem->flags;
> if (flags & MEM_Int) {
> *i = pMem->u.i;
> + *is_neg = true;
> + return 0;
> + } else if (flags & MEM_UInt) {
> + *i = pMem->u.u;
> + *is_neg = false;
> return 0;
> } else if (flags & MEM_Real) {
> - return doubleToInt64(pMem->u.r, i);
> + return doubleToInt64(pMem->u.r, i, is_neg);
> } else if (flags & (MEM_Str)) {
> assert(pMem->z || pMem->n == 0);
> - bool is_neg;
> - if (sql_atoi64(pMem->z, (int64_t *)i, &is_neg, pMem->n) == 0)
> + if (sql_atoi64(pMem->z, (int64_t *)i, is_neg, pMem->n) == 0)
22. I do not understand these casts. 'i' is already int64_t *. Why do
you cast it?
> return 0;
> }
> return -1;
> @@ -489,7 +497,11 @@ sqlVdbeRealValue(Mem * pMem, double *v)
> } else if (pMem->flags & MEM_Int) {
> *v = (double)pMem->u.i;
> return 0;
> - } else if (pMem->flags & MEM_Str) {
> + } else if (pMem->flags & MEM_UInt) {
> + *v = (double)pMem->u.u;
> + return 0;
> + }
> + else if (pMem->flags & MEM_Str) {
23. Stray line wrap.
> if (sqlAtoF(pMem->z, v, pMem->n))
> return 0;
> }
> @@ -777,42 +792,29 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
> pMem->z = 0;
> }
>
> -/*
> - * The pMem is known to contain content that needs to be destroyed prior
> - * to a value change. So invoke the destructor, then set the value to
> - * a 64-bit integer.
> - */
> -static SQL_NOINLINE void
> -vdbeReleaseAndSetInt64(Mem * pMem, i64 val)
> +void
> +mem_set_bool(struct Mem *mem, bool value)
24. Please, move this function back below mem_set_int to
avoid diff padding out.
> {
> - sqlVdbeMemSetNull(pMem);
> - pMem->u.i = val;
> - pMem->flags = MEM_Int;
> + sqlVdbeMemSetNull(mem);
> + mem->u.b = value;
> + mem->flags = MEM_Bool;
> }
>
> -/*
> - * Delete any previous value and set the value stored in *pMem to val,
> - * manifest type INTEGER.
> - */
> void
> -sqlVdbeMemSetInt64(Mem * pMem, i64 val)
> +mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
> {
> - if (VdbeMemDynamic(pMem)) {
> - vdbeReleaseAndSetInt64(pMem, val);
> + if (VdbeMemDynamic(mem))
> + sqlVdbeMemSetNull(mem);
> + if (is_neg) {
> + assert(value < 0);
> + mem->u.i = value;
> + MemSetTypeFlag(mem, MEM_Int);
> } else {
> - pMem->u.i = val;
> - pMem->flags = MEM_Int;
> + mem->u.u = value;
> + MemSetTypeFlag(mem, MEM_UInt);
> }
> }
>
> -void
> -mem_set_bool(struct Mem *mem, bool value)
> -{
> - sqlVdbeMemSetNull(mem);
> - mem->u.b = value;
> - mem->flags = MEM_Bool;
> -}
25. I was confused by several places where you use Mem.u.i not
as a negative integer, but as a buffer of both integers. To avoid
further confusion I think we should add another alias to Mem.u -
'ib' - integer buffer. It should be uint64_t, but due to its name
it will be clear that in places of its usage we just want to take
raw bytes of the stored integer, separately from its sign.
Besides, in a couple of places 'ib' usage will be faster.
1) sqlVdbeMemCast - here the only thing which matters if 'ib' is 0.
Now you check it twice - for 'u.u' and 'u.i'.
2) OP_SeekGT, here:
if ((pIn3->flags & MEM_Int) == MEM_Int) {
i = pIn3->u.i;
is_neg = true;
} else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
i = pIn3->u.u;
is_neg = false;
In this place you could write 'i = pIn3->u.ib; is_neg = flags == MEM_Int;'.
Please, consider my review fixes here and on the branch in a separate commit:
=============================================================
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7e7c12ee4..dffdcf55d 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -347,10 +347,9 @@ mem_apply_type(struct Mem *record, enum field_type type)
* an integer or real representation (BLOB and
* NULL do not get converted).
*/
- if ((record->flags & MEM_Str) == 0) {
- if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)))
- sqlVdbeMemStringify(record, 1);
- }
+ if ((record->flags & MEM_Str) == 0 &&
+ (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
+ sqlVdbeMemStringify(record, 1);
record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
return 0;
case FIELD_TYPE_SCALAR:
@@ -380,7 +379,7 @@ sql_value_apply_type(
*/
static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
{
- assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real))==0);
+ assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
return 0;
@@ -399,9 +398,8 @@ static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
*/
static u16 numericType(Mem *pMem)
{
- if (pMem->flags & (MEM_Int| MEM_UInt | MEM_Real)) {
+ if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
- }
if (pMem->flags & (MEM_Str|MEM_Blob)) {
return computeNumericType(pMem);
}
@@ -861,9 +859,7 @@ case OP_Gosub: { /* jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
memAboutToChange(p, pIn1);
- pIn1->flags = MEM_UInt;
- assert((pOp-aOp) >= 0);
- pIn1->u.u = (int)(pOp-aOp);
+ mem_set_int(pIn1, pOp - aOp, false);
REGISTER_TRACE(pOp->p1, pIn1);
/* Most jump operations do a goto to this spot in order to update
@@ -904,8 +900,7 @@ case OP_InitCoroutine: { /* jump */
assert(pOp->p3>=0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
assert(!VdbeMemDynamic(pOut));
- pOut->u.u = pOp->p3 - 1;
- pOut->flags = MEM_UInt;
+ mem_set_int(pOut, pOp->p3 - 1, false);
if (pOp->p2) goto jump_to_p2;
break;
}
@@ -945,12 +940,10 @@ case OP_EndCoroutine: { /* in1 */
* See also: InitCoroutine
*/
case OP_Yield: { /* in1, jump */
- int pcDest;
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
- pIn1->flags = MEM_UInt;
- pcDest = (int)pIn1->u.u;
- pIn1->u.u = (int)(pOp - aOp);
+ int pcDest = (int)pIn1->u.u;
+ mem_set_int(pIn1, pOp - aOp, false);
REGISTER_TRACE(pOp->p1, pIn1);
pOp = &aOp[pcDest];
break;
@@ -1078,12 +1071,10 @@ case OP_Halt: {
*/
case OP_Integer: { /* out2 */
pOut = out2Prerelease(p, pOp);
- if (pOp->p1 < 0) {
+ if (pOp->p1 < 0)
pOut->u.i = pOp->p1;
- } else {
- pOut->u.u = pOp->p1;
- MemSetTypeFlag(pOut, MEM_UInt);
- }
+ else
+ mem_set_int(pOut, pOp->p1, false);
break;
}
@@ -1392,9 +1383,9 @@ case OP_SCopy: { /* out2 */
*/
case OP_IntCopy: { /* out2 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & (MEM_Int | MEM_UInt))!=0);
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
pOut = &aMem[pOp->p2];
- mem_set_int(pOut, pIn1->u.i, pIn1->flags & MEM_Int);
+ mem_set_int(pOut, pIn1->u.i, (pIn1->flags & MEM_Int) != 0);
break;
}
@@ -1617,8 +1608,8 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
pOut = &aMem[pOp->p3];
flags = pIn1->flags | pIn2->flags;
if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
- if ((type1 & (MEM_Int | MEM_UInt)) !=0 &&
- (type2 & (MEM_Int | MEM_UInt)) !=0) {
+ if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
+ (type2 & (MEM_Int | MEM_UInt)) != 0) {
iA = pIn1->u.i;
iB = pIn2->u.i;
bIntint = 1;
@@ -1954,7 +1945,7 @@ case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if ((pIn1->flags & (MEM_Int | MEM_UInt))== 0) {
+ if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
if (pOp->p2==0) {
rc = SQL_MISMATCH;
goto abort_due_to_error;
@@ -2171,12 +2162,12 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (sql_type_is_numeric(type)) {
if ((flags1 | flags3)&MEM_Str) {
- if ((flags1 & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((flags1 & MEM_Str) == MEM_Str) {
mem_apply_numeric_type(pIn1);
testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
flags3 = pIn3->flags;
}
- if ((flags3 & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((flags3 & MEM_Str) == MEM_Str) {
if (mem_apply_numeric_type(pIn3) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
@@ -2191,14 +2182,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
/* Handle the common case of integer comparison here, as an
* optimization, to avoid a call to sqlMemCompare()
*/
- if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt))!=0) {
+ if ((pIn1->flags & pIn3->flags &
+ (MEM_Int | MEM_UInt)) != 0) {
if (pIn3->u.i > pIn1->u.i) { res = +1; goto compare_op; }
if (pIn3->u.i < pIn1->u.i) { res = -1; goto compare_op; }
res = 0;
goto compare_op;
}
} else if (type == FIELD_TYPE_STRING) {
- if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_UInt|MEM_Real))!=0) {
+ if ((flags1 & MEM_Str) == 0 &&
+ (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
testcase( pIn1->flags & MEM_Int);
testcase( pIn1->flags & MEM_Real);
sqlVdbeMemStringify(pIn1, 1);
@@ -2206,7 +2199,8 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
assert(pIn1!=pIn3);
}
- if ((flags3 & MEM_Str)==0 && (flags3 & (MEM_Int|MEM_UInt|MEM_Real))!=0) {
+ if ((flags3 & MEM_Str) == 0 &&
+ (flags3 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
testcase( pIn3->flags & MEM_Int);
testcase( pIn3->flags & MEM_Real);
sqlVdbeMemStringify(pIn3, 1);
@@ -2755,7 +2749,7 @@ case OP_Column: {
pDest->flags = MEM_Blob|MEM_Ephem|MEM_Subtype;
pDest->subtype = SQL_SUBTYPE_MSGPACK;
}
- if ((pDest->flags & (MEM_Int|MEM_UInt)) != 0 &&
+ if ((pDest->flags & (MEM_Int | MEM_UInt)) != 0 &&
pC->eCurType == CURTYPE_TARANTOOL) {
enum field_type f = FIELD_TYPE_ANY;
/*
@@ -2959,9 +2953,7 @@ case OP_Count: { /* out2 */
}
if (rc) goto abort_due_to_error;
pOut = out2Prerelease(p, pOp);
- assert(nEntry >= 0);
- pOut->u.u = nEntry;
- pOut->flags = MEM_UInt;
+ mem_set_int(pOut, nEntry, false);
break;
}
@@ -3515,9 +3507,9 @@ case OP_SeekGT: { /* jump, in3 */
* the seek, so convert it.
*/
pIn3 = &aMem[reg_ipk];
- if ((pIn3->flags & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
+ if ((pIn3->flags & (MEM_Int | MEM_UInt | MEM_Real |
+ MEM_Str)) == MEM_Str)
mem_apply_numeric_type(pIn3);
- }
int64_t i;
if ((pIn3->flags & MEM_Int) == MEM_Int) {
i = pIn3->u.i;
@@ -3813,9 +3805,7 @@ case OP_Sequence: { /* out2 */
assert(p->apCsr[pOp->p1]!=0);
pOut = out2Prerelease(p, pOp);
int64_t seq_val = p->apCsr[pOp->p1]->seqCount++;
- assert(seq_val >= 0);
- pOut->u.u = seq_val;
- pOut->flags = MEM_UInt;
+ mem_set_int(pOut, seq_val, false);
break;
}
@@ -3864,8 +3854,7 @@ case OP_NextIdEphemeral: {
goto abort_due_to_error;
}
pOut = &aMem[pOp->p2];
- pOut->u.u = rowid;
- pOut->flags = MEM_UInt;
+ mem_set_int(pOut, rowid, false);
break;
}
@@ -3900,8 +3889,7 @@ case OP_FCopy: { /* out2 */
assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
pOut = &aMem[pOp->p2];
- mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_UInt ?
- false : true);
+ mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
}
break;
}
@@ -5131,8 +5119,8 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
pOut = out2Prerelease(p, pOp);
- assert(pIn1->flags & (MEM_Int | MEM_UInt));
- assert(pIn3->flags & (MEM_Int | MEM_UInt));
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ assert((pIn3->flags & (MEM_Int | MEM_UInt)) != 0);
x = pIn1->u.i;
if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
/* If the LIMIT is less than or equal to zero, loop forever. This
@@ -5143,8 +5131,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
* it would take nearly 300 years to actually reach the limit. So
* looping forever is a reasonable approximation.
*/
- pOut->u.i = -1;
- pOut->flags = MEM_Int;
+ mem_set_int(pOut, -1, true);
} else {
mem_set_int(pOut, x, x < 0);
}
@@ -5161,7 +5148,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags&(MEM_Int|MEM_UInt));
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
VdbeBranchTaken(pIn1->u.i<0, 2);
if (pIn1->u.i) {
if (pIn1->u.i>0) pIn1->u.i--;
@@ -5178,7 +5165,7 @@ case OP_IfNotZero: { /* jump, in1 */
*/
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags&(MEM_Int|MEM_UInt)) != 0);
+ assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
if (pIn1->u.i>SMALLEST_INT64) pIn1->u.i--;
VdbeBranchTaken(pIn1->u.i==0, 2);
if (pIn1->u.i==0) goto jump_to_p2;
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index abed46486..d29f3323b 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -231,7 +231,7 @@ struct Mem {
#define MEM_Real 0x0008 /* Value is a real number */
#define MEM_Blob 0x0010 /* Value is a BLOB */
#define MEM_Bool 0x0020 /* Value is a bool */
-#define MEM_UInt 0x0040 /* Value is a unsigned integer */
+#define MEM_UInt 0x0040 /* Value is an unsigned integer */
#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
#define MEM_Undefined 0x0100 /* Value is undefined */
#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
@@ -264,10 +264,10 @@ enum {
MEM_PURE_TYPE_MASK = 0x7f
};
-static_assert((int) MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int |
- MEM_Real | MEM_Blob | MEM_Bool |
- MEM_UInt),
- "value of type mask must consist of corresponding to memory type bits");
+static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
+ MEM_Blob | MEM_Bool | MEM_UInt),
+ "value of type mask must consist of corresponding to memory "\
+ "type bits");
/**
* Simple type to str convertor. It is used to simplify
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 393782c23..3c0a9cc59 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -388,7 +388,7 @@ sql_result_error(sql_context * pCtx, const char *z, int n)
void
sql_result_int(sql_context * pCtx, int iVal)
{
- mem_set_int(pCtx->pOut, (i64) iVal, iVal < 0);
+ mem_set_int(pCtx->pOut, iVal, iVal < 0);
}
void
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 5a71e1801..de5fa5047 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1639,9 +1639,8 @@ sqlVdbeList(Vdbe * p)
pOp = &apSub[j]->aOp[i];
}
if (p->explain == 1) {
- pMem->flags = MEM_Int;
- pMem->u.i = i; /* Program counter */
- mem_set_int(pMem, i, i < 0);
+ assert(i >= 0);
+ mem_set_int(pMem, i, false);
pMem++;
@@ -3373,7 +3372,7 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
/* At least one of the two values is a number
*/
- if (combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) {
+ if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
if ((f1 & f2 & MEM_Int) != 0) {
if (pMem1->u.i < pMem2->u.i)
return -1;
@@ -3590,8 +3589,7 @@ sqlVdbeCompareMsgpack(const char **key1,
case MP_INT:{
mem1.u.i = mp_decode_int(&aKey1);
do_int:
- if ((pKey2->flags & MEM_Int) ||
- (pKey2->flags & MEM_UInt)) {
+ if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
if (mem1.u.i < pKey2->u.i) {
rc = -1;
} else if (mem1.u.i > pKey2->u.i) {
@@ -3612,7 +3610,7 @@ sqlVdbeCompareMsgpack(const char **key1,
case MP_DOUBLE:{
mem1.u.r = mp_decode_double(&aKey1);
do_float:
- if (pKey2->flags & MEM_Int || pKey2->flags & MEM_UInt) {
+ if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
rc = -sqlIntFloatCompare(pKey2->u.i,
mem1.u.r);
} else if (pKey2->flags & MEM_Real) {
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 48cb556e9..9b70e61a4 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -291,7 +291,7 @@ sqlVdbeMemStringify(Mem * pMem, u8 bForce)
return SQL_OK;
assert(!(fg & MEM_Zero));
- assert(fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool));
+ assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool)) != 0);
assert(EIGHT_BYTE_ALIGNMENT(pMem));
if (sqlVdbeMemClearAndResize(pMem, nByte)) {
@@ -497,11 +497,10 @@ sqlVdbeRealValue(Mem * pMem, double *v)
} else if (pMem->flags & MEM_Int) {
*v = (double)pMem->u.i;
return 0;
- } else if (pMem->flags & MEM_UInt) {
+ } else if ((pMem->flags & MEM_UInt) != 0) {
*v = (double)pMem->u.u;
return 0;
- }
- else if (pMem->flags & MEM_Str) {
+ } else if (pMem->flags & MEM_Str) {
if (sqlAtoF(pMem->z, v, pMem->n))
return 0;
}
@@ -596,9 +595,9 @@ sqlVdbeMemNumerify(Mem * pMem)
if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) == 0) {
assert((pMem->flags & (MEM_Blob | MEM_Str)) != 0);
bool is_neg;
- if (sql_atoi64(pMem->z, &pMem->u.i, &is_neg, pMem->n) == 0) {
- int flag = is_neg ? MEM_Int : MEM_UInt;
- MemSetTypeFlag(pMem, flag);
+ int64_t i;
+ if (sql_atoi64(pMem->z, &i, &is_neg, pMem->n) == 0) {
+ mem_set_int(pMem, i, is_neg);
} else {
double v;
if (sqlVdbeRealValue(pMem, &v))
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-01 14:21 ` n.pettik
2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-01 14:21 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index bb7405e68..f4c1cbcca 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -111,6 +111,7 @@ typeofFunc(sql_context * context, int NotUsed, sql_value ** argv)
>> UNUSED_PARAMETER(NotUsed);
>> switch (sql_value_type(argv[0])) {
>> case MP_INT:
>> + case MP_UINT:
>
> 1. Why on MP_UINT do you return 'integer' instead of
> 'unsigned’?
It is fixed in the next patch (and the rest of built-in functions).
>> z = "integer";
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 6ecdb26fc..d141397a0 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -900,8 +904,8 @@ case OP_InitCoroutine: { /* jump */
>> assert(pOp->p3>=0 && pOp->p3<p->nOp);
>> pOut = &aMem[pOp->p1];
>> assert(!VdbeMemDynamic(pOut));
>> - pOut->u.i = pOp->p3 - 1;
>> - pOut->flags = MEM_Int;
>> + pOut->u.u = pOp->p3 - 1;
>
> 2. Why are you sure, that p3 >= 1? According to
> the assertion above it is >= 0, but we don't know
> anything about p3 >= 1.
P3 is an address of opcode. OP_Init features 0 address
(always), ergo other instructions have addresses >= 1.
>> @@ -1099,7 +1108,7 @@ case OP_Bool: { /* out2 */
>> case OP_Int64: { /* out2 */
>> pOut = out2Prerelease(p, pOp);
>> assert(pOp->p4.pI64!=0);
>> - pOut->u.i = *pOp->p4.pI64;
>> + mem_set_int(pOut, *pOp->p4.pI64, *pOp->p4.pI64 < 0);
>
> 3. Sorry, but such places are hell. You pass separately a value
> and a flag if it is negative. Please, introduce separate functions
>
> /* Set a value, check sign inside. */
> mem_set_i64(int64_t value);
>
> /* Set a non-negative value. */
> mem_set_u64(uint64_t value);
>
> /* Set a big integer with sign passed separately. */
> mem_set_int(uint64_t value, bool is_negative);
>
> In most places you will use mem_set_i64 and mem_set_u64. It
> will look shorter and somewhere work faster.
Done. Note that I’ve also rebased onto the fresh master,
so real diff may turn out to be slightly different.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 602b39eb9..6d6300f97 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -313,7 +313,7 @@ mem_apply_type(struct Mem *record, enum field_type type)
if ((record->flags & MEM_Real) == MEM_Real) {
int64_t i = (int64_t) record->u.r;
if (i == record->u.r)
- mem_set_int(record, i, i < 0);
+ mem_set_i64(record, i);
return 0;
}
return sqlVdbeMemIntegerify(record, false);
@@ -898,7 +898,7 @@ case OP_Gosub: { /* jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
memAboutToChange(p, pIn1);
- mem_set_int(pIn1, pOp - aOp, false);
+ mem_set_u64(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
/* Most jump operations do a goto to this spot in order to update
@@ -939,7 +939,7 @@ case OP_InitCoroutine: { /* jump */
assert(pOp->p3>=0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
assert(!VdbeMemDynamic(pOut));
- mem_set_int(pOut, pOp->p3 - 1, false);
+ mem_set_u64(pOut, pOp->p3 - 1);
if (pOp->p2) goto jump_to_p2;
break;
}
@@ -982,7 +982,7 @@ case OP_Yield: { /* in1, jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
int pcDest = (int)pIn1->u.u;
- mem_set_int(pIn1, pOp - aOp, false);
+ mem_set_u64(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
pOp = &aOp[pcDest];
break;
@@ -1064,7 +1064,7 @@ case OP_Integer: { /* out2 */
if (pOp->p1 < 0)
pOut->u.i = pOp->p1;
else
- mem_set_int(pOut, pOp->p1, false);
+ mem_set_u64(pOut, pOp->p1);
break;
}
@@ -1089,7 +1089,7 @@ case OP_Bool: { /* out2 */
case OP_Int64: { /* out2 */
pOut = out2Prerelease(p, pOp);
assert(pOp->p4.pI64!=0);
- mem_set_int(pOut, *pOp->p4.pI64, *pOp->p4.pI64 < 0);
+ mem_set_i64(pOut, *pOp->p4.pI64);
break;
}
@@ -1173,7 +1173,7 @@ case OP_NextAutoincValue: {
goto abort_due_to_error;
pOut = out2Prerelease(p, pOp);
- mem_set_int(pOut, value, value < 0);
+ mem_set_i64(pOut, value);
break;
}
@@ -1605,7 +1605,7 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
break;
}
}
- mem_set_int(pOut, iB, iB < 0);
+ mem_set_i64(pOut, iB);
} else {
bIntint = 0;
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
@@ -1870,7 +1870,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
memcpy(&iA, &uA, sizeof(iA));
}
}
- mem_set_int(pOut, iA, iA < 0);
+ mem_set_i64(pOut, iA);
break;
}
@@ -2437,7 +2437,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
sql_value_text(pIn1), "integer");
goto abort_due_to_error;
}
- mem_set_int(pOut, ~i, ~i < 0);
+ mem_set_i64(pOut, ~i);
}
break;
}
@@ -2798,7 +2798,7 @@ case OP_Count: { /* out2 */
nEntry = tarantoolsqlEphemeralCount(pCrsr);
}
pOut = out2Prerelease(p, pOp);
- mem_set_int(pOut, nEntry, false);
+ mem_set_u64(pOut, nEntry);
break;
}
@@ -3603,7 +3603,7 @@ case OP_Sequence: { /* out2 */
assert(p->apCsr[pOp->p1]!=0);
pOut = out2Prerelease(p, pOp);
int64_t seq_val = p->apCsr[pOp->p1]->seqCount++;
- mem_set_int(pOut, seq_val, false);
+ mem_set_u64(pOut, seq_val);
break;
}
@@ -3619,8 +3619,7 @@ case OP_NextSequenceId: {
uint64_t id;
tarantoolSqlNextSeqId(&id);
id++;
- pOut->u.u = id;
- pOut->flags = MEM_UInt;
+ mem_set_u64(pOut, id);
break;
}
@@ -3650,7 +3649,7 @@ case OP_NextIdEphemeral: {
goto abort_due_to_error;
}
pOut = &aMem[pOp->p2];
- mem_set_int(pOut, rowid, false);
+ mem_set_u64(pOut, rowid);
break;
}
@@ -4904,9 +4903,9 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
* it would take nearly 300 years to actually reach the limit. So
* looping forever is a reasonable approximation.
*/
- mem_set_int(pOut, -1, true);
+ mem_set_i64(pOut, -1);
} else {
- mem_set_int(pOut, x, false);
+ mem_set_u64(pOut, x);
}
break;
}
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 8df34bb27..b83926a66 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -470,6 +470,21 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_ptr(struct Mem *mem, void *ptr);
+/**
+ * Set integer value. Depending on its sign MEM_Int (in case
+ * of negative value) or MEM_UInt flag is set.
+ */
+void
+mem_set_i64(struct Mem *mem, int64_t value);
+
+/** Set unsigned value and MEM_UInt flag. */
+void
+mem_set_u64(struct Mem *mem, uint64_t value);
+
+/**
+ * Set integer value. According to is_neg flag value is considered
+ * to be signed or unsigned.
+ */
void
mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 57bf25ed8..705e869bc 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -322,7 +322,7 @@ sql_result_double(sql_context * pCtx, double rVal)
void
sql_result_int(sql_context * pCtx, int iVal)
{
- mem_set_int(pCtx->pOut, iVal, iVal < 0);
+ mem_set_i64(pCtx->pOut, iVal);
}
void
@@ -334,7 +334,7 @@ sql_result_bool(struct sql_context *ctx, bool value)
void
sql_result_int64(sql_context * pCtx, i64 iVal)
{
- mem_set_int(pCtx->pOut, iVal, iVal < 0);
+ mem_set_i64(pCtx->pOut, iVal);
}
void
@@ -998,7 +998,7 @@ sql_bind_int64(sql_stmt * pStmt, int i, sql_int64 iValue)
if (vdbeUnbind(p, i) != 0)
return -1;
int rc = sql_bind_type(p, i, "INTEGER");
- mem_set_int(&p->aVar[i - 1], iValue, iValue < 0);
+ mem_set_i64(&p->aVar[i - 1], iValue);
return rc;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 60a4e8669..d5ecd8f43 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1430,7 +1430,7 @@ sqlVdbeList(Vdbe * p)
}
if (p->explain == 1) {
assert(i >= 0);
- mem_set_int(pMem, i, false);
+ mem_set_u64(pMem, i);
pMem++;
@@ -1463,13 +1463,13 @@ sqlVdbeList(Vdbe * p)
}
}
- mem_set_int(pMem, pOp->p1, pOp->p1 < 0);
+ mem_set_i64(pMem, pOp->p1);
pMem++;
- mem_set_int(pMem, pOp->p2, pOp->p2 < 0);
+ mem_set_i64(pMem, pOp->p2);
pMem++;
- mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
+ mem_set_i64(pMem, pOp->p3);
pMem++;
if (sqlVdbeMemClearAndResize(pMem, 256)) {
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 51d4787f7..5387c6f33 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -798,6 +798,25 @@ mem_set_bool(struct Mem *mem, bool value)
mem->flags = MEM_Bool;
}
+void
+mem_set_i64(struct Mem *mem, int64_t value)
+{
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ mem->u.i = value;
+ int flag = value < 0 ? MEM_Int : MEM_UInt;
+ MemSetTypeFlag(mem, flag);
+}
+
+void
+mem_set_u64(struct Mem *mem, uint64_t value)
+{
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ mem->u.u = value;
+ MemSetTypeFlag(mem, MEM_UInt);
+}
+
void
mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
{
@@ -1376,8 +1395,7 @@ valueFromExpr(sql * db, /* The database connection */
if (pVal == 0)
goto no_mem;
if (ExprHasProperty(pExpr, EP_IntValue)) {
- mem_set_int(pVal, (i64) pExpr->u.iValue * negInt,
- pExpr->u.iValue * negInt < 0);
+ mem_set_i64(pVal, (i64) pExpr->u.iValue * negInt);
} else {
zVal =
sqlMPrintf(db, "%s%s", zNeg, pExpr->u.zToken);
>> @@ -1609,7 +1617,8 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
>> pOut = &aMem[pOp->p3];
>> flags = pIn1->flags | pIn2->flags;
>> if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
>> - if ((type1 & type2 & MEM_Int)!=0) {
>> + if ((type1 & (MEM_Int | MEM_UInt)) !=0 &&
>> + (type2 & (MEM_Int | MEM_UInt)) !=0) {
>> iA = pIn1->u.i;
>> iB = pIn2->u.i;
>
> 4. How can you access pIn1/2->u.i if the types can be MEM_UInt?
There’s no opportunity to operate on values in the range
[INT64_MAX + 1, UINT64_MAX] in scope of this patch.
So, basically it is safe to use values as integers members
of union. It is fixed in the next patch as well as other arithmetic
operations.
>> bIntint = 1;
>> @@ -2485,14 +2490,14 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
>> sqlVdbeMemSetNull(pOut);
>> if ((pIn1->flags & MEM_Null)==0) {
>> int64_t i;
>> - if (sqlVdbeIntValue(pIn1, &i) != 0) {
>> + bool is_neg;
>> + if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
>> diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
>> sql_value_text(pIn1), "integer");
>> rc = SQL_TARANTOOL_ERROR;
>> goto abort_due_to_error;
>> }
>> - pOut->flags = MEM_Int;
>> - pOut->u.i = ~i;
>> + mem_set_int(pOut, ~i, ~i < 0);
>
> 5. Why do you compare 'i' with 0? What if it was a big
> unsigned value?
See answer above: in this patch “i” can’t be “big unsigned value”.
> You have 'is_neg' flag for this, don't you?
>
>> }
>> break;
>> }
>> @@ -3507,12 +3515,16 @@ case OP_SeekGT: { /* jump, in3 */
>> * the seek, so convert it.
>> */
>> pIn3 = &aMem[reg_ipk];
>> - if ((pIn3->flags & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
>> + if ((pIn3->flags & (MEM_Int|MEM_UInt|MEM_Real|MEM_Str))==MEM_Str) {
>
> 6. Please, remind me how is it possible, that a value is at the
> same time string and number? Why not to check for flags & MEM_Str != 0
> only?
It definitely could be some time ago (when string was converted to a number).
I do not remember reason of that behaviour. Now it seems that this situation
doesn’t appear anymore, so we really can simplify this check. Fixed by
applying your diff.
>> @@ -3814,10 +3828,10 @@ case OP_Sequence: { /* out2 */
>> */
>> case OP_NextSequenceId: {
>> pOut = &aMem[pOp->p2];
>> - tarantoolSqlNextSeqId((uint64_t *) &pOut->u.i);
>> -
>> - pOut->u.i += 1;
>> - pOut->flags = MEM_Int;
>> + int64_t id;
>> + tarantoolSqlNextSeqId((uint64_t *) &id);
>> + id++;
>> + mem_set_int(pOut, id, id < 0);
>
> 7. tarantoolSqlNextSeqId returns _sequence.id field. The
> field has unsigned type. How is it possible, that it is < 0?
No way. Fixed:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7e7c12ee4..47f558011 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -3828,10 +3828,11 @@ case OP_Sequence: { /* out2 */
*/
case OP_NextSequenceId: {
pOut = &aMem[pOp->p2];
- int64_t id;
- tarantoolSqlNextSeqId((uint64_t *) &id);
+ uint64_t id;
+ tarantoolSqlNextSeqId(&id);
id++;
- mem_set_int(pOut, id, id < 0);
+ mem_set_u64(pOut, id);
break;
}
>> @@ -5079,10 +5092,17 @@ case OP_FkIfZero: { /* jump */
>> */
>> case OP_IfPos: { /* jump, in1 */
>> pIn1 = &aMem[pOp->p1];
>> - assert(pIn1->flags&MEM_Int);
>> - VdbeBranchTaken( pIn1->u.i>0, 2);
>> - if (pIn1->u.i>0) {
>> - pIn1->u.i -= pOp->p3;
>> + assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
>> + if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
>> + assert(pOp->p3 >= 0);
>> + uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
>> + /*
>> + * To not bother setting integer flag in case
>> + * result of subtraction is negative, just
>> + * use saturated arithmetic.
>> + */
>> + res &= -(res <= pIn1->u.u);
>
> 8. I do not understand. Why do you compare res and pIn1->u.u?
> You compare 'u.u' and 'u.u - p3'. Obvioiusly, the latter is
> smaller. Probably, this is because you rely on 'res' being
> overflowed will be bigger than 'u.u’?
Yep.
>
> In such a case, I think it would be cleaner to compare 'u.u' and
> 'p3'.
>
> res &= -(u.u > p3);
Sorry, can’t understand what you mean or how it
should be implemented. This is common way to
implement saturated subtraction.
> Additionally, now this opcode's comment is wrong - P1 is not equal
> to P1 - P3 from this moment.
Ok, fixed a bit comment:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index b93e12b9a..fcc3b250c 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -5115,7 +5115,8 @@ case OP_FkIfZero: { /* jump */
*
* Register P1 must contain an integer.
* If the value of register P1 is 1 or greater, subtract P3 from the
- * value in P1 and jump to P2.
+ * value in P1 and jump to P2. If the result of subtraction is
+ * negative, then register P1 will hold 0.
*
* If the initial value of register P1 is less than 1, then the
* value is unchanged and control passes through to the next instruction.
>> @@ -5111,10 +5131,10 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>> pIn1 = &aMem[pOp->p1];
>> pIn3 = &aMem[pOp->p3];
>> pOut = out2Prerelease(p, pOp);
>> - assert(pIn1->flags & MEM_Int);
>> - assert(pIn3->flags & MEM_Int);
>> + assert(pIn1->flags & (MEM_Int | MEM_UInt));
>> + assert(pIn3->flags & (MEM_Int | MEM_UInt));
>> x = pIn1->u.i;
>> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i>0?pIn3->u.i:0)) {
>> + if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
>
> 9. Why do you touch pIn3.u.i without checking its type? In case it
> is a big unsigned, you will get 'u.i < 0'. Additionally, as I understand,
> now no one should treat 'u.i' as an unsigned. It is supposed to be
> negative always.
Due to the reason I’ve already mentioned: in this patch there’s
no chance that values in range [INT64_MAX + 1, …] get to VDBE.
>> /* If the LIMIT is less than or equal to zero, loop forever. This
>> * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
>> * also loop forever. This is undocumented. In fact, one could argue
>> @@ -5124,8 +5144,9 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>> * looping forever is a reasonable approximation.
>> */
>> pOut->u.i = -1;
>> + pOut->flags = MEM_Int;
>> } else {
>> - pOut->u.i = x;
>> + mem_set_int(pOut, x, x < 0);
>
> 10. Here 'x' is always >= 0.
Fixed:
@@ -5178,7 +5179,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
mem_set_int(pOut, -1, true);
} else {
- mem_set_int(pOut, x, x < 0);
+ mem_set_u64(pOut, x,);
}
break;
}
>> @@ -5140,7 +5161,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>> */
>> case OP_IfNotZero: { /* jump, in1 */
>> pIn1 = &aMem[pOp->p1];
>> - assert(pIn1->flags&MEM_Int);
>> + assert(pIn1->flags&(MEM_Int|MEM_UInt));
>> VdbeBranchTaken(pIn1->u.i<0, 2);
>> if (pIn1->u.i) {
>> if (pIn1->u.i>0) pIn1->u.i--;
>
> 11. You can't treat 'u.i' as a positive value without
> checking the flags.
P1 is LIMIT counter (according to the only usage of this opcode)
and it can be only positive value. It is checked in
computeLimitRegisters(). I’ve fixed an assertion:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index a08194178..f4d311927 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -5149,7 +5149,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ assert((pIn1->flags & MEM_UInt) != 0);
- VdbeBranchTaken(pIn1->u.i<0, 2);
- if (pIn1->u.i) {
- if (pIn1->u.i>0) pIn1->u.i--;
+ VdbeBranchTaken(pIn1->u.u<0, 2);
+ if (pIn1->u.u) {
+ if (pIn1->u.u > 0) pIn1->u.u--;
>> @@ -5157,7 +5178,7 @@ case OP_IfNotZero: { /* jump, in1 */
>> */
>> case OP_DecrJumpZero: { /* jump, in1 */
>> pIn1 = &aMem[pOp->p1];
>> - assert(pIn1->flags&MEM_Int);
>> + assert((pIn1->flags&(MEM_Int|MEM_UInt)) != 0);
>> if (pIn1->u.i>SMALLEST_INT64) pIn1->u.i--;
>
> 12. The same.
The same is here: P1 is the LIMIT counter, so I’ve fixed
an assertion:
@@ -5166,7 +5166,7 @@ case OP_IfNotZero: { /* jump, in1 */
*/
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ assert((pIn1->flags & MEM_UInt) != 0);
- if (pIn1->u.i>SMALLEST_INT64) pIn1->u.i--;
+ if (pIn1->u.i>SMALLEST_INT64) pIn1->u.u--;
VdbeBranchTaken(pIn1->u.i==0, 2);
- if (pIn1->u.i==0) goto jump_to_p2;
+ if (pIn1->u.u == 0) goto jump_to_p2;
break;
}
>> @@ -5396,12 +5417,12 @@ case OP_Init: { /* jump */
>> case OP_IncMaxid: {
>> assert(pOp->p1 > 0);
>> pOut = &aMem[pOp->p1];
>> -
>> - rc = tarantoolsqlIncrementMaxid((uint64_t*) &pOut->u.i);
>> + int64_t id;
>> + rc = tarantoolsqlIncrementMaxid((uint64_t*) &id);
>
> 13. Id is always non-negative.
Fixed:
@@ -5417,12 +5418,13 @@ case OP_Init: { /* jump */
case OP_IncMaxid: {
assert(pOp->p1 > 0);
pOut = &aMem[pOp->p1];
- int64_t id;
- rc = tarantoolsqlIncrementMaxid((uint64_t*) &id);
+ uint64_t id;
+ rc = tarantoolsqlIncrementMaxid(&id);
if (rc!=SQL_OK) {
goto abort_due_to_error;
}
- mem_set_int(pOut, id, id < 0);
+ mem_set_u64(pOut, id);
break;
}
>> diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
>> index a3100e513..abed46486 100644
>> --- a/src/box/sql/vdbeInt.h
>> +++ b/src/box/sql/vdbeInt.h
>> @@ -259,9 +261,14 @@ struct Mem {
>> * auxiliary flags.
>> */
>> enum {
>> - MEM_PURE_TYPE_MASK = 0x3f
>> + MEM_PURE_TYPE_MASK = 0x7f
>> };
>>
>> +static_assert((int) MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int |
>
> 14. Why do you need the cast to 'int’?
Well, in fact I don’t need to do that. Fixed by applying your diff.
>
>> + MEM_Real | MEM_Blob | MEM_Bool |
>> + MEM_UInt),
>> + "value of type mask must consist of corresponding to memory type bits");
>
> 15. Out of 80.
Same.
>> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
>> index e480ae720..393782c23 100644
>> --- a/src/box/sql/vdbeapi.c
>> +++ b/src/box/sql/vdbeapi.c
>> @@ -233,7 +234,8 @@ sql_int64
>> sql_value_int64(sql_value * pVal)
>> {
>> int64_t i = 0;
>> - sqlVdbeIntValue((Mem *) pVal, &i);
>> + bool is_neg;
>> + sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
>> return i;
>
> 16. The result integer is invalid in case it was a
> big unsigned value.
In the next patches I’m introducing sql_value_uint(),
so it’s OK. However, I’d like to rename is_neg to unused
to underline that the usage is correct:
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 3c0a9cc59..5ba7981ec 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -234,8 +234,8 @@ sql_int64
sql_value_int64(sql_value * pVal)
{
int64_t i = 0;
- bool is_neg;
- sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
+ bool unused;
+ sqlVdbeIntValue((Mem *) pVal, &i, &unused);
return i;
}
>> @@ -385,7 +388,7 @@ sql_result_error(sql_context * pCtx, const char *z, int n)
>> void
>> sql_result_int(sql_context * pCtx, int iVal)
>> {
>> - sqlVdbeMemSetInt64(pCtx->pOut, (i64) iVal);
>> + mem_set_int(pCtx->pOut, (i64) iVal, iVal < 0);
>
> 17. int to int64 cast works implicitly. You don't need an
> explicit cast.
Ok, fixed in your diff.
>> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
>> index 25d4cd759..5a71e1801 100644
>> --- a/src/box/sql/vdbeaux.c
>> +++ b/src/box/sql/vdbeaux.c
>> @@ -1639,6 +1641,8 @@ sqlVdbeList(Vdbe * p)
>> if (p->explain == 1) {
>> pMem->flags = MEM_Int;
>> pMem->u.i = i; /* Program counter */
>> + mem_set_int(pMem, i, i < 0);
>> +
>> pMem++;
>
> 18. Stray empty line + you do mem_set_int() twice - once you
> call the function, and second you set flag and value manually.
> Besides, program counter is always > 0.
Fixed:
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 5a71e1801..de5fa5047 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1639,9 +1639,8 @@ sqlVdbeList(Vdbe * p)
pOp = &apSub[j]->aOp[i];
}
if (p->explain == 1) {
- pMem->flags = MEM_Int;
- pMem->u.i = i; /* Program counter */
- mem_set_int(pMem, i, i < 0);
+ assert(i >= 0);
+ mem_set_int(pMem, i, false);
pMem++;
>> @@ -3395,10 +3403,23 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
>> return -1;
>> }
>> }
>> + if ((f1 & MEM_UInt) != 0) {
>> + if ((f2 & MEM_Real) != 0) {
>> + return sqlIntFloatCompare(pMem1->u.i,
>> + pMem2->u.r);
>> + } else if ((f2 & MEM_Int) != 0) {
>> + return +1;
>> + } else {
>> + return -1;
>> + }
>> + }
>> if ((f1 & MEM_Real) != 0) {
>> if ((f2 & MEM_Int) != 0) {
>> return -sqlIntFloatCompare(pMem2->u.i,
>> pMem1->u.r);
>> + } else if ((f2 & MEM_UInt) != 0) {
>> + return -sqlIntFloatCompare(pMem2->u.u,
>
> 19. You treat 'u.u' here as int64_t. Why?
Same reason and fixed in the next patch.
>> @@ -3563,13 +3584,14 @@ sqlVdbeCompareMsgpack(const char **key1,
>> mem1.u.r = (double)v;
>> goto do_float;
>> }
>> - mem1.u.i = v;
>> + mem1.u.u = v;
>> goto do_int;
>> }
>> case MP_INT:{
>> mem1.u.i = mp_decode_int(&aKey1);
>> do_int:
>> - if (pKey2->flags & MEM_Int) {
>> + if ((pKey2->flags & MEM_Int) ||
>> + (pKey2->flags & MEM_UInt)) {
>> if (mem1.u.i < pKey2->u.i) {
>
> 20. How can you touch pKey2->u.i, if you don't know
> if it is MEM_Int or MEM_UInt? The only thing you know is
> that it is an integer. You don't know which.
It is fixed in the next patch.
>> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
>> index 3a361d066..25119ff16 100644
>> --- a/src/box/sql/vdbemem.c
>> +++ b/src/box/sql/vdbemem.c
>> @@ -418,7 +422,7 @@ sqlVdbeMemRelease(Mem * p)
>> * return the closest available 64-bit signed integer.
>> */
>> static int
>> -doubleToInt64(double r, int64_t *i)
>> +doubleToInt64(double r, int64_t *i, bool *is_neg)
>
> 21. I understand 'is_neg' in the most of other places,
> but why here? In a caller function you can always check
> 'r < 0' if necessary.
Fair, removed:
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 48cb556e9..a8ac9c806 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -422,7 +422,7 @@ sqlVdbeMemRelease(Mem * p)
* return the closest available 64-bit signed integer.
*/
static int
-doubleToInt64(double r, int64_t *i, bool *is_neg)
+doubleToInt64(double r, int64_t *i)
{
/*
* Many compilers we encounter do not define constants for the
@@ -433,7 +433,6 @@ doubleToInt64(double r, int64_t *i, bool *is_neg)
*/
static const int64_t maxInt = LARGEST_INT64;
static const int64_t minInt = SMALLEST_INT64;
- *is_neg = r < 0.f;
if (r <= (double)minInt) {
*i = minInt;
return -1;
@@ -472,7 +471,9 @@ sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
*is_neg = false;
return 0;
} else if (flags & MEM_Real) {
- return doubleToInt64(pMem->u.r, i, is_neg);
+ *is_neg = pMem->u.r < 0;
+ return doubleToInt64(pMem->u.r, i);
} else if (flags & (MEM_Str)) {
assert(pMem->z || pMem->n == 0);
if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
@@ -500,8 +501,7 @@ sqlVdbeRealValue(Mem * pMem, double *v)
} else if (pMem->flags & MEM_UInt) {
*v = (double)pMem->u.u;
return 0;
- }
- else if (pMem->flags & MEM_Str) {
+ } else if (pMem->flags & MEM_Str) {
if (sqlAtoF(pMem->z, v, pMem->n))
return 0;
}
@@ -530,9 +530,8 @@ mem_apply_integer_type(Mem *pMem)
assert(pMem->flags & MEM_Real);
assert(EIGHT_BYTE_ALIGNMENT(pMem));
- bool is_neg;
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix, &is_neg)) == 0)
- mem_set_int(pMem, ix, is_neg);
+ if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
+ mem_set_int(pMem, ix, pMem->u.r <= -1);
return rc;
}
>> @@ -454,20 +458,24 @@ doubleToInt64(double r, int64_t *i)
>> * If pMem represents a string value, its encoding might be changed.
>> */
>> int
>> -sqlVdbeIntValue(Mem * pMem, int64_t *i)
>> +sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
>> {
>> int flags;
>> assert(EIGHT_BYTE_ALIGNMENT(pMem));
>> flags = pMem->flags;
>> if (flags & MEM_Int) {
>> *i = pMem->u.i;
>> + *is_neg = true;
>> + return 0;
>> + } else if (flags & MEM_UInt) {
>> + *i = pMem->u.u;
>> + *is_neg = false;
>> return 0;
>> } else if (flags & MEM_Real) {
>> - return doubleToInt64(pMem->u.r, i);
>> + return doubleToInt64(pMem->u.r, i, is_neg);
>> } else if (flags & (MEM_Str)) {
>> assert(pMem->z || pMem->n == 0);
>> - bool is_neg;
>> - if (sql_atoi64(pMem->z, (int64_t *)i, &is_neg, pMem->n) == 0)
>> + if (sql_atoi64(pMem->z, (int64_t *)i, is_neg, pMem->n) == 0)
>
> 22. I do not understand these casts. 'i' is already int64_t *. Why do
> you cast it?
Indeed, there’s no need to cast here. Removed in the previous patch.
>> @@ -489,7 +497,11 @@ sqlVdbeRealValue(Mem * pMem, double *v)
>> } else if (pMem->flags & MEM_Int) {
>> *v = (double)pMem->u.i;
>> return 0;
>> - } else if (pMem->flags & MEM_Str) {
>> + } else if (pMem->flags & MEM_UInt) {
>> + *v = (double)pMem->u.u;
>> + return 0;
>> + }
>> + else if (pMem->flags & MEM_Str) {
>
> 23. Stray line wrap.
Fixed:
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 48cb556e9..241a8f78a 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -500,8 +500,7 @@ sqlVdbeRealValue(Mem * pMem, double *v)
} else if (pMem->flags & MEM_UInt) {
*v = (double)pMem->u.u;
return 0;
- }
- else if (pMem->flags & MEM_Str) {
+ } else if (pMem->flags & MEM_Str) {
if (sqlAtoF(pMem->z, v, pMem->n))
return 0;
}
>> @@ -777,42 +792,29 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
>> pMem->z = 0;
>> }
>>
>> -/*
>> - * The pMem is known to contain content that needs to be destroyed prior
>> - * to a value change. So invoke the destructor, then set the value to
>> - * a 64-bit integer.
>> - */
>> -static SQL_NOINLINE void
>> -vdbeReleaseAndSetInt64(Mem * pMem, i64 val)
>> +void
>> +mem_set_bool(struct Mem *mem, bool value)
>
> 24. Please, move this function back below mem_set_int to
> avoid diff padding out.
But it is not used anymore, so compilation would
result in error.
> 25. I was confused by several places where you use Mem.u.i not
> as a negative integer, but as a buffer of both integers. To avoid
> further confusion I think we should add another alias to Mem.u -
> 'ib' - integer buffer. It should be uint64_t, but due to its name
> it will be clear that in places of its usage we just want to take
> raw bytes of the stored integer, separately from its sign.
>
> Besides, in a couple of places 'ib' usage will be faster.
Sounds extremely dubious.
> 1) sqlVdbeMemCast - here the only thing which matters if 'ib' is 0.
> Now you check it twice - for 'u.u' and 'u.i'.
>
> 2) OP_SeekGT, here:
>
> if ((pIn3->flags & MEM_Int) == MEM_Int) {
> i = pIn3->u.i;
> is_neg = true;
> } else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
> i = pIn3->u.u;
> is_neg = false;
>
> In this place you could write 'i = pIn3->u.ib; is_neg = flags == MEM_Int;’.
I don’t see difference between “i” and “ib”. IMHO it would only
complicate code maintenance.
> Please, consider my review fixes here and on the branch in a separate commit:
Applied.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints
2019-07-01 14:21 ` n.pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
2019-07-05 16:33 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
Consider new ones below, and on the branch
in a separate commit.
=====================================================
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 83c0e90f1..2be42faf2 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -260,7 +260,7 @@ allocateCursor(
int
mem_apply_numeric_type(struct Mem *record)
{
- if ((record->flags & (MEM_Str | MEM_Int | MEM_Real | MEM_UInt)) != MEM_Str)
+ if ((record->flags & MEM_Str) == 0)
return -1;
int64_t integer_value;
bool is_neg;
@@ -936,7 +936,7 @@ case OP_Return: { /* in1 */
case OP_InitCoroutine: { /* jump */
assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
assert(pOp->p2>=0 && pOp->p2<p->nOp);
- assert(pOp->p3>=0 && pOp->p3<p->nOp);
+ assert(pOp->p3>0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
assert(!VdbeMemDynamic(pOut));
mem_set_u64(pOut, pOp->p3 - 1);
@@ -1061,10 +1061,12 @@ case OP_Halt: {
*/
case OP_Integer: { /* out2 */
pOut = out2Prerelease(p, pOp);
- if (pOp->p1 < 0)
+ if (pOp->p1 < 0) {
pOut->u.i = pOp->p1;
- else
+ assert((pOut->flags & MEM_Int) != 0);
+ } else {
mem_set_u64(pOut, pOp->p1);
+ }
break;
}
@@ -3307,8 +3309,7 @@ case OP_SeekGT: { /* jump, in3 */
* the seek, so convert it.
*/
pIn3 = &aMem[reg_ipk];
- if ((pIn3->flags & (MEM_Int | MEM_UInt | MEM_Real |
- MEM_Str)) == MEM_Str)
+ if ((pIn3->flags & MEM_Str) != 0)
mem_apply_numeric_type(pIn3);
int64_t i;
if ((pIn3->flags & MEM_Int) == MEM_Int) {
@@ -4921,9 +4922,8 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
assert((pIn1->flags & MEM_UInt) != 0);
- VdbeBranchTaken(pIn1->u.u<0, 2);
- if (pIn1->u.u) {
- if (pIn1->u.u > 0) pIn1->u.u--;
+ if (pIn1->u.u > 0) {
+ pIn1->u.u--;
goto jump_to_p2;
}
break;
@@ -4938,8 +4938,8 @@ case OP_IfNotZero: { /* jump, in1 */
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
assert((pIn1->flags & MEM_UInt) != 0);
- if (pIn1->u.i>SMALLEST_INT64) pIn1->u.u--;
- VdbeBranchTaken(pIn1->u.i==0, 2);
+ if (pIn1->u.u > 0)
+ pIn1->u.u--;
if (pIn1->u.u == 0) goto jump_to_p2;
break;
}
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
2019-06-07 15:37 ` [tarantool-patches] [PATCH 1/6] sql: refactor sql_atoi64() Nikita Pettik
2019-06-07 15:37 ` [tarantool-patches] [PATCH 2/6] sql: separate VDBE memory holding positive and negative ints Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
2019-06-07 15:37 ` [tarantool-patches] [PATCH 4/6] sql: make built-in functions operate on unsigned values Nikita Pettik
` (3 subsequent siblings)
6 siblings, 1 reply; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Let's patch internal VDBE routines which add, subtract, multiply, divide
and calculate the remainder of division to allow them take operands of
unsigned type. In this respect, each operator now accepts signs of both
operands and return sign of result.
Part of #3810
Part of #4015
---
src/box/sql/func.c | 14 ++-
src/box/sql/sqlInt.h | 35 +++++-
src/box/sql/util.c | 233 +++++++++++++++++++++++++++----------
src/box/sql/vdbe.c | 77 +++++++++---
test/sql-tap/func.test.lua | 28 +++--
test/sql/integer-overflow.result | 15 ++-
test/sql/integer-overflow.test.lua | 3 +
7 files changed, 309 insertions(+), 96 deletions(-)
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index f4c1cbcca..457c9b92b 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1582,7 +1582,9 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
typedef struct SumCtx SumCtx;
struct SumCtx {
double rSum; /* Floating point sum */
- i64 iSum; /* Integer sum */
+ int64_t iSum; /* Integer sum */
+ /** True if iSum < 0. */
+ bool is_neg;
i64 cnt; /* Number of elements summed */
u8 overflow; /* True if integer overflow seen */
u8 approx; /* True if non-integer value was input to the sum */
@@ -1620,9 +1622,13 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
p->cnt++;
if (type == MP_INT || type == MP_UINT) {
int64_t v = sql_value_int64(argv[0]);
- p->rSum += v;
+ if (type == MP_INT)
+ p->rSum += v;
+ else
+ p->rSum += (uint64_t) v;
if ((p->approx | p->overflow) == 0 &&
- sqlAddInt64(&p->iSum, v) != 0) {
+ sql_add_int(p->iSum, p->is_neg, v, v < 0, &p->iSum,
+ &p->is_neg) != 0) {
p->overflow = 1;
}
} else {
@@ -1642,7 +1648,7 @@ sumFinalize(sql_context * context)
} else if (p->approx) {
sql_result_double(context, p->rSum);
} else {
- sql_result_int64(context, p->iSum);
+ mem_set_int(context->pOut, p->iSum, p->is_neg);
}
}
}
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index da17e24ed..c0e2ab699 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -4461,10 +4461,37 @@ Expr *sqlExprAddCollateString(Parse *, Expr *, const char *);
Expr *sqlExprSkipCollate(Expr *);
int sqlCheckIdentifierName(Parse *, char *);
void sqlVdbeSetChanges(sql *, int);
-int sqlAddInt64(i64 *, i64);
-int sqlSubInt64(i64 *, i64);
-int sqlMulInt64(i64 *, i64);
-int sqlAbsInt32(int);
+
+/**
+ * Attempt to add, subtract, multiply or get the remainder of
+ * 64-bit integer values. There functions are able to operate
+ * on signed as well as unsigned integers. If result of operation
+ * is greater 0, then it is assumed to be unsigned and can take
+ * values in range up to 2^64 - 1. If the result is negative,
+ * then its minimum value is -2^63.
+ * Return 0 on success. Or if the operation would have resulted
+ * in an overflow, return -1.
+ */
+int
+sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg);
+
+int
+sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg);
+
+int
+sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg);
+
+int
+sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg);
+
+int
+sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg);
+
u8 sqlGetBoolean(const char *z, u8);
const void *sqlValueText(sql_value *);
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 2e7c298e1..ee6a83ad5 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -1194,89 +1194,196 @@ sqlSafetyCheckSickOrOk(sql * db)
}
}
-/*
- * Attempt to add, substract, or multiply the 64-bit signed value iB against
- * the other 64-bit signed integer at *pA and store the result in *pA.
- * Return 0 on success. Or if the operation would have resulted in an
- * overflow, leave *pA unchanged and return 1.
- */
int
-sqlAddInt64(i64 * pA, i64 iB)
+sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg)
{
- i64 iA = *pA;
- testcase(iA == 0);
- testcase(iA == 1);
- testcase(iB == -1);
- testcase(iB == 0);
- if (iB >= 0) {
- testcase(iA > 0 && LARGEST_INT64 - iA == iB);
- testcase(iA > 0 && LARGEST_INT64 - iA == iB - 1);
- if (iA > 0 && LARGEST_INT64 - iA < iB)
- return 1;
- } else {
- testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 1);
- testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 2);
- if (iA < 0 && -(iA + LARGEST_INT64) > iB + 1)
- return 1;
+ /* Addition of two negative integers. */
+ if (is_lhs_neg && is_rhs_neg) {
+ assert(lhs < 0 && rhs < 0);
+ /* This is the same as (lhs + rhs) < INT64_MIN */
+ if (-(lhs + INT64_MAX) > rhs + 1)
+ return -1;
+ *is_res_neg = true;
+ *res = lhs + rhs;
+ return 0;
}
- *pA += iB;
+ /* Both are unsigned integers. */
+ if (!is_lhs_neg && !is_rhs_neg) {
+ uint64_t u_lhs = (uint64_t) lhs;
+ uint64_t u_rhs = (uint64_t) rhs;
+ /* This is the same as (lhs + rhs) > UINT64_MAX */
+ if (UINT64_MAX - u_lhs < u_rhs)
+ return -1;
+ *is_res_neg = false;
+ *res = lhs + rhs;
+ return 0;
+ }
+ /*
+ * Make sure we've got only one combination of
+ * positive and negative operands.
+ */
+ if (is_lhs_neg) {
+ SWAP(is_lhs_neg, is_rhs_neg);
+ SWAP(lhs, rhs);
+ }
+ assert(! is_lhs_neg && is_rhs_neg);
+ *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ *res = lhs + rhs;
return 0;
}
int
-sqlSubInt64(i64 * pA, i64 iB)
+sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg)
{
- testcase(iB == SMALLEST_INT64 + 1);
- if (iB == SMALLEST_INT64) {
- testcase((*pA) == (-1));
- testcase((*pA) == 0);
- if ((*pA) >= 0)
- return 1;
- *pA -= iB;
+ if (!is_lhs_neg && !is_rhs_neg) {
+ uint64_t u_lhs = (uint64_t) lhs;
+ uint64_t u_rhs = (uint64_t) rhs;
+ /*
+ * Unsigned arithmetic has no overflows, so to
+ * check if lhs is less than rhs we can compare
+ * result of their subtraction with lhs.
+ */
+ *is_res_neg = u_lhs < (uint64_t)(u_lhs - u_rhs);
+ if (! *is_res_neg) {
+ *res = u_lhs - u_rhs;
+ return 0;
+ }
+ if (u_rhs > (uint64_t) INT64_MAX + 1 &&
+ u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
+ return -1;
+ *res = lhs - rhs;
return 0;
- } else {
- return sqlAddInt64(pA, -iB);
}
+ if (is_rhs_neg) {
+ return sql_add_int(lhs, is_lhs_neg, -rhs, false, res,
+ is_res_neg);
+ }
+ assert(is_lhs_neg && !is_rhs_neg);
+ /*
+ * (lhs - rhs) < 0, lhs < 0, rhs > 0: in this case their
+ * difference must be less than INT64_MIN.
+ */
+ if ((uint64_t) -lhs + (uint64_t) rhs > (uint64_t) INT64_MAX + 1)
+ return -1;
+ *is_res_neg = true;
+ *res = lhs - rhs;
+ return 0;
}
int
-sqlMulInt64(i64 * pA, i64 iB)
+sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg)
{
- i64 iA = *pA;
- if (iB > 0) {
- if (iA > LARGEST_INT64 / iB)
- return 1;
- if (iA < SMALLEST_INT64 / iB)
- return 1;
- } else if (iB < 0) {
- if (iA > 0) {
- if (iB < SMALLEST_INT64 / iA)
- return 1;
- } else if (iA < 0) {
- if (iB == SMALLEST_INT64)
- return 1;
- if (iA == SMALLEST_INT64)
- return 1;
- if (-iA > LARGEST_INT64 / -iB)
- return 1;
- }
+ if (lhs == 0 || rhs == 0) {
+ *res = 0;
+ *is_res_neg = false;
+ return 0;
+ }
+ /*
+ * Multiplication of integers with same sign leads to
+ * unsigned result.
+ */
+ if (! (is_lhs_neg ^ is_rhs_neg)) {
+ uint64_t u_res = is_lhs_neg ?
+ (uint64_t) (-lhs) * (uint64_t) (-rhs) :
+ (uint64_t) lhs * (uint64_t) (rhs);
+ /*
+ * Overflow detection is quite primitive due to
+ * the absence of overflow with unsigned values:
+ * lhs * rhs == res --> rhs == res / lhs;
+ * If this predicate is false, then result was
+ * reduced modulo UINT_MAX + 1.
+ */
+ if ((is_lhs_neg && u_res / (uint64_t) (-lhs) !=
+ (uint64_t) (-rhs)) ||
+ (!is_lhs_neg && u_res / (uint64_t) lhs != (uint64_t) rhs))
+ return -1;
+ *is_res_neg = false;
+ *res = u_res;
+ return 0;
+ }
+ /*
+ * Make sure we've got only one combination of
+ * positive and negative operands.
+ */
+ if (is_lhs_neg) {
+ SWAP(is_lhs_neg, is_rhs_neg);
+ SWAP(lhs, rhs);
}
- *pA = iA * iB;
+ assert(! is_lhs_neg && is_rhs_neg);
+ uint64_t u_rhs = (uint64_t) (-rhs);
+ uint64_t u_res = u_rhs * (uint64_t) lhs;
+ if (u_res / u_rhs != (uint64_t) lhs ||
+ u_rhs * (uint64_t) lhs > (uint64_t) INT64_MAX + 1)
+ return -1;
+ *is_res_neg = true;
+ *res = lhs * rhs;
+ return 0;
+}
+
+int
+sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg)
+{
+ if (lhs == 0) {
+ *res = 0;
+ *is_res_neg = false;
+ return 0;
+ }
+ /*
+ * The only possible overflow situations is when operands
+ * of different signs and result turns out to be less
+ * than INT64_MIN.
+ */
+ if (is_lhs_neg ^ is_rhs_neg) {
+ uint64_t u_res = is_lhs_neg ?
+ (uint64_t) (-lhs) / (uint64_t) rhs :
+ (uint64_t) lhs / (uint64_t) (-rhs);
+ if (u_res > (uint64_t) INT64_MAX + 1)
+ return -1;
+ *is_res_neg = u_res != 0 ? true : false;
+ *res = -u_res;
+ return 0;
+ }
+ *is_res_neg = false;
+ /*
+ * Another one special case: INT64_MIN / -1
+ * Signed division leads to program termination due
+ * to overflow.
+ */
+ if (is_lhs_neg && lhs == INT64_MIN && rhs == -1) {
+ *res = (uint64_t) INT64_MAX + 1;
+ return 0;
+ }
+ *res = is_lhs_neg ? (uint64_t) (lhs / rhs) :
+ (uint64_t) lhs / (uint64_t) rhs;
return 0;
}
-/*
- * Compute the absolute value of a 32-bit signed integer, of possible. Or
- * if the integer has a value of -2147483648, return +2147483647
- */
int
-sqlAbsInt32(int x)
+sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+ int64_t *res, bool *is_res_neg)
{
- if (x >= 0)
- return x;
- if (x == (int)0x80000000)
- return 0x7fffffff;
- return -x;
+ if (is_lhs_neg) {
+ uint64_t u_lhs = (uint64_t) (-lhs);
+ uint64_t u_res = u_lhs % (uint64_t) rhs;
+ if (u_res > (uint64_t) INT64_MAX + 1)
+ return -1;
+ *res = -u_res;
+ *is_res_neg = true;
+ return 0;
+ }
+ /*
+ * While calculating remainder we always ignore sign of
+ * rhs - it doesn't affect the result.
+ * */
+ uint64_t u_lhs = (uint64_t) lhs;
+ uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) : (uint64_t) rhs;
+ *res = u_lhs % u_rhs;
+ *is_res_neg = false;
+ return 0;
}
/*
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index d141397a0..9c28b9131 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1621,27 +1621,48 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
(type2 & (MEM_Int | MEM_UInt)) !=0) {
iA = pIn1->u.i;
iB = pIn2->u.i;
+ bool is_lhs_neg = pIn1->flags & MEM_Int;
+ bool is_rhs_neg = pIn2->flags & MEM_Int;
+ bool is_res_neg;
bIntint = 1;
switch( pOp->opcode) {
- case OP_Add: if (sqlAddInt64(&iB,iA)) goto integer_overflow; break;
- case OP_Subtract: if (sqlSubInt64(&iB,iA)) goto integer_overflow; break;
- case OP_Multiply: if (sqlMulInt64(&iB,iA)) goto integer_overflow; break;
+ case OP_Add: {
+ if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
+ (int64_t *) &iB, &is_res_neg) != 0)
+ goto integer_overflow;
+ break;
+ }
+ case OP_Subtract: {
+ if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
+ (int64_t *) &iB, &is_res_neg) != 0)
+ goto integer_overflow;
+ break;
+ }
+ case OP_Multiply: {
+ if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
+ (int64_t *) &iB, &is_res_neg) != 0)
+ goto integer_overflow;
+ break;
+ }
case OP_Divide: {
if (iA == 0)
goto division_by_zero;
- if (iA==-1 && iB==SMALLEST_INT64) goto integer_overflow;
- iB /= iA;
+ if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
+ (int64_t *) &iB, &is_res_neg) != 0)
+ goto integer_overflow;
break;
}
default: {
if (iA == 0)
goto division_by_zero;
if (iA==-1) iA = 1;
- iB %= iA;
+ if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
+ (int64_t *) &iB, &is_res_neg) != 0)
+ goto integer_overflow;
break;
}
}
- mem_set_int(pOut, iB, iB < 0);
+ mem_set_int(pOut, iB, is_res_neg);
} else {
bIntint = 0;
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
@@ -1883,13 +1904,13 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
break;
}
bool unused;
- if (sqlVdbeIntValue(pIn2, &iA, &unused) != 0) {
+ if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn2), "integer");
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
- if (sqlVdbeIntValue(pIn1, &iB, &unused) != 0) {
+ if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn1), "integer");
rc = SQL_TARANTOOL_ERROR;
@@ -2191,10 +2212,31 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
/* Handle the common case of integer comparison here, as an
* optimization, to avoid a call to sqlMemCompare()
*/
- if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt))!=0) {
- if (pIn3->u.i > pIn1->u.i) { res = +1; goto compare_op; }
- if (pIn3->u.i < pIn1->u.i) { res = -1; goto compare_op; }
- res = 0;
+ if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt)) != 0) {
+ if ((pIn1->flags & pIn3->flags & MEM_Int) != 0) {
+ if (pIn3->u.i > pIn1->u.i)
+ res = +1;
+ else if (pIn3->u.i < pIn1->u.i)
+ res = -1;
+ else
+ res = 0;
+ goto compare_op;
+ }
+ if ((pIn1->flags & pIn3->flags & MEM_UInt) != 0) {
+ if (pIn3->u.u > pIn1->u.u)
+ res = +1;
+ else if (pIn3->u.u < pIn1->u.u)
+ res = -1;
+ else
+ res = 0;
+ goto compare_op;
+ }
+ if ((pIn1->flags & MEM_UInt) != 0 &&
+ (pIn3->flags & MEM_Int) != 0) {
+ res = -1;
+ goto compare_op;
+ }
+ res = 1;
goto compare_op;
}
} else if (type == FIELD_TYPE_STRING) {
@@ -3900,8 +3942,7 @@ case OP_FCopy: { /* out2 */
assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
pOut = &aMem[pOp->p2];
- mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_UInt ?
- false : true);
+ mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
}
break;
}
@@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
assert(pIn1->flags & (MEM_Int | MEM_UInt));
assert(pIn3->flags & (MEM_Int | MEM_UInt));
x = pIn1->u.i;
- if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
+ int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
+ bool unused;
+ if ((x == 0 || pIn1->flags & MEM_Int) ||
+ sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
+ (int64_t *) &x, &unused) != 0) {
/* If the LIMIT is less than or equal to zero, loop forever. This
* is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
* also loop forever. This is undocumented. In fact, one could argue
diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
index 09b1cf967..314c528ab 100755
--- a/test/sql-tap/func.test.lua
+++ b/test/sql-tap/func.test.lua
@@ -1,6 +1,6 @@
#!/usr/bin/env tarantool
test = require("sqltester")
-test:plan(14590)
+test:plan(14591)
--!./tcltestrunner.lua
-- 2001 September 15
@@ -1591,14 +1591,14 @@ test:do_execsql_test(
-- </func-18.11>
})
-test:do_catchsql_test(
+test:do_execsql_test(
"func-18.12",
[[
INSERT INTO t6 VALUES(3, 1<<62);
SELECT sum(x) - ((1<<62)*2.0+1) from t6;
]], {
-- <func-18.12>
- 1, "Failed to execute SQL statement: integer overflow"
+ 0
-- </func-18.12>
})
@@ -1645,18 +1645,30 @@ test:do_catchsql_test(
-- </func-18.17>
})
-test:do_catchsql_test(
- "func-18.15",
+test:do_execsql_test(
+ "func-18.15.1",
[[
SELECT sum(x) FROM
(SELECT 9223372036854775807 AS x UNION ALL
SELECT 10 AS x);
]], {
- -- <func-18.15>
- 1, "Failed to execute SQL statement: integer overflow"
- -- </func-18.15>
+ -- <func-18.15.1>
+ 9223372036854775817LL
+ -- </func-18.15.1>
})
+test:do_catchsql_test(
+ "func-18.15.2",
+ [[
+ SELECT sum(x) FROM
+ (SELECT 9223372036854775807 AS x UNION ALL SELECT 9223372036854775807 AS x
+ UNION ALL SELECT 10 AS x);
+ ]], {
+ -- <func-18.15.2>
+ 1, "Failed to execute SQL statement: integer overflow"
+ -- </func-18.15.2>
+})
+
test:do_catchsql_test(
"func-18.18",
[[
diff --git a/test/sql/integer-overflow.result b/test/sql/integer-overflow.result
index d6ca66ec9..40962ac5c 100644
--- a/test/sql/integer-overflow.result
+++ b/test/sql/integer-overflow.result
@@ -10,6 +10,7 @@ box.execute('pragma sql_default_engine=\''..engine..'\'')
...
-- gh-3735: make sure that integer overflows errors are
-- handled during VDBE execution.
+-- gh-3810: range of integer is extended up to 2^64 - 1.
--
box.execute('SELECT (2147483647 * 2147483647 * 2147483647);')
---
@@ -17,7 +18,11 @@ box.execute('SELECT (2147483647 * 2147483647 * 2147483647);')
...
box.execute('SELECT (-9223372036854775808 / -1);')
---
-- error: 'Failed to execute SQL statement: integer is overflowed'
+- metadata:
+ - name: (-9223372036854775808 / -1)
+ type: integer
+ rows:
+ - [9223372036854775808]
...
box.execute('SELECT (-9223372036854775808 - 1);')
---
@@ -25,6 +30,14 @@ box.execute('SELECT (-9223372036854775808 - 1);')
...
box.execute('SELECT (9223372036854775807 + 1);')
---
+- metadata:
+ - name: (9223372036854775807 + 1)
+ type: integer
+ rows:
+ - [9223372036854775808]
+...
+box.execute('SELECT (9223372036854775807 + 9223372036854775807 + 2);')
+---
- error: 'Failed to execute SQL statement: integer is overflowed'
...
-- Literals are checked right after parsing.
diff --git a/test/sql/integer-overflow.test.lua b/test/sql/integer-overflow.test.lua
index 4339edf39..7727f368c 100644
--- a/test/sql/integer-overflow.test.lua
+++ b/test/sql/integer-overflow.test.lua
@@ -4,11 +4,14 @@ box.execute('pragma sql_default_engine=\''..engine..'\'')
-- gh-3735: make sure that integer overflows errors are
-- handled during VDBE execution.
+-- gh-3810: range of integer is extended up to 2^64 - 1.
--
box.execute('SELECT (2147483647 * 2147483647 * 2147483647);')
box.execute('SELECT (-9223372036854775808 / -1);')
box.execute('SELECT (-9223372036854775808 - 1);')
box.execute('SELECT (9223372036854775807 + 1);')
+box.execute('SELECT (9223372036854775807 + 9223372036854775807 + 2);')
+
-- Literals are checked right after parsing.
--
box.execute('SELECT 9223372036854775808;')
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-06-07 15:37 ` [tarantool-patches] [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints Nikita Pettik
@ 2019-06-11 21:11 ` Vladislav Shpilevoy
2019-07-01 14:21 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-06-11 21:11 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 14 comments below, review fixes
at the end of the email, and in a separate commit on the
branch.
On 07/06/2019 18:37, Nikita Pettik wrote:
> Let's patch internal VDBE routines which add, subtract, multiply, divide
> and calculate the remainder of division to allow them take operands of
> unsigned type. In this respect, each operator now accepts signs of both
> operands and return sign of result.
>
> Part of #3810
> Part of #4015
> ---
> src/box/sql/func.c | 14 ++-
> src/box/sql/sqlInt.h | 35 +++++-
> src/box/sql/util.c | 233 +++++++++++++++++++++++++++----------
> src/box/sql/vdbe.c | 77 +++++++++---
> test/sql-tap/func.test.lua | 28 +++--
> test/sql/integer-overflow.result | 15 ++-
> test/sql/integer-overflow.test.lua | 3 +
> 7 files changed, 309 insertions(+), 96 deletions(-)
>
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index f4c1cbcca..457c9b92b 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -1620,9 +1622,13 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
> p->cnt++;
> if (type == MP_INT || type == MP_UINT) {
> int64_t v = sql_value_int64(argv[0]);
> - p->rSum += v;
> + if (type == MP_INT)
> + p->rSum += v;
> + else
> + p->rSum += (uint64_t) v;
> if ((p->approx | p->overflow) == 0 &&
> - sqlAddInt64(&p->iSum, v) != 0) {
> + sql_add_int(p->iSum, p->is_neg, v, v < 0, &p->iSum,
1. As I understand, now sql_value_int64() returns an integer buffer,
and its result can't be compared with 0. How do you do it here with
'v'?
> + &p->is_neg) != 0) {
> p->overflow = 1;
> }
> } else {
> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> index da17e24ed..c0e2ab699 100644
> --- a/src/box/sql/sqlInt.h
> +++ b/src/box/sql/sqlInt.h
> @@ -4461,10 +4461,37 @@ Expr *sqlExprAddCollateString(Parse *, Expr *, const char *);
> Expr *sqlExprSkipCollate(Expr *);
> int sqlCheckIdentifierName(Parse *, char *);
> void sqlVdbeSetChanges(sql *, int);
> -int sqlAddInt64(i64 *, i64);
> -int sqlSubInt64(i64 *, i64);
> -int sqlMulInt64(i64 *, i64);
> -int sqlAbsInt32(int);
> +
> +/**
> + * Attempt to add, subtract, multiply or get the remainder of
> + * 64-bit integer values. There functions are able to operate
> + * on signed as well as unsigned integers. If result of operation
> + * is greater 0, then it is assumed to be unsigned and can take
> + * values in range up to 2^64 - 1. If the result is negative,
> + * then its minimum value is -2^63.
> + * Return 0 on success. Or if the operation would have resulted
> + * in an overflow, return -1.
> + */
> +int
> +sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg);
> +
> +int
> +sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg);
> +
> +int
> +sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg);
> +
> +int
> +sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg);
> +
> +int
> +sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg);
2. That function is usually called 'mod', not 'rem'.
> +
> u8 sqlGetBoolean(const char *z, u8);
>
> const void *sqlValueText(sql_value *);
> diff --git a/src/box/sql/util.c b/src/box/sql/util.c
> index 2e7c298e1..ee6a83ad5 100644
> --- a/src/box/sql/util.c
> +++ b/src/box/sql/util.c
> @@ -1194,89 +1194,196 @@ sqlSafetyCheckSickOrOk(sql * db)
> }
> }
>
> -/*
> - * Attempt to add, substract, or multiply the 64-bit signed value iB against
> - * the other 64-bit signed integer at *pA and store the result in *pA.
> - * Return 0 on success. Or if the operation would have resulted in an
> - * overflow, leave *pA unchanged and return 1.
> - */
> int
> -sqlAddInt64(i64 * pA, i64 iB)
> +sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg)
> {
> - i64 iA = *pA;
> - testcase(iA == 0);
> - testcase(iA == 1);
> - testcase(iB == -1);
> - testcase(iB == 0);
> - if (iB >= 0) {
> - testcase(iA > 0 && LARGEST_INT64 - iA == iB);
> - testcase(iA > 0 && LARGEST_INT64 - iA == iB - 1);
> - if (iA > 0 && LARGEST_INT64 - iA < iB)
> - return 1;
> - } else {
> - testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 1);
> - testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 2);
> - if (iA < 0 && -(iA + LARGEST_INT64) > iB + 1)
> - return 1;
> + /* Addition of two negative integers. */
> + if (is_lhs_neg && is_rhs_neg) {
> + assert(lhs < 0 && rhs < 0);
> + /* This is the same as (lhs + rhs) < INT64_MIN */
> + if (-(lhs + INT64_MAX) > rhs + 1)
> + return -1;
3. Why so complex? Why not 'lhs < INT64_MIN - rhs'? I did it, and the
tests pass. If they should not, then please, provide a test.
> + *is_res_neg = true;
> + *res = lhs + rhs;
> + return 0;
> }
> - *pA += iB;
> + /* Both are unsigned integers. */
> + if (!is_lhs_neg && !is_rhs_neg) {
> + uint64_t u_lhs = (uint64_t) lhs;
> + uint64_t u_rhs = (uint64_t) rhs;
> + /* This is the same as (lhs + rhs) > UINT64_MAX */
> + if (UINT64_MAX - u_lhs < u_rhs)
> + return -1;
> + *is_res_neg = false;
> + *res = lhs + rhs;
> + return 0;
> + }
> + /*
> + * Make sure we've got only one combination of
> + * positive and negative operands.
> + */
> + if (is_lhs_neg) {
> + SWAP(is_lhs_neg, is_rhs_neg);
> + SWAP(lhs, rhs);
> + }
> + assert(! is_lhs_neg && is_rhs_neg);
4. You have spent more code lines on the SWAP and its
explanation, than could on avoidance of this SWAP. In
addition, SWAP version is slower obviously.
- /*
- * Make sure we've got only one combination of
- * positive and negative operands.
- */
- if (is_lhs_neg) {
- SWAP(is_lhs_neg, is_rhs_neg);
- SWAP(lhs, rhs);
- }
- assert(! is_lhs_neg && is_rhs_neg);
- *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ if (is_rhs_neg)
+ *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ else
+ *is_res_neg = (uint64_t)(-lhs) > (uint64_t) rhs;
*res = lhs + rhs;
return 0;
> + *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
> + *res = lhs + rhs;
> return 0;
> }
>
> int
> -sqlSubInt64(i64 * pA, i64 iB)
> +sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg)
> {
> - testcase(iB == SMALLEST_INT64 + 1);
> - if (iB == SMALLEST_INT64) {
> - testcase((*pA) == (-1));
> - testcase((*pA) == 0);
> - if ((*pA) >= 0)
> - return 1;
> - *pA -= iB;
> + if (!is_lhs_neg && !is_rhs_neg) {
> + uint64_t u_lhs = (uint64_t) lhs;
> + uint64_t u_rhs = (uint64_t) rhs;
> + /*
> + * Unsigned arithmetic has no overflows, so to
> + * check if lhs is less than rhs we can compare
> + * result of their subtraction with lhs.
5. Why not to compare just the values? They are of the same
sign, use '<'.
> + */
> + *is_res_neg = u_lhs < (uint64_t)(u_lhs - u_rhs);
> + if (! *is_res_neg) {
> + *res = u_lhs - u_rhs;
> + return 0;
> + }
> + if (u_rhs > (uint64_t) INT64_MAX + 1 &&
> + u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
6. Too complex. You wanted to check u_lhs - u_rhs < INT64_MIN.
Then check it, in a one comparison.
u_lhs - u_rhs < INT64_MIN
But u_lhs is < u_rhs, so swap them and change the signs:
u_rhs - u_lhs > (uint64_t) INT64_MAX + 1
No overflows are possible here.
> + return -1;
> + *res = lhs - rhs;
> return 0;
> - } else {
> - return sqlAddInt64(pA, -iB);
> }
> + if (is_rhs_neg) {
> + return sql_add_int(lhs, is_lhs_neg, -rhs, false, res,
> + is_res_neg);
> + }
> + assert(is_lhs_neg && !is_rhs_neg);
> + /*
> + * (lhs - rhs) < 0, lhs < 0, rhs > 0: in this case their
> + * difference must be less than INT64_MIN.
> + */
> + if ((uint64_t) -lhs + (uint64_t) rhs > (uint64_t) INT64_MAX + 1)
> + return -1;
> + *is_res_neg = true;
> + *res = lhs - rhs;
> + return 0;
> }
>
> int
> -sqlMulInt64(i64 * pA, i64 iB)
> +sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg)
> {
> - i64 iA = *pA;
> - if (iB > 0) {
> - if (iA > LARGEST_INT64 / iB)
> - return 1;
> - if (iA < SMALLEST_INT64 / iB)
> - return 1;
> - } else if (iB < 0) {
> - if (iA > 0) {
> - if (iB < SMALLEST_INT64 / iA)
> - return 1;
> - } else if (iA < 0) {
> - if (iB == SMALLEST_INT64)
> - return 1;
> - if (iA == SMALLEST_INT64)
> - return 1;
> - if (-iA > LARGEST_INT64 / -iB)
> - return 1;
> - }
> + if (lhs == 0 || rhs == 0) {
> + *res = 0;
> + *is_res_neg = false;
> + return 0;
> + }
> + /*
> + * Multiplication of integers with same sign leads to
> + * unsigned result.
> + */
> + if (! (is_lhs_neg ^ is_rhs_neg)) {
7. Dear God, what is it? What about '!=' operator? Is it
disabled, or !^ is faster?
> + uint64_t u_res = is_lhs_neg ?
> + (uint64_t) (-lhs) * (uint64_t) (-rhs) :
> + (uint64_t) lhs * (uint64_t) (rhs);
> + /*
> + * Overflow detection is quite primitive due to
> + * the absence of overflow with unsigned values:
> + * lhs * rhs == res --> rhs == res / lhs;
> + * If this predicate is false, then result was
> + * reduced modulo UINT_MAX + 1.
> + */
> + if ((is_lhs_neg && u_res / (uint64_t) (-lhs) !=
> + (uint64_t) (-rhs)) ||
> + (!is_lhs_neg && u_res / (uint64_t) lhs != (uint64_t) rhs))
> + return -1;
> + *is_res_neg = false;
> + *res = u_res;
> + return 0;
> + }
> + /*
> + * Make sure we've got only one combination of
> + * positive and negative operands.
> + */
> + if (is_lhs_neg) {
> + SWAP(is_lhs_neg, is_rhs_neg);
> + SWAP(lhs, rhs);
> }
> - *pA = iA * iB;
> + assert(! is_lhs_neg && is_rhs_neg);
> + uint64_t u_rhs = (uint64_t) (-rhs);
> + uint64_t u_res = u_rhs * (uint64_t) lhs;
> + if (u_res / u_rhs != (uint64_t) lhs ||
> + u_rhs * (uint64_t) lhs > (uint64_t) INT64_MAX + 1)
8. You have already calculated that multiplication, it is
u_res.
> + return -1;
> + *is_res_neg = true;
> + *res = lhs * rhs;
> + return 0;
> +}
> +
> +int
> +sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg)
> +{
> + if (lhs == 0) {
> + *res = 0;
> + *is_res_neg = false;
> + return 0;
> + }
> + /*
> + * The only possible overflow situations is when operands
> + * of different signs and result turns out to be less
> + * than INT64_MIN.
> + */
> + if (is_lhs_neg ^ is_rhs_neg) {
9. 😢
> + uint64_t u_res = is_lhs_neg ?
> + (uint64_t) (-lhs) / (uint64_t) rhs :
> + (uint64_t) lhs / (uint64_t) (-rhs);
> + if (u_res > (uint64_t) INT64_MAX + 1)
> + return -1;
> + *is_res_neg = u_res != 0 ? true : false;
10. Does not it look tautological to you? 'if <bool> then true else false'.
> + *res = -u_res;
> + return 0;
> + }
> + *is_res_neg = false;
> + /*
> + * Another one special case: INT64_MIN / -1
> + * Signed division leads to program termination due
> + * to overflow.
> + */
> + if (is_lhs_neg && lhs == INT64_MIN && rhs == -1) {
> + *res = (uint64_t) INT64_MAX + 1;
> + return 0;
> + }
> + *res = is_lhs_neg ? (uint64_t) (lhs / rhs) :
11. Why do you need that cast? *res is int64_t, so it makes no
sense to cast the rvalue to uint64_t.
> + (uint64_t) lhs / (uint64_t) rhs;
> return 0;
> }
>
> -/*
> - * Compute the absolute value of a 32-bit signed integer, of possible. Or
> - * if the integer has a value of -2147483648, return +2147483647
> - */
> int
> -sqlAbsInt32(int x)
> +sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
> + int64_t *res, bool *is_res_neg)
> {
> - if (x >= 0)
> - return x;
> - if (x == (int)0x80000000)
> - return 0x7fffffff;
> - return -x;
> + if (is_lhs_neg) {
> + uint64_t u_lhs = (uint64_t) (-lhs);
> + uint64_t u_res = u_lhs % (uint64_t) rhs;
12. Why do you ignore sign of 'rhs'? You can't just cast it,
if it is negative.
> + if (u_res > (uint64_t) INT64_MAX + 1)
> + return -1;
> + *res = -u_res;
> + *is_res_neg = true;
> + return 0;
> + }
> + /*
> + * While calculating remainder we always ignore sign of
> + * rhs - it doesn't affect the result.
> + * */
> + uint64_t u_lhs = (uint64_t) lhs;
> + uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) : (uint64_t) rhs;
> + *res = u_lhs % u_rhs;
> + *is_res_neg = false;
> + return 0;
> }
>
> /*
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index d141397a0..9c28b9131 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1621,27 +1621,48 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
> (type2 & (MEM_Int | MEM_UInt)) !=0) {
> iA = pIn1->u.i;
> iB = pIn2->u.i;
> + bool is_lhs_neg = pIn1->flags & MEM_Int;
> + bool is_rhs_neg = pIn2->flags & MEM_Int;
> + bool is_res_neg;
> bIntint = 1;
> switch( pOp->opcode) {
> - case OP_Add: if (sqlAddInt64(&iB,iA)) goto integer_overflow; break;
> - case OP_Subtract: if (sqlSubInt64(&iB,iA)) goto integer_overflow; break;
> - case OP_Multiply: if (sqlMulInt64(&iB,iA)) goto integer_overflow; break;
> + case OP_Add: {
> + if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
> + (int64_t *) &iB, &is_res_neg) != 0)
13. Why do you need all these casts to (int64_t *)? These variables already
are signed 64 bit integers.
> + goto integer_overflow;
> + break;
> + }
> + case OP_Subtract: {
> + if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
> + (int64_t *) &iB, &is_res_neg) != 0)
> + goto integer_overflow;
> + break;
> + }
> + case OP_Multiply: {
> + if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
> + (int64_t *) &iB, &is_res_neg) != 0)
> + goto integer_overflow;
> + break;
> + }
> case OP_Divide: {
> if (iA == 0)
> goto division_by_zero;
> - if (iA==-1 && iB==SMALLEST_INT64) goto integer_overflow;
> - iB /= iA;
> + if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
> + (int64_t *) &iB, &is_res_neg) != 0)
> + goto integer_overflow;
> break;
> }
> default: {
> if (iA == 0)
> goto division_by_zero;
> if (iA==-1) iA = 1;
> - iB %= iA;
> + if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
> + (int64_t *) &iB, &is_res_neg) != 0)
> + goto integer_overflow;
> break;
> }
> }
> - mem_set_int(pOut, iB, iB < 0);
> + mem_set_int(pOut, iB, is_res_neg);
> } else {
> bIntint = 0;
> if (sqlVdbeRealValue(pIn1, &rA) != 0) {
> @@ -1883,13 +1904,13 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
> break;
> }
> bool unused;
> - if (sqlVdbeIntValue(pIn2, &iA, &unused) != 0) {
> + if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
> diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> sql_value_text(pIn2), "integer");
> rc = SQL_TARANTOOL_ERROR;
> goto abort_due_to_error;
> }
> - if (sqlVdbeIntValue(pIn1, &iB, &unused) != 0) {
> + if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
> diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> sql_value_text(pIn1), "integer");
> rc = SQL_TARANTOOL_ERROR;
> @@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
> assert(pIn1->flags & (MEM_Int | MEM_UInt));
> assert(pIn3->flags & (MEM_Int | MEM_UInt));
> x = pIn1->u.i;
> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
> + int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
> + bool unused;
> + if ((x == 0 || pIn1->flags & MEM_Int) ||
> + sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
14. If you get to this line, then (pIn1->flags & MEM_Int) is already
0 and can be inlined.
> + (int64_t *) &x, &unused) != 0) {
> /* If the LIMIT is less than or equal to zero, loop forever. This
> * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
> * also loop forever. This is undocumented. In fact, one could argue
Consider my review fixes here and on the branch in a
separate commit. These changes and the comments are arguable,
probably I am wrong somewhere.
=============================================================
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index c0e2ab699..de63d4a6f 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -4489,7 +4489,7 @@ sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
int64_t *res, bool *is_res_neg);
int
-sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+sql_mod_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
int64_t *res, bool *is_res_neg);
u8 sqlGetBoolean(const char *z, u8);
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 0688087d4..1cba4142b 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -1194,8 +1194,8 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (is_lhs_neg && is_rhs_neg) {
assert(lhs < 0 && rhs < 0);
/* This is the same as (lhs + rhs) < INT64_MIN */
- if (-(lhs + INT64_MAX) > rhs + 1)
- return -1;
+ if (lhs < INT64_MIN - rhs)
+ return -1;
*is_res_neg = true;
*res = lhs + rhs;
return 0;
@@ -1211,16 +1211,10 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
*res = lhs + rhs;
return 0;
}
- /*
- * Make sure we've got only one combination of
- * positive and negative operands.
- */
- if (is_lhs_neg) {
- SWAP(is_lhs_neg, is_rhs_neg);
- SWAP(lhs, rhs);
- }
- assert(! is_lhs_neg && is_rhs_neg);
- *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ if (is_rhs_neg)
+ *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ else
+ *is_res_neg = (uint64_t)(-lhs) > (uint64_t) rhs;
*res = lhs + rhs;
return 0;
}
@@ -1232,19 +1226,14 @@ sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (!is_lhs_neg && !is_rhs_neg) {
uint64_t u_lhs = (uint64_t) lhs;
uint64_t u_rhs = (uint64_t) rhs;
- /*
- * Unsigned arithmetic has no overflows, so to
- * check if lhs is less than rhs we can compare
- * result of their subtraction with lhs.
- */
- *is_res_neg = u_lhs < (uint64_t)(u_lhs - u_rhs);
- if (! *is_res_neg) {
+ if (u_lhs >= u_rhs) {
+ *is_res_neg = false;
*res = u_lhs - u_rhs;
return 0;
}
- if (u_rhs > (uint64_t) INT64_MAX + 1 &&
- u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
+ if ((uint64_t) INT64_MAX + 1 < u_rhs - u_lhs)
return -1;
+ *is_res_neg = true;
*res = lhs - rhs;
return 0;
}
@@ -1277,7 +1266,7 @@ sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
* Multiplication of integers with same sign leads to
* unsigned result.
*/
- if (! (is_lhs_neg ^ is_rhs_neg)) {
+ if (is_lhs_neg == is_rhs_neg) {
uint64_t u_res = is_lhs_neg ?
(uint64_t) (-lhs) * (uint64_t) (-rhs) :
(uint64_t) lhs * (uint64_t) (rhs);
@@ -1308,7 +1297,7 @@ sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
uint64_t u_rhs = (uint64_t) (-rhs);
uint64_t u_res = u_rhs * (uint64_t) lhs;
if (u_res / u_rhs != (uint64_t) lhs ||
- u_rhs * (uint64_t) lhs > (uint64_t) INT64_MAX + 1)
+ u_res > (uint64_t) INT64_MAX + 1)
return -1;
*is_res_neg = true;
*res = lhs * rhs;
@@ -1329,13 +1318,13 @@ sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
* of different signs and result turns out to be less
* than INT64_MIN.
*/
- if (is_lhs_neg ^ is_rhs_neg) {
+ if (is_lhs_neg != is_rhs_neg) {
uint64_t u_res = is_lhs_neg ?
(uint64_t) (-lhs) / (uint64_t) rhs :
(uint64_t) lhs / (uint64_t) (-rhs);
if (u_res > (uint64_t) INT64_MAX + 1)
return -1;
- *is_res_neg = u_res != 0 ? true : false;
+ *is_res_neg = u_res != 0;
*res = -u_res;
return 0;
}
@@ -1349,13 +1338,12 @@ sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
*res = (uint64_t) INT64_MAX + 1;
return 0;
}
- *res = is_lhs_neg ? (uint64_t) (lhs / rhs) :
- (uint64_t) lhs / (uint64_t) rhs;
+ *res = is_lhs_neg ? lhs / rhs : (uint64_t) lhs / (uint64_t) rhs;
return 0;
}
int
-sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
+sql_mod_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
int64_t *res, bool *is_res_neg)
{
if (is_lhs_neg) {
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4379bd123..7103db1c9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1619,19 +1619,19 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
switch( pOp->opcode) {
case OP_Add: {
if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
+ &iB, &is_res_neg) != 0)
goto integer_overflow;
break;
}
case OP_Subtract: {
if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
+ &iB, &is_res_neg) != 0)
goto integer_overflow;
break;
}
case OP_Multiply: {
if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
+ &iB, &is_res_neg) != 0)
goto integer_overflow;
break;
}
@@ -1639,7 +1639,7 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
if (iA == 0)
goto division_by_zero;
if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
+ &iB, &is_res_neg) != 0)
goto integer_overflow;
break;
}
@@ -1647,8 +1647,8 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
if (iA == 0)
goto division_by_zero;
if (iA==-1) iA = 1;
- if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
+ if (sql_mod_int(iB, is_rhs_neg, iA, is_lhs_neg,
+ &iB, &is_res_neg) != 0)
goto integer_overflow;
break;
}
@@ -1895,13 +1895,13 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
break;
}
bool unused;
- if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
+ if (sqlVdbeIntValue(pIn2, &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn2), "integer");
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
- if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
+ if (sqlVdbeIntValue(pIn1, &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sql_value_text(pIn1), "integer");
rc = SQL_TARANTOOL_ERROR;
@@ -5165,9 +5165,8 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
x = pIn1->u.i;
int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
bool unused;
- if ((x == 0 || pIn1->flags & MEM_Int) ||
- sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
- (int64_t *) &x, &unused) != 0) {
+ if (x == 0 || (pIn1->flags & MEM_Int) != 0 ||
+ sql_add_int(x, false, rhs, false, &x, &unused) != 0) {
/* If the LIMIT is less than or equal to zero, loop forever. This
* is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
* also loop forever. This is undocumented. In fact, one could argue
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-01 14:21 ` n.pettik
2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-01 14:21 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index f4c1cbcca..457c9b92b 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -1620,9 +1622,13 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
>> p->cnt++;
>> if (type == MP_INT || type == MP_UINT) {
>> int64_t v = sql_value_int64(argv[0]);
>> - p->rSum += v;
>> + if (type == MP_INT)
>> + p->rSum += v;
>> + else
>> + p->rSum += (uint64_t) v;
>> if ((p->approx | p->overflow) == 0 &&
>> - sqlAddInt64(&p->iSum, v) != 0) {
>> + sql_add_int(p->iSum, p->is_neg, v, v < 0, &p->iSum,
>
> 1. As I understand, now sql_value_int64() returns an integer buffer,
> and its result can't be compared with 0. How do you do it here with
> 'v’?
It is fixed in the next patch. Fix itself is obvious. Moved this change
to current patch.
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 457c9b92b..77340c7fb 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1626,8 +1626,9 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
p->rSum += v;
else
p->rSum += (uint64_t) v;
+
if ((p->approx | p->overflow) == 0 &&
- sql_add_int(p->iSum, p->is_neg, v, v < 0, &p->iSum,
+ sql_add_int(p->iSum, p->is_neg, v, type == MP_INT, &p->iSum,
&p->is_neg) != 0) {
p->overflow = 1;
>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>> index da17e24ed..c0e2ab699 100644
>> --- a/src/box/sql/sqlInt.h
>> +++ b/src/box/sql/sqlInt.h
>> @@ -4461,10 +4461,37 @@ Expr *sqlExprAddCollateString(Parse *, Expr *, const char *);
>> +int
>> +sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg);
>> +
>> +int
>> +sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg);
>> +
>> +int
>> +sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg);
>> +
>> +int
>> +sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg);
>> +
>> +int
>> +sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg);
>
> 2. That function is usually called 'mod', not 'rem’.
Opcode is called OP_Reminder, so to avoid confusions
I guess it would be better to keep _rem_ name. If you
insist I will rename it.
>> diff --git a/src/box/sql/util.c b/src/box/sql/util.c
>> index 2e7c298e1..ee6a83ad5 100644
>> --- a/src/box/sql/util.c
>> +++ b/src/box/sql/util.c
>> @@ -1194,89 +1194,196 @@ sqlSafetyCheckSickOrOk(sql * db)
>> }
>> }
>>
>> -/*
>> - * Attempt to add, substract, or multiply the 64-bit signed value iB against
>> - * the other 64-bit signed integer at *pA and store the result in *pA.
>> - * Return 0 on success. Or if the operation would have resulted in an
>> - * overflow, leave *pA unchanged and return 1.
>> - */
>> int
>> -sqlAddInt64(i64 * pA, i64 iB)
>> +sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg)
>> {
>> - i64 iA = *pA;
>> - testcase(iA == 0);
>> - testcase(iA == 1);
>> - testcase(iB == -1);
>> - testcase(iB == 0);
>> - if (iB >= 0) {
>> - testcase(iA > 0 && LARGEST_INT64 - iA == iB);
>> - testcase(iA > 0 && LARGEST_INT64 - iA == iB - 1);
>> - if (iA > 0 && LARGEST_INT64 - iA < iB)
>> - return 1;
>> - } else {
>> - testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 1);
>> - testcase(iA < 0 && -(iA + LARGEST_INT64) == iB + 2);
>> - if (iA < 0 && -(iA + LARGEST_INT64) > iB + 1)
>> - return 1;
>> + /* Addition of two negative integers. */
>> + if (is_lhs_neg && is_rhs_neg) {
>> + assert(lhs < 0 && rhs < 0);
>> + /* This is the same as (lhs + rhs) < INT64_MIN */
>> + if (-(lhs + INT64_MAX) > rhs + 1)
>> + return -1;
>
> 3. Why so complex? Why not 'lhs < INT64_MIN - rhs'? I did it, and the
> tests pass. If they should not, then please, provide a test.
Well, it is the same condition. Let’s use your variant:
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 3cbcda04f..73d7b95b6 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -1199,7 +1199,7 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (is_lhs_neg && is_rhs_neg) {
assert(lhs < 0 && rhs < 0);
/* This is the same as (lhs + rhs) < INT64_MIN */
- if (-(lhs + INT64_MAX) > rhs + 1)
+ if (lhs < INT64_MIN - rhs)
return -1;
*is_res_neg = true;
*res = lhs + rhs;
>> + /*
>> + * Make sure we've got only one combination of
>> + * positive and negative operands.
>> + */
>> + if (is_lhs_neg) {
>> + SWAP(is_lhs_neg, is_rhs_neg);
>> + SWAP(lhs, rhs);
>> + }
>> + assert(! is_lhs_neg && is_rhs_neg);
>
> 4. You have spent more code lines on the SWAP and its
> explanation, than could on avoidance of this SWAP. In
> addition, SWAP version is slower obviously.
Ok, np, refactored:
@@ -1216,16 +1216,8 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
*res = lhs + rhs;
return 0;
}
- /*
- * Make sure we've got only one combination of
- * positive and negative operands.
- */
- if (is_lhs_neg) {
- SWAP(is_lhs_neg, is_rhs_neg);
- SWAP(lhs, rhs);
- }
- assert(! is_lhs_neg && is_rhs_neg);
- *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
+ *is_res_neg = is_rhs_neg ? (uint64_t)(-rhs) > (uint64_t) lhs :
+ (uint64_t)(-lhs) > (uint64_t) rhs;
*res = lhs + rhs;
return 0;
}
>
> - /*
> - * Make sure we've got only one combination of
> - * positive and negative operands.
> - */
> - if (is_lhs_neg) {
> - SWAP(is_lhs_neg, is_rhs_neg);
> - SWAP(lhs, rhs);
> - }
> - assert(! is_lhs_neg && is_rhs_neg);
> - *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
> + if (is_rhs_neg)
> + *is_res_neg = (uint64_t)(-rhs) > (uint64_t) lhs;
> + else
> + *is_res_neg = (uint64_t)(-lhs) > (uint64_t) rhs;
> *res = lhs + rhs;
> return 0;
>> int
>> -sqlSubInt64(i64 * pA, i64 iB)
>> +sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg)
>> {
>> - testcase(iB == SMALLEST_INT64 + 1);
>> - if (iB == SMALLEST_INT64) {
>> - testcase((*pA) == (-1));
>> - testcase((*pA) == 0);
>> - if ((*pA) >= 0)
>> - return 1;
>> - *pA -= iB;
>> + if (!is_lhs_neg && !is_rhs_neg) {
>> + uint64_t u_lhs = (uint64_t) lhs;
>> + uint64_t u_rhs = (uint64_t) rhs;
>> + /*
>> + * Unsigned arithmetic has no overflows, so to
>> + * check if lhs is less than rhs we can compare
>> + * result of their subtraction with lhs.
>
> 5. Why not to compare just the values? They are of the same
> sign, use '<‘.
Thx, fixed:
@@ -1237,19 +1229,15 @@ sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (!is_lhs_neg && !is_rhs_neg) {
uint64_t u_lhs = (uint64_t) lhs;
uint64_t u_rhs = (uint64_t) rhs;
- /*
- * Unsigned arithmetic has no overflows, so to
- * check if lhs is less than rhs we can compare
- * result of their subtraction with lhs.
- */
- *is_res_neg = u_lhs < (uint64_t)(u_lhs - u_rhs);
- if (! *is_res_neg) {
+ if (u_lhs >= u_rhs) {
+ *is_res_neg = false;
*res = u_lhs - u_rhs;
return 0;
}
if (u_rhs > (uint64_t) INT64_MAX + 1 &&
u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
return -1;
+ *is_res_neg = true;
*res = lhs - rhs;
return 0;
}
>> + */
>> + *is_res_neg = u_lhs < (uint64_t)(u_lhs - u_rhs);
>> + if (! *is_res_neg) {
>> + *res = u_lhs - u_rhs;
>> + return 0;
>> + }
>> + if (u_rhs > (uint64_t) INT64_MAX + 1 &&
>> + u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
>
> 6. Too complex. You wanted to check u_lhs - u_rhs < INT64_MIN.
> Then check it, in a one comparison.
>
> u_lhs - u_rhs < INT64_MIN
>
> But u_lhs is < u_rhs, so swap them and change the signs:
>
> u_rhs - u_lhs > (uint64_t) INT64_MAX + 1
>
> No overflows are possible here.
Thx! Applied this diff:
- if (u_rhs > (uint64_t) INT64_MAX + 1 &&
- u_lhs < (uint64_t)(u_rhs - INT64_MAX - 1))
+ if (u_rhs - u_lhs > (uint64_t) INT64_MAX + 1)
return -1;
>> int
>> -sqlMulInt64(i64 * pA, i64 iB)
>> +sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg)
>> {
>> - i64 iA = *pA;
>> - if (iB > 0) {
>> - if (iA > LARGEST_INT64 / iB)
>> - return 1;
>> - if (iA < SMALLEST_INT64 / iB)
>> - return 1;
>> - } else if (iB < 0) {
>> - if (iA > 0) {
>> - if (iB < SMALLEST_INT64 / iA)
>> - return 1;
>> - } else if (iA < 0) {
>> - if (iB == SMALLEST_INT64)
>> - return 1;
>> - if (iA == SMALLEST_INT64)
>> - return 1;
>> - if (-iA > LARGEST_INT64 / -iB)
>> - return 1;
>> - }
>> + if (lhs == 0 || rhs == 0) {
>> + *res = 0;
>> + *is_res_neg = false;
>> + return 0;
>> + }
>> + /*
>> + * Multiplication of integers with same sign leads to
>> + * unsigned result.
>> + */
>> + if (! (is_lhs_neg ^ is_rhs_neg)) {
>
> 7. Dear God, what is it? What about '!=' operator? Is it
> disabled, or !^ is faster?
Ok, replaced:
@@ -1282,7 +1269,7 @@ sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
* Multiplication of integers with same sign leads to
* unsigned result.
*/
- if (! (is_lhs_neg ^ is_rhs_neg)) {
+ if (is_lhs_neg == is_rhs_neg) {
uint64_t u_res = is_lhs_neg ?
(uint64_t) (-lhs) * (uint64_t) (-rhs) :
(uint64_t) lhs * (uint64_t) (rhs);
>> + uint64_t u_res = is_lhs_neg ?
>> + (uint64_t) (-lhs) * (uint64_t) (-rhs) :
>> + (uint64_t) lhs * (uint64_t) (rhs);
>> + /*
>> + * Overflow detection is quite primitive due to
>> + * the absence of overflow with unsigned values:
>> + * lhs * rhs == res --> rhs == res / lhs;
>> + * If this predicate is false, then result was
>> + * reduced modulo UINT_MAX + 1.
>> + */
>> + if ((is_lhs_neg && u_res / (uint64_t) (-lhs) !=
>> + (uint64_t) (-rhs)) ||
>> + (!is_lhs_neg && u_res / (uint64_t) lhs != (uint64_t) rhs))
>> + return -1;
>> + *is_res_neg = false;
>> + *res = u_res;
>> + return 0;
>> + }
>> + /*
>> + * Make sure we've got only one combination of
>> + * positive and negative operands.
>> + */
>> + if (is_lhs_neg) {
>> + SWAP(is_lhs_neg, is_rhs_neg);
>> + SWAP(lhs, rhs);
>> }
>> - *pA = iA * iB;
>> + assert(! is_lhs_neg && is_rhs_neg);
>> + uint64_t u_rhs = (uint64_t) (-rhs);
>> + uint64_t u_res = u_rhs * (uint64_t) lhs;
>> + if (u_res / u_rhs != (uint64_t) lhs ||
>> + u_rhs * (uint64_t) lhs > (uint64_t) INT64_MAX + 1)
>
> 8. You have already calculated that multiplication, it is
> u_res.
@@ -1313,7 +1300,7 @@ sql_mul_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
uint64_t u_rhs = (uint64_t) (-rhs);
uint64_t u_res = u_rhs * (uint64_t) lhs;
if (u_res / u_rhs != (uint64_t) lhs ||
- u_rhs * (uint64_t) lhs > (uint64_t) INT64_MAX + 1)
+ u_res > (uint64_t) INT64_MAX + 1)
return -1;
*is_res_neg = true;
*res = lhs * rhs;
>> +int
>> +sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg)
>> +{
>> + if (lhs == 0) {
>> + *res = 0;
>> + *is_res_neg = false;
>> + return 0;
>> + }
>> + /*
>> + * The only possible overflow situations is when operands
>> + * of different signs and result turns out to be less
>> + * than INT64_MIN.
>> + */
>> + if (is_lhs_neg ^ is_rhs_neg) {
>
> 9. 😢
@@ -1334,7 +1321,7 @@ sql_div_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
* of different signs and result turns out to be less
* than INT64_MIN.
*/
- if (is_lhs_neg ^ is_rhs_neg) {
+ if (is_lhs_neg != is_rhs_neg) {
uint64_t u_res = is_lhs_neg ?
(uint64_t) (-lhs) / (uint64_t) rhs :
(uint64_t) lhs / (uint64_t) (-rhs);
>> + uint64_t u_res = is_lhs_neg ?
>> + (uint64_t) (-lhs) / (uint64_t) rhs :
>> + (uint64_t) lhs / (uint64_t) (-rhs);
>> + if (u_res > (uint64_t) INT64_MAX + 1)
>> + return -1;
>> + *is_res_neg = u_res != 0 ? true : false;
>
> 10. Does not it look tautological to you? 'if <bool> then true else false’.
if (u_res > (uint64_t) INT64_MAX + 1)
return -1;
- *is_res_neg = u_res != 0 ? true : false;
+ *is_res_neg = u_res != 0;
*res = -u_res;
return 0;
}
>> + *res = -u_res;
>> + return 0;
>> + }
>> + *is_res_neg = false;
>> + /*
>> + * Another one special case: INT64_MIN / -1
>> + * Signed division leads to program termination due
>> + * to overflow.
>> + */
>> + if (is_lhs_neg && lhs == INT64_MIN && rhs == -1) {
>> + *res = (uint64_t) INT64_MAX + 1;
>> + return 0;
>> + }
>> + *res = is_lhs_neg ? (uint64_t) (lhs / rhs) :
>
> 11. Why do you need that cast? *res is int64_t, so it makes no
> sense to cast the rvalue to uint64_t.
Otherwise compilation error is raised (on linux target):
/tarantool/src/box/sql/util.c: In function ‘sql_div_int’:
/tarantool/src/box/sql/util.c:1342:34: error: signed and unsigned type in conditional expression [-Werror=sign-compare]
*res = is_lhs_neg ? (lhs / rhs) : (uint64_t) lhs / (uint64_t) rhs;
>> + (uint64_t) lhs / (uint64_t) rhs;
>> return 0;
>> }
>>
>> -/*
>> - * Compute the absolute value of a 32-bit signed integer, of possible. Or
>> - * if the integer has a value of -2147483648, return +2147483647
>> - */
>> int
>> -sqlAbsInt32(int x)
>> +sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
>> + int64_t *res, bool *is_res_neg)
>> {
>> - if (x >= 0)
>> - return x;
>> - if (x == (int)0x80000000)
>> - return 0x7fffffff;
>> - return -x;
>> + if (is_lhs_neg) {
>> + uint64_t u_lhs = (uint64_t) (-lhs);
>> + uint64_t u_res = u_lhs % (uint64_t) rhs;
>
> 12. Why do you ignore sign of 'rhs'? You can't just cast it,
> if it is negative.
Yep, you are right, fixed:
@@ -1365,7 +1351,9 @@ sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
{
if (is_lhs_neg) {
uint64_t u_lhs = (uint64_t) (-lhs);
- uint64_t u_res = u_lhs % (uint64_t) rhs;
+ uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) :
+ (uint64_t) rhs;
+ uint64_t u_res = u_lhs % u_rhs;
if (u_res > (uint64_t) INT64_MAX + 1)
return -1;
*res = -u_res;
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index d141397a0..9c28b9131 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -1621,27 +1621,48 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
>> (type2 & (MEM_Int | MEM_UInt)) !=0) {
>> iA = pIn1->u.i;
>> iB = pIn2->u.i;
>> + bool is_lhs_neg = pIn1->flags & MEM_Int;
>> + bool is_rhs_neg = pIn2->flags & MEM_Int;
>> + bool is_res_neg;
>> bIntint = 1;
>> switch( pOp->opcode) {
>> - case OP_Add: if (sqlAddInt64(&iB,iA)) goto integer_overflow; break;
>> - case OP_Subtract: if (sqlSubInt64(&iB,iA)) goto integer_overflow; break;
>> - case OP_Multiply: if (sqlMulInt64(&iB,iA)) goto integer_overflow; break;
>> + case OP_Add: {
>> + if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
>> + (int64_t *) &iB, &is_res_neg) != 0)
>
> 13. Why do you need all these casts to (int64_t *)? These variables already
> are signed 64 bit integers.
On linux (Travis targets) I get compilation errors since there
they (i64 and int64_t) are considered to be different types.
>> @@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>> assert(pIn1->flags & (MEM_Int | MEM_UInt));
>> assert(pIn3->flags & (MEM_Int | MEM_UInt));
>> x = pIn1->u.i;
>> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
>> + int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
>> + bool unused;
>> + if ((x == 0 || pIn1->flags & MEM_Int) ||
>> + sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
>
> 14. If you get to this line, then (pIn1->flags & MEM_Int) is already
> 0 and can be inlined.
Wait, why? If x == 0 then pIn1->flags == MEM_UInt -
we consider 0 as an unsigned value.
>> + (int64_t *) &x, &unused) != 0) {
>> /* If the LIMIT is less than or equal to zero, loop forever. This
>> * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
>> * also loop forever. This is undocumented. In fact, one could argue
>
> Consider my review fixes here and on the branch in a
> separate commit. These changes and the comments are arguable,
> probably I am wrong somewhere.
Almost all applied (see diffs above).
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-07-01 14:21 ` n.pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
2019-07-05 16:36 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
>>> @@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>>> assert(pIn1->flags & (MEM_Int | MEM_UInt));
>>> assert(pIn3->flags & (MEM_Int | MEM_UInt));
>>> x = pIn1->u.i;
>>> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
>>> + int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
>>> + bool unused;
>>> + if ((x == 0 || pIn1->flags & MEM_Int) ||
>>> + sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
>>
>> 14. If you get to this line, then (pIn1->flags & MEM_Int) is already
>> 0 and can be inlined.
>
> Wait, why? If x == 0 then pIn1->flags == MEM_UInt -
> we consider 0 as an unsigned value.
Because you can only get to sql_add_int(), if x != 0 and
pIn1->flags & MEM_Int == 0. It is the C standard. In an
expression (a || b) 'b' is executed iff 'a' is false.
Looks like that place is not tested at all. The tests pass
regardless of how I call sql_add_int: with pIn1->flags & MEM_Int -> false
or true.
Please, inline the value (false), and add a test, which would fail, if
I put here true.
> if (is_lhs_neg) {
> uint64_t u_lhs = (uint64_t) (-lhs);
> uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) :
> (uint64_t) rhs;
> uint64_t u_res = u_lhs % u_rhs;
> if (u_res > (uint64_t) INT64_MAX + 1)
> return -1;
Please, add a test for this error. I've removed that check,
and the tests passed.
Consider new fixes below, and on the branch
in a separate commit.
=====================================================
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index d58c0c6e6..1bdaa24e5 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -952,7 +952,6 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
/* Addition of two negative integers. */
if (is_lhs_neg && is_rhs_neg) {
assert(lhs < 0 && rhs < 0);
- /* This is the same as (lhs + rhs) < INT64_MIN */
if (lhs < INT64_MIN - rhs)
return -1;
*is_res_neg = true;
@@ -963,7 +962,6 @@ sql_add_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (!is_lhs_neg && !is_rhs_neg) {
uint64_t u_lhs = (uint64_t) lhs;
uint64_t u_rhs = (uint64_t) rhs;
- /* This is the same as (lhs + rhs) > UINT64_MAX */
if (UINT64_MAX - u_lhs < u_rhs)
return -1;
*is_res_neg = false;
@@ -1001,7 +999,7 @@ sql_sub_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
assert(is_lhs_neg && !is_rhs_neg);
/*
* (lhs - rhs) < 0, lhs < 0, rhs > 0: in this case their
- * difference must be less than INT64_MIN.
+ * difference must not be less than INT64_MIN.
*/
if ((uint64_t) -lhs + (uint64_t) rhs > (uint64_t) INT64_MAX + 1)
return -1;
@@ -1104,10 +1102,9 @@ int
sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
int64_t *res, bool *is_res_neg)
{
+ uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) : (uint64_t) rhs;
if (is_lhs_neg) {
uint64_t u_lhs = (uint64_t) (-lhs);
- uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) :
- (uint64_t) rhs;
uint64_t u_res = u_lhs % u_rhs;
if (u_res > (uint64_t) INT64_MAX + 1)
return -1;
@@ -1120,7 +1117,6 @@ sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
* rhs - it doesn't affect the result.
* */
uint64_t u_lhs = (uint64_t) lhs;
- uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) : (uint64_t) rhs;
*res = u_lhs % u_rhs;
*is_res_neg = false;
return 0;
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-07-01 21:53 ` Vladislav Shpilevoy
@ 2019-07-05 16:36 ` n.pettik
2019-07-10 22:49 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-05 16:36 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 2 Jul 2019, at 00:53, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Thanks for the fixes!
>
>>>> @@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>>>> assert(pIn1->flags & (MEM_Int | MEM_UInt));
>>>> assert(pIn3->flags & (MEM_Int | MEM_UInt));
>>>> x = pIn1->u.i;
>>>> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
>>>> + int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
>>>> + bool unused;
>>>> + if ((x == 0 || pIn1->flags & MEM_Int) ||
>>>> + sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
>>>
>>> 14. If you get to this line, then (pIn1->flags & MEM_Int) is already
>>> 0 and can be inlined.
>>
>> Wait, why? If x == 0 then pIn1->flags == MEM_UInt -
>> we consider 0 as an unsigned value.
>
> Because you can only get to sql_add_int(), if x != 0 and
> pIn1->flags & MEM_Int == 0. It is the C standard. In an
> expression (a || b) 'b' is executed iff 'a' is false.
My bad, never mind.
> Looks like that place is not tested at all. The tests pass
> regardless of how I call sql_add_int: with pIn1->flags & MEM_Int -> false
> or true.
In fact, the reason is the same as for OP_DecrJumpZero and
OP_IfNotZero: P1 is a limit counter and P3 is an offset counter.
Hence, both are >= 0, it is checked before these opcodes are
executed. Added fix to the previous patch:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index f864ef950..81005d14a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4929,18 +4929,16 @@ case OP_IfPos: { /* jump, in1 */
* Otherwise, r[P2] is set to the sum of r[P1] and r[P3].
*/
case OP_OffsetLimit: { /* in1, out2, in3 */
- i64 x;
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
pOut = out2Prerelease(p, pOp);
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
- assert((pIn3->flags & (MEM_Int | MEM_UInt)) != 0);
- x = pIn1->u.i;
- int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
+ assert((pIn1->flags & MEM_UInt) != 0);
+ assert((pIn3->flags & MEM_UInt) != 0);
+ uint64_t x = pIn1->u.u;
+ uint64_t rhs = pIn3->u.u;
bool unused;
- if ((x == 0 || pIn1->flags & MEM_Int) ||
- sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
- (int64_t *) &x, &unused) != 0) {
+ if (x == 0 || sql_add_int(x, false, rhs, false, (int64_t *) &x,
+ &unused) != 0) {
> Please, inline the value (false), and add a test, which would fail, if
> I put here true.
>
>> if (is_lhs_neg) {
>> uint64_t u_lhs = (uint64_t) (-lhs);
>> uint64_t u_rhs = is_rhs_neg ? (uint64_t) (-rhs) :
>> (uint64_t) rhs;
>> uint64_t u_res = u_lhs % u_rhs;
>> if (u_res > (uint64_t) INT64_MAX + 1)
>> return -1;
>
> Please, add a test for this error. I've removed that check,
> and the tests passed.
Indeed, this check is redundant: we are ignoring the sign
of rhs, so if lhs is negative then the result is negative as
well. Hence, it is always less than INT64_MAX:
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index 1bdaa24e5..161c1f607 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -1106,8 +1106,6 @@ sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
if (is_lhs_neg) {
uint64_t u_lhs = (uint64_t) (-lhs);
uint64_t u_res = u_lhs % u_rhs;
- if (u_res > (uint64_t) INT64_MAX + 1)
- return -1;
*res = -u_res;
*is_res_neg = true;
return 0;
> Consider new fixes below, and on the branch
> in a separate commit.
Applied.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-07-05 16:36 ` n.pettik
@ 2019-07-10 22:49 ` Vladislav Shpilevoy
2019-07-17 12:24 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-10 22:49 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi! Thanks for the fixes!
On 05/07/2019 18:36, n.pettik wrote:
>
>
>> On 2 Jul 2019, at 00:53, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>>
>> Thanks for the fixes!
>>
>>>>> @@ -5134,7 +5175,11 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
>>>>> assert(pIn1->flags & (MEM_Int | MEM_UInt));
>>>>> assert(pIn3->flags & (MEM_Int | MEM_UInt));
>>>>> x = pIn1->u.i;
>>>>> - if (x<=0 || sqlAddInt64(&x, pIn3->u.i > 0 ? pIn3->u.i : 0)) {
>>>>> + int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
>>>>> + bool unused;
>>>>> + if ((x == 0 || pIn1->flags & MEM_Int) ||
>>>>> + sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
>>>>
>>>> 14. If you get to this line, then (pIn1->flags & MEM_Int) is already
>>>> 0 and can be inlined.
>>>
>>> Wait, why? If x == 0 then pIn1->flags == MEM_UInt -
>>> we consider 0 as an unsigned value.
>>
>> Because you can only get to sql_add_int(), if x != 0 and
>> pIn1->flags & MEM_Int == 0. It is the C standard. In an
>> expression (a || b) 'b' is executed iff 'a' is false.
>
> My bad, never mind.
>
>> Looks like that place is not tested at all. The tests pass
>> regardless of how I call sql_add_int: with pIn1->flags & MEM_Int -> false
>> or true.
>
> In fact, the reason is the same as for OP_DecrJumpZero and
> OP_IfNotZero: P1 is a limit counter and P3 is an offset counter.
> Hence, both are >= 0, it is checked before these opcodes are
> executed. Added fix to the previous patch:
>
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index f864ef950..81005d14a 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -4929,18 +4929,16 @@ case OP_IfPos: { /* jump, in1 */
> * Otherwise, r[P2] is set to the sum of r[P1] and r[P3].
> */
> case OP_OffsetLimit: { /* in1, out2, in3 */
> - i64 x;
> pIn1 = &aMem[pOp->p1];
> pIn3 = &aMem[pOp->p3];
> pOut = out2Prerelease(p, pOp);
> - assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
> - assert((pIn3->flags & (MEM_Int | MEM_UInt)) != 0);
> - x = pIn1->u.i;
> - int64_t rhs = pIn3->flags & MEM_Int ? 0 : pIn3->u.u;
> + assert((pIn1->flags & MEM_UInt) != 0);
> + assert((pIn3->flags & MEM_UInt) != 0);
> + uint64_t x = pIn1->u.u;
> + uint64_t rhs = pIn3->u.u;
> bool unused;
> - if ((x == 0 || pIn1->flags & MEM_Int) ||
> - sql_add_int(x, pIn1->flags & MEM_Int, rhs, false,
> - (int64_t *) &x, &unused) != 0) {
> + if (x == 0 || sql_add_int(x, false, rhs, false, (int64_t *) &x,
> + &unused) != 0) {
>
>> Please, inline the value (false), and add a test, which would fail, if
>> I put here true.
>>
I did this:
@@ -4943,7 +4943,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
uint64_t x = pIn1->u.u;
uint64_t rhs = pIn3->u.u;
bool unused;
- if (x == 0 || sql_add_int(x, false, rhs, false, (int64_t *) &x,
+ if (x == 0 || sql_add_int(x, true, rhs, false, (int64_t *) &x,
&unused) != 0) {
/* If the LIMIT is less than or equal to zero, loop forever. This
* is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
And the tests passed. Looks like a test was not added.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints
2019-07-10 22:49 ` Vladislav Shpilevoy
@ 2019-07-17 12:24 ` n.pettik
0 siblings, 0 replies; 49+ messages in thread
From: n.pettik @ 2019-07-17 12:24 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> I did this:
>
> @@ -4943,7 +4943,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
> uint64_t x = pIn1->u.u;
> uint64_t rhs = pIn3->u.u;
> bool unused;
> - if (x == 0 || sql_add_int(x, false, rhs, false, (int64_t *) &x,
> + if (x == 0 || sql_add_int(x, true, rhs, false, (int64_t *) &x,
> &unused) != 0) {
> /* If the LIMIT is less than or equal to zero, loop forever. This
> * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
>
> And the tests passed. Looks like a test was not added.
Ok, I’ve dived a bit into this code and added auxiliary commit:
commit b954fac26809286f764e4509335c0ee7b63d42c2 (HEAD)
Author: Nikita Pettik <korablev@tarantool.org>
Date: Tue Jul 16 19:25:05 2019 +0300
sql: refactor VDBE opcode OP_OffsetLimit
OP_OffsetLimit instruction calculates sum of OFFSET and LIMIT values
when they are present. This sum serves as a counter of entries to be
inserted to temporary space during VDBE execution. Consider query like:
SELECT * FROM t ORDER BY x LIMIT 5 OFFSET 2;
To perform ordering alongside with applying limit and offset
restrictions, first 7 (5 + 2) entries are inserted into temporary space.
They are sorted and then first two tuples are skipped according to offset
clause. The rest tuples from temporary space get to result set.
When sum of LIMIT and OFFSET values is big enough to cause integer
overflow, we can't apply this approach. Previously, counter was simply
set to -1 which means that all entries from base table will be transferred
to ephemeral space. As a result, LIMIT clause was ignored and the result
of query would be incorrect. Motivation for this obviously wrong step was
that to perform query with such huge limit and offset values too many time
is required (like years). What is more, ephemeral spaces support
auto-generated IDs in the range up to 2^32, so there's even no opportunity
to process such queries in theory. Nevertheless, to make code cleaner
let's fix this tricky place and just raise an overflow error if result
of addition exceeds integer range.
This patch fixes obsolete comments saying that in case of overflow
execution won't stop; now limit and offset counter are always >= 0, so
removed redundant branching.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 181f71deb..e349af68f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4861,7 +4861,7 @@ case OP_IfPos: { /* jump, in1 */
}
/* Opcode: OffsetLimit P1 P2 P3 * *
- * Synopsis: if r[P1]>0 then r[P2]=r[P1]+max(0,r[P3]) else r[P2]=(-1)
+ * Synopsis: r[P2]=r[P1]+r[P3]
*
* This opcode performs a commonly used computation associated with
* LIMIT and OFFSET process. r[P1] holds the limit counter. r[P3]
@@ -4870,35 +4870,23 @@ case OP_IfPos: { /* jump, in1 */
* value computed is the total number of rows that will need to be
* visited in order to complete the query.
*
- * If r[P3] is zero or negative, that means there is no OFFSET
- * and r[P2] is set to be the value of the LIMIT, r[P1].
- *
- * if r[P1] is zero or negative, that means there is no LIMIT
- * and r[P2] is set to -1.
- *
- * Otherwise, r[P2] is set to the sum of r[P1] and r[P3].
+ * Otherwise, r[P2] is set to the sum of r[P1] and r[P3]. If the
+ * sum is larger than 2^63-1 (i.e. overflow takes place) then
+ * error is raised.
*/
case OP_OffsetLimit: { /* in1, out2, in3 */
- i64 x;
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
pOut = out2Prerelease(p, pOp);
assert(pIn1->flags & MEM_Int);
assert(pIn3->flags & MEM_Int);
- x = pIn1->u.i;
- if (x<=0 || sqlAddInt64(&x, pIn3->u.i>0?pIn3->u.i:0)) {
- /* If the LIMIT is less than or equal to zero, loop forever. This
- * is documented. But also, if the LIMIT+OFFSET exceeds 2^63 then
- * also loop forever. This is undocumented. In fact, one could argue
- * that the loop should terminate. But assuming 1 billion iterations
- * per second (far exceeding the capabilities of any current hardware)
- * it would take nearly 300 years to actually reach the limit. So
- * looping forever is a reasonable approximation.
- */
- pOut->u.i = -1;
- } else {
- pOut->u.i = x;
+ i64 x = pIn1->u.i;
+ if (sqlAddInt64(&x, pIn3->u.i) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "sum of LIMIT and OFFSET "
+ "values should not result in integer overflow");
+ goto abort_due_to_error;
}
+ pOut->u.i = x;
break;
}
diff --git a/test/sql-tap/limit.test.lua b/test/sql-tap/limit.test.lua
index 40b787b52..84d0798db 100755
--- a/test/sql-tap/limit.test.lua
+++ b/test/sql-tap/limit.test.lua
@@ -1,6 +1,6 @@
#!/usr/bin/env tarantool
test = require("sqltester")
-test:plan(111)
+test:plan(113)
--!./tcltestrunner.lua
-- 2001 November 6
@@ -1354,4 +1354,18 @@ test:do_execsql_test(
-- </limit-14.7.2>
})
+-- Sum of LIMIT and OFFSET values should not cause integer overflow.
+--
+test:do_catchsql_test(
+ "limit-15.1",
+ [[
+ SELECT 1 LIMIT 9223372036854775807 OFFSET 1;
+ ]], { 1, "Failed to execute SQL statement: sum of LIMIT and OFFSET values should not result in integer overflow" } )
+
+test:do_catchsql_test(
+ "limit-15.2",
+ [[
+ SELECT 1 LIMIT 1 OFFSET 9223372036854775807;
+ ]], { 1, "Failed to execute SQL statement: sum of LIMIT and OFFSET values should not result in integer overflow "} )
+
test:finish_test()
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
` (2 preceding siblings ...)
2019-06-07 15:37 ` [tarantool-patches] [PATCH 3/6] sql: refactor arithmetic operations to support unsigned ints Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
2019-06-07 15:37 ` [tarantool-patches] [PATCH 5/6] sql: introduce extended range for INTEGER type Nikita Pettik
` (2 subsequent siblings)
6 siblings, 1 reply; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
As a part of introduction unsigned type in SQL, let's patch all built-in
function to make them accept and operate on unsigned value, i.e. values
which come with MEM_UInt VDBE memory type.
Part of #3810
Part of #4015
---
src/box/sql/func.c | 70 ++++++++++++++++++-------------------------
src/box/sql/main.c | 3 +-
src/box/sql/sqlInt.h | 9 ++++--
src/box/sql/vdbeInt.h | 2 +-
src/box/sql/vdbeapi.c | 22 ++++++++++----
test/sql-tap/cast.test.lua | 8 ++---
test/sql-tap/check.test.lua | 2 +-
test/sql-tap/cse.test.lua | 6 ++--
test/sql-tap/default.test.lua | 2 +-
test/sql-tap/fkey2.test.lua | 4 +--
test/sql-tap/func.test.lua | 18 ++++++++---
test/sql-tap/table.test.lua | 2 +-
test/sql-tap/tkt3493.test.lua | 4 +--
13 files changed, 82 insertions(+), 70 deletions(-)
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 457c9b92b..365f8f9d2 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -111,9 +111,11 @@ typeofFunc(sql_context * context, int NotUsed, sql_value ** argv)
UNUSED_PARAMETER(NotUsed);
switch (sql_value_type(argv[0])) {
case MP_INT:
- case MP_UINT:
z = "integer";
break;
+ case MP_UINT:
+ z = "unsigned";
+ break;
case MP_STR:
z = "string";
break;
@@ -148,8 +150,7 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv)
case MP_INT:
case MP_UINT:
case MP_DOUBLE:{
- sql_result_int(context,
- sql_value_bytes(argv[0]));
+ sql_result_uint(context, sql_value_bytes(argv[0]));
break;
}
case MP_STR:{
@@ -157,7 +158,7 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv)
if (z == 0)
return;
len = sql_utf8_char_count(z, sql_value_bytes(argv[0]));
- sql_result_int(context, len);
+ sql_result_uint(context, len);
break;
}
default:{
@@ -179,24 +180,15 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
assert(argc == 1);
UNUSED_PARAMETER(argc);
switch (sql_value_type(argv[0])) {
- case MP_UINT:
+ case MP_UINT: {
+ sql_result_uint(context, sql_value_uint64(argv[0]));
+ break;
+ }
case MP_INT:{
- i64 iVal = sql_value_int64(argv[0]);
- if (iVal < 0) {
- if (iVal == SMALLEST_INT64) {
- /* IMP: R-31676-45509 If X is the integer -9223372036854775808
- * then abs(X) throws an integer overflow error since there is no
- * equivalent positive 64-bit two complement value.
- */
- sql_result_error(context,
- "integer overflow",
- -1);
- return;
- }
- iVal = -iVal;
- }
- sql_result_int64(context, iVal);
- break;
+ i64 iVal = sql_value_int64(argv[0]);
+ assert(iVal < 0);
+ sql_result_uint(context, -iVal);
+ break;
}
case MP_NIL:{
/* IMP: R-37434-19929 Abs(X) returns NULL if X is NULL. */
@@ -358,7 +350,8 @@ position_func(struct sql_context *context, int argc, struct Mem **argv)
}
}
finish:
- sql_result_int(context, position);
+ assert(position >= 0);
+ sql_result_uint(context, position);
}
/*
@@ -645,21 +638,13 @@ ICU_CASE_CONVERT(Upper);
static void
randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
{
- sql_int64 r;
+ int64_t r;
UNUSED_PARAMETER2(NotUsed, NotUsed2);
sql_randomness(sizeof(r), &r);
- if (r < 0) {
- /* We need to prevent a random number of 0x8000000000000000
- * (or -9223372036854775808) since when you do abs() of that
- * number of you get the same value back again. To do this
- * in a way that is testable, mask the sign bit off of negative
- * values, resulting in a positive value. Then take the
- * 2s complement of that positive value. The end result can
- * therefore be no less than -9223372036854775807.
- */
- r = -(r & LARGEST_INT64);
- }
- sql_result_int64(context, r);
+ if (r < 0)
+ sql_result_int(context, r);
+ else
+ sql_result_uint(context, r);
}
/*
@@ -1126,7 +1111,7 @@ unicodeFunc(sql_context * context, int argc, sql_value ** argv)
const unsigned char *z = sql_value_text(argv[0]);
(void)argc;
if (z && z[0])
- sql_result_int(context, sqlUtf8Read(&z));
+ sql_result_uint(context, sqlUtf8Read(&z));
}
/*
@@ -1145,10 +1130,13 @@ charFunc(sql_context * context, int argc, sql_value ** argv)
return;
}
for (i = 0; i < argc; i++) {
- sql_int64 x;
+ uint64_t x;
unsigned c;
- x = sql_value_int64(argv[i]);
- if (x < 0 || x > 0x10ffff)
+ if (sql_value_type(argv[i]) == MP_INT)
+ x = 0xfffd;
+ else
+ x = sql_value_uint64(argv[i]);
+ if (x > 0x10ffff)
x = 0xfffd;
c = (unsigned)(x & 0x1fffff);
if (c < 0x00080) {
@@ -1627,7 +1615,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
else
p->rSum += (uint64_t) v;
if ((p->approx | p->overflow) == 0 &&
- sql_add_int(p->iSum, p->is_neg, v, v < 0, &p->iSum,
+ sql_add_int(p->iSum, p->is_neg, v, type == MP_INT, &p->iSum,
&p->is_neg) != 0) {
p->overflow = 1;
}
@@ -1698,7 +1686,7 @@ countFinalize(sql_context * context)
{
CountCtx *p;
p = sql_aggregate_context(context, 0);
- sql_result_int64(context, p ? p->n : 0);
+ sql_result_uint(context, p ? p->n : 0);
}
/*
diff --git a/src/box/sql/main.c b/src/box/sql/main.c
index fe1135a71..6a08a9302 100644
--- a/src/box/sql/main.c
+++ b/src/box/sql/main.c
@@ -250,7 +250,8 @@ sql_row_count(struct sql_context *context, MAYBE_UNUSED int unused1,
MAYBE_UNUSED sql_value **unused2)
{
sql *db = sql_context_db_handle(context);
- sql_result_int(context, db->nChange);
+ assert(db->nChange >= 0);
+ sql_result_uint(context, db->nChange);
}
/*
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index c0e2ab699..4697e3003 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -455,6 +455,9 @@ sql_column_subtype(struct sql_stmt *stmt, int i);
sql_int64
sql_value_int64(sql_value *);
+uint64_t
+sql_value_uint64(sql_value *val);
+
const unsigned char *
sql_value_text(sql_value *);
@@ -496,13 +499,13 @@ void
sql_result_error_code(sql_context *, int);
void
-sql_result_int(sql_context *, int);
+sql_result_int(sql_context *, int64_t);
void
-sql_result_bool(struct sql_context *ctx, bool value);
+sql_result_uint(sql_context *ctx, int64_t u_val);
void
-sql_result_int64(sql_context *, sql_int64);
+sql_result_bool(struct sql_context *ctx, bool value);
void
sql_result_null(sql_context *);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index abed46486..0c86ed0f3 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -501,7 +501,7 @@ void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemNulTerminate(Mem *);
int sqlVdbeMemSetStr(Mem *, const char *, int, u8, void (*)(void *));
-void sqlVdbeMemSetInt64(Mem *, i64);
+void sqlVdbeMemSetInt64(Mem *, i64, bool is_neg);
void
mem_set_bool(struct Mem *mem, bool value);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 393782c23..752efeecb 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -239,6 +239,16 @@ sql_value_int64(sql_value * pVal)
return i;
}
+uint64_t
+sql_value_uint64(sql_value *val)
+{
+ int64_t i = 0;
+ bool is_neg;
+ sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
+ assert(!is_neg);
+ return i;
+}
+
enum sql_subtype
sql_value_subtype(sql_value * pVal)
{
@@ -386,21 +396,21 @@ sql_result_error(sql_context * pCtx, const char *z, int n)
}
void
-sql_result_int(sql_context * pCtx, int iVal)
+sql_result_int(sql_context * pCtx, int64_t iVal)
{
- mem_set_int(pCtx->pOut, (i64) iVal, iVal < 0);
+ mem_set_int(pCtx->pOut, (i64) iVal, true);
}
void
-sql_result_bool(struct sql_context *ctx, bool value)
+sql_result_uint(sql_context *ctx, int64_t u_val)
{
- mem_set_bool(ctx->pOut, value);
+ mem_set_int(ctx->pOut, u_val, false);
}
void
-sql_result_int64(sql_context * pCtx, i64 iVal)
+sql_result_bool(struct sql_context *ctx, bool value)
{
- mem_set_int(pCtx->pOut, iVal, iVal < 0);
+ mem_set_bool(ctx->pOut, value);
}
void
diff --git a/test/sql-tap/cast.test.lua b/test/sql-tap/cast.test.lua
index 531c310d2..315f49d08 100755
--- a/test/sql-tap/cast.test.lua
+++ b/test/sql-tap/cast.test.lua
@@ -220,7 +220,7 @@ test:do_execsql_test(
SELECT typeof(123)
]], {
-- <cast-1.22>
- "integer"
+ "unsigned"
-- </cast-1.22>
})
@@ -280,7 +280,7 @@ test:do_execsql_test(
SELECT typeof(CAST(123 AS SCALAR))
]], {
-- <cast-1.28>
- "integer"
+ "unsigned"
-- </cast-1.28>
})
@@ -300,7 +300,7 @@ test:do_execsql_test(
SELECT typeof(CAST(123 AS integer))
]], {
-- <cast-1.30>
- "integer"
+ "unsigned"
-- </cast-1.30>
})
@@ -400,7 +400,7 @@ test:do_execsql_test(
SELECT typeof(CAST(123.456 AS integer))
]], {
-- <cast-1.38>
- "integer"
+ "unsigned"
-- </cast-1.38>
})
diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua
index b01afca7c..8fd722924 100755
--- a/test/sql-tap/check.test.lua
+++ b/test/sql-tap/check.test.lua
@@ -206,7 +206,7 @@ test:do_execsql_test(
[[
CREATE TABLE t2(
id INT primary key,
- x INTEGER CONSTRAINT one CHECK( typeof(coalesce(x,0))=='integer'),
+ x INTEGER CONSTRAINT one CHECK( typeof(coalesce(x,0))=='unsigned'),
y REAL CONSTRAINT two CHECK( typeof(coalesce(y,0.1))=='number' ),
z TEXT CONSTRAINT three CHECK( typeof(coalesce(z,''))=='string' )
);
diff --git a/test/sql-tap/cse.test.lua b/test/sql-tap/cse.test.lua
index 78d0d2046..f83cde9bd 100755
--- a/test/sql-tap/cse.test.lua
+++ b/test/sql-tap/cse.test.lua
@@ -165,7 +165,7 @@ test:do_execsql_test(
SELECT CAST(b AS integer), typeof(b), CAST(b AS text), typeof(b) FROM t1
]], {
-- <cse-1.10>
- 11, "integer", "11", "integer", 21, "integer", "21", "integer"
+ 11, "unsigned", "11", "unsigned", 21, "unsigned", "21", "unsigned"
-- </cse-1.10>
})
@@ -201,7 +201,7 @@ test:do_execsql_test(
SELECT upper(b), typeof(b), b FROM t1
]], {
-- <cse-1.13>
- "11", "integer", 11, "21", "integer", 21
+ "11", "unsigned", 11, "21", "unsigned", 21
-- </cse-1.13>
})
@@ -211,7 +211,7 @@ test:do_execsql_test(
SELECT b, typeof(b), upper(b), typeof(b), b FROM t1
]], {
-- <cse-1.14>
- 11, "integer", "11", "integer", 11, 21, "integer", "21", "integer", 21
+ 11, "unsigned", "11", "unsigned", 11, 21, "unsigned", "21", "unsigned", 21
-- </cse-1.14>
})
diff --git a/test/sql-tap/default.test.lua b/test/sql-tap/default.test.lua
index 55d16bcd7..0353796d5 100755
--- a/test/sql-tap/default.test.lua
+++ b/test/sql-tap/default.test.lua
@@ -119,7 +119,7 @@ test:do_execsql_test(
-- In current situation I don't know what to do, need Kirill's
-- advice.
-- Bulat
- 1, "integer", 5, "integer", "row1", "string", 5.25, "number", 8.67, "number", "321", "string", 432, "integer"
+ 1, "unsigned", 5, "unsigned", "row1", "string", 5.25, "number", 8.67, "number", "321", "string", 432, "unsigned"
-- </default-3.1>
})
diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua
index 54e5059b3..8e786edec 100755
--- a/test/sql-tap/fkey2.test.lua
+++ b/test/sql-tap/fkey2.test.lua
@@ -320,10 +320,10 @@ test:do_execsql_test(
CREATE TABLE j(j INT PRIMARY KEY REFERENCES i);
INSERT INTO i VALUES(35);
INSERT INTO j VALUES(35);
- SELECT j, typeof(j) FROM j;
+ SELECT j FROM j;
]], {
-- <fkey2-2.1>
- 35, "integer"
+ 35
-- </fkey2-2.1>
})
diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
index 314c528ab..9cf9c2f67 100755
--- a/test/sql-tap/func.test.lua
+++ b/test/sql-tap/func.test.lua
@@ -1,6 +1,6 @@
#!/usr/bin/env tarantool
test = require("sqltester")
-test:plan(14591)
+test:plan(14592)
--!./tcltestrunner.lua
-- 2001 September 15
@@ -908,7 +908,7 @@ test:do_execsql_test(
UNION ALL SELECT -9223372036854775807)
]], {
-- <func-8.6>
- "integer"
+ "unsigned"
-- </func-8.6>
})
@@ -1587,7 +1587,7 @@ test:do_execsql_test(
SELECT typeof(sum(x)) FROM t6
]], {
-- <func-18.11>
- "integer"
+ "unsigned"
-- </func-18.11>
})
@@ -1742,10 +1742,20 @@ test:do_catchsql_test(
SELECT abs(-9223372036854775807-1);
]], {
-- <func-18.32>
- 1, "Failed to execute SQL statement: integer overflow"
+ 0, {9223372036854775808LL}
-- </func-18.32>
})
+test:do_catchsql_test(
+ "func-18.33",
+ [[
+ SELECT abs(-9223372036854775807-1);
+ ]], {
+ -- <func-18.32>
+ 0, {9223372036854775808LL}
+ -- </func-18.32>
+})
+
-- The MATCH function exists but is only a stub and always throws an error.
--
test:do_execsql_test(
diff --git a/test/sql-tap/table.test.lua b/test/sql-tap/table.test.lua
index 5b793c0fc..5ed4a83fa 100755
--- a/test/sql-tap/table.test.lua
+++ b/test/sql-tap/table.test.lua
@@ -906,7 +906,7 @@ test:do_execsql_test(
FROM t7 LIMIT 1;
]], {
-- <table-11.1>
- "integer", "null", "null", "null", "null", "null", "null", "null"
+ "unsigned", "null", "null", "null", "null", "null", "null", "null"
-- </table-11.1>
})
diff --git a/test/sql-tap/tkt3493.test.lua b/test/sql-tap/tkt3493.test.lua
index 67e79da72..df6bb4d36 100755
--- a/test/sql-tap/tkt3493.test.lua
+++ b/test/sql-tap/tkt3493.test.lua
@@ -229,7 +229,7 @@ test:do_execsql_test(
SELECT typeof(b), b FROM t1 GROUP BY a HAVING b='456'
]], {
-- <tkt3493-2.5.1>
- "integer", 456
+ "unsigned", 456
-- </tkt3493-2.5.1>
})
@@ -239,7 +239,7 @@ test:do_execsql_test(
SELECT typeof(b), b FROM t1 GROUP BY b HAVING b='456'
]], {
-- <tkt3493-2.5.2>
- "integer", 456
+ "unsigned", 456
-- </tkt3493-2.5.2>
})
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-06-07 15:37 ` [tarantool-patches] [PATCH 4/6] sql: make built-in functions operate on unsigned values Nikita Pettik
@ 2019-06-11 21:11 ` Vladislav Shpilevoy
2019-07-01 14:21 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-06-11 21:11 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 7 comments below, review
fixes at the end of the email, and on the branch in
a separate commit.
On 07/06/2019 18:37, Nikita Pettik wrote:
> As a part of introduction unsigned type in SQL, let's patch all built-in
> function to make them accept and operate on unsigned value, i.e. values
> which come with MEM_UInt VDBE memory type.
>
> Part of #3810
> Part of #4015
> ---
> src/box/sql/func.c | 70 ++++++++++++++++++-------------------------
> src/box/sql/main.c | 3 +-
> src/box/sql/sqlInt.h | 9 ++++--
> src/box/sql/vdbeInt.h | 2 +-
> src/box/sql/vdbeapi.c | 22 ++++++++++----
> test/sql-tap/cast.test.lua | 8 ++---
> test/sql-tap/check.test.lua | 2 +-
> test/sql-tap/cse.test.lua | 6 ++--
> test/sql-tap/default.test.lua | 2 +-
> test/sql-tap/fkey2.test.lua | 4 +--
> test/sql-tap/func.test.lua | 18 ++++++++---
> test/sql-tap/table.test.lua | 2 +-
> test/sql-tap/tkt3493.test.lua | 4 +--
> 13 files changed, 82 insertions(+), 70 deletions(-)
>
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index 457c9b92b..365f8f9d2 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -645,21 +638,13 @@ ICU_CASE_CONVERT(Upper);
> static void
> randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
> {
> - sql_int64 r;
> + int64_t r;
> UNUSED_PARAMETER2(NotUsed, NotUsed2);
> sql_randomness(sizeof(r), &r);
> - if (r < 0) {
> - /* We need to prevent a random number of 0x8000000000000000
> - * (or -9223372036854775808) since when you do abs() of that
> - * number of you get the same value back again. To do this
> - * in a way that is testable, mask the sign bit off of negative
> - * values, resulting in a positive value. Then take the
> - * 2s complement of that positive value. The end result can
> - * therefore be no less than -9223372036854775807.
> - */
> - r = -(r & LARGEST_INT64);
> - }
> - sql_result_int64(context, r);
> + if (r < 0)
> + sql_result_int(context, r);
> + else
> + sql_result_uint(context, r);
1. To avoid such mess I propose you the same as for mem_set_int - make
several functions:
/* Return an int64 value, sign is detected inside. */
sql_result_i64(int64_t value);
/* Return a uint64 value. */
sql_result_u64(uint64_t value);
/* Return an integer value with explicit sign. */
sql_result_int(int64_t value, bool is_negative);
> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> index c0e2ab699..4697e3003 100644
> --- a/src/box/sql/sqlInt.h
> +++ b/src/box/sql/sqlInt.h
> @@ -455,6 +455,9 @@ sql_column_subtype(struct sql_stmt *stmt, int i);
> sql_int64
> sql_value_int64(sql_value *);
>
> +uint64_t
> +sql_value_uint64(sql_value *val);
> +
> const unsigned char *
> sql_value_text(sql_value *);
>
> @@ -496,13 +499,13 @@ void
> sql_result_error_code(sql_context *, int);
>
> void
> -sql_result_int(sql_context *, int);
> +sql_result_int(sql_context *, int64_t);
>
> void
> -sql_result_bool(struct sql_context *ctx, bool value);
> +sql_result_uint(sql_context *ctx, int64_t u_val);
2. Why result_uint takes int instead of uint?
3. Please, move 'result_uint' above 'result_int' to
avoid messing diff with 'result_bool'.
>
> void
> -sql_result_int64(sql_context *, sql_int64);
> +sql_result_bool(struct sql_context *ctx, bool value);
>
> void
> sql_result_null(sql_context *);
> diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
> index abed46486..0c86ed0f3 100644
> --- a/src/box/sql/vdbeInt.h
> +++ b/src/box/sql/vdbeInt.h
> @@ -501,7 +501,7 @@ void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
> void sqlVdbeMemMove(Mem *, Mem *);
> int sqlVdbeMemNulTerminate(Mem *);
> int sqlVdbeMemSetStr(Mem *, const char *, int, u8, void (*)(void *));
> -void sqlVdbeMemSetInt64(Mem *, i64);
> +void sqlVdbeMemSetInt64(Mem *, i64, bool is_neg);
4. That function does not exist. And looks like its existence has
ceased not in this commit, but in one of the previous ones. Please,
drop.
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index 393782c23..752efeecb 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -239,6 +239,16 @@ sql_value_int64(sql_value * pVal)
> return i;
> }
>
> +uint64_t
> +sql_value_uint64(sql_value *val)
> +{
> + int64_t i = 0;
> + bool is_neg;
> + sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
> + assert(!is_neg);
5. Why did not you add a similar assertion to sql_value_int64()?
> + return i;
> +}
> +
> enum sql_subtype
> sql_value_subtype(sql_value * pVal)
> {
> diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua
> index 54e5059b3..8e786edec 100755
> --- a/test/sql-tap/fkey2.test.lua
> +++ b/test/sql-tap/fkey2.test.lua
> @@ -320,10 +320,10 @@ test:do_execsql_test(
> CREATE TABLE j(j INT PRIMARY KEY REFERENCES i);
> INSERT INTO i VALUES(35);
> INSERT INTO j VALUES(35);
> - SELECT j, typeof(j) FROM j;
> + SELECT j FROM j;
> ]], {
> -- <fkey2-2.1>
> - 35, "integer"
> + 35
6. Why did you drop 'typeof' instead of having it
fixed?
> -- </fkey2-2.1>
> })
>
> diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
> index 314c528ab..9cf9c2f67 100755
> --- a/test/sql-tap/func.test.lua
> +++ b/test/sql-tap/func.test.lua
> @@ -1742,10 +1742,20 @@ test:do_catchsql_test(
> SELECT abs(-9223372036854775807-1);
> ]], {
> -- <func-18.32>
> - 1, "Failed to execute SQL statement: integer overflow"
> + 0, {9223372036854775808LL}
> -- </func-18.32>
> })
>
> +test:do_catchsql_test(
> + "func-18.33",
> + [[
> + SELECT abs(-9223372036854775807-1);
> + ]], {
> + -- <func-18.32>
> + 0, {9223372036854775808LL}
> + -- </func-18.32>
> +})
7. But this test is exactly the same as the previous one. Why
do you need it?
Consider my review fixes here and on the branch in a
separate commit.
=============================================================
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 365f8f9d2..a4f513ae8 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -184,12 +184,12 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
sql_result_uint(context, sql_value_uint64(argv[0]));
break;
}
- case MP_INT:{
- i64 iVal = sql_value_int64(argv[0]);
- assert(iVal < 0);
- sql_result_uint(context, -iVal);
+ case MP_INT: {
+ int64_t value = sql_value_int64(argv[0]);
+ assert(value < 0);
+ sql_result_uint(context, -value);
break;
- }
+ }
case MP_NIL:{
/* IMP: R-37434-19929 Abs(X) returns NULL if X is NULL. */
sql_result_null(context);
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-01 14:21 ` n.pettik
2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-01 14:21 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index 457c9b92b..365f8f9d2 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -645,21 +638,13 @@ ICU_CASE_CONVERT(Upper);
>> static void
>> randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
>> {
>> - sql_int64 r;
>> + int64_t r;
>> UNUSED_PARAMETER2(NotUsed, NotUsed2);
>> sql_randomness(sizeof(r), &r);
>> - if (r < 0) {
>> - /* We need to prevent a random number of 0x8000000000000000
>> - * (or -9223372036854775808) since when you do abs() of that
>> - * number of you get the same value back again. To do this
>> - * in a way that is testable, mask the sign bit off of negative
>> - * values, resulting in a positive value. Then take the
>> - * 2s complement of that positive value. The end result can
>> - * therefore be no less than -9223372036854775807.
>> - */
>> - r = -(r & LARGEST_INT64);
>> - }
>> - sql_result_int64(context, r);
>> + if (r < 0)
>> + sql_result_int(context, r);
>> + else
>> + sql_result_uint(context, r);
>
> 1. To avoid such mess I propose you the same as for mem_set_int - make
> several functions:
What kind of mess do you mean? Here we have two functions:
one sets unsigned result, another one - signed. Pretty straight
approach, isn’t it? What is more, this is the only place where
branching is required.
> /* Return an int64 value, sign is detected inside. */
> sql_result_i64(int64_t value);
>
> /* Return a uint64 value. */
> sql_result_u64(uint64_t value);
>
> /* Return an integer value with explicit sign. */
> sql_result_int(int64_t value, bool is_negative);
>
>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>> index c0e2ab699..4697e3003 100644
>> --- a/src/box/sql/sqlInt.h
>> +++ b/src/box/sql/sqlInt.h
>> @@ -455,6 +455,9 @@ sql_column_subtype(struct sql_stmt *stmt, int i);
>> sql_int64
>> sql_value_int64(sql_value *);
>>
>> +uint64_t
>> +sql_value_uint64(sql_value *val);
>> +
>> const unsigned char *
>> sql_value_text(sql_value *);
>>
>> @@ -496,13 +499,13 @@ void
>> sql_result_error_code(sql_context *, int);
>>
>> void
>> -sql_result_int(sql_context *, int);
>> +sql_result_int(sql_context *, int64_t);
>>
>> void
>> -sql_result_bool(struct sql_context *ctx, bool value);
>> +sql_result_uint(sql_context *ctx, int64_t u_val);
>
> 2. Why result_uint takes int instead of uint?
Because mem_set_int() accepts int64 and to avoid
extra casts to uint (since almost al used values are singed).
> 3. Please, move 'result_uint' above 'result_int' to
> avoid messing diff with 'result_bool’.
… Ok, done.
>> diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
>> index abed46486..0c86ed0f3 100644
>> --- a/src/box/sql/vdbeInt.h
>> +++ b/src/box/sql/vdbeInt.h
>> @@ -501,7 +501,7 @@ void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
>> void sqlVdbeMemMove(Mem *, Mem *);
>> int sqlVdbeMemNulTerminate(Mem *);
>> int sqlVdbeMemSetStr(Mem *, const char *, int, u8, void (*)(void *));
>> -void sqlVdbeMemSetInt64(Mem *, i64);
>> +void sqlVdbeMemSetInt64(Mem *, i64, bool is_neg);
>
> 4. That function does not exist. And looks like its existence has
> ceased not in this commit, but in one of the previous ones. Please,
> drop.
Removed in previous patch.
>> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
>> index 393782c23..752efeecb 100644
>> --- a/src/box/sql/vdbeapi.c
>> +++ b/src/box/sql/vdbeapi.c
>> @@ -239,6 +239,16 @@ sql_value_int64(sql_value * pVal)
>> return i;
>> }
>>
>> +uint64_t
>> +sql_value_uint64(sql_value *val)
>> +{
>> + int64_t i = 0;
>> + bool is_neg;
>> + sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
>> + assert(!is_neg);
>
> 5. Why did not you add a similar assertion to sql_value_int64()?
Because it can accept both signed and unsigned values.
>> diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua
>> index 54e5059b3..8e786edec 100755
>> --- a/test/sql-tap/fkey2.test.lua
>> +++ b/test/sql-tap/fkey2.test.lua
>> @@ -320,10 +320,10 @@ test:do_execsql_test(
>> CREATE TABLE j(j INT PRIMARY KEY REFERENCES i);
>> INSERT INTO i VALUES(35);
>> INSERT INTO j VALUES(35);
>> - SELECT j, typeof(j) FROM j;
>> + SELECT j FROM j;
>> ]], {
>> -- <fkey2-2.1>
>> - 35, "integer"
>> + 35
>
> 6. Why did you drop 'typeof' instead of having it
> fixed?
Because typeof() function will be fixed: it should return
type of column, not type of particular value. To avoid
abusing test files, I decided to drop it (since it is going
to change again soon).
>> diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
>> index 314c528ab..9cf9c2f67 100755
>> --- a/test/sql-tap/func.test.lua
>> +++ b/test/sql-tap/func.test.lua
>> @@ -1742,10 +1742,20 @@ test:do_catchsql_test(
>> SELECT abs(-9223372036854775807-1);
>> ]], {
>> -- <func-18.32>
>> - 1, "Failed to execute SQL statement: integer overflow"
>> + 0, {9223372036854775808LL}
>> -- </func-18.32>
>> })
>>
>> +test:do_catchsql_test(
>> + "func-18.33",
>> + [[
>> + SELECT abs(-9223372036854775807-1);
>> + ]], {
>> + -- <func-18.32>
>> + 0, {9223372036854775808LL}
>> + -- </func-18.32>
>> +})
>
> 7. But this test is exactly the same as the previous one. Why
> do you need it?
Probably I wanted to do this:
diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
index 4d96f2fed..3ff623e82 100755
--- a/test/sql-tap/func.test.lua
+++ b/test/sql-tap/func.test.lua
@@ -1749,10 +1749,10 @@ test:do_catchsql_test(
test:do_catchsql_test(
"func-18.33",
[[
- SELECT abs(-9223372036854775807-1);
+ SELECT abs(-9223372036854775807-2);
]], {
-- <func-18.32>
- 0, {9223372036854775808LL}
+ 1, "Failed to execute SQL statement: integer is overflowed"
-- </func-18.32>
})
>
> Consider my review fixes here and on the branch in a
> separate commit.
Applied.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-07-01 14:21 ` n.pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
2019-07-05 16:36 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
On 01/07/2019 16:21, n.pettik wrote:
>
>>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>>> index 457c9b92b..365f8f9d2 100644
>>> --- a/src/box/sql/func.c
>>> +++ b/src/box/sql/func.c
>>> @@ -645,21 +638,13 @@ ICU_CASE_CONVERT(Upper);
>>> static void
>>> randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
>>> {
>>> - sql_int64 r;
>>> + int64_t r;
>>> UNUSED_PARAMETER2(NotUsed, NotUsed2);
>>> sql_randomness(sizeof(r), &r);
>>> - if (r < 0) {
>>> - /* We need to prevent a random number of 0x8000000000000000
>>> - * (or -9223372036854775808) since when you do abs() of that
>>> - * number of you get the same value back again. To do this
>>> - * in a way that is testable, mask the sign bit off of negative
>>> - * values, resulting in a positive value. Then take the
>>> - * 2s complement of that positive value. The end result can
>>> - * therefore be no less than -9223372036854775807.
>>> - */
>>> - r = -(r & LARGEST_INT64);
>>> - }
>>> - sql_result_int64(context, r);
>>> + if (r < 0)
>>> + sql_result_int(context, r);
>>> + else
>>> + sql_result_uint(context, r);
>>
>> 1. To avoid such mess I propose you the same as for mem_set_int - make
>> several functions:
>
> What kind of mess do you mean? Here we have two functions:
> one sets unsigned result, another one - signed. Pretty straight
> approach, isn’t it? What is more, this is the only place where
> branching is required.
If it is so straight, then why do you need branching at all?
I see, that sql_result_int() calls mem_set_i64, which is able
to take positive values.
I've done this and the tests passed. Why?
@@ -621,10 +621,7 @@ randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
int64_t r;
UNUSED_PARAMETER2(NotUsed, NotUsed2);
sql_randomness(sizeof(r), &r);
- if (r < 0)
- sql_result_int(context, r);
- else
- sql_result_uint(context, r);
+ sql_result_int(context, r);
}
>
>> /* Return an int64 value, sign is detected inside. */
>> sql_result_i64(int64_t value);
>>
>> /* Return a uint64 value. */
>> sql_result_u64(uint64_t value);
>>
>> /* Return an integer value with explicit sign. */
>> sql_result_int(int64_t value, bool is_negative);
>>
>>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>>> index c0e2ab699..4697e3003 100644
>>> --- a/src/box/sql/sqlInt.h
>>> +++ b/src/box/sql/sqlInt.h
>>> @@ -455,6 +455,9 @@ sql_column_subtype(struct sql_stmt *stmt, int i);
>>> sql_int64
>>> sql_value_int64(sql_value *);
>>>
>>> +uint64_t
>>> +sql_value_uint64(sql_value *val);
>>> +
>>> const unsigned char *
>>> sql_value_text(sql_value *);
>>>
>>> @@ -496,13 +499,13 @@ void
>>> sql_result_error_code(sql_context *, int);
>>>
>>> void
>>> -sql_result_int(sql_context *, int);
>>> +sql_result_int(sql_context *, int64_t);
>>>
>>> void
>>> -sql_result_bool(struct sql_context *ctx, bool value);
>>> +sql_result_uint(sql_context *ctx, int64_t u_val);
>>
>> 2. Why result_uint takes int instead of uint?
>
> Because mem_set_int() accepts int64 and to avoid
> extra casts to uint (since almost al used values are singed).
This function has nothing to do with mem_set_int().
This function calls mem_set_u64(), and its argument is
unsigned. So what is a problem, again?
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-07-01 21:53 ` Vladislav Shpilevoy
@ 2019-07-05 16:36 ` n.pettik
2019-07-10 22:49 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-05 16:36 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 2 Jul 2019, at 00:53, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
> Thanks for the fixes!
> On 01/07/2019 16:21, n.pettik wrote:
>>
>>>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>>>> index 457c9b92b..365f8f9d2 100644
>>>> --- a/src/box/sql/func.c
>>>> +++ b/src/box/sql/func.c
>>>> @@ -645,21 +638,13 @@ ICU_CASE_CONVERT(Upper);
>>>> static void
>>>> randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
>>>> {
>>>> - sql_int64 r;
>>>> + int64_t r;
>>>> UNUSED_PARAMETER2(NotUsed, NotUsed2);
>>>> sql_randomness(sizeof(r), &r);
>>>> - if (r < 0) {
>>>> - /* We need to prevent a random number of 0x8000000000000000
>>>> - * (or -9223372036854775808) since when you do abs() of that
>>>> - * number of you get the same value back again. To do this
>>>> - * in a way that is testable, mask the sign bit off of negative
>>>> - * values, resulting in a positive value. Then take the
>>>> - * 2s complement of that positive value. The end result can
>>>> - * therefore be no less than -9223372036854775807.
>>>> - */
>>>> - r = -(r & LARGEST_INT64);
>>>> - }
>>>> - sql_result_int64(context, r);
>>>> + if (r < 0)
>>>> + sql_result_int(context, r);
>>>> + else
>>>> + sql_result_uint(context, r);
>>>
>>> 1. To avoid such mess I propose you the same as for mem_set_int - make
>>> several functions:
>>
>> What kind of mess do you mean? Here we have two functions:
>> one sets unsigned result, another one - signed. Pretty straight
>> approach, isn’t it? What is more, this is the only place where
>> branching is required.
>
> If it is so straight, then why do you need branching at all?
> I see, that sql_result_int() calls mem_set_i64, which is able
> to take positive values.
>
> I've done this and the tests passed. Why?
You had already answered your question:
> I see, that sql_result_int() calls mem_set_i64, which is able
> to take positive values.
Anyway sql_randomness() doesn’t return sign of random number.
So actually, your diff is completely correct and I’ve applied it.
> @@ -621,10 +621,7 @@ randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
> int64_t r;
> UNUSED_PARAMETER2(NotUsed, NotUsed2);
> sql_randomness(sizeof(r), &r);
> - if (r < 0)
> - sql_result_int(context, r);
> - else
> - sql_result_uint(context, r);
> + sql_result_int(context, r);
> }
>
>>
>>> /* Return an int64 value, sign is detected inside. */
>>> sql_result_i64(int64_t value);
>>>
>>> /* Return a uint64 value. */
>>> sql_result_u64(uint64_t value);
>>>
>>> /* Return an integer value with explicit sign. */
>>> sql_result_int(int64_t value, bool is_negative);
>>>
>>>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>>>> index c0e2ab699..4697e3003 100644
>>>> --- a/src/box/sql/sqlInt.h
>>>> +++ b/src/box/sql/sqlInt.h
>>>> @@ -455,6 +455,9 @@ sql_column_subtype(struct sql_stmt *stmt, int i);
>>>> sql_int64
>>>> sql_value_int64(sql_value *);
>>>>
>>>> +uint64_t
>>>> +sql_value_uint64(sql_value *val);
>>>> +
>>>> const unsigned char *
>>>> sql_value_text(sql_value *);
>>>>
>>>> @@ -496,13 +499,13 @@ void
>>>> sql_result_error_code(sql_context *, int);
>>>>
>>>> void
>>>> -sql_result_int(sql_context *, int);
>>>> +sql_result_int(sql_context *, int64_t);
>>>>
>>>> void
>>>> -sql_result_bool(struct sql_context *ctx, bool value);
>>>> +sql_result_uint(sql_context *ctx, int64_t u_val);
>>>
>>> 2. Why result_uint takes int instead of uint?
>>
>> Because mem_set_int() accepts int64 and to avoid
>> extra casts to uint (since almost al used values are singed).
>
> This function has nothing to do with mem_set_int().
> This function calls mem_set_u64(), and its argument is
> unsigned. So what is a problem, again?
Sorry, answered before I introduced mem_set_u64().
Now there’s no problem:
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 088cfbc51..976b4486a 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -398,7 +398,7 @@ void
sql_result_double(sql_context *, double);
void
-sql_result_uint(sql_context *ctx, int64_t u_val);
+sql_result_uint(sql_context *ctx, uint64_t u_val);
void
sql_result_int(sql_context *, int64_t);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 2b1115074..ff7ce658b 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -330,7 +330,7 @@ sql_result_double(sql_context * pCtx, double rVal)
}
void
-sql_result_uint(sql_context *ctx, int64_t u_val)
+sql_result_uint(sql_context *ctx, uint64_t u_val)
{
mem_set_u64(ctx->pOut, u_val);
}
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-07-05 16:36 ` n.pettik
@ 2019-07-10 22:49 ` Vladislav Shpilevoy
2019-07-17 0:53 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-10 22:49 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes! See 2 comments below.
> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> index 2e8ac55e9..976b4486a 100644
> --- a/src/box/sql/sqlInt.h
> +++ b/src/box/sql/sqlInt.h
> @@ -395,13 +398,13 @@ void
> sql_result_double(sql_context *, double);
>
> void
> -sql_result_int(sql_context *, int);
> +sql_result_uint(sql_context *ctx, uint64_t u_val);
>
> void
> -sql_result_bool(struct sql_context *ctx, bool value);
> +sql_result_int(sql_context *, int64_t);
1. Please, do not omit parameter names.
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index 705e869bc..ff7ce658b 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -320,21 +330,21 @@ sql_result_double(sql_context * pCtx, double rVal)
> }
>
> void
> -sql_result_int(sql_context * pCtx, int iVal)
> +sql_result_uint(sql_context *ctx, uint64_t u_val)
> {
> - mem_set_i64(pCtx->pOut, iVal);
> + mem_set_u64(ctx->pOut, u_val);
> }
>
> void
> -sql_result_bool(struct sql_context *ctx, bool value)
> +sql_result_int(sql_context * pCtx, int64_t iVal)
2. In new code we usually use Tarantool code style - no
spaces after '*', and no camel case.
> {
> - mem_set_bool(ctx->pOut, value);
> + mem_set_i64(pCtx->pOut, iVal);
> }
>
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 4/6] sql: make built-in functions operate on unsigned values
2019-07-10 22:49 ` Vladislav Shpilevoy
@ 2019-07-17 0:53 ` n.pettik
0 siblings, 0 replies; 49+ messages in thread
From: n.pettik @ 2019-07-17 0:53 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>> index 2e8ac55e9..976b4486a 100644
>> --- a/src/box/sql/sqlInt.h
>> +++ b/src/box/sql/sqlInt.h
>> @@ -395,13 +398,13 @@ void
>> sql_result_double(sql_context *, double);
>>
>> void
>> -sql_result_int(sql_context *, int);
>> +sql_result_uint(sql_context *ctx, uint64_t u_val);
>>
>> void
>> -sql_result_bool(struct sql_context *ctx, bool value);
>> +sql_result_int(sql_context *, int64_t);
>
> 1. Please, do not omit parameter names.
>
>> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
>> index 705e869bc..ff7ce658b 100644
>> --- a/src/box/sql/vdbeapi.c
>> +++ b/src/box/sql/vdbeapi.c
>> @@ -320,21 +330,21 @@ sql_result_double(sql_context * pCtx, double rVal)
>> }
>>
>> void
>> -sql_result_int(sql_context * pCtx, int iVal)
>> +sql_result_uint(sql_context *ctx, uint64_t u_val)
>> {
>> - mem_set_i64(pCtx->pOut, iVal);
>> + mem_set_u64(ctx->pOut, u_val);
>> }
>>
>> void
>> -sql_result_bool(struct sql_context *ctx, bool value)
>> +sql_result_int(sql_context * pCtx, int64_t iVal)
>
> 2. In new code we usually use Tarantool code style - no
> spaces after '*', and no camel case.
Fixes for both comments:
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 976b4486a..3a1a63b22 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -401,7 +401,7 @@ void
sql_result_uint(sql_context *ctx, uint64_t u_val);
void
-sql_result_int(sql_context *, int64_t);
+sql_result_int(sql_context *ctx, int64_t val);
void
sql_result_bool(struct sql_context *ctx, bool value);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index ff7ce658b..44f01fb87 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -336,9 +336,9 @@ sql_result_uint(sql_context *ctx, uint64_t u_val)
}
void
-sql_result_int(sql_context * pCtx, int64_t iVal)
+sql_result_int(sql_context *ctx, int64_t val)
{
- mem_set_i64(pCtx->pOut, iVal);
+ mem_set_i64(ctx->pOut, val);
}
void
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
` (3 preceding siblings ...)
2019-06-07 15:37 ` [tarantool-patches] [PATCH 4/6] sql: make built-in functions operate on unsigned values Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
2019-07-24 15:59 ` Konstantin Osipov
2019-06-07 15:37 ` [tarantool-patches] [PATCH 6/6] sql: allow to specify UNSIGNED column type Nikita Pettik
2019-07-24 13:01 ` [tarantool-patches] Re: [PATCH 0/6] Introduce UNSIGNED type in SQL Kirill Yukhin
6 siblings, 2 replies; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
This patch allows to operate on integer values in range [2^63, 2^64 - 1]
It means that:
- One can use literals from 9223372036854775808 to 18446744073709551615
- One can pass values from mentioned range to bindings
- One can insert and select values from mentioned range
Support of built-in functions and operators has been introduced in
previous patches.
Closes #3810
Part of #4015
---
src/box/bind.c | 13 +-
src/box/bind.h | 1 +
src/box/errcode.h | 2 +-
src/box/execute.c | 2 +-
src/box/lua/execute.c | 9 +-
src/box/lua/lua_sql.c | 2 +-
src/box/sql/build.c | 4 +-
src/box/sql/expr.c | 7 +-
src/box/sql/pragma.c | 2 +-
src/box/sql/sqlInt.h | 6 +
src/box/sql/util.c | 2 -
src/box/sql/vdbe.c | 2 +-
src/box/sql/vdbe.h | 3 +-
src/box/sql/vdbeapi.c | 23 +-
src/box/sql/vdbeaux.c | 14 +-
test/sql-tap/func.test.lua | 2 +-
test/sql-tap/hexlit.test.lua | 2 +-
test/sql-tap/sql-errors.test.lua | 4 +-
test/sql/bind.result | 46 ++-
test/sql/bind.test.lua | 4 +
test/sql/gh-2347-max-int-literals.result | 39 ---
test/sql/gh-2347-max-int-literals.test.lua | 11 -
test/sql/integer-overflow.result | 62 +++-
test/sql/integer-overflow.test.lua | 7 +
test/sql/iproto.result | 6 +-
test/sql/types.result | 503 +++++++++++++++++++++++++++++
test/sql/types.test.lua | 98 ++++++
27 files changed, 761 insertions(+), 115 deletions(-)
delete mode 100644 test/sql/gh-2347-max-int-literals.result
delete mode 100644 test/sql/gh-2347-max-int-literals.test.lua
diff --git a/src/box/bind.c b/src/box/bind.c
index f15915377..9e952857f 100644
--- a/src/box/bind.c
+++ b/src/box/bind.c
@@ -69,13 +69,8 @@ sql_bind_decode(struct sql_bind *bind, int i, const char **packet)
switch (type) {
case MP_UINT: {
uint64_t n = mp_decode_uint(packet);
- if (n > INT64_MAX) {
- diag_set(ClientError, ER_SQL_BIND_VALUE,
- sql_bind_name(bind), "INTEGER");
- return -1;
- }
- bind->i64 = (int64_t) n;
- bind->bytes = sizeof(bind->i64);
+ bind->u64 = n;
+ bind->bytes = sizeof(bind->u64);
break;
}
case MP_INT:
@@ -173,9 +168,11 @@ sql_bind_column(struct sql_stmt *stmt, const struct sql_bind *p,
}
switch (p->type) {
case MP_INT:
- case MP_UINT:
rc = sql_bind_int64(stmt, pos, p->i64);
break;
+ case MP_UINT:
+ rc = sql_bind_uint64(stmt, pos, p->u64);
+ break;
case MP_BOOL:
rc = sql_bind_boolean(stmt, pos, p->b);
break;
diff --git a/src/box/bind.h b/src/box/bind.h
index d9823431e..568c558f3 100644
--- a/src/box/bind.h
+++ b/src/box/bind.h
@@ -64,6 +64,7 @@ struct sql_bind {
bool b;
double d;
int64_t i64;
+ uint64_t u64;
/** For string or blob. */
const char *s;
};
diff --git a/src/box/errcode.h b/src/box/errcode.h
index f4aba2f54..417029adb 100644
--- a/src/box/errcode.h
+++ b/src/box/errcode.h
@@ -242,7 +242,7 @@ struct errcode_record {
/*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
/*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
/*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
- /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %lld") \
+ /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %llu") \
/*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \
/*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \
/*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \
diff --git a/src/box/execute.c b/src/box/execute.c
index f5aead391..2e76724a6 100644
--- a/src/box/execute.c
+++ b/src/box/execute.c
@@ -146,7 +146,7 @@ sql_column_to_messagepack(struct sql_stmt *stmt, int i,
break;
}
case MP_UINT: {
- uint64_t n = sql_column_int64(stmt, i);
+ uint64_t n = sql_column_uint64(stmt, i);
size = mp_sizeof_uint(n);
char *pos = (char *) region_alloc(region, size);
if (pos == NULL)
diff --git a/src/box/lua/execute.c b/src/box/lua/execute.c
index 239aba47b..eb6516096 100644
--- a/src/box/lua/execute.c
+++ b/src/box/lua/execute.c
@@ -140,12 +140,9 @@ lua_sql_bind_decode(struct lua_State *L, struct sql_bind *bind, int idx, int i)
return -1;
switch (field.type) {
case MP_UINT:
- if ((uint64_t) field.ival > INT64_MAX) {
- diag_set(ClientError, ER_SQL_BIND_VALUE,
- sql_bind_name(bind), "INTEGER");
- return -1;
- }
- FALLTHROUGH;
+ bind->u64 = field.ival;
+ bind->bytes = sizeof(bind->u64);
+ break;
case MP_INT:
bind->i64 = field.ival;
bind->bytes = sizeof(bind->i64);
diff --git a/src/box/lua/lua_sql.c b/src/box/lua/lua_sql.c
index 59ea260bf..89e72e6d2 100644
--- a/src/box/lua/lua_sql.c
+++ b/src/box/lua/lua_sql.c
@@ -61,7 +61,7 @@ lua_sql_call(sql_context *pCtx, int nVal, sql_value **apVal) {
luaL_pushint64(L, sql_value_int64(param));
break;
case MP_UINT:
- luaL_pushuint64(L, sql_value_int64(param));
+ luaL_pushuint64(L, sql_value_uint64(param));
break;
case MP_DOUBLE:
lua_pushnumber(L, sql_value_double(param));
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index e2353d8cc..75d6579ca 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -918,10 +918,10 @@ emitNewSysSequenceRecord(Parse *pParse, int reg_seq_id, const char *seq_name)
/* 5. Minimum */
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 5, 0,
- (unsigned char*)&min_usigned_long_long, P4_INT64);
+ (unsigned char*)&min_usigned_long_long, P4_UINT64);
/* 6. Maximum */
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 6, 0,
- (unsigned char*)&max_usigned_long_long, P4_INT64);
+ (unsigned char*)&max_usigned_long_long, P4_UINT64);
/* 7. Start */
sqlVdbeAddOp2(v, OP_Integer, 1, first_col + 7);
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 158631416..4da297a5f 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -1203,7 +1203,7 @@ sqlExprAssignVarNumber(Parse * pParse, Expr * pExpr, u32 n)
testcase(i == 1);
testcase(i == SQL_BIND_PARAMETER_MAX - 1);
testcase(i == SQL_BIND_PARAMETER_MAX);
- if (i < 1) {
+ if (is_neg || i < 1) {
diag_set(ClientError, ER_SQL_PARSER_GENERIC,
"Index of binding slots must start "\
"from 1");
@@ -3367,12 +3367,13 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
if (rc != 0) {
int_overflow:
diag_set(ClientError, ER_INT_LITERAL_MAX, z, INT64_MIN,
- INT64_MAX);
+ UINT64_MAX);
parse->is_aborted = true;
return;
}
}
- sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0, (u8 *)&value, P4_INT64);
+ sqlVdbeAddOp4Dup8(v, OP_Int64, 0, mem, 0, (u8 *) &value,
+ is_neg ? P4_INT64 : P4_UINT64);
}
/*
diff --git a/src/box/sql/pragma.c b/src/box/sql/pragma.c
index 53524b617..18b8f351b 100644
--- a/src/box/sql/pragma.c
+++ b/src/box/sql/pragma.c
@@ -134,7 +134,7 @@ static void
returnSingleInt(Vdbe * v, i64 value)
{
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, 1, 0, (const u8 *)&value,
- P4_INT64);
+ value < 0 ? P4_INT64 : P4_UINT64);
sqlVdbeAddOp2(v, OP_ResultRow, 1, 1);
}
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 4697e3003..89b7356d8 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -594,6 +594,9 @@ sql_column_boolean(struct sql_stmt *stmt, int column);
sql_int64
sql_column_int64(sql_stmt *, int iCol);
+uint64_t
+sql_column_uint64(struct sql_stmt *stmt, int column);
+
const unsigned char *
sql_column_text(sql_stmt *,
int iCol);
@@ -908,6 +911,9 @@ sql_bind_int(sql_stmt *, int, int);
int
sql_bind_int64(sql_stmt *, int, sql_int64);
+int
+sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value);
+
int
sql_bind_null(sql_stmt *, int);
diff --git a/src/box/sql/util.c b/src/box/sql/util.c
index ee6a83ad5..3af2d6d1f 100644
--- a/src/box/sql/util.c
+++ b/src/box/sql/util.c
@@ -611,8 +611,6 @@ sql_atoi64(const char *z, int64_t *val, bool *is_neg, int length)
*val = strtoll(z, &end, 10);
} else {
uint64_t u_val = strtoull(z, &end, 10);
- if (u_val > INT64_MAX)
- return -1;
*val = u_val;
}
/* No digits were found, e.g. an empty string. */
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9c28b9131..dae9f903d 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1108,7 +1108,7 @@ case OP_Bool: { /* out2 */
case OP_Int64: { /* out2 */
pOut = out2Prerelease(p, pOp);
assert(pOp->p4.pI64!=0);
- mem_set_int(pOut, *pOp->p4.pI64, *pOp->p4.pI64 < 0);
+ mem_set_int(pOut, *pOp->p4.pI64, pOp->p4type == P4_INT64);
break;
}
diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h
index f9bb96f09..b37daa897 100644
--- a/src/box/sql/vdbe.h
+++ b/src/box/sql/vdbe.h
@@ -70,7 +70,7 @@ struct VdbeOp {
int i; /* Integer value if p4type==P4_INT32 */
void *p; /* Generic pointer */
char *z; /* Pointer to data for string (char array) types */
- i64 *pI64; /* Used when p4type is P4_INT64 */
+ i64 *pI64; /* Used when p4type is P4_INT64/UINT64 */
double *pReal; /* Used when p4type is P4_REAL */
FuncDef *pFunc; /* Used when p4type is P4_FUNCDEF */
sql_context *pCtx; /* Used when p4type is P4_FUNCCTX */
@@ -127,6 +127,7 @@ struct SubProgram {
#define P4_TRANSIENT 0 /* P4 is a pointer to a transient string */
#define P4_REAL (-9) /* P4 is a 64-bit floating point value */
#define P4_INT64 (-10) /* P4 is a 64-bit signed integer */
+#define P4_UINT64 (-8) /* P4 is a 64-bit signed integer */
#define P4_INT32 (-11) /* P4 is a 32-bit signed integer */
#define P4_INTARRAY (-12) /* P4 is a vector of 32-bit integers */
#define P4_SUBPROGRAM (-13) /* P4 is a pointer to a SubProgram structure */
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 752efeecb..751aead70 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -1027,6 +1027,14 @@ sql_column_int64(sql_stmt * pStmt, int i)
return val;
}
+uint64_t
+sql_column_uint64(sql_stmt * pStmt, int i)
+{
+ sql_int64 val = sql_value_uint64(columnMem(pStmt, i));
+ columnMallocFailure(pStmt);
+ return val;
+}
+
const unsigned char *
sql_column_text(sql_stmt * pStmt, int i)
{
@@ -1387,9 +1395,22 @@ sql_bind_int64(sql_stmt * pStmt, int i, sql_int64 iValue)
int rc;
Vdbe *p = (Vdbe *) pStmt;
rc = vdbeUnbind(p, i);
+ assert(iValue < 0);
if (rc == SQL_OK) {
rc = sql_bind_type(p, i, "INTEGER");
- mem_set_int(&p->aVar[i - 1], iValue, iValue < 0);
+ mem_set_int(&p->aVar[i - 1], iValue, true);
+ }
+ return rc;
+}
+
+int
+sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value)
+{
+ struct Vdbe *p = (struct Vdbe *) stmt;
+ int rc = vdbeUnbind(p, i);
+ if (rc == SQL_OK) {
+ rc = sql_bind_type(p, i, "UNSIGNED");
+ mem_set_int(&p->aVar[i - 1], value, false);
}
return rc;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 5a71e1801..7ca2d89d8 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -386,8 +386,8 @@ sqlVdbeAddOp4(Vdbe * p, /* Add the opcode to this VM */
}
/*
- * Add an opcode that includes the p4 value with a P4_INT64 or
- * P4_REAL type.
+ * Add an opcode that includes the p4 value with a P4_INT64/UINT64
+ * or P4_REAL type.
*/
int
sqlVdbeAddOp4Dup8(Vdbe * p, /* Add the opcode to this VM */
@@ -863,6 +863,7 @@ freeP4(sql * db, int p4type, void *p4)
}
case P4_REAL:
case P4_INT64:
+ case P4_UINT64:
case P4_DYNAMIC:
case P4_INTARRAY:{
sqlDbFree(db, p4);
@@ -1355,6 +1356,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
sqlXPrintf(&x, "%lld", *pOp->p4.pI64);
break;
}
+ case P4_UINT64: {
+ sqlXPrintf(&x, "%llu", (uint64_t)*pOp->p4.pI64);
+ break;
+ }
case P4_INT32:{
sqlXPrintf(&x, "%d", pOp->p4.i);
break;
@@ -3735,11 +3740,6 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
}
case MP_UINT: {
uint64_t v = mp_decode_uint(&buf);
- if (v > INT64_MAX) {
- diag_set(ClientError, ER_SQL_EXECUTE,
- "integer is overflowed");
- return -1;
- }
mem->u.u = v;
mem->flags = MEM_UInt;
break;
diff --git a/test/sql-tap/func.test.lua b/test/sql-tap/func.test.lua
index 9cf9c2f67..217b137a4 100755
--- a/test/sql-tap/func.test.lua
+++ b/test/sql-tap/func.test.lua
@@ -919,7 +919,7 @@ test:do_execsql_test(
UNION ALL SELECT -9223372036854775807)
]], {
-- <func-8.7>
- "number"
+ "unsigned"
-- </func-8.7>
})
diff --git a/test/sql-tap/hexlit.test.lua b/test/sql-tap/hexlit.test.lua
index 288d823d9..220aa9117 100755
--- a/test/sql-tap/hexlit.test.lua
+++ b/test/sql-tap/hexlit.test.lua
@@ -91,7 +91,7 @@ hexlit1(160, "0X1000000000000000", 1152921504606846976LL)
hexlit1(161, "0x2000000000000000", 2305843009213693952LL)
hexlit1(162, "0X4000000000000000", 4611686018427387904LL)
hexlit1(163, "0x8000000000000000", -9223372036854775808LL)
-hexlit1(164, "0XFFFFFFFFFFFFFFFF", -1)
+hexlit1(164, "0XFFFFFFFFFFFFFFFF", 18446744073709551615LL)
for n = 1, 0x10 -1, 1 do
hexlit1("200."..n..".1", "0X"..string.format("%03X",n), n)
hexlit1("200."..n..".2", "0x"..string.format("%03X",n), n)
diff --git a/test/sql-tap/sql-errors.test.lua b/test/sql-tap/sql-errors.test.lua
index 9357c406b..4201899fd 100755
--- a/test/sql-tap/sql-errors.test.lua
+++ b/test/sql-tap/sql-errors.test.lua
@@ -139,10 +139,10 @@ test:do_catchsql_test(
test:do_catchsql_test(
"sql-errors-1.13",
[[
- SELECT 9223372036854775808;
+ SELECT 18446744073709551616;
]], {
-- <sql-errors-1.13>
- 1,"Integer literal 9223372036854775808 exceeds the supported range -9223372036854775808 - 9223372036854775807"
+ 1,"Integer literal 18446744073709551616 exceeds the supported range -9223372036854775808 - 18446744073709551615"
-- </sql-errors-1.13>
})
diff --git a/test/sql/bind.result b/test/sql/bind.result
index 076bf8319..209daff85 100644
--- a/test/sql/bind.result
+++ b/test/sql/bind.result
@@ -72,11 +72,11 @@ execute('SELECT ?, ?, ?', {1, 2, 3})
---
- metadata:
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: '?'
- type: INTEGER
+ type: UNSIGNED
rows:
- [1, 2, 3]
...
@@ -102,11 +102,11 @@ execute('SELECT ?, :value1, @value2', parameters)
---
- metadata:
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: :value1
- type: INTEGER
+ type: UNSIGNED
- name: '@value2'
- type: INTEGER
+ type: UNSIGNED
rows:
- [10, 11, 12]
...
@@ -144,21 +144,21 @@ execute('SELECT :value3, ?, :value1, ?, ?, @value2, ?, :value3', parameters)
---
- metadata:
- name: :value3
- type: INTEGER
+ type: UNSIGNED
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: :value1
- type: INTEGER
+ type: UNSIGNED
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: '?'
- type: INTEGER
+ type: UNSIGNED
- name: '@value2'
- type: INTEGER
+ type: UNSIGNED
- name: '?'
type: boolean
- name: :value3
- type: INTEGER
+ type: UNSIGNED
rows:
- [1, 2, 3, 4, 5, 6, null, 1]
...
@@ -187,9 +187,9 @@ execute('SELECT ? AS kek, ? AS kek2', {1, 2})
---
- metadata:
- name: KEK
- type: INTEGER
+ type: UNSIGNED
- name: KEK2
- type: INTEGER
+ type: UNSIGNED
rows:
- [1, 2]
...
@@ -236,7 +236,11 @@ execute(sql, parameters)
-- suitable method in its bind API.
execute('SELECT ? AS big_uint', {0xefffffffffffffff})
---
-- error: Bind value for parameter 1 is out of range for type INTEGER
+- metadata:
+ - name: BIG_UINT
+ type: UNSIGNED
+ rows:
+ - [17293822569102704640]
...
-- Bind incorrect parameters.
ok, err = pcall(execute, 'SELECT ?', { {1, 2, 3} })
@@ -275,6 +279,16 @@ execute('SELECT :value', parameters)
---
- error: Bind value type MAP for parameter ':value' is not supported
...
+-- gh-3810: bind values of integer in range up to 2^64 - 1.
+--
+execute('SELECT ? ', {18446744073709551615ULL})
+---
+- metadata:
+ - name: '?'
+ type: UNSIGNED
+ rows:
+ - [18446744073709551615]
+...
test_run:cmd("setopt delimiter ';'")
---
- true
diff --git a/test/sql/bind.test.lua b/test/sql/bind.test.lua
index 229207d3a..ea71e7fe7 100644
--- a/test/sql/bind.test.lua
+++ b/test/sql/bind.test.lua
@@ -89,6 +89,10 @@ parameters[1] = {}
parameters[1][':value'] = {kek = 300}
execute('SELECT :value', parameters)
+-- gh-3810: bind values of integer in range up to 2^64 - 1.
+--
+execute('SELECT ? ', {18446744073709551615ULL})
+
test_run:cmd("setopt delimiter ';'")
if remote then
cn:close()
diff --git a/test/sql/gh-2347-max-int-literals.result b/test/sql/gh-2347-max-int-literals.result
deleted file mode 100644
index e6c4d9992..000000000
--- a/test/sql/gh-2347-max-int-literals.result
+++ /dev/null
@@ -1,39 +0,0 @@
-test_run = require('test_run').new()
----
-...
-engine = test_run:get_cfg('engine')
----
-...
-box.execute('pragma sql_default_engine=\''..engine..'\'')
----
-- row_count: 0
-...
-box.cfg{}
----
-...
-box.execute("select (9223372036854775807)")
----
-- metadata:
- - name: (9223372036854775807)
- type: integer
- rows:
- - [9223372036854775807]
-...
-box.execute("select (-9223372036854775808)")
----
-- metadata:
- - name: (-9223372036854775808)
- type: integer
- rows:
- - [-9223372036854775808]
-...
-box.execute("select (9223372036854775808)")
----
-- error: Integer literal 9223372036854775808 exceeds the supported range -9223372036854775808
- - 9223372036854775807
-...
-box.execute("select (-9223372036854775809)")
----
-- error: Integer literal -9223372036854775809 exceeds the supported range -9223372036854775808
- - 9223372036854775807
-...
diff --git a/test/sql/gh-2347-max-int-literals.test.lua b/test/sql/gh-2347-max-int-literals.test.lua
deleted file mode 100644
index 8331f0333..000000000
--- a/test/sql/gh-2347-max-int-literals.test.lua
+++ /dev/null
@@ -1,11 +0,0 @@
-test_run = require('test_run').new()
-engine = test_run:get_cfg('engine')
-box.execute('pragma sql_default_engine=\''..engine..'\'')
-
-box.cfg{}
-
-box.execute("select (9223372036854775807)")
-box.execute("select (-9223372036854775808)")
-
-box.execute("select (9223372036854775808)")
-box.execute("select (-9223372036854775809)")
diff --git a/test/sql/integer-overflow.result b/test/sql/integer-overflow.result
index 40962ac5c..915914eac 100644
--- a/test/sql/integer-overflow.result
+++ b/test/sql/integer-overflow.result
@@ -40,28 +40,67 @@ box.execute('SELECT (9223372036854775807 + 9223372036854775807 + 2);')
---
- error: 'Failed to execute SQL statement: integer is overflowed'
...
+box.execute('SELECT 18446744073709551615 * 2;')
+---
+- error: 'Failed to execute SQL statement: integer is overflowed'
+...
+box.execute('SELECT (-9223372036854775807 * (-2));')
+---
+- metadata:
+ - name: (-9223372036854775807 * (-2))
+ type: integer
+ rows:
+ - [18446744073709551614]
+...
-- Literals are checked right after parsing.
--
box.execute('SELECT 9223372036854775808;')
---
-- error: Integer literal 9223372036854775808 exceeds the supported range -9223372036854775808
- - 9223372036854775807
+- metadata:
+ - name: '9223372036854775808'
+ type: integer
+ rows:
+ - [9223372036854775808]
...
box.execute('SELECT -9223372036854775809;')
---
- error: Integer literal -9223372036854775809 exceeds the supported range -9223372036854775808
- - 9223372036854775807
+ - 18446744073709551615
...
box.execute('SELECT 9223372036854775808 - 1;')
---
-- error: Integer literal 9223372036854775808 exceeds the supported range -9223372036854775808
- - 9223372036854775807
+- metadata:
+ - name: 9223372036854775808 - 1
+ type: integer
+ rows:
+ - [9223372036854775807]
+...
+box.execute('SELECT 18446744073709551615;')
+---
+- metadata:
+ - name: '18446744073709551615'
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+box.execute('SELECT 18446744073709551616;')
+---
+- error: Integer literal 18446744073709551616 exceeds the supported range -9223372036854775808
+ - 18446744073709551615
...
-- Test that CAST may also leads to overflow.
--
box.execute('SELECT CAST(\'9223372036854775808\' AS INTEGER);')
---
-- error: 'Type mismatch: can not convert 9223372036854775808 to integer'
+- metadata:
+ - name: CAST('9223372036854775808' AS INTEGER)
+ type: integer
+ rows:
+ - [9223372036854775808]
+...
+box.execute('SELECT CAST(\'18446744073709551616\' AS INTEGER);')
+---
+- error: 'Type mismatch: can not convert 18446744073709551616 to integer'
...
-- Due to inexact represantation of large integers in terms of
-- floating point numbers, numerics with value < INT64_MAX
@@ -85,9 +124,18 @@ box.space.T:insert({9223372036854775809})
---
- [9223372036854775808]
...
+box.space.T:insert({18446744073709551615ULL})
+---
+- [18446744073709551615]
+...
box.execute('SELECT * FROM t;')
---
-- error: 'Failed to execute SQL statement: integer is overflowed'
+- metadata:
+ - name: ID
+ type: integer
+ rows:
+ - [9223372036854775808]
+ - [18446744073709551615]
...
box.space.T:drop()
---
diff --git a/test/sql/integer-overflow.test.lua b/test/sql/integer-overflow.test.lua
index 7727f368c..246465da0 100644
--- a/test/sql/integer-overflow.test.lua
+++ b/test/sql/integer-overflow.test.lua
@@ -11,15 +11,21 @@ box.execute('SELECT (-9223372036854775808 / -1);')
box.execute('SELECT (-9223372036854775808 - 1);')
box.execute('SELECT (9223372036854775807 + 1);')
box.execute('SELECT (9223372036854775807 + 9223372036854775807 + 2);')
+box.execute('SELECT 18446744073709551615 * 2;')
+box.execute('SELECT (-9223372036854775807 * (-2));')
-- Literals are checked right after parsing.
--
box.execute('SELECT 9223372036854775808;')
box.execute('SELECT -9223372036854775809;')
box.execute('SELECT 9223372036854775808 - 1;')
+box.execute('SELECT 18446744073709551615;')
+box.execute('SELECT 18446744073709551616;')
+
-- Test that CAST may also leads to overflow.
--
box.execute('SELECT CAST(\'9223372036854775808\' AS INTEGER);')
+box.execute('SELECT CAST(\'18446744073709551616\' AS INTEGER);')
-- Due to inexact represantation of large integers in terms of
-- floating point numbers, numerics with value < INT64_MAX
-- have INT64_MAX + 1 value in integer representation:
@@ -33,5 +39,6 @@ box.execute('SELECT CAST(9223372036854775807.0 AS INTEGER);')
--
box.execute('CREATE TABLE t (id INT PRIMARY KEY);')
box.space.T:insert({9223372036854775809})
+box.space.T:insert({18446744073709551615ULL})
box.execute('SELECT * FROM t;')
box.space.T:drop()
diff --git a/test/sql/iproto.result b/test/sql/iproto.result
index e734872b2..10f42d936 100644
--- a/test/sql/iproto.result
+++ b/test/sql/iproto.result
@@ -368,11 +368,11 @@ cn:execute('select $2, $1, $3', parameters)
---
- metadata:
- name: $2
- type: INTEGER
+ type: UNSIGNED
- name: $1
- type: INTEGER
+ type: UNSIGNED
- name: $3
- type: INTEGER
+ type: UNSIGNED
rows:
- [22, 11, 33]
...
diff --git a/test/sql/types.result b/test/sql/types.result
index 4670fd38a..e12515345 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -963,3 +963,506 @@ box.execute('SELECT ?', {true})
rows:
- [true]
...
+-- gh-3810: range of integer is extended up to 2^64 - 1.
+--
+box.execute("SELECT 18446744073709551615 > 18446744073709551614;")
+---
+- metadata:
+ - name: 18446744073709551615 > 18446744073709551614
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT 18446744073709551615 = 18446744073709551615;")
+---
+- metadata:
+ - name: 18446744073709551615 = 18446744073709551615
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT 18446744073709551615 > -9223372036854775808;")
+---
+- metadata:
+ - name: 18446744073709551615 > -9223372036854775808
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT -1 < 18446744073709551615;")
+---
+- metadata:
+ - name: -1 < 18446744073709551615
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT 18446744073709551610 - 18446744073709551615;")
+---
+- metadata:
+ - name: 18446744073709551610 - 18446744073709551615
+ type: integer
+ rows:
+ - [-5]
+...
+box.execute("SELECT 18446744073709551615 = null;")
+---
+- metadata:
+ - name: 18446744073709551615 = null
+ type: boolean
+ rows:
+ - [null]
+...
+box.execute("SELECT 18446744073709551615 = 18446744073709551615.0;")
+---
+- metadata:
+ - name: 18446744073709551615 = 18446744073709551615.0
+ type: boolean
+ rows:
+ - [false]
+...
+box.execute("SELECT 18446744073709551615.0 > 18446744073709551615")
+---
+- metadata:
+ - name: 18446744073709551615.0 > 18446744073709551615
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT 18446744073709551615 IN ('18446744073709551615', 18446744073709551615.0)")
+---
+- metadata:
+ - name: 18446744073709551615 IN ('18446744073709551615', 18446744073709551615.0)
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT 1 LIMIT 18446744073709551615;")
+---
+- metadata:
+ - name: '1'
+ type: integer
+ rows:
+ - [1]
+...
+box.execute("SELECT 1 LIMIT 1 OFFSET 18446744073709551615;")
+---
+- metadata:
+ - name: '1'
+ type: integer
+ rows: []
+...
+box.execute("SELECT CAST('18446744073' || '709551616' AS INTEGER);")
+---
+- error: 'Type mismatch: can not convert 18446744073709551616 to integer'
+...
+box.execute("SELECT CAST('18446744073' || '709551615' AS INTEGER);")
+---
+- metadata:
+ - name: CAST('18446744073' || '709551615' AS INTEGER)
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+box.execute("SELECT 18446744073709551610 + 5;")
+---
+- metadata:
+ - name: 18446744073709551610 + 5
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+box.execute("SELECT 18446744073709551615 * 1;")
+---
+- metadata:
+ - name: 18446744073709551615 * 1
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+box.execute("SELECT 1 / 18446744073709551615;")
+---
+- metadata:
+ - name: 1 / 18446744073709551615
+ type: integer
+ rows:
+ - [0]
+...
+box.execute("SELECT 18446744073709551615 / 18446744073709551615;")
+---
+- metadata:
+ - name: 18446744073709551615 / 18446744073709551615
+ type: integer
+ rows:
+ - [1]
+...
+box.execute("SELECT 18446744073709551615 / -9223372036854775808;")
+---
+- metadata:
+ - name: 18446744073709551615 / -9223372036854775808
+ type: integer
+ rows:
+ - [-1]
+...
+box.execute("SELECT 0 - 18446744073709551610;")
+---
+- error: 'Failed to execute SQL statement: integer is overflowed'
+...
+box.execute("CREATE TABLE t (id INT PRIMARY KEY, i INT);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (2, 18446744073709551614);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (3, 18446744073709551613)")
+---
+- row_count: 1
+...
+box.execute("SELECT i FROM t;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551615]
+ - [18446744073709551614]
+ - [18446744073709551613]
+...
+box.execute("SELECT i FROM t WHERE i = 18446744073709551615;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+box.execute("SELECT i FROM t WHERE i BETWEEN 18446744073709551613 AND 18446744073709551615;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551615]
+ - [18446744073709551614]
+ - [18446744073709551613]
+...
+box.execute("SELECT i FROM t ORDER BY i;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551613]
+ - [18446744073709551614]
+ - [18446744073709551615]
+...
+box.execute("SELECT i FROM t ORDER BY -i;")
+---
+- error: 'Failed to execute SQL statement: integer is overflowed'
+...
+box.execute("SELECT i FROM t ORDER BY i LIMIT 1;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551613]
+...
+-- Test that built-in functions are capable of handling unsigneds.
+--
+box.execute("DELETE FROM t WHERE i > 18446744073709551613;")
+---
+- row_count: 2
+...
+box.execute("INSERT INTO t VALUES (1, 1);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (2, -1);")
+---
+- row_count: 1
+...
+box.execute("SELECT sum(i) FROM t;")
+---
+- metadata:
+ - name: sum(i)
+ type: number
+ rows:
+ - [18446744073709551613]
+...
+box.execute("SELECT avg(i) FROM t;")
+---
+- metadata:
+ - name: avg(i)
+ type: number
+ rows:
+ - [6148914691236516864]
+...
+box.execute("SELECT total(i) FROM t;")
+---
+- metadata:
+ - name: total(i)
+ type: number
+ rows:
+ - [1.844674407371e+19]
+...
+box.execute("SELECT min(i) FROM t;")
+---
+- metadata:
+ - name: min(i)
+ type: scalar
+ rows:
+ - [-1]
+...
+box.execute("SELECT max(i) FROM t;")
+---
+- metadata:
+ - name: max(i)
+ type: scalar
+ rows:
+ - [18446744073709551613]
+...
+box.execute("SELECT count(i) FROM t;")
+---
+- metadata:
+ - name: count(i)
+ type: integer
+ rows:
+ - [3]
+...
+box.execute("SELECT group_concat(i) FROM t;")
+---
+- metadata:
+ - name: group_concat(i)
+ type: string
+ rows:
+ - ['1,-1,18446744073709551613']
+...
+box.execute("DELETE FROM t WHERE i < 18446744073709551613;")
+---
+- row_count: 2
+...
+box.execute("SELECT lower(i) FROM t;")
+---
+- metadata:
+ - name: lower(i)
+ type: string
+ rows:
+ - ['18446744073709551613']
+...
+box.execute("SELECT upper(i) FROM t;")
+---
+- metadata:
+ - name: upper(i)
+ type: string
+ rows:
+ - ['18446744073709551613']
+...
+box.execute("SELECT abs(i) FROM t;")
+---
+- metadata:
+ - name: abs(i)
+ type: number
+ rows:
+ - [18446744073709551613]
+...
+box.execute("SELECT typeof(i) FROM t;")
+---
+- metadata:
+ - name: typeof(i)
+ type: string
+ rows:
+ - ['unsigned']
+...
+box.execute("SELECT quote(i) FROM t;")
+---
+- metadata:
+ - name: quote(i)
+ type: string
+ rows:
+ - [18446744073709551613]
+...
+box.execute("SELECT min(-1, i) FROM t;")
+---
+- metadata:
+ - name: min(-1, i)
+ type: scalar
+ rows:
+ - [-1]
+...
+box.execute("SELECT quote(i) FROM t;")
+---
+- metadata:
+ - name: quote(i)
+ type: string
+ rows:
+ - [18446744073709551613]
+...
+box.execute("CREATE INDEX i ON t(i);")
+---
+- row_count: 1
+...
+box.execute("SELECT i FROM t WHERE i = 18446744073709551613;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551613]
+...
+box.execute("SELECT i FROM t WHERE i >= 18446744073709551613 ORDER BY i;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551613]
+...
+box.execute("UPDATE t SET i = 18446744073709551615 WHERE i = 18446744073709551613;")
+---
+- row_count: 1
+...
+box.execute("SELECT i FROM t;")
+---
+- metadata:
+ - name: I
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
+-- Test constraints functionality.
+--
+box.execute("CREATE TABLE parent (id INT PRIMARY KEY, a INT UNIQUE);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO parent VALUES (1, 18446744073709551613);")
+---
+- row_count: 1
+...
+box.space.T:truncate()
+---
+...
+box.execute("ALTER TABLE t ADD CONSTRAINT fk1 FOREIGN KEY (i) REFERENCES parent (a);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+---
+- error: 'Failed to execute SQL statement: FOREIGN KEY constraint failed'
+...
+box.execute("INSERT INTO parent VALUES (2, 18446744073709551615);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+---
+- row_count: 1
+...
+box.execute("ALTER TABLE t DROP CONSTRAINT fk1;")
+---
+- row_count: 1
+...
+box.space.PARENT:drop()
+---
+...
+box.space.T:drop()
+---
+...
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a INT CHECK (a > 18446744073709551612));")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t1 VALUES (1, 18446744073709551611);")
+---
+- error: 'Failed to execute SQL statement: CHECK constraint failed: T1'
+...
+box.execute("INSERT INTO t1 VALUES (1, -1);")
+---
+- error: 'Failed to execute SQL statement: CHECK constraint failed: T1'
+...
+box.space.T1:drop()
+---
+...
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a INT DEFAULT 18446744073709551615);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t1 (id) VALUES (1);")
+---
+- row_count: 1
+...
+box.space.T1:select()
+---
+- - [1, 18446744073709551615]
+...
+box.space.T1:drop()
+---
+...
+-- Test that autoincrement accepts only max 2^63 - 1 .
+--
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY AUTOINCREMENT);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t1 VALUES (18446744073709551615);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t1 VALUES (NULL);")
+---
+- autoincrement_ids:
+ - 1
+ row_count: 1
+...
+box.space.T1:drop()
+---
+...
+-- Test CAST facilities.
+--
+box.execute("SELECT CAST(18446744073709551615 AS FLOAT);")
+---
+- metadata:
+ - name: CAST(18446744073709551615 AS FLOAT)
+ type: number
+ rows:
+ - [1.844674407371e+19]
+...
+box.execute("SELECT CAST(18446744073709551615 AS TEXT);")
+---
+- metadata:
+ - name: CAST(18446744073709551615 AS TEXT)
+ type: string
+ rows:
+ - ['18446744073709551615']
+...
+box.execute("SELECT CAST(18446744073709551615 AS SCALAR);")
+---
+- metadata:
+ - name: CAST(18446744073709551615 AS SCALAR)
+ type: scalar
+ rows:
+ - [18446744073709551615]
+...
+box.execute("SELECT CAST(18446744073709551615 AS BOOLEAN);")
+---
+- metadata:
+ - name: CAST(18446744073709551615 AS BOOLEAN)
+ type: boolean
+ rows:
+ - [true]
+...
+box.execute("SELECT CAST('18446744073709551615' AS INTEGER);")
+---
+- metadata:
+ - name: CAST('18446744073709551615' AS INTEGER)
+ type: integer
+ rows:
+ - [18446744073709551615]
+...
diff --git a/test/sql/types.test.lua b/test/sql/types.test.lua
index ae1a0ab72..ccdd1f3d4 100644
--- a/test/sql/types.test.lua
+++ b/test/sql/types.test.lua
@@ -234,3 +234,101 @@ box.execute('SELECT \'9223372036854\' + 1;')
-- Fix BOOLEAN bindings.
box.execute('SELECT ?', {true})
+
+-- gh-3810: range of integer is extended up to 2^64 - 1.
+--
+box.execute("SELECT 18446744073709551615 > 18446744073709551614;")
+box.execute("SELECT 18446744073709551615 = 18446744073709551615;")
+box.execute("SELECT 18446744073709551615 > -9223372036854775808;")
+box.execute("SELECT -1 < 18446744073709551615;")
+box.execute("SELECT 18446744073709551610 - 18446744073709551615;")
+box.execute("SELECT 18446744073709551615 = null;")
+box.execute("SELECT 18446744073709551615 = 18446744073709551615.0;")
+box.execute("SELECT 18446744073709551615.0 > 18446744073709551615")
+box.execute("SELECT 18446744073709551615 IN ('18446744073709551615', 18446744073709551615.0)")
+box.execute("SELECT 1 LIMIT 18446744073709551615;")
+box.execute("SELECT 1 LIMIT 1 OFFSET 18446744073709551615;")
+box.execute("SELECT CAST('18446744073' || '709551616' AS INTEGER);")
+box.execute("SELECT CAST('18446744073' || '709551615' AS INTEGER);")
+box.execute("SELECT 18446744073709551610 + 5;")
+box.execute("SELECT 18446744073709551615 * 1;")
+box.execute("SELECT 1 / 18446744073709551615;")
+box.execute("SELECT 18446744073709551615 / 18446744073709551615;")
+box.execute("SELECT 18446744073709551615 / -9223372036854775808;")
+box.execute("SELECT 0 - 18446744073709551610;")
+box.execute("CREATE TABLE t (id INT PRIMARY KEY, i INT);")
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+box.execute("INSERT INTO t VALUES (2, 18446744073709551614);")
+box.execute("INSERT INTO t VALUES (3, 18446744073709551613)")
+box.execute("SELECT i FROM t;")
+box.execute("SELECT i FROM t WHERE i = 18446744073709551615;")
+box.execute("SELECT i FROM t WHERE i BETWEEN 18446744073709551613 AND 18446744073709551615;")
+box.execute("SELECT i FROM t ORDER BY i;")
+box.execute("SELECT i FROM t ORDER BY -i;")
+box.execute("SELECT i FROM t ORDER BY i LIMIT 1;")
+-- Test that built-in functions are capable of handling unsigneds.
+--
+box.execute("DELETE FROM t WHERE i > 18446744073709551613;")
+box.execute("INSERT INTO t VALUES (1, 1);")
+box.execute("INSERT INTO t VALUES (2, -1);")
+box.execute("SELECT sum(i) FROM t;")
+box.execute("SELECT avg(i) FROM t;")
+box.execute("SELECT total(i) FROM t;")
+box.execute("SELECT min(i) FROM t;")
+box.execute("SELECT max(i) FROM t;")
+box.execute("SELECT count(i) FROM t;")
+box.execute("SELECT group_concat(i) FROM t;")
+
+box.execute("DELETE FROM t WHERE i < 18446744073709551613;")
+box.execute("SELECT lower(i) FROM t;")
+box.execute("SELECT upper(i) FROM t;")
+box.execute("SELECT abs(i) FROM t;")
+box.execute("SELECT typeof(i) FROM t;")
+box.execute("SELECT quote(i) FROM t;")
+box.execute("SELECT min(-1, i) FROM t;")
+box.execute("SELECT quote(i) FROM t;")
+
+box.execute("CREATE INDEX i ON t(i);")
+box.execute("SELECT i FROM t WHERE i = 18446744073709551613;")
+box.execute("SELECT i FROM t WHERE i >= 18446744073709551613 ORDER BY i;")
+
+box.execute("UPDATE t SET i = 18446744073709551615 WHERE i = 18446744073709551613;")
+box.execute("SELECT i FROM t;")
+
+-- Test constraints functionality.
+--
+box.execute("CREATE TABLE parent (id INT PRIMARY KEY, a INT UNIQUE);")
+box.execute("INSERT INTO parent VALUES (1, 18446744073709551613);")
+box.space.T:truncate()
+box.execute("ALTER TABLE t ADD CONSTRAINT fk1 FOREIGN KEY (i) REFERENCES parent (a);")
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+box.execute("INSERT INTO parent VALUES (2, 18446744073709551615);")
+box.execute("INSERT INTO t VALUES (1, 18446744073709551615);")
+box.execute("ALTER TABLE t DROP CONSTRAINT fk1;")
+box.space.PARENT:drop()
+box.space.T:drop()
+
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a INT CHECK (a > 18446744073709551612));")
+box.execute("INSERT INTO t1 VALUES (1, 18446744073709551611);")
+box.execute("INSERT INTO t1 VALUES (1, -1);")
+box.space.T1:drop()
+
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a INT DEFAULT 18446744073709551615);")
+box.execute("INSERT INTO t1 (id) VALUES (1);")
+box.space.T1:select()
+box.space.T1:drop()
+
+-- Test that autoincrement accepts only max 2^63 - 1 .
+--
+box.execute("CREATE TABLE t1 (id INT PRIMARY KEY AUTOINCREMENT);")
+box.execute("INSERT INTO t1 VALUES (18446744073709551615);")
+box.execute("INSERT INTO t1 VALUES (NULL);")
+box.space.T1:drop()
+
+-- Test CAST facilities.
+--
+box.execute("SELECT CAST(18446744073709551615 AS FLOAT);")
+box.execute("SELECT CAST(18446744073709551615 AS TEXT);")
+box.execute("SELECT CAST(18446744073709551615 AS SCALAR);")
+box.execute("SELECT CAST(18446744073709551615 AS BOOLEAN);")
+box.execute("SELECT CAST('18446744073709551615' AS INTEGER);")
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-06-07 15:37 ` [tarantool-patches] [PATCH 5/6] sql: introduce extended range for INTEGER type Nikita Pettik
@ 2019-06-11 21:11 ` Vladislav Shpilevoy
2019-07-01 14:21 ` n.pettik
2019-07-24 15:59 ` Konstantin Osipov
1 sibling, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-06-11 21:11 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 1 comment below, review
fixes at the end of the email, and on the branch in
a separate commit.
> diff --git a/src/box/errcode.h b/src/box/errcode.h
> index f4aba2f54..417029adb 100644
> --- a/src/box/errcode.h
> +++ b/src/box/errcode.h
> @@ -242,7 +242,7 @@ struct errcode_record {
> /*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
> /*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
> /*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
> - /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %lld") \
> + /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %llu") \
That error is used only once and with the constants. Please, inline them, and
make that error argument-less. Also, alongside I propose you to write the range in
[start, end] instead of 'start - end'. The latter looks like a minus.
Please, consider my review fixes here and on the branch in a separate commit:
=============================================================
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 75d6579ca..93aab8501 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -918,10 +918,10 @@ emitNewSysSequenceRecord(Parse *pParse, int reg_seq_id, const char *seq_name)
/* 5. Minimum */
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 5, 0,
- (unsigned char*)&min_usigned_long_long, P4_UINT64);
+ (unsigned char *) &min_usigned_long_long, P4_UINT64);
/* 6. Maximum */
sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 6, 0,
- (unsigned char*)&max_usigned_long_long, P4_UINT64);
+ (unsigned char *) &max_usigned_long_long, P4_UINT64);
/* 7. Start */
sqlVdbeAddOp2(v, OP_Integer, 1, first_col + 7);
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-01 14:21 ` n.pettik
2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-01 14:21 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/errcode.h b/src/box/errcode.h
>> index f4aba2f54..417029adb 100644
>> --- a/src/box/errcode.h
>> +++ b/src/box/errcode.h
>> @@ -242,7 +242,7 @@ struct errcode_record {
>> /*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
>> /*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
>> /*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
>> - /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %lld") \
>> + /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s exceeds the supported range %lld - %llu") \
>
> That error is used only once and with the constants. Please, inline them, and
> make that error argument-less. Also, alongside I propose you to write the range in
> [start, end] instead of 'start - end'. The latter looks like a minus.
Ok, I don’t mind:
diff --git a/src/box/errcode.h b/src/box/errcode.h
index 65fb1a011..601cb9670 100644
--- a/src/box/errcode.h
+++ b/src/box/errcode.h
@@ -242,7 +242,7 @@ struct errcode_record {
/*187 */_(ER_SQL_ANALYZE_ARGUMENT, "ANALYZE statement argument %s is not a base table") \
/*188 */_(ER_SQL_COLUMN_COUNT_MAX, "Failed to create space '%s': space column count %d exceeds the limit (%d)") \
/*189 */_(ER_HEX_LITERAL_MAX, "Hex literal %s%s length %d exceeds the supported limit (%d)") \
- /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range %lld - %llu") \
+ /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range [-9223372036854775808, 18446744073709551615]") \
/*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \
/*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \
/*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \
diff --git a/test/sql-tap/sql-errors.test.lua b/test/sql-tap/sql-errors.test.lua
index 4201899fd..30bd7d6ae 100755
--- a/test/sql-tap/sql-errors.test.lua
+++ b/test/sql-tap/sql-errors.test.lua
@@ -142,7 +142,7 @@ test:do_catchsql_test(
SELECT 18446744073709551616;
]], {
-- <sql-errors-1.13>
- 1,"Integer literal 18446744073709551616 exceeds the supported range -9223372036854775808 - 18446744073709551615"
+ 1,"Integer literal 18446744073709551616 exceeds the supported range [-9223372036854775808, 18446744073709551615]"
-- </sql-errors-1.13>
})
> Please, consider my review fixes here and on the branch in a separate commit:
Applied.
> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
> index 75d6579ca..93aab8501 100644
> --- a/src/box/sql/build.c
> +++ b/src/box/sql/build.c
> @@ -918,10 +918,10 @@ emitNewSysSequenceRecord(Parse *pParse, int reg_seq_id, const char *seq_name)
>
> /* 5. Minimum */
> sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 5, 0,
> - (unsigned char*)&min_usigned_long_long, P4_UINT64);
> + (unsigned char *) &min_usigned_long_long, P4_UINT64);
> /* 6. Maximum */
> sqlVdbeAddOp4Dup8(v, OP_Int64, 0, first_col + 6, 0,
> - (unsigned char*)&max_usigned_long_long, P4_UINT64);
> + (unsigned char *) &max_usigned_long_long, P4_UINT64);
> /* 7. Start */
> sqlVdbeAddOp2(v, OP_Integer, 1, first_col + 7);
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-07-01 14:21 ` n.pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
0 siblings, 0 replies; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
Consider new one below, and on the branch
in a separate commit.
=====================================================
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index fca22c371..8854fec06 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3311,8 +3311,7 @@ expr_code_int(struct Parse *parse, struct Expr *expr, bool is_neg,
if (sql_atoi64(z, &value, &unused, len) != 0 ||
(is_neg && (uint64_t) value > (uint64_t) INT64_MAX + 1)) {
int_overflow:
- diag_set(ClientError, ER_INT_LITERAL_MAX, sign, z,
- INT64_MIN, UINT64_MAX);
+ diag_set(ClientError, ER_INT_LITERAL_MAX, sign, z);
parse->is_aborted = true;
return;
}
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-06-07 15:37 ` [tarantool-patches] [PATCH 5/6] sql: introduce extended range for INTEGER type Nikita Pettik
2019-06-11 21:11 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-24 15:59 ` Konstantin Osipov
2019-07-24 16:54 ` n.pettik
1 sibling, 1 reply; 49+ messages in thread
From: Konstantin Osipov @ 2019-07-24 15:59 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
* Nikita Pettik <korablev@tarantool.org> [19/06/07 23:30]:
> @@ -72,11 +72,11 @@ execute('SELECT ?, ?, ?', {1, 2, 3})
> ---
> - metadata:
> - name: '?'
> - type: INTEGER
> + type: UNSIGNED
> - name: '?'
> - type: INTEGER
> + type: UNSIGNED
> - name: '?'
> - type: INTEGER
> + type: UNSIGNED
> rows:
> - [1, 2, 3]
Nikita,
I know this decision came after you coded most of the patch, but
we agreed that unsigned integer literals should still have
"INTEGER" type, not "UNSIGNED", since INTEGER in tarantool fully
covers UNSIGNED range.
This leads to fewer surprises to users, e.g. when calculating
expression types. Could you please fix this?
>
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-07-24 15:59 ` Konstantin Osipov
@ 2019-07-24 16:54 ` n.pettik
2019-07-24 17:09 ` Konstantin Osipov
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-24 16:54 UTC (permalink / raw)
To: tarantool-patches; +Cc: Konstantin Osipov
> On 24 Jul 2019, at 18:59, Konstantin Osipov <kostja@tarantool.org> wrote:
>
> * Nikita Pettik <korablev@tarantool.org> [19/06/07 23:30]:
>> @@ -72,11 +72,11 @@ execute('SELECT ?, ?, ?', {1, 2, 3})
>> ---
>> - metadata:
>> - name: '?'
>> - type: INTEGER
>> + type: UNSIGNED
>> - name: '?'
>> - type: INTEGER
>> + type: UNSIGNED
>> - name: '?'
>> - type: INTEGER
>> + type: UNSIGNED
>> rows:
>> - [1, 2, 3]
>
> Nikita,
>
> I know this decision came after you coded most of the patch, but
> we agreed that unsigned integer literals should still have
> "INTEGER" type, not "UNSIGNED", since INTEGER in tarantool fully
> covers UNSIGNED range.
Yep, I remember that and fixed that for literals:
tarantool> select 1
---
- metadata:
- name: '1'
type: integer
rows:
- [1]
…
But failed (simply forgot) to fix that for binding values.
So, juts want to clarify: we never set “UNSIGNED” type for
literals, even if they are in range [2^63, 2^64-1], right?
If so, patch turns out to be pretty simple.
> This leads to fewer surprises to users, e.g. when calculating
> expression types. Could you please fix this?
>
> --
> Konstantin Osipov, Moscow, Russia
>
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 5/6] sql: introduce extended range for INTEGER type
2019-07-24 16:54 ` n.pettik
@ 2019-07-24 17:09 ` Konstantin Osipov
0 siblings, 0 replies; 49+ messages in thread
From: Konstantin Osipov @ 2019-07-24 17:09 UTC (permalink / raw)
To: n.pettik; +Cc: tarantool-patches
* n.pettik <korablev@tarantool.org> [19/07/24 19:54]:
>
> But failed (simply forgot) to fix that for binding values.
> So, juts want to clarify: we never set “UNSIGNED” type for
> literals, even if they are in range [2^63, 2^64-1], right?
>
> If so, patch turns out to be pretty simple.
Yes, I think UNSIGNED is only storage type, we never should deliberately use
UNSIGNED values in vdbe, INTEGER is powerful enough to always use it.
Of course if we get UNSIGNED from the storage layer, we need to
preserve its type.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
` (4 preceding siblings ...)
2019-06-07 15:37 ` [tarantool-patches] [PATCH 5/6] sql: introduce extended range for INTEGER type Nikita Pettik
@ 2019-06-07 15:37 ` Nikita Pettik
2019-07-01 21:53 ` [tarantool-patches] " Vladislav Shpilevoy
2019-07-24 13:01 ` [tarantool-patches] Re: [PATCH 0/6] Introduce UNSIGNED type in SQL Kirill Yukhin
6 siblings, 1 reply; 49+ messages in thread
From: Nikita Pettik @ 2019-06-07 15:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Since all preparations concerning internal handling of unsigned values
have been done, now nothing prevents us from using UNSIGNED type in SQL.
This patch allows to specify UNSIGNED as a column type and adds CAST
rules, which are the same as for casual INTEGER, but with additional
check - result must be positive. Otherwise, error is raised.
Closes #4015
---
| 1 +
src/box/sql/parse.y | 1 +
src/box/sql/vdbemem.c | 10 ++++++-
test/sql/types.result | 72 +++++++++++++++++++++++++++++++++++++++++++++++++
test/sql/types.test.lua | 17 ++++++++++++
5 files changed, 100 insertions(+), 1 deletion(-)
--git a/extra/mkkeywordhash.c b/extra/mkkeywordhash.c
index 2ad74eddc..ff59d60cf 100644
--- a/extra/mkkeywordhash.c
+++ b/extra/mkkeywordhash.c
@@ -209,6 +209,7 @@ static Keyword aKeywordTable[] = {
{ "UNION", "TK_UNION", COMPOUND, true },
{ "UNIQUE", "TK_UNIQUE", ALWAYS, true },
{ "UNKNOWN", "TK_NULL", ALWAYS, true },
+ { "UNSIGNED", "TK_UNSIGNED", ALWAYS, true },
{ "UPDATE", "TK_UPDATE", ALWAYS, true },
{ "USING", "TK_USING", ALWAYS, true },
{ "VALUES", "TK_VALUES", ALWAYS, true },
diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y
index f241b8d52..ee0d6ce79 100644
--- a/src/box/sql/parse.y
+++ b/src/box/sql/parse.y
@@ -1766,6 +1766,7 @@ typedef(A) ::= VARCHAR char_len(B) . {
typedef(A) ::= number_typedef(A) .
number_typedef(A) ::= FLOAT_KW|REAL|DOUBLE . { A.type = FIELD_TYPE_NUMBER; }
number_typedef(A) ::= INT|INTEGER_KW . { A.type = FIELD_TYPE_INTEGER; }
+number_typedef(A) ::= UNSIGNED . { A.type = FIELD_TYPE_UNSIGNED; }
/**
* NUMERIC type is temporary disabled. To be enabled when
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 25119ff16..cef5196dd 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -699,12 +699,15 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return 0;
return -1;
case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
if ((pMem->flags & MEM_Blob) != 0) {
bool is_neg;
int64_t val;
if (sql_atoi64(pMem->z, (int64_t *) &val, &is_neg,
pMem->n) != 0)
return -1;
+ if (type == FIELD_TYPE_UNSIGNED && is_neg)
+ return -1;
mem_set_int(pMem, val, is_neg);
return 0;
}
@@ -713,7 +716,12 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
MemSetTypeFlag(pMem, MEM_UInt);
return 0;
}
- return sqlVdbeMemIntegerify(pMem, true);
+ if (sqlVdbeMemIntegerify(pMem, true) != 0)
+ return -1;
+ if (type == FIELD_TYPE_UNSIGNED &&
+ (pMem->flags & MEM_UInt) == 0)
+ return -1;
+ return 0;
case FIELD_TYPE_NUMBER:
return sqlVdbeMemRealify(pMem);
default:
diff --git a/test/sql/types.result b/test/sql/types.result
index e12515345..6a09bac8a 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -1466,3 +1466,75 @@ box.execute("SELECT CAST('18446744073709551615' AS INTEGER);")
rows:
- [18446744073709551615]
...
+-- gh-4015: introduce unsigned type in SQL.
+--
+box.execute("CREATE TABLE t1 (id UNSIGNED PRIMARY KEY);")
+---
+- row_count: 1
+...
+box.execute("INSERT INTO t1 VALUES (0), (1), (2);")
+---
+- row_count: 3
+...
+box.execute("INSERT INTO t1 VALUES (-3);")
+---
+- error: 'Failed to execute SQL statement: Tuple field 1 type does not match one required
+ by operation: expected unsigned'
+...
+box.execute("SELECT id FROM t1;")
+---
+- metadata:
+ - name: ID
+ type: unsigned
+ rows:
+ - [0]
+ - [1]
+ - [2]
+...
+box.execute("SELECT CAST(123 AS UNSIGNED);")
+---
+- metadata:
+ - name: CAST(123 AS UNSIGNED)
+ type: unsigned
+ rows:
+ - [123]
+...
+box.execute("SELECT CAST(-123 AS UNSIGNED);")
+---
+- error: 'Type mismatch: can not convert -123 to unsigned'
+...
+box.execute("SELECT CAST(1.5 AS UNSIGNED);")
+---
+- metadata:
+ - name: CAST(1.5 AS UNSIGNED)
+ type: unsigned
+ rows:
+ - [1]
+...
+box.execute("SELECT CAST(-1.5 AS UNSIGNED);")
+---
+- error: 'Type mismatch: can not convert -1 to unsigned'
+...
+box.execute("SELECT CAST(true AS UNSIGNED);")
+---
+- metadata:
+ - name: CAST(true AS UNSIGNED)
+ type: unsigned
+ rows:
+ - [1]
+...
+box.execute("SELECT CAST('123' AS UNSIGNED);")
+---
+- metadata:
+ - name: CAST('123' AS UNSIGNED)
+ type: unsigned
+ rows:
+ - [123]
+...
+box.execute("SELECT CAST('-123' AS UNSIGNED);")
+---
+- error: 'Type mismatch: can not convert -123 to unsigned'
+...
+box.space.T1:drop()
+---
+...
diff --git a/test/sql/types.test.lua b/test/sql/types.test.lua
index ccdd1f3d4..ddd19cde2 100644
--- a/test/sql/types.test.lua
+++ b/test/sql/types.test.lua
@@ -332,3 +332,20 @@ box.execute("SELECT CAST(18446744073709551615 AS TEXT);")
box.execute("SELECT CAST(18446744073709551615 AS SCALAR);")
box.execute("SELECT CAST(18446744073709551615 AS BOOLEAN);")
box.execute("SELECT CAST('18446744073709551615' AS INTEGER);")
+
+-- gh-4015: introduce unsigned type in SQL.
+--
+box.execute("CREATE TABLE t1 (id UNSIGNED PRIMARY KEY);")
+box.execute("INSERT INTO t1 VALUES (0), (1), (2);")
+box.execute("INSERT INTO t1 VALUES (-3);")
+box.execute("SELECT id FROM t1;")
+
+box.execute("SELECT CAST(123 AS UNSIGNED);")
+box.execute("SELECT CAST(-123 AS UNSIGNED);")
+box.execute("SELECT CAST(1.5 AS UNSIGNED);")
+box.execute("SELECT CAST(-1.5 AS UNSIGNED);")
+box.execute("SELECT CAST(true AS UNSIGNED);")
+box.execute("SELECT CAST('123' AS UNSIGNED);")
+box.execute("SELECT CAST('-123' AS UNSIGNED);")
+
+box.space.T1:drop()
--
2.15.1
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-06-07 15:37 ` [tarantool-patches] [PATCH 6/6] sql: allow to specify UNSIGNED column type Nikita Pettik
@ 2019-07-01 21:53 ` Vladislav Shpilevoy
2019-07-05 16:36 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-01 21:53 UTC (permalink / raw)
To: Nikita Pettik, tarantool-patches
There still are places, which access Mem.u.i/u without
checking flags properly. Please, for each place either
explain why it is correct, or fix it and add a test.
--------------------------------------------------------------
vdbe.c:2659:
if ((pDest->flags & (MEM_Int | MEM_UInt)) != 0) {
if (field_type == FIELD_TYPE_NUMBER)
sqlVdbeMemSetDouble(pDest, pDest->u.i);
}
You can't access pDest->u.i - it can be a big unsigned.
--------------------------------------------------------------
vdbe.c:2955:
pIn1 = &aMem[pOp->p1];
uint32_t space_id = pIn1->u.i;
Here you touch u.i, but it is always
unsigned. You should use u.u, I think.
--------------------------------------------------------------
vdbeaux.c:2428:
if (flags & MEM_Int) {
...
i64 i = pMem->u.i;
u64 u;
if (i < 0) {
u = ~i;
} else {
u = i;
}
Why do you check 'i < 0', if you already
see its flag is 'Int'. It should be < 0 by
definition.
--------------------------------------------------------------
vdbeaux.c:2589:
u64 v;
u32 i;
if (serial_type == 7) {
assert(sizeof(v) == sizeof(pMem->u.r));
memcpy(&v, &pMem->u.r, sizeof(v));
swapMixedEndianFloat(v);
} else {
v = pMem->u.i;
}
Why are you sure it is safe to store u.i into
uint64_t variable?
--------------------------------------------------------------
vdbeaux.c:2644:
u64 x = FOUR_BYTE_UINT(buf);
u32 y = FOUR_BYTE_UINT(buf + 4);
x = (x << 32) + y;
if (serial_type == 6) {
/* EVIDENCE-OF: R-29851-52272 Value is a big-endian 64-bit
* twos-complement integer.
*/
pMem->u.i = *(i64 *) & x;
pMem->flags = MEM_Int;
Vice versa - why is it safe to store uint64_t into u.i,
and set its flag to 'Int' (== value is negative).
Actually, all this 'serial' shit looks broken. Please,
verify that code.
--------------------------------------------------------------
vdbeaux.c:2998:
if ((f1 & MEM_UInt) != 0) {
if ((f2 & MEM_Real) != 0) {
return sqlIntFloatCompare(pMem1->u.i,
pMem1 is unsigned, according to the first check,
but you use u.i. Why?
--------------------------------------------------------------
vdbeaux.c:3184:
do_int:
if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
if (mem1.u.i < pKey2->u.i) {
rc = -1;
pKey2 is an integer, but you don't know the sign. And use u.i,
assuming it is negative, or at least small enough. Why?
Line 3207 is the same.
--------------------------------------------------------------
vdbemem.c:1431:
} else if (pVal->u.i == SMALLEST_INT64) {
pVal->u.r = -(double)SMALLEST_INT64;
MemSetTypeFlag(pVal, MEM_Real);
} else {
pVal->u.i = -pVal->u.i;
}
You compare u.i and SMALLEST_INT64, but you can't
be sure, that u.i is not a big unsigned, can you?
--------------------------------------------------------------
I will verify u.u, MEM_Int, and MEM_UInt usages, when we
finish with u.i.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-01 21:53 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-07-05 16:36 ` n.pettik
2019-07-10 22:49 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-05 16:36 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 2 Jul 2019, at 00:53, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> There still are places, which access Mem.u.i/u without
> checking flags properly. Please, for each place either
> explain why it is correct, or fix it and add a test.
>
> --------------------------------------------------------------
> vdbe.c:2659:
> if ((pDest->flags & (MEM_Int | MEM_UInt)) != 0) {
> if (field_type == FIELD_TYPE_NUMBER)
> sqlVdbeMemSetDouble(pDest, pDest->u.i);
> }
>
> You can't access pDest->u.i - it can be a big unsigned.
Test case:
create table "t"(id int primary key, "a" number);
box.space.t:insert({2, 18446744073709551615ULL})
select * from “t”;
---
- metadata:
- name: ID
type: integer
- name: a
type: number
rows:
- [2, -1]
…
Diff:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 97cc1fa8a..cabc852fa 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2616,8 +2616,12 @@ case OP_Column: {
sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
}
if ((pDest->flags & (MEM_Int | MEM_UInt)) != 0) {
- if (field_type == FIELD_TYPE_NUMBER)
- sqlVdbeMemSetDouble(pDest, pDest->u.i);
+ if (field_type == FIELD_TYPE_NUMBER) {
+ if ((pDest->flags & MEM_Int) != 0)
+ sqlVdbeMemSetDouble(pDest, pDest->u.i);
+ else
+ sqlVdbeMemSetDouble(pDest, pDest->u.u);
+ }
}
op_column_out:
REGISTER_TRACE(p, pOp->p3, pDest);
diff --git a/test/sql/integer-overflow.result b/test/sql/integer-overflow.result
index 7ca2f336d..528e7a6d6 100644
--- a/test/sql/integer-overflow.result
+++ b/test/sql/integer-overflow.result
@@ -140,3 +140,32 @@ box.execute('SELECT * FROM t;')
box.space.T:drop()
---
...
+-- Make sure that integers stored in NUMBER field are converted
+-- to floating point properly.
+--
+box.execute("CREATE TABLE t(id INT PRIMARY KEY, a FLOAT);")
+---
+- row_count: 1
+...
+box.space.T:insert({1, 18446744073709551615ULL})
+---
+- [1, 18446744073709551615]
+...
+box.space.T:insert({2, -1})
+---
+- [2, -1]
+...
+box.execute("SELECT * FROM t;")
+---
+- metadata:
+ - name: ID
+ type: integer
+ - name: A
+ type: number
+ rows:
+ - [1, 1.844674407371e+19]
+ - [2, -1]
+...
+box.space.T:drop()
+---
+...
diff --git a/test/sql/integer-overflow.test.lua b/test/sql/integer-overflow.test.lua
index 246465da0..d5635d5af 100644
--- a/test/sql/integer-overflow.test.lua
+++ b/test/sql/integer-overflow.test.lua
@@ -42,3 +42,12 @@ box.space.T:insert({9223372036854775809})
box.space.T:insert({18446744073709551615ULL})
box.execute('SELECT * FROM t;')
box.space.T:drop()
+
+-- Make sure that integers stored in NUMBER field are converted
+-- to floating point properly.
+--
+box.execute("CREATE TABLE t(id INT PRIMARY KEY, a FLOAT);")
+box.space.T:insert({1, 18446744073709551615ULL})
+box.space.T:insert({2, -1})
+box.execute("SELECT * FROM t;")
+box.space.T:drop()
> --------------------------------------------------------------
> vdbe.c:2955:
> pIn1 = &aMem[pOp->p1];
> uint32_t space_id = pIn1->u.i;
>
> Here you touch u.i, but it is always
> unsigned. You should use u.u, I think.
Fair (but space id always fits into 32 bits and always non-negative,
so technically it is correct usage):
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index cabc852fa..0626983f0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2915,7 +2915,8 @@ case OP_Savepoint: {
case OP_CheckViewReferences: {
assert(pOp->p1 > 0);
pIn1 = &aMem[pOp->p1];
- uint32_t space_id = pIn1->u.i;
+ uint64_t space_id = pIn1->u.u;
+ assert(space_id <= INT32_MAX);
struct space *space = space_by_id(space_id);
assert(space != NULL);
if (space->def->view_ref_count > 0) {
> --------------------------------------------------------------
> vdbeaux.c:2428:
> if (flags & MEM_Int) {
>
> ...
>
> i64 i = pMem->u.i;
> u64 u;
> if (i < 0) {
> u = ~i;
> } else {
> u = i;
> }
>
> Why do you check 'i < 0', if you already
> see its flag is 'Int'. It should be < 0 by
> definition.
I’m not sure this code is still in usage. It looks like
dead code. Will remove it in a follow-up patch.
> --------------------------------------------------------------
> vdbeaux.c:2589:
> u64 v;
> u32 i;
> if (serial_type == 7) {
> assert(sizeof(v) == sizeof(pMem->u.r));
> memcpy(&v, &pMem->u.r, sizeof(v));
> swapMixedEndianFloat(v);
> } else {
> v = pMem->u.i;
> }
>
> Why are you sure it is safe to store u.i into
> uint64_t variable?
Same.
> --------------------------------------------------------------
> vdbeaux.c:2644:
> u64 x = FOUR_BYTE_UINT(buf);
> u32 y = FOUR_BYTE_UINT(buf + 4);
> x = (x << 32) + y;
> if (serial_type == 6) {
> /* EVIDENCE-OF: R-29851-52272 Value is a big-endian 64-bit
> * twos-complement integer.
> */
> pMem->u.i = *(i64 *) & x;
> pMem->flags = MEM_Int;
>
> Vice versa - why is it safe to store uint64_t into u.i,
> and set its flag to 'Int' (== value is negative).
>
> Actually, all this 'serial' shit looks broken. Please,
> verify that code.
Same.
> --------------------------------------------------------------
> vdbeaux.c:2998:
> if ((f1 & MEM_UInt) != 0) {
> if ((f2 & MEM_Real) != 0) {
> return sqlIntFloatCompare(pMem1->u.i,
>
> pMem1 is unsigned, according to the first check,
> but you use u.i. Why?
Thx, I’ve fixed series of similar places and extended sql/types.test.lua:
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 325c54c18..b6b5cd0bf 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2887,43 +2887,50 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
return n1 - n2;
}
-/*
- * Do a comparison between a 64-bit signed integer and a 64-bit floating-point
- * number. Return negative, zero, or positive if the first (i64) is less than,
- * equal to, or greater than the second (double).
+/**
+ * Do a comparison between a 64-bit unsigned/signed integer and a
+ * 64-bit floating-point number. Return negative, zero, or
+ * positive if the first (integer) is less than, equal to, or
+ * greater than the second (double).
*/
static int
-sqlIntFloatCompare(i64 i, double r)
+compare_uint_float(uint64_t u, double r)
{
- if (sizeof(LONGDOUBLE_TYPE) > 8) {
- LONGDOUBLE_TYPE x = (LONGDOUBLE_TYPE) i;
- if (x < r)
- return -1;
- if (x > r)
- return +1;
- return 0;
- } else {
- i64 y;
- double s;
- if (r < -9223372036854775808.0)
- return +1;
- if (r > 9223372036854775807.0)
- return -1;
- y = (i64) r;
- if (i < y)
- return -1;
- if (i > y) {
- if (y == SMALLEST_INT64 && r > 0.0)
- return -1;
- return +1;
- }
- s = (double)i;
- if (s < r)
- return -1;
- if (s > r)
- return +1;
- return 0;
- }
+ if (r > (double) UINT64_MAX)
+ return -1;
+ if (r < 0.0)
+ return +1;
+ uint64_t y = (uint64_t) r;
+ if (u < y)
+ return -1;
+ if (u > y)
+ return +1;
+ double s = (double) u;
+ if (s < r)
+ return -1;
+ if (s > r)
+ return +1;
+ return 0;
+}
+
+static int
+compare_int_float(int64_t i, double r)
+{
+ if (r < (double) INT64_MIN)
+ return +1;
+ if (r >= 0.0)
+ return -1;
+ int64_t y = (int64_t) r;
+ if (i < y)
+ return -1;
+ if (i > y)
+ return +1;
+ double s = (double) i;
+ if (s < r)
+ return -1;
+ if (s > r)
+ return +1;
+ return 0;
}
/*
@@ -2989,15 +2996,15 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
}
if ((f1 & MEM_Int) != 0) {
if ((f2 & MEM_Real) != 0) {
- return sqlIntFloatCompare(pMem1->u.i,
- pMem2->u.r);
+ return compare_int_float(pMem1->u.i,
+ pMem2->u.r);
} else {
return -1;
}
}
if ((f1 & MEM_UInt) != 0) {
if ((f2 & MEM_Real) != 0) {
- return sqlIntFloatCompare(pMem1->u.i,
+ return compare_uint_float(pMem1->u.u,
pMem2->u.r);
} else if ((f2 & MEM_Int) != 0) {
return +1;
@@ -3007,10 +3014,10 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
}
if ((f1 & MEM_Real) != 0) {
if ((f2 & MEM_Int) != 0) {
- return -sqlIntFloatCompare(pMem2->u.i,
- pMem1->u.r);
+ return -compare_int_float(pMem2->u.i,
+ pMem1->u.r);
} else if ((f2 & MEM_UInt) != 0) {
- return -sqlIntFloatCompare(pMem2->u.u,
+ return -compare_uint_float(pMem2->u.u,
pMem1->u.r);
} else {
return -1;
@@ -3171,26 +3178,33 @@ sqlVdbeCompareMsgpack(const char **key1,
break;
}
case MP_UINT:{
- uint64_t v = mp_decode_uint(&aKey1);
- if (v > INT64_MAX) {
- mem1.u.r = (double)v;
- goto do_float;
+ mem1.u.u = mp_decode_uint(&aKey1);
+ if ((pKey2->flags & MEM_Int) != 0) {
+ rc = +1;
+ } else if ((pKey2->flags & MEM_UInt) != 0) {
+ if (mem1.u.u < pKey2->u.u)
+ rc = -1;
+ else if (mem1.u.u > pKey2->u.u)
+ rc = +1;
+ } else if ((pKey2->flags & MEM_Real) != 0) {
+ rc = compare_uint_float(mem1.u.u, pKey2->u.r);
+ } else {
+ rc = (pKey2->flags & MEM_Null) ? +1 : -1;
}
- mem1.u.u = v;
- goto do_int;
+ break;
}
case MP_INT:{
mem1.u.i = mp_decode_int(&aKey1);
- do_int:
- if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
+ if ((pKey2->flags & MEM_UInt) != 0) {
+ rc = -1;
+ } else if ((pKey2->flags & MEM_Int) != 0) {
if (mem1.u.i < pKey2->u.i) {
rc = -1;
} else if (mem1.u.i > pKey2->u.i) {
rc = +1;
}
} else if (pKey2->flags & MEM_Real) {
- rc = sqlIntFloatCompare(mem1.u.i,
- pKey2->u.r);
+ rc = compare_int_float(mem1.u.i, pKey2->u.r);
} else {
rc = (pKey2->flags & MEM_Null) ? +1 : -1;
}
@@ -3203,9 +3217,10 @@ sqlVdbeCompareMsgpack(const char **key1,
case MP_DOUBLE:{
mem1.u.r = mp_decode_double(&aKey1);
do_float:
- if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
- rc = -sqlIntFloatCompare(pKey2->u.i,
- mem1.u.r);
+ if ((pKey2->flags & MEM_Int) != 0) {
+ rc = -compare_int_float(pKey2->u.i, mem1.u.r);
+ } else if (pKey2->flags & MEM_UInt) {
+ rc = -compare_uint_float(pKey2->u.u, mem1.u.r);
} else if (pKey2->flags & MEM_Real) {
if (mem1.u.r < pKey2->u.r) {
rc = -1;
> --------------------------------------------------------------
> vdbeaux.c:3184:
> do_int:
> if ((pKey2->flags & (MEM_Int | MEM_UInt)) != 0) {
> if (mem1.u.i < pKey2->u.i) {
> rc = -1;
>
> pKey2 is an integer, but you don't know the sign. And use u.i,
> assuming it is negative, or at least small enough. Why?
>
> Line 3207 is the same.
Fixed, see diff above.
> --------------------------------------------------------------
> vdbemem.c:1431:
> } else if (pVal->u.i == SMALLEST_INT64) {
> pVal->u.r = -(double)SMALLEST_INT64;
> MemSetTypeFlag(pVal, MEM_Real);
> } else {
> pVal->u.i = -pVal->u.i;
> }
>
> You compare u.i and SMALLEST_INT64, but you can't
> be sure, that u.i is not a big unsigned, can you?
Fixed:
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index f8673912e..64acb5d41 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -1428,11 +1428,15 @@ valueFromExpr(sql * db, /* The database connection */
return rc;
if (pVal->flags & MEM_Real) {
pVal->u.r = -pVal->u.r;
- } else if (pVal->u.i == SMALLEST_INT64) {
- pVal->u.r = -(double)SMALLEST_INT64;
- MemSetTypeFlag(pVal, MEM_Real);
- } else {
- pVal->u.i = -pVal->u.i;
+ } else if ((pVal->flags & MEM_Int) != 0) {
+ mem_set_u64(pVal, (uint64_t)(-pVal->u.i));
+ } else if ((pVal->flags & MEM_UInt) != 0) {
+ if (pVal->u.u > (uint64_t) INT64_MAX + 1) {
+ pVal->u.r = -(double) pVal->u.u;
+ MemSetTypeFlag(pVal, MEM_Real);
+ } else {
+ mem_set_i64(pVal, (int64_t)(-pVal->u.u));
+ }
}
sql_value_apply_type(pVal, type);
}
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-05 16:36 ` n.pettik
@ 2019-07-10 22:49 ` Vladislav Shpilevoy
2019-07-11 21:25 ` Vladislav Shpilevoy
` (2 more replies)
0 siblings, 3 replies; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-10 22:49 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
>> --------------------------------------------------------------
>> vdbeaux.c:2998:
>> if ((f1 & MEM_UInt) != 0) {
>> if ((f2 & MEM_Real) != 0) {
>> return sqlIntFloatCompare(pMem1->u.i,
>>
>> pMem1 is unsigned, according to the first check,
>> but you use u.i. Why?
>
> Thx, I’ve fixed series of similar places and extended sql/types.test.lua:
>
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 325c54c18..b6b5cd0bf 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -2887,43 +2887,50 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
> return n1 - n2;
> }
>
> -/*
> - * Do a comparison between a 64-bit signed integer and a 64-bit floating-point
> - * number. Return negative, zero, or positive if the first (i64) is less than,
> - * equal to, or greater than the second (double).
> +/**
> + * Do a comparison between a 64-bit unsigned/signed integer and a
> + * 64-bit floating-point number. Return negative, zero, or
> + * positive if the first (integer) is less than, equal to, or
> + * greater than the second (double).
> */
> static int
> -sqlIntFloatCompare(i64 i, double r)
> +compare_uint_float(uint64_t u, double r)
Unfortunately, it is not as simple as you implemented it.
See mp_compare_double_uint64 in tuple_compare.cc for details.
In short, your function works wrong when numbers are near
uint maximum. Perhaps, it is worth moving this comparator
from tuple_compare.cc to a header. Like trivia/util.h.
> {
> - if (sizeof(LONGDOUBLE_TYPE) > 8) {
> - LONGDOUBLE_TYPE x = (LONGDOUBLE_TYPE) i;
> - if (x < r)
> - return -1;
> - if (x > r)
> - return +1;
> - return 0;
> - } else {
> - i64 y;
> - double s;
> - if (r < -9223372036854775808.0)
> - return +1;
> - if (r > 9223372036854775807.0)
> - return -1;
> - y = (i64) r;
> - if (i < y)
> - return -1;
> - if (i > y) {
> - if (y == SMALLEST_INT64 && r > 0.0)
> - return -1;
> - return +1;
> - }
> - s = (double)i;
> - if (s < r)
> - return -1;
> - if (s > r)
> - return +1;
> - return 0;
> - }
> + if (r > (double) UINT64_MAX)
> + return -1;
> + if (r < 0.0)
> + return +1;
> + uint64_t y = (uint64_t) r;
> + if (u < y)
> + return -1;
> + if (u > y)
> + return +1;
> + double s = (double) u;
> + if (s < r)
> + return -1;
> + if (s > r)
> + return +1;
> + return 0;
> +}
> +
> +static int
> +compare_int_float(int64_t i, double r)
It has the same problems as the previous function,
but can be fixed by calling compare_uint_float()
with a modulo of 'int64_t i' and reversed result,
if the value was negative. This is what mp_compare_double_any_int
does.
> +{
> + if (r < (double) INT64_MIN)
> + return +1;
> + if (r >= 0.0)
> + return -1;
> + int64_t y = (int64_t) r;
> + if (i < y)
> + return -1;
> + if (i > y)
> + return +1;
> + double s = (double) i;
> + if (s < r)
> + return -1;
> + if (s > r)
> + return +1;
> + return 0;
> }
>> --------------------------------------------------------------
>> vdbemem.c:1431:
>> } else if (pVal->u.i == SMALLEST_INT64) {
>> pVal->u.r = -(double)SMALLEST_INT64;
>> MemSetTypeFlag(pVal, MEM_Real);
>> } else {
>> pVal->u.i = -pVal->u.i;
>> }
>>
>> You compare u.i and SMALLEST_INT64, but you can't
>> be sure, that u.i is not a big unsigned, can you?
>
> Fixed:
>
> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
> index f8673912e..64acb5d41 100644
> --- a/src/box/sql/vdbemem.c
> +++ b/src/box/sql/vdbemem.c
> @@ -1428,11 +1428,15 @@ valueFromExpr(sql * db, /* The database connection */
> return rc;
> if (pVal->flags & MEM_Real) {
> pVal->u.r = -pVal->u.r;
> - } else if (pVal->u.i == SMALLEST_INT64) {
> - pVal->u.r = -(double)SMALLEST_INT64;
> - MemSetTypeFlag(pVal, MEM_Real);
> - } else {
> - pVal->u.i = -pVal->u.i;
> + } else if ((pVal->flags & MEM_Int) != 0) {
> + mem_set_u64(pVal, (uint64_t)(-pVal->u.i));
> + } else if ((pVal->flags & MEM_UInt) != 0) {
> + if (pVal->u.u > (uint64_t) INT64_MAX + 1) {
> + pVal->u.r = -(double) pVal->u.u;
> + MemSetTypeFlag(pVal, MEM_Real);
Won't we have a problem here, that an expression '--value' won't be
equal to 'value' in case the value is bigger than INT64_MAX + 1?
> + } else {
> + mem_set_i64(pVal, (int64_t)(-pVal->u.u));
> + }
> }
> sql_value_apply_type(pVal, type);
> }
>
Note for Kirill: this is not a final review. I will get back a bit later to
check other places where Mem.u.u, Mem.u.i, MEM_UInt, MEM_Int are used.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-10 22:49 ` Vladislav Shpilevoy
@ 2019-07-11 21:25 ` Vladislav Shpilevoy
2019-07-17 0:53 ` n.pettik
2019-07-17 0:54 ` n.pettik
2019-08-06 19:36 ` n.pettik
2 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-11 21:25 UTC (permalink / raw)
To: n.pettik, tarantool-patches
As I promised, below are other comments regarding Mem.u.i/u,
MEM_Int/UInt. Perhaps, they are final.
-------------------------
vdbemem.c:661
> if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
> bool is_neg;
> if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
> pMem->n) == 0) {
> MemSetTypeFlag(pMem, MEM_Real);
> pMem->u.r = pMem->u.i;
> return 0;
> }
> return ! sqlAtoF(pMem->z, &pMem->u.r, pMem->n);
> }
Here you use assign 'pMem.u.i' to 'pMem.u.r' without checking
its sign. In theory something like that should work wrong now:
box.execute("SELECT CAST('9223372036854775837' AS REAL)")
But you need to somehow write that number as blob.
-------------------------
vdbetrace.c:90
Function sqlVdbeExpandSql stringifies integers, but
misses a version for big unsigned.
-------------------------
vdbe.c:307
> case FIELD_TYPE_INTEGER:
> case FIELD_TYPE_UNSIGNED:
> if ((record->flags & MEM_Int) == MEM_Int)
> return 0;
> if ((record->flags & MEM_UInt) == MEM_UInt)
> return 0;
> if ((record->flags & MEM_Real) == MEM_Real) {
> int64_t i = (int64_t) record->u.r;
> if (i == record->u.r)
> mem_set_int(record, record->u.r,
> record->u.r <= -1);
> return 0;
> }
It is a part of function mem_apply_type. When target type is
UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
it is possible to pass here a negative value, and CAST UNSIGNED
would do nothing.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-11 21:25 ` Vladislav Shpilevoy
@ 2019-07-17 0:53 ` n.pettik
2019-07-18 20:18 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-17 0:53 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> -------------------------
> vdbemem.c:661
>
>> if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
>> bool is_neg;
>> if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
>> pMem->n) == 0) {
>> MemSetTypeFlag(pMem, MEM_Real);
>> pMem->u.r = pMem->u.i;
>> return 0;
>> }
>> return ! sqlAtoF(pMem->z, &pMem->u.r, pMem->n);
>> }
>
> Here you use assign 'pMem.u.i' to 'pMem.u.r' without checking
> its sign. In theory something like that should work wrong now:
>
> box.execute("SELECT CAST('9223372036854775837' AS REAL)")
>
> But you need to somehow write that number as blob.
Thanks, you are absolutely right:
tarantool> SELECT CAST(x'3138343436373434303733373039353531363135' AS real)
---
- metadata:
- name: CAST(x'3138343436373434303733373039353531363135' AS real)
type: number
rows:
- [-1]
...
Fix:
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 64acb5d41..087cd20f1 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -658,7 +658,10 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg,
pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
- pMem->u.r = pMem->u.i;
+ if (is_neg)
+ pMem->u.r = pMem->u.i;
+ else
+ pMem->u.r = pMem->u.u;
return 0;
}
return ! sqlAtoF(pMem->z, &pMem->u.r, pMem->n);
diff --git a/test/sql-tap/cast.test.lua b/test/sql-tap/cast.test.lua
index 315f49d08..3027657a5 100755
--- a/test/sql-tap/cast.test.lua
+++ b/test/sql-tap/cast.test.lua
@@ -1,6 +1,6 @@
#!/usr/bin/env tarantool
test = require("sqltester")
-test:plan(82)
+test:plan(84)
--!./tcltestrunner.lua
-- 2005 June 25
@@ -793,6 +793,18 @@ if true then --test:execsql("PRAGMA encoding")[1][1]=="UTF-8" then
})
end
+test:do_execsql_test(
+ "case-3.25",
+ [[
+ SELECT CAST(x'3138343436373434303733373039353531363135' AS REAL);
+ ]], { 1.844674407371e+19 } )
+
+test:do_execsql_test(
+ "case-3.26",
+ [[
+ SELECT CAST(x'3138343436373434303733373039353531363135' AS INT);
+ ]], { 18446744073709551615LL } )
+
test:do_execsql_test(
"case-3.31",
[[
> -------------------------
> vdbetrace.c:90
>
> Function sqlVdbeExpandSql stringifies integers, but
> misses a version for big unsigned.
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index f78d5b093..2ee9f668c 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -151,6 +151,8 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
sqlStrAccumAppend(&out, "NULL", 4);
} else if (pVar->flags & MEM_Int) {
sqlXPrintf(&out, "%lld", pVar->u.i);
+ } else if (pVar->flags & MEM_UInt) {
+ sqlXPrintf(&out, "%llu", pVar->u.u);
} else if (pVar->flags & MEM_Real) {
sqlXPrintf(&out, "%!.15g", pVar->u.r);
} else if (pVar->flags & MEM_Str) {
> -------------------------
> vdbe.c:307
>
>> case FIELD_TYPE_INTEGER:
>> case FIELD_TYPE_UNSIGNED:
>> if ((record->flags & MEM_Int) == MEM_Int)
>> return 0;
>> if ((record->flags & MEM_UInt) == MEM_UInt)
>> return 0;
>> if ((record->flags & MEM_Real) == MEM_Real) {
>> int64_t i = (int64_t) record->u.r;
>> if (i == record->u.r)
>> mem_set_int(record, record->u.r,
>> record->u.r <= -1);
>> return 0;
>> }
>
> It is a part of function mem_apply_type. When target type is
> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
> it is possible to pass here a negative value, and CAST UNSIGNED
> would do nothing.
Basically, this function implements sort of implicit cast
which takes place before comparison/assignment.
For comparisons it makes no sense - we can compare
integer with unsigned value - the latter is always greater.
For assignment it is also meaningless: if we attempt
at inserting negative values to unsigned field appropriate
error will be raised anyway. If you can come up with
specific example, let’s discuss it.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-17 0:53 ` n.pettik
@ 2019-07-18 20:18 ` Vladislav Shpilevoy
2019-07-18 20:56 ` n.pettik
0 siblings, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-18 20:18 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi!
Thanks for the fixes!
>> -------------------------
>> vdbe.c:307
>>
>>> case FIELD_TYPE_INTEGER:
>>> case FIELD_TYPE_UNSIGNED:
>>> if ((record->flags & MEM_Int) == MEM_Int)
>>> return 0;
>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>> return 0;
>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>> int64_t i = (int64_t) record->u.r;
>>> if (i == record->u.r)
>>> mem_set_int(record, record->u.r,
>>> record->u.r <= -1);
>>> return 0;
>>> }
>>
>> It is a part of function mem_apply_type. When target type is
>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>> it is possible to pass here a negative value, and CAST UNSIGNED
>> would do nothing.
>
> Basically, this function implements sort of implicit cast
> which takes place before comparison/assignment.
> For comparisons it makes no sense - we can compare
> integer with unsigned value - the latter is always greater.
> For assignment it is also meaningless: if we attempt
> at inserting negative values to unsigned field appropriate
> error will be raised anyway. If you can come up with
> specific example, let’s discuss it.
>
I can't provide a test. But the function is named mem_apply_type,
and it doesn't apply type, when it is unsigned, and a value is
negative. Doesn't it look wrong to you?
If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
to int should raise an error here. Otherwise this function can't
be named 'apply_type' because it ignores negative -> unsigned case.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-18 20:18 ` Vladislav Shpilevoy
@ 2019-07-18 20:56 ` n.pettik
2019-07-18 21:08 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-18 20:56 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
[-- Attachment #1: Type: text/plain, Size: 2215 bytes --]
> On 18 Jul 2019, at 23:18, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Hi!
>
> Thanks for the fixes!
>
>>> -------------------------
>>> vdbe.c:307
>>>
>>>> case FIELD_TYPE_INTEGER:
>>>> case FIELD_TYPE_UNSIGNED:
>>>> if ((record->flags & MEM_Int) == MEM_Int)
>>>> return 0;
>>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>>> return 0;
>>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>>> int64_t i = (int64_t) record->u.r;
>>>> if (i == record->u.r)
>>>> mem_set_int(record, record->u.r,
>>>> record->u.r <= -1);
>>>> return 0;
>>>> }
>>>
>>> It is a part of function mem_apply_type. When target type is
>>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>>> it is possible to pass here a negative value, and CAST UNSIGNED
>>> would do nothing.
>>
>> Basically, this function implements sort of implicit cast
>> which takes place before comparison/assignment.
>> For comparisons it makes no sense - we can compare
>> integer with unsigned value - the latter is always greater.
>> For assignment it is also meaningless: if we attempt
>> at inserting negative values to unsigned field appropriate
>> error will be raised anyway. If you can come up with
>> specific example, let’s discuss it.
>>
>
> I can't provide a test. But the function is named mem_apply_type,
> and it doesn't apply type, when it is unsigned, and a value is
> negative. Doesn't it look wrong to you?
>
> If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
> instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
> to int should raise an error here. Otherwise this function can't
> be named 'apply_type' because it ignores negative -> unsigned case.
Okay, let’s rename it. I can suggest these options:
mem_cast_implicit()
mem_cast_implicit_to_type()
mem_implicit_cast_to_type()
mem_convert_implicit()
mem_convert_to_type()
mem_type_coerce_implicit()
mem_type_implicit_coercion()
mem_type_coercion_implicit()
mem_implicit_type_juggling()
mem_implicit_juggle_to_type()
mem_do_implicit_conversion()
mem_do_implicit_coercion()
Or any other combination :)
[-- Attachment #2: Type: text/html, Size: 13183 bytes --]
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-18 20:56 ` n.pettik
@ 2019-07-18 21:08 ` Vladislav Shpilevoy
2019-07-18 21:13 ` Vladislav Shpilevoy
2019-07-22 10:20 ` n.pettik
0 siblings, 2 replies; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-18 21:08 UTC (permalink / raw)
To: n.pettik, tarantool-patches
On 18/07/2019 22:56, n.pettik wrote:
>
>
>> On 18 Jul 2019, at 23:18, Vladislav Shpilevoy <v.shpilevoy@tarantool.org <mailto:v.shpilevoy@tarantool.org>> wrote:
>>
>> Hi!
>>
>> Thanks for the fixes!
>>
>>>> -------------------------
>>>> vdbe.c:307
>>>>
>>>>> case FIELD_TYPE_INTEGER:
>>>>> case FIELD_TYPE_UNSIGNED:
>>>>> if ((record->flags & MEM_Int) == MEM_Int)
>>>>> return 0;
>>>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>>>> return 0;
>>>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>>>> int64_t i = (int64_t) record->u.r;
>>>>> if (i == record->u.r)
>>>>> mem_set_int(record, record->u.r,
>>>>> record->u.r <= -1);
>>>>> return 0;
>>>>> }
>>>>
>>>> It is a part of function mem_apply_type. When target type is
>>>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>>>> it is possible to pass here a negative value, and CAST UNSIGNED
>>>> would do nothing.
>>>
>>> Basically, this function implements sort of implicit cast
>>> which takes place before comparison/assignment.
>>> For comparisons it makes no sense - we can compare
>>> integer with unsigned value - the latter is always greater.
>>> For assignment it is also meaningless: if we attempt
>>> at inserting negative values to unsigned field appropriate
>>> error will be raised anyway. If you can come up with
>>> specific example, let’s discuss it.
>>>
>>
>> I can't provide a test. But the function is named mem_apply_type,
>> and it doesn't apply type, when it is unsigned, and a value is
>> negative. Doesn't it look wrong to you?
>>
>> If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
>> instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
>> to int should raise an error here. Otherwise this function can't
>> be named 'apply_type' because it ignores negative -> unsigned case.
>
> Okay, let’s rename it. I can suggest these options:
>
> mem_cast_implicit()
> mem_cast_implicit_to_type()
> mem_implicit_cast_to_type()
> mem_convert_implicit()
> mem_convert_to_type()
> mem_type_coerce_implicit()
> mem_type_implicit_coercion()
> mem_type_coercion_implicit()
> mem_implicit_type_juggling()
> mem_implicit_juggle_to_type()
> mem_do_implicit_conversion()
> mem_do_implicit_coercion()
>
> Or any other combination :)
>
But it is not implicit. It just does not work, when a value is negative,
and type is unsigned.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-18 21:08 ` Vladislav Shpilevoy
@ 2019-07-18 21:13 ` Vladislav Shpilevoy
2019-07-22 10:20 ` n.pettik
2019-07-22 10:20 ` n.pettik
1 sibling, 1 reply; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-18 21:13 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Why are you keep trying not to fix this obvious bug?
If that path is unreachable, then either delete it,
or add an assertion. Or fix this place.
On 18/07/2019 23:08, Vladislav Shpilevoy wrote:
>
>
> On 18/07/2019 22:56, n.pettik wrote:
>>
>>
>>> On 18 Jul 2019, at 23:18, Vladislav Shpilevoy <v.shpilevoy@tarantool.org <mailto:v.shpilevoy@tarantool.org>> wrote:
>>>
>>> Hi!
>>>
>>> Thanks for the fixes!
>>>
>>>>> -------------------------
>>>>> vdbe.c:307
>>>>>
>>>>>> case FIELD_TYPE_INTEGER:
>>>>>> case FIELD_TYPE_UNSIGNED:
>>>>>> if ((record->flags & MEM_Int) == MEM_Int)
>>>>>> return 0;
>>>>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>>>>> return 0;
>>>>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>>>>> int64_t i = (int64_t) record->u.r;
>>>>>> if (i == record->u.r)
>>>>>> mem_set_int(record, record->u.r,
>>>>>> record->u.r <= -1);
>>>>>> return 0;
>>>>>> }
>>>>>
>>>>> It is a part of function mem_apply_type. When target type is
>>>>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>>>>> it is possible to pass here a negative value, and CAST UNSIGNED
>>>>> would do nothing.
>>>>
>>>> Basically, this function implements sort of implicit cast
>>>> which takes place before comparison/assignment.
>>>> For comparisons it makes no sense - we can compare
>>>> integer with unsigned value - the latter is always greater.
>>>> For assignment it is also meaningless: if we attempt
>>>> at inserting negative values to unsigned field appropriate
>>>> error will be raised anyway. If you can come up with
>>>> specific example, let’s discuss it.
>>>>
>>>
>>> I can't provide a test. But the function is named mem_apply_type,
>>> and it doesn't apply type, when it is unsigned, and a value is
>>> negative. Doesn't it look wrong to you?
>>>
>>> If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
>>> instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
>>> to int should raise an error here. Otherwise this function can't
>>> be named 'apply_type' because it ignores negative -> unsigned case.
>>
>> Okay, let’s rename it. I can suggest these options:
>>
>> mem_cast_implicit()
>> mem_cast_implicit_to_type()
>> mem_implicit_cast_to_type()
>> mem_convert_implicit()
>> mem_convert_to_type()
>> mem_type_coerce_implicit()
>> mem_type_implicit_coercion()
>> mem_type_coercion_implicit()
>> mem_implicit_type_juggling()
>> mem_implicit_juggle_to_type()
>> mem_do_implicit_conversion()
>> mem_do_implicit_coercion()
>>
>> Or any other combination :)
>>
>
> But it is not implicit. It just does not work, when a value is negative,
> and type is unsigned.
>
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-18 21:13 ` Vladislav Shpilevoy
@ 2019-07-22 10:20 ` n.pettik
2019-07-22 19:17 ` Vladislav Shpilevoy
0 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-22 10:20 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 19 Jul 2019, at 00:13, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Why are you keep trying not to fix this obvious bug?
> If that path is unreachable, then either delete it,
> or add an assertion. Or fix this place.
Sorry, message threads now seem to be entangled a bit,
so I guess you are talking about valueFromExpr() func.
I’ve removed the single reachable to this function path,
but left function itself (valueFromExpr()) since it seem
to be quite meaningful and can be applied somewhere
later. In the next patch I’ve added assertion fault to the
place of conversion of double negative values (-(-5)):
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index cf738dcc7..f296ed329 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -1424,8 +1424,15 @@ valueFromExpr(sql * db, /* The database connection */
mem_set_u64(pVal, (uint64_t)(-pVal->u.i));
} else if ((pVal->flags & MEM_UInt) != 0) {
if (pVal->u.u > (uint64_t) INT64_MAX + 1) {
- pVal->u.r = -(double) pVal->u.u;
- MemSetTypeFlag(pVal, MEM_Real);
+ /*
+ * FIXME: while resurrecting this func
+ * come up with way of dealing with
+ * this situation. In previous
+ * implementation it was conversion to
+ * double, but in this case
+ * -(UINT) x -> (DOUBLE) y and -y != x.
+ */
+ unreachable();
} else {
mem_set_i64(pVal, (int64_t)(-pVal->u.u));
}
Author: Nikita Pettik <korablev@tarantool.org>
Date: Mon Jul 22 02:54:25 2019 +0300
sql: remove sqlColumnDefault() function
This routine implements obsolete mechanism to retrieve default column
value. Since it is not used anymore, let's remove it. Note that related
functions valieFromExpr()/valueFromFunction() are not erased since they
look pretty useful and may be involved later.
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index 9af23e985..4623589f0 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -1025,9 +1025,8 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
*/
for (int i = 0; i < part_count; i++) {
uint32_t tabl_col = idx->def->key_def->parts[i].fieldno;
- sqlExprCodeGetColumnOfTable(v, space->def,
- tab_cursor, tabl_col,
- col_reg + i);
+ sqlVdbeAddOp3(v, OP_Column, tab_cursor, tabl_col,
+ col_reg + i);
}
sqlVdbeAddOp3(v, OP_MakeRecord, col_reg, part_count,
sample_reg);
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index ba0bc2c52..454899592 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -294,11 +294,8 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
if (!is_view) {
struct key_part_def *part = pk_info->parts;
for (int i = 0; i < pk_len; i++, part++) {
- struct space_def *def = space->def;
- sqlExprCodeGetColumnOfTable(v, def,
- tab_cursor,
- part->fieldno,
- reg_pk + i);
+ sqlVdbeAddOp3(v, OP_Column, tab_cursor,
+ part->fieldno, reg_pk + i);
}
} else {
for (int i = 0; i < pk_len; i++) {
@@ -482,10 +479,8 @@ sql_generate_row_delete(struct Parse *parse, struct space *space,
sqlVdbeAddOp2(v, OP_Copy, reg_pk, first_old_reg);
for (int i = 0; i < (int)space->def->field_count; i++) {
if (column_mask_fieldno_is_set(mask, i)) {
- sqlExprCodeGetColumnOfTable(v, space->def,
- cursor, i,
- first_old_reg +
- i + 1);
+ sqlVdbeAddOp3(v, OP_Column, cursor, i,
+ first_old_reg + i + 1);
}
}
@@ -579,8 +574,7 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
continue;
}
uint32_t tabl_col = index->def->key_def->parts[j].fieldno;
- sqlExprCodeGetColumnOfTable(v, space->def, cursor, tabl_col,
- reg_base + j);
+ sqlVdbeAddOp3(v, OP_Column, cursor, tabl_col, reg_base + j);
/*
* If the column type is NUMBER but the number
* is an integer, then it might be stored in the
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index bf347b401..f4f4383ab 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3480,19 +3480,8 @@ sqlExprCachePinRegister(Parse * pParse, int iReg)
}
}
-void
-sqlExprCodeGetColumnOfTable(Vdbe *v, struct space_def *space_def,
- int iTabCur, int iCol, int regOut)
-{
- sqlVdbeAddOp3(v, OP_Column, iTabCur, iCol, regOut);
- if (iCol >= 0) {
- sqlColumnDefault(v, space_def, iCol, regOut);
- }
-}
-
int
-sqlExprCodeGetColumn(Parse *pParse, struct space_def *space_def,
- int iColumn, int iTable, int iReg, u8 p5)
+sqlExprCodeGetColumn(Parse *pParse, int iColumn, int iTable, int iReg, u8 p5)
{
Vdbe *v = pParse->pVdbe;
int i;
@@ -3507,7 +3496,7 @@ sqlExprCodeGetColumn(Parse *pParse, struct space_def *space_def,
}
}
assert(v != 0);
- sqlExprCodeGetColumnOfTable(v, space_def, iTable, iColumn, iReg);
+ sqlVdbeAddOp3(v, OP_Column, iTable, iColumn, iReg);
if (p5) {
sqlVdbeChangeP5(v, p5);
} else {
@@ -3517,12 +3506,10 @@ sqlExprCodeGetColumn(Parse *pParse, struct space_def *space_def,
}
void
-sqlExprCodeGetColumnToReg(Parse * pParse, struct space_def * space_def,
- int iColumn, int iTable, int iReg)
+sqlExprCodeGetColumnToReg(Parse * pParse, int iColumn, int iTable, int iReg)
{
int r1 =
- sqlExprCodeGetColumn(pParse, space_def, iColumn, iTable,
- iReg, 0);
+ sqlExprCodeGetColumn(pParse, iColumn, iTable, iReg, 0);
if (r1 != iReg)
sqlVdbeAddOp2(pParse->pVdbe, OP_SCopy, r1, iReg);
}
@@ -3721,10 +3708,8 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
iTab = pParse->iSelfTab;
}
}
- return sqlExprCodeGetColumn(pParse,
- pExpr->space_def, col,
- iTab, target,
- pExpr->op2);
+ return sqlExprCodeGetColumn(pParse, col, iTab, target,
+ pExpr->op2);
}
case TK_INTEGER:{
expr_code_int(pParse, pExpr, false, target);
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index a257e7204..68c5731bd 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -6069,8 +6069,7 @@ sqlSelect(Parse * pParse, /* The parser context */
if (pCol->iSorterColumn >= j) {
int r1 = j + regBase;
sqlExprCodeGetColumnToReg
- (pParse, pCol->space_def,
- pCol->iColumn,
+ (pParse, pCol->iColumn,
pCol->iTable, r1);
j++;
}
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 98b4c651f..5842eeec6 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -3183,7 +3183,6 @@ int sqlWhereOkOnePass(WhereInfo *, int *);
* stored in any register. But the result is guaranteed to land
* in register iReg for GetColumnToReg().
* @param pParse Parsing and code generating context.
- * @param space_def Space definition.
* @param iColumn Index of the table column.
* @param iTable The cursor pointing to the table.
* @param iReg Store results here.
@@ -3191,31 +3190,19 @@ int sqlWhereOkOnePass(WhereInfo *, int *);
* @return iReg value.
*/
int
-sqlExprCodeGetColumn(Parse *, struct space_def *, int, int, int, u8);
+sqlExprCodeGetColumn(Parse *, int, int, int, u8);
/**
* Generate code that will extract the iColumn-th column from
* table defined by space_def and store the column value in
* a register, copy the result.
* @param pParse Parsing and code generating context.
- * @param space_def Space definition.
* @param iColumn Index of the table column.
* @param iTable The cursor pointing to the table.
* @param iReg Store results here.
*/
void
-sqlExprCodeGetColumnToReg(Parse *, struct space_def *, int, int, int);
-
-/**
- * Generate code to extract the value of the iCol-th column of a table.
- * @param v The VDBE under construction.
- * @param space_def Space definition.
- * @param iTabCur The PK cursor.
- * @param iCol Index of the column to extract.
- * @param regOut Extract the value into this register.
- */
-void
-sqlExprCodeGetColumnOfTable(Vdbe *, struct space_def *, int, int, int);
+sqlExprCodeGetColumnToReg(Parse *, int, int, int);
void sqlExprCodeMove(Parse *, int, int, int);
void sqlExprCacheStore(Parse *, int, int, int);
@@ -4111,44 +4098,6 @@ int sqlResolveExprListNames(NameContext *, ExprList *);
void sqlResolveSelectNames(Parse *, Select *, NameContext *);
int sqlResolveOrderGroupBy(Parse *, Select *, ExprList *, const char *);
-/**
- * Generate code for default value.
- * The most recently coded instruction was an OP_Column to retrieve the
- * i-th column of table defined by space_def. This routine sets
- * the P4 parameter of the OP_Column to the default value, if any.
- *
- * The default value of a column is specified by a DEFAULT clause in the
- * column definition. This was either supplied by the user when the table
- * was created, or added later to the table definition by an ALTER TABLE
- * command. If the latter, then the row-records in the table btree on disk
- * may not contain a value for the column and the default value, taken
- * from the P4 parameter of the OP_Column instruction, is returned instead.
- * If the former, then all row-records are guaranteed to include a value
- * for the column and the P4 value is not required.
- *
- * Column definitions created by an ALTER TABLE command may only have
- * literal default values specified: a number, null or a string. (If a more
- * complicated default expression value was provided, it is evaluated
- * when the ALTER TABLE is executed and one of the literal values written
- * into the schema.)
- *
- * Therefore, the P4 parameter is only required if the default value for
- * the column is a literal number, string or null. The sqlValueFromExpr()
- * function is capable of transforming these types of expressions into
- * sql_value objects.
- *
- * If parameter iReg is not negative, code an OP_Realify instruction
- * on register iReg. This is used when an equivalent integer value is
- * stored in place of an 8-byte floating point value in order to save
- * space.
- * @param v Vdbe object.
- * @param def space definition object.
- * @param i column index.
- * @param iReg register index.
- */
-void
-sqlColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg);
-
char* rename_trigger(sql *, char const *, char const *, bool *);
/**
* Find a collation by name. Set error in @a parser if not found.
diff --git a/src/box/sql/update.c b/src/box/sql/update.c
index d77bee051..c5ba56afa 100644
--- a/src/box/sql/update.c
+++ b/src/box/sql/update.c
@@ -38,31 +38,6 @@
#include "box/tuple_format.h"
#include "box/schema.h"
-void
-sqlColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg)
-{
- assert(def != 0);
- if (!def->opts.is_view) {
- sql_value *pValue = 0;
- enum field_type type = def->fields[i].type;
- VdbeComment((v, "%s.%s", def->name, def->fields[i].name));
- assert(i < (int)def->field_count);
-
- Expr *expr = NULL;
- assert(def->fields != NULL && i < (int)def->field_count);
- if (def->fields != NULL)
- expr = def->fields[i].default_value_expr;
- sqlValueFromExpr(sqlVdbeDb(v), expr, type,
- &pValue);
- if (pValue) {
- sqlVdbeAppendP4(v, pValue, P4_MEM);
- }
- if (type == FIELD_TYPE_NUMBER) {
- sqlVdbeAddOp1(v, OP_Realify, ireg);
- }
- }
-}
-
/*
* Process an UPDATE statement.
*
@@ -266,10 +241,9 @@ sqlUpdate(Parse * pParse, /* The parser context */
}
} else {
for (i = 0; i < (int) pk_part_count; i++) {
- sqlExprCodeGetColumnOfTable(v, def, pk_cursor,
- pPk->def->key_def->
- parts[i].fieldno,
- iPk + i);
+ sqlVdbeAddOp3(v, OP_Column, pk_cursor,
+ pPk->def->key_def->parts[i].fieldno,
+ iPk + i);
}
}
@@ -341,8 +315,8 @@ sqlUpdate(Parse * pParse, /* The parser context */
for (i = 0; i < (int)def->field_count; i++) {
if (column_mask_fieldno_is_set(oldmask, i) ||
sql_space_column_is_in_pk(space, i)) {
- sqlExprCodeGetColumnOfTable(v, def, pk_cursor,
- i, regOld + i);
+ sqlVdbeAddOp3(v, OP_Column, pk_cursor, i,
+ regOld + i);
} else {
sqlVdbeAddOp2(v, OP_Null, 0, regOld + i);
}
@@ -377,8 +351,8 @@ sqlUpdate(Parse * pParse, /* The parser context */
* if there are one or more BEFORE triggers that use this value via
* a new.* reference in a trigger program.
*/
- sqlExprCodeGetColumnToReg(pParse, def, i,
- pk_cursor, regNew + i);
+ sqlExprCodeGetColumnToReg(pParse, i, pk_cursor,
+ regNew + i);
} else {
sqlVdbeAddOp2(v, OP_Null, 0, regNew + i);
}
@@ -416,9 +390,8 @@ sqlUpdate(Parse * pParse, /* The parser context */
*/
for (i = 0; i < (int)def->field_count; i++) {
if (aXRef[i] < 0) {
- sqlExprCodeGetColumnOfTable(v, def,
- pk_cursor, i,
- regNew + i);
+ sqlVdbeAddOp3(v, OP_Column, pk_cursor, i,
+ regNew + i);
}
}
}
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 7d5dd46b0..19c9740ae 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -1415,7 +1415,6 @@ sqlWhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about the W
fieldno;
sqlExprCodeGetColumnToReg
(pParse,
- space->def,
fieldno,
iCur,
r + iPk);
Also, Konstantin asked me to fix typeof() behaviour
so that it returns “integer” for integer values > 0 as well
(since basically range of integers in Tarantool is
up to 2^64 - 1). I’ve applied corresponding fixes
to the patch "sql: make built-in functions operate on unsigned values"
> On 18/07/2019 23:08, Vladislav Shpilevoy wrote:
>>
>>
>> On 18/07/2019 22:56, n.pettik wrote:
>>>
>>>
>>>> On 18 Jul 2019, at 23:18, Vladislav Shpilevoy <v.shpilevoy@tarantool.org <mailto:v.shpilevoy@tarantool.org>> wrote:
>>>>
>>>> Hi!
>>>>
>>>> Thanks for the fixes!
>>>>
>>>>>> -------------------------
>>>>>> vdbe.c:307
>>>>>>
>>>>>>> case FIELD_TYPE_INTEGER:
>>>>>>> case FIELD_TYPE_UNSIGNED:
>>>>>>> if ((record->flags & MEM_Int) == MEM_Int)
>>>>>>> return 0;
>>>>>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>>>>>> return 0;
>>>>>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>>>>>> int64_t i = (int64_t) record->u.r;
>>>>>>> if (i == record->u.r)
>>>>>>> mem_set_int(record, record->u.r,
>>>>>>> record->u.r <= -1);
>>>>>>> return 0;
>>>>>>> }
>>>>>>
>>>>>> It is a part of function mem_apply_type. When target type is
>>>>>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>>>>>> it is possible to pass here a negative value, and CAST UNSIGNED
>>>>>> would do nothing.
>>>>>
>>>>> Basically, this function implements sort of implicit cast
>>>>> which takes place before comparison/assignment.
>>>>> For comparisons it makes no sense - we can compare
>>>>> integer with unsigned value - the latter is always greater.
>>>>> For assignment it is also meaningless: if we attempt
>>>>> at inserting negative values to unsigned field appropriate
>>>>> error will be raised anyway. If you can come up with
>>>>> specific example, let’s discuss it.
>>>>>
>>>>
>>>> I can't provide a test. But the function is named mem_apply_type,
>>>> and it doesn't apply type, when it is unsigned, and a value is
>>>> negative. Doesn't it look wrong to you?
>>>>
>>>> If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
>>>> instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
>>>> to int should raise an error here. Otherwise this function can't
>>>> be named 'apply_type' because it ignores negative -> unsigned case.
>>>
>>> Okay, let’s rename it. I can suggest these options:
>>>
>>> mem_cast_implicit()
>>> mem_cast_implicit_to_type()
>>> mem_implicit_cast_to_type()
>>> mem_convert_implicit()
>>> mem_convert_to_type()
>>> mem_type_coerce_implicit()
>>> mem_type_implicit_coercion()
>>> mem_type_coercion_implicit()
>>> mem_implicit_type_juggling()
>>> mem_implicit_juggle_to_type()
>>> mem_do_implicit_conversion()
>>> mem_do_implicit_coercion()
>>>
>>> Or any other combination :)
>>>
>>
>> But it is not implicit. It just does not work, when a value is negative,
>> and type is unsigned.
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-22 10:20 ` n.pettik
@ 2019-07-22 19:17 ` Vladislav Shpilevoy
0 siblings, 0 replies; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-22 19:17 UTC (permalink / raw)
To: n.pettik, tarantool-patches, Kirill Yukhin
Hi!
TLDR: LGTM.
Detailed answer below.
On 22/07/2019 12:20, n.pettik wrote:
>
>
>> On 19 Jul 2019, at 00:13, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>>
>> Why are you keep trying not to fix this obvious bug?
>> If that path is unreachable, then either delete it,
>> or add an assertion. Or fix this place.
>
> Sorry, message threads now seem to be entangled a bit,
> so I guess you are talking about valueFromExpr() func.
> I’ve removed the single reachable to this function path,
> but left function itself (valueFromExpr()) since it seem
> to be quite meaningful and can be applied somewhere
> later. In the next patch I’ve added assertion fault to the
> place of conversion of double negative values (-(-5)):
No, I was talking about mem_apply_type. With the place you
are talking above I agree.
Sorry, I will bring your answer from another email here:
> So, you want to see smth like this:
>
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 73a1f321b..835544d44 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -306,8 +306,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
> switch (type) {
> case FIELD_TYPE_INTEGER:
> case FIELD_TYPE_UNSIGNED:
> - if ((record->flags & MEM_Int) == MEM_Int)
> - return 0;
> if ((record->flags & MEM_UInt) == MEM_UInt)
> return 0;
> if ((record->flags & MEM_Real) == MEM_Real) {
> @@ -317,7 +315,14 @@ mem_apply_type(struct Mem *record, enum field_type type)
> record->u.r <= -1);
> return 0;
> }
> - return sqlVdbeMemIntegerify(record, false);
> + if (sqlVdbeMemIntegerify(record, false) != 0)
> + return -1;
> + if ((record->flags & MEM_Int) == MEM_Int) {
> + if (type == FIELD_TYPE_UNSIGNED)
> + return -1;
> + return 0;
> + }
> + return 0;
>
> The difference can be seen in queries like this:
>
> box.execute("CREATE TABLE t1 (id UNSIGNED PRIMARY KEY);")
> box.execute("INSERT INTO t1 VALUES (-3);")
> ---
> - error: 'Type mismatch: can not convert -3 to unsigned'
> …
>
> Without this change we got:
> 'Tuple field 1 type does not match one required by operation: expected unsigned’
>
> I consider this check to be a bit inappropriate since It is redundant.
> Comparison between unsigned and signed is well defined;
> insertion to the unsigned field is prohibited and checked in more
> low level core mechanisms. I would say I don’t mind applying this
> change (and already applied to speed up review process), but on
> the other side I don’t understand why do we need to add extra checks
> on top (SQL) layer.
I agree, all the checks should be done by box when possible. The only
single problem I am talking about is not a check, but a function name
mismatching what it is doing. 'mem_apply_type' does not apply unsigned
type to a negative value. I just feel uncomfortable about that. The
function just ignores this case.
I was thinking, that perhaps some day the function mem_apply_type will
evolve and consume MemCast and other functions related to type conversions,
and we will forget to fix this place, and will got a bug.
At this moment it is not a bug, because mem_apply_type is used only
before calling comparators, which check types in box. But if someday it
will be used for something else, we will forget that it ignores negative
-> unsigned conversion.
Anyway, never mind. The patchset LGTM.
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-18 21:08 ` Vladislav Shpilevoy
2019-07-18 21:13 ` Vladislav Shpilevoy
@ 2019-07-22 10:20 ` n.pettik
1 sibling, 0 replies; 49+ messages in thread
From: n.pettik @ 2019-07-22 10:20 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 19 Jul 2019, at 00:08, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
> On 18/07/2019 22:56, n.pettik wrote:
>>
>>> On 18 Jul 2019, at 23:18, Vladislav Shpilevoy <v.shpilevoy@tarantool.org <mailto:v.shpilevoy@tarantool.org>> wrote:
>>>>> -------------------------
>>>>> vdbe.c:307
>>>>>
>>>>>> case FIELD_TYPE_INTEGER:
>>>>>> case FIELD_TYPE_UNSIGNED:
>>>>>> if ((record->flags & MEM_Int) == MEM_Int)
>>>>>> return 0;
>>>>>> if ((record->flags & MEM_UInt) == MEM_UInt)
>>>>>> return 0;
>>>>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>>>>> int64_t i = (int64_t) record->u.r;
>>>>>> if (i == record->u.r)
>>>>>> mem_set_int(record, record->u.r,
>>>>>> record->u.r <= -1);
>>>>>> return 0;
>>>>>> }
>>>>>
>>>>> It is a part of function mem_apply_type. When target type is
>>>>> UNSIGNED, and a value is MEM_Int, you do nothing. Why? Looks like
>>>>> it is possible to pass here a negative value, and CAST UNSIGNED
>>>>> would do nothing.
>>>>
>>>> Basically, this function implements sort of implicit cast
>>>> which takes place before comparison/assignment.
>>>> For comparisons it makes no sense - we can compare
>>>> integer with unsigned value - the latter is always greater.
>>>> For assignment it is also meaningless: if we attempt
>>>> at inserting negative values to unsigned field appropriate
>>>> error will be raised anyway. If you can come up with
>>>> specific example, let’s discuss it.
>>>>
>>>
>>> I can't provide a test. But the function is named mem_apply_type,
>>> and it doesn't apply type, when it is unsigned, and a value is
>>> negative. Doesn't it look wrong to you?
>>>
>>> If some code wants to get an integer, it can apply FIELD_TYPE_INTEGER
>>> instead of FIELD_TYPE_UNSIGNED. IMO, an attempt to apply unsigned
>>> to int should raise an error here. Otherwise this function can't
>>> be named 'apply_type' because it ignores negative -> unsigned case.
>>
>> Okay, let’s rename it. I can suggest these options:
>>
>> mem_cast_implicit()
>> mem_cast_implicit_to_type()
>> mem_implicit_cast_to_type()
>> mem_convert_implicit()
>> mem_convert_to_type()
>> mem_type_coerce_implicit()
>> mem_type_implicit_coercion()
>> mem_type_coercion_implicit()
>> mem_implicit_type_juggling()
>> mem_implicit_juggle_to_type()
>> mem_do_implicit_conversion()
>> mem_do_implicit_coercion()
>>
>> Or any other combination :)
>>
>
> But it is not implicit. It just does not work, when a value is negative,
> and type is unsigned.
So, you want to see smth like this:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 73a1f321b..835544d44 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -306,8 +306,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
switch (type) {
case FIELD_TYPE_INTEGER:
case FIELD_TYPE_UNSIGNED:
- if ((record->flags & MEM_Int) == MEM_Int)
- return 0;
if ((record->flags & MEM_UInt) == MEM_UInt)
return 0;
if ((record->flags & MEM_Real) == MEM_Real) {
@@ -317,7 +315,14 @@ mem_apply_type(struct Mem *record, enum field_type type)
record->u.r <= -1);
return 0;
}
- return sqlVdbeMemIntegerify(record, false);
+ if (sqlVdbeMemIntegerify(record, false) != 0)
+ return -1;
+ if ((record->flags & MEM_Int) == MEM_Int) {
+ if (type == FIELD_TYPE_UNSIGNED)
+ return -1;
+ return 0;
+ }
+ return 0;
The difference can be seen in queries like this:
box.execute("CREATE TABLE t1 (id UNSIGNED PRIMARY KEY);")
box.execute("INSERT INTO t1 VALUES (-3);")
---
- error: 'Type mismatch: can not convert -3 to unsigned'
…
Without this change we got:
'Tuple field 1 type does not match one required by operation: expected unsigned’
I consider this check to be a bit inappropriate since It is redundant.
Comparison between unsigned and signed is well defined;
insertion to the unsigned field is prohibited and checked in more
low level core mechanisms. I would say I don’t mind applying this
change (and already applied to speed up review process), but on
the other side I don’t understand why do we need to add extra checks
on top (SQL) layer.
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-10 22:49 ` Vladislav Shpilevoy
2019-07-11 21:25 ` Vladislav Shpilevoy
@ 2019-07-17 0:54 ` n.pettik
2019-07-18 20:18 ` Vladislav Shpilevoy
2019-08-06 19:36 ` n.pettik
2 siblings, 1 reply; 49+ messages in thread
From: n.pettik @ 2019-07-17 0:54 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>> --------------------------------------------------------------
>>> vdbeaux.c:2998:
>>> if ((f1 & MEM_UInt) != 0) {
>>> if ((f2 & MEM_Real) != 0) {
>>> return sqlIntFloatCompare(pMem1->u.i,
>>>
>>> pMem1 is unsigned, according to the first check,
>>> but you use u.i. Why?
>>
>> Thx, I’ve fixed series of similar places and extended sql/types.test.lua:
>>
>> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
>> index 325c54c18..b6b5cd0bf 100644
>> --- a/src/box/sql/vdbeaux.c
>> +++ b/src/box/sql/vdbeaux.c
>> @@ -2887,43 +2887,50 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
>> return n1 - n2;
>> }
>>
>> -/*
>> - * Do a comparison between a 64-bit signed integer and a 64-bit floating-point
>> - * number. Return negative, zero, or positive if the first (i64) is less than,
>> - * equal to, or greater than the second (double).
>> +/**
>> + * Do a comparison between a 64-bit unsigned/signed integer and a
>> + * 64-bit floating-point number. Return negative, zero, or
>> + * positive if the first (integer) is less than, equal to, or
>> + * greater than the second (double).
>> */
>> static int
>> -sqlIntFloatCompare(i64 i, double r)
>> +compare_uint_float(uint64_t u, double r)
>
> Unfortunately, it is not as simple as you implemented it.
> See mp_compare_double_uint64 in tuple_compare.cc for details.
> In short, your function works wrong when numbers are near
> uint maximum. Perhaps, it is worth moving this comparator
> from tuple_compare.cc to a header. Like trivia/util.h.
Yep, I’m realising that :)
Unfortunately, sqlite uses such simplified version and there
are tests verifying this behaviour (boundary1/2/3 test suits).
Patch fixing it (as you suggest) is trivial, but it requires monotonous
test changes. Don’t you mind if I send it as follow-up after this
patch-set hits the trunk?
>> {
>> - if (sizeof(LONGDOUBLE_TYPE) > 8) {
>> - LONGDOUBLE_TYPE x = (LONGDOUBLE_TYPE) i;
>> - if (x < r)
>> - return -1;
>> - if (x > r)
>> - return +1;
>> - return 0;
>> - } else {
>> - i64 y;
>> - double s;
>> - if (r < -9223372036854775808.0)
>> - return +1;
>> - if (r > 9223372036854775807.0)
>> - return -1;
>> - y = (i64) r;
>> - if (i < y)
>> - return -1;
>> - if (i > y) {
>> - if (y == SMALLEST_INT64 && r > 0.0)
>> - return -1;
>> - return +1;
>> - }
>> - s = (double)i;
>> - if (s < r)
>> - return -1;
>> - if (s > r)
>> - return +1;
>> - return 0;
>> - }
>> + if (r > (double) UINT64_MAX)
>> + return -1;
>> + if (r < 0.0)
>> + return +1;
>> + uint64_t y = (uint64_t) r;
>> + if (u < y)
>> + return -1;
>> + if (u > y)
>> + return +1;
>> + double s = (double) u;
>> + if (s < r)
>> + return -1;
>> + if (s > r)
>> + return +1;
>> + return 0;
>> +}
>> +
>> +static int
>> +compare_int_float(int64_t i, double r)
>
> It has the same problems as the previous function,
> but can be fixed by calling compare_uint_float()
> with a modulo of 'int64_t i' and reversed result,
> if the value was negative. This is what mp_compare_double_any_int
> does.
>
>> +{
>> + if (r < (double) INT64_MIN)
>> + return +1;
>> + if (r >= 0.0)
>> + return -1;
>> + int64_t y = (int64_t) r;
>> + if (i < y)
>> + return -1;
>> + if (i > y)
>> + return +1;
>> + double s = (double) i;
>> + if (s < r)
>> + return -1;
>> + if (s > r)
>> + return +1;
>> + return 0;
>> }
>>> --------------------------------------------------------------
>>> vdbemem.c:1431:
>>> } else if (pVal->u.i == SMALLEST_INT64) {
>>> pVal->u.r = -(double)SMALLEST_INT64;
>>> MemSetTypeFlag(pVal, MEM_Real);
>>> } else {
>>> pVal->u.i = -pVal->u.i;
>>> }
>>>
>>> You compare u.i and SMALLEST_INT64, but you can't
>>> be sure, that u.i is not a big unsigned, can you?
>>
>> Fixed:
>>
>> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
>> index f8673912e..64acb5d41 100644
>> --- a/src/box/sql/vdbemem.c
>> +++ b/src/box/sql/vdbemem.c
>> @@ -1428,11 +1428,15 @@ valueFromExpr(sql * db, /* The database connection */
>> return rc;
>> if (pVal->flags & MEM_Real) {
>> pVal->u.r = -pVal->u.r;
>> - } else if (pVal->u.i == SMALLEST_INT64) {
>> - pVal->u.r = -(double)SMALLEST_INT64;
>> - MemSetTypeFlag(pVal, MEM_Real);
>> - } else {
>> - pVal->u.i = -pVal->u.i;
>> + } else if ((pVal->flags & MEM_Int) != 0) {
>> + mem_set_u64(pVal, (uint64_t)(-pVal->u.i));
>> + } else if ((pVal->flags & MEM_UInt) != 0) {
>> + if (pVal->u.u > (uint64_t) INT64_MAX + 1) {
>> + pVal->u.r = -(double) pVal->u.u;
>> + MemSetTypeFlag(pVal, MEM_Real);
>
> Won't we have a problem here, that an expression '--value' won't be
> equal to 'value' in case the value is bigger than INT64_MAX + 1?
I guess it’s OK according to the original code. What is more, I doubt
that this path is reachable at all: both select -(-18446744073709551615)
and select -(18446744073709551615) queries result in error while
processing expr_code_int()..
^ permalink raw reply [flat|nested] 49+ messages in thread* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-17 0:54 ` n.pettik
@ 2019-07-18 20:18 ` Vladislav Shpilevoy
0 siblings, 0 replies; 49+ messages in thread
From: Vladislav Shpilevoy @ 2019-07-18 20:18 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
On 17/07/2019 02:54, n.pettik wrote:
>
>>>> --------------------------------------------------------------
>>>> vdbeaux.c:2998:
>>>> if ((f1 & MEM_UInt) != 0) {
>>>> if ((f2 & MEM_Real) != 0) {
>>>> return sqlIntFloatCompare(pMem1->u.i,
>>>>
>>>> pMem1 is unsigned, according to the first check,
>>>> but you use u.i. Why?
>>>
>>> Thx, I’ve fixed series of similar places and extended sql/types.test.lua:
>>>
>>> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
>>> index 325c54c18..b6b5cd0bf 100644
>>> --- a/src/box/sql/vdbeaux.c
>>> +++ b/src/box/sql/vdbeaux.c
>>> @@ -2887,43 +2887,50 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
>>> return n1 - n2;
>>> }
>>>
>>> -/*
>>> - * Do a comparison between a 64-bit signed integer and a 64-bit floating-point
>>> - * number. Return negative, zero, or positive if the first (i64) is less than,
>>> - * equal to, or greater than the second (double).
>>> +/**
>>> + * Do a comparison between a 64-bit unsigned/signed integer and a
>>> + * 64-bit floating-point number. Return negative, zero, or
>>> + * positive if the first (integer) is less than, equal to, or
>>> + * greater than the second (double).
>>> */
>>> static int
>>> -sqlIntFloatCompare(i64 i, double r)
>>> +compare_uint_float(uint64_t u, double r)
>>
>> Unfortunately, it is not as simple as you implemented it.
>> See mp_compare_double_uint64 in tuple_compare.cc for details.
>> In short, your function works wrong when numbers are near
>> uint maximum. Perhaps, it is worth moving this comparator
>> from tuple_compare.cc to a header. Like trivia/util.h.
>
> Yep, I’m realising that :)
> Unfortunately, sqlite uses such simplified version and there
> are tests verifying this behaviour (boundary1/2/3 test suits).
> Patch fixing it (as you suggest) is trivial, but it requires monotonous
> test changes. Don’t you mind if I send it as follow-up after this
> patch-set hits the trunk?
It is ok. Please, create an issue.
>>>> --------------------------------------------------------------
>>>> vdbemem.c:1431:
>>> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
>>> index f8673912e..64acb5d41 100644
>>> --- a/src/box/sql/vdbemem.c
>>> +++ b/src/box/sql/vdbemem.c
>>> @@ -1428,11 +1428,15 @@ valueFromExpr(sql * db, /* The database connection */
>>> return rc;
>>> if (pVal->flags & MEM_Real) {
>>> pVal->u.r = -pVal->u.r;
>>> - } else if (pVal->u.i == SMALLEST_INT64) {
>>> - pVal->u.r = -(double)SMALLEST_INT64;
>>> - MemSetTypeFlag(pVal, MEM_Real);
>>> - } else {
>>> - pVal->u.i = -pVal->u.i;
>>> + } else if ((pVal->flags & MEM_Int) != 0) {
>>> + mem_set_u64(pVal, (uint64_t)(-pVal->u.i));
>>> + } else if ((pVal->flags & MEM_UInt) != 0) {
>>> + if (pVal->u.u > (uint64_t) INT64_MAX + 1) {
>>> + pVal->u.r = -(double) pVal->u.u;
>>> + MemSetTypeFlag(pVal, MEM_Real);
>>
>> Won't we have a problem here, that an expression '--value' won't be
>> equal to 'value' in case the value is bigger than INT64_MAX + 1?
>
> I guess it’s OK according to the original code. What is more, I doubt
> that this path is reachable at all: both select -(-18446744073709551615)
> and select -(18446744073709551615) queries result in error while
> processing expr_code_int()..
>
Then why do we keep that code, if it is unreachable?
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] Re: [PATCH 6/6] sql: allow to specify UNSIGNED column type
2019-07-10 22:49 ` Vladislav Shpilevoy
2019-07-11 21:25 ` Vladislav Shpilevoy
2019-07-17 0:54 ` n.pettik
@ 2019-08-06 19:36 ` n.pettik
2 siblings, 0 replies; 49+ messages in thread
From: n.pettik @ 2019-08-06 19:36 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
>> index 325c54c18..b6b5cd0bf 100644
>> --- a/src/box/sql/vdbeaux.c
>> +++ b/src/box/sql/vdbeaux.c
>> @@ -2887,43 +2887,50 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
>> return n1 - n2;
>> }
>>
>> -/*
>> - * Do a comparison between a 64-bit signed integer and a 64-bit floating-point
>> - * number. Return negative, zero, or positive if the first (i64) is less than,
>> - * equal to, or greater than the second (double).
>> +/**
>> + * Do a comparison between a 64-bit unsigned/signed integer and a
>> + * 64-bit floating-point number. Return negative, zero, or
>> + * positive if the first (integer) is less than, equal to, or
>> + * greater than the second (double).
>> */
>> static int
>> -sqlIntFloatCompare(i64 i, double r)
>> +compare_uint_float(uint64_t u, double r)
>
> Unfortunately, it is not as simple as you implemented it.
> See mp_compare_double_uint64 in tuple_compare.cc for details.
> In short, your function works wrong when numbers are near
> uint maximum. Perhaps, it is worth moving this comparator
> from tuple_compare.cc to a header. Like trivia/util.h.
Hi,
I remember your concern about compare_uint_float().
At the moment of review I said that integrating
mp_compare_double_uint64() into SQL code results in
test fails. In fact, that’s not true: I accidentally added
bug when substituting them. Now I've carefully reviewed
patch and it seems that all tests except for one (which
exactly covers that border case) are passed. What is more,
fix seems to be quite trivial:
@@ -2895,7 +2897,7 @@ sqlBlobCompare(const Mem * pB1, const Mem * pB2)
static int
compare_uint_float(uint64_t u, double r)
{
- if (r > (double) UINT64_MAX)
+ if (r >= exp2(64))
return -1;
if (r < 0.0)
return +1;
With this fix I’ve failed to come up with counter-example
demonstrating different from mp_compare_double_uint64
results. Anyway, I replaced compare_uint_float/compare_int_float
with analogue from comparators so that we have unified way of
int-float comparison. Patch is on its way.
^ permalink raw reply [flat|nested] 49+ messages in thread
* [tarantool-patches] Re: [PATCH 0/6] Introduce UNSIGNED type in SQL
2019-06-07 15:37 [tarantool-patches] [PATCH 0/6] Introduce UNSIGNED type in SQL Nikita Pettik
` (5 preceding siblings ...)
2019-06-07 15:37 ` [tarantool-patches] [PATCH 6/6] sql: allow to specify UNSIGNED column type Nikita Pettik
@ 2019-07-24 13:01 ` Kirill Yukhin
6 siblings, 0 replies; 49+ messages in thread
From: Kirill Yukhin @ 2019-07-24 13:01 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Hello,
On 07 Jun 18:37, Nikita Pettik wrote:
> Branch: https://github.com/tarantool/tarantool/tree/np/introduce-uint
> Issues:
> https://github.com/tarantool/tarantool/issues/3810
> https://github.com/tarantool/tarantool/issues/4015
>
> Current patch-set allows using UNSIGNED as a column type and operating
> on integer values in range up to [2^64 - 1]. This means that result of
> all basic routines such as arithmetic operations and built-in functions
> is extended to unsigned range.
> The basic idea under the hood is separating storing positive and
> negative integers into different memory cells. We've added new
> type of VDBE memory - MEM_UInt. Now all positive values are stored with
> featuring this flag; in turn, negative integers have MEM_Int type.
> It's quite similar to MP_* format types.
>
> For more details see content of patches.
>
> Nikita Pettik (6):
> sql: refactor sql_atoi64()
> sql: separate VDBE memory holding positive and negative ints
> sql: refactor arithmetic operations to support unsigned ints
> sql: make built-in functions operate on unsigned values
> sql: introduce extended range for INTEGER type
> sql: allow to specify UNSIGNED column type
I've checked the patchset into master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 49+ messages in thread