* [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches
2021-04-11 17:42 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 02/52] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches
` (40 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:51 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Hi! Thank you for the review! My answer and new patch below. I didn't include
diffs in answers since due to merge conflicts they are partly useless.
On 30.03.2021 01:57, Vladislav Shpilevoy wrote:
> Hi! I appreciate the work you did here!
>
> Truly huge patch. But it does the important thing which I think
> should give a huge help in the task of SQL code rework.
>
> On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote:
>> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
>> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
>> MP_ARRAY fields. Also, it doesn't set field_type.
>
> AFAIR, field type wasn't set deliberately. Because we use it only for
> strictly typed values obtained from formatted space fields. It wasn't
> applied to plain non-formatted values on purpose.
>
> Why do you change that?
I didn't know about that. I thought that all MEMs that contains data should have
field_type set. However, I tried to understand where did this field come from
and found that it was added for two purposes: to show difference between NUMBER
and INTEGER in MEM before DOUBLE was added and to show column name instead of
type determined from MP-type in typeof(). I believe that both these purposes are
not needed now and that this field should be removed from struct MEM. I created
an issue for this: #5957. However, I was prohibited to remove this field for now
by @tsafin, who believes that this field is actually important.
New patch:
commit 3fda2029b186d0cea67ba01269dc3d6e209b3c89
Author: Mergen Imeev <imeevma@gmail.com>
Date: Thu Mar 4 17:17:18 2021 +0300
sql: enhance vdbe_decode_msgpack_into_mem()
Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
properly initialized in case msgpack contains MP_EXT, MP_MAP, or
MP_ARRAY fields. This patch makes it so that after execution of this
function, all created MEMs are properly initialized.
Needed for #5818
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 3b3b1f01d..9a4f38bb9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -846,16 +846,6 @@ vdbe_field_ref_fetch_data(struct vdbe_field_ref *field_ref, uint32_t fieldno)
return field_begin;
}
-static inline enum field_type
-vdbe_field_ref_fetch_type(struct vdbe_field_ref *field_ref, uint32_t fieldno)
-{
- const struct tuple_field *tf =
- vdbe_field_ref_fetch_field(field_ref, fieldno);
- if (tf == NULL || tf->type == FIELD_TYPE_ANY)
- return field_type_MAX;
- return tf->type;
-}
-
/**
* Fetch field by fieldno using vdbe_field_ref and store result
* in dest_mem.
@@ -879,17 +869,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
return -1;
- /*
- * MsgPack map, array or extension (unsupported in sql).
- * Wrap it in a blob verbatim.
- */
- if (dest_mem->flags == 0) {
- dest_mem->z = (char *) data;
- dest_mem->n = vdbe_field_ref_fetch_data(field_ref,
- fieldno + 1) - data;
- dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
- dest_mem->subtype = SQL_SUBTYPE_MSGPACK;
- }
/*
* Add 0 termination (at most for strings)
* Not sure why do we check MEM_Ephem
@@ -909,7 +888,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
dest_mem->flags |= MEM_Term;
}
UPDATE_MAX_BLOBSIZE(dest_mem);
- dest_mem->field_type = vdbe_field_ref_fetch_type(field_ref, fieldno);
return 0;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 91b64316e..772476377 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2793,38 +2793,62 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
{
const char *start_buf = buf;
switch (mp_typeof(*buf)) {
- case MP_ARRAY:
- case MP_MAP:
- case MP_EXT:
- default: {
- mem->flags = 0;
+ case MP_ARRAY: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_ARRAY;
+ break;
+ }
+ case MP_MAP: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_MAP;
+ break;
+ }
+ case MP_EXT: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
break;
}
case MP_NIL: {
mp_decode_nil(&buf);
mem->flags = MEM_Null;
+ mem->field_type = field_type_MAX;
break;
}
case MP_BOOL: {
mem->u.b = mp_decode_bool(&buf);
mem->flags = MEM_Bool;
+ mem->field_type = FIELD_TYPE_BOOLEAN;
break;
}
case MP_UINT: {
uint64_t v = mp_decode_uint(&buf);
mem->u.u = v;
mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_INTEGER;
break;
}
case MP_INT: {
mem->u.i = mp_decode_int(&buf);
mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
break;
}
case MP_STR: {
/* XXX u32->int */
mem->n = (int) mp_decode_strl(&buf);
mem->flags = MEM_Str | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_STRING;
install_blob:
mem->z = (char *)buf;
buf += mem->n;
@@ -2834,18 +2858,33 @@ install_blob:
/* XXX u32->int */
mem->n = (int) mp_decode_binl(&buf);
mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
goto install_blob;
}
case MP_FLOAT: {
mem->u.r = mp_decode_float(&buf);
- mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
break;
}
case MP_DOUBLE: {
mem->u.r = mp_decode_double(&buf);
- mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
break;
}
+ default:
+ unreachable();
}
*len = (uint32_t)(buf - start_buf);
return 0;
@@ -2868,15 +2907,8 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec
pMem->z = 0;
uint32_t sz = 0;
vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
- if (sz == 0) {
- /* MsgPack array, map or ext. Treat as blob. */
- pMem->z = (char *)zParse;
- mp_next(&zParse);
- pMem->n = zParse - pMem->z;
- pMem->flags = MEM_Blob | MEM_Ephem;
- } else {
- zParse += sz;
- }
+ assert(sz != 0);
+ zParse += sz;
pMem++;
}
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-04-11 17:42 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 17:42 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Hi! Good job on the patch!
On 09.04.2021 18:51, Mergen Imeev via Tarantool-patches wrote:
> Hi! Thank you for the review! My answer and new patch below. I didn't include
> diffs in answers since due to merge conflicts they are partly useless.
>
> On 30.03.2021 01:57, Vladislav Shpilevoy wrote:
>> Hi! I appreciate the work you did here!
>>
>> Truly huge patch. But it does the important thing which I think
>> should give a huge help in the task of SQL code rework.
>>
>> On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote:
>>> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
>>> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
>>> MP_ARRAY fields. Also, it doesn't set field_type.
>>
>> AFAIR, field type wasn't set deliberately. Because we use it only for
>> strictly typed values obtained from formatted space fields. It wasn't
>> applied to plain non-formatted values on purpose.
>>
>> Why do you change that?
>
> I didn't know about that. I thought that all MEMs that contains data should have
> field_type set. However, I tried to understand where did this field come from
> and found that it was added for two purposes: to show difference between NUMBER
> and INTEGER in MEM before DOUBLE was added and to show column name instead of
> type determined from MP-type in typeof(). I believe that both these purposes are
> not needed now and that this field should be removed from struct MEM. I created
> an issue for this: #5957. However, I was prohibited to remove this field for now
> by @tsafin, who believes that this field is actually important.
I tend to agree, that it must be done separately. Although probably could be done
before this patchset to make it kind of simpler. Anyway, thanks for the
explanation.
^ permalink raw reply [flat|nested] 107+ messages in thread
* Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-11 17:42 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches
2021-04-13 12:12 ` Mergen Imeev via Tarantool-patches
2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 2 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 12:01 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! I'a sorry to ask you so late, but could you look at
new version of the patch? It was decided that this patch should be pushed to
2.6 and 2.7 along with master. Only this patch of the patch-set.
Diff and new version below. Please note, that I added additional SELECT to
the first test since currently sql-tap test system does not support binary
result. Also, in the last two tests I created new table for each since this way
we could see proper error. If I used space T there then in the last test there
would be the same error as in the test 5 due to UUID being before DECIMAL in
space format. To remove dependence of their position, I added two new spaces.
On Sun, Apr 11, 2021 at 07:42:22PM +0200, Vladislav Shpilevoy wrote:
> Hi! Good job on the patch!
>
> On 09.04.2021 18:51, Mergen Imeev via Tarantool-patches wrote:
> > Hi! Thank you for the review! My answer and new patch below. I didn't include
> > diffs in answers since due to merge conflicts they are partly useless.
> >
> > On 30.03.2021 01:57, Vladislav Shpilevoy wrote:
> >> Hi! I appreciate the work you did here!
> >>
> >> Truly huge patch. But it does the important thing which I think
> >> should give a huge help in the task of SQL code rework.
> >>
> >> On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote:
> >>> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
> >>> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
> >>> MP_ARRAY fields. Also, it doesn't set field_type.
> >>
> >> AFAIR, field type wasn't set deliberately. Because we use it only for
> >> strictly typed values obtained from formatted space fields. It wasn't
> >> applied to plain non-formatted values on purpose.
> >>
> >> Why do you change that?
> >
> > I didn't know about that. I thought that all MEMs that contains data should have
> > field_type set. However, I tried to understand where did this field come from
> > and found that it was added for two purposes: to show difference between NUMBER
> > and INTEGER in MEM before DOUBLE was added and to show column name instead of
> > type determined from MP-type in typeof(). I believe that both these purposes are
> > not needed now and that this field should be removed from struct MEM. I created
> > an issue for this: #5957. However, I was prohibited to remove this field for now
> > by @tsafin, who believes that this field is actually important.
>
> I tend to agree, that it must be done separately. Although probably could be done
> before this patchset to make it kind of simpler. Anyway, thanks for the
> explanation.
Diff:
diff --git a/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
new file mode 100755
index 000000000..60978c2b5
--- /dev/null
+++ b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
@@ -0,0 +1,99 @@
+#!/usr/bin/env tarantool
+local test = require("sqltester")
+test:plan(6)
+
+local uuid = require("uuid").fromstr("11111111-1111-1111-1111-111111111111")
+local decimal = require("decimal").new(111.111)
+
+box.schema.create_space('T')
+box.space.T:format({{name = "I", type = "integer"}, {name = "U", type = "uuid"},
+ {name = "D", type = "decimal"}})
+box.space.T:create_index("primary")
+box.space.T:insert({1, uuid, decimal})
+
+--
+-- Make sure that there is no segmentation fault on select from field that
+-- contains UUID or DECIMAL. Currently SQL does not support UUID and DECIMAL,
+-- so they treated as VARBINARY.
+--
+test:do_execsql_test(
+ "gh-5913-1",
+ [[
+ SELECT i, u, d FROM t;
+ SELECT i from t;
+ ]], {
+ 1
+ })
+
+box.schema.create_space('T1')
+box.space.T1:format({{name = "I", type = "integer"},
+ {name = "U", type = "uuid", is_nullable = true},
+ {name = "D", type = "decimal", is_nullable = true}})
+box.space.T1:create_index("primary")
+
+--
+-- Since SQL does not support UUID and DECIMAL and they treated as VARBINARY,
+-- they cannot be inserted from SQL.
+--
+test:do_catchsql_test(
+ "gh-5913-2",
+ [[
+ INSERT INTO t1 SELECT i, u, NULL FROM t;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to uuid"
+ })
+
+test:do_catchsql_test(
+ "gh-5913-3",
+ [[
+ INSERT INTO t1 SELECT i, NULL, d FROM t;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to decimal"
+ })
+
+--
+-- Still, if UUID or DECIMAL fields does not selected directly, insert is
+-- working properly.
+--
+test:do_execsql_test(
+ "gh-5913-4",
+ [[
+ INSERT INTO t1 SELECT * FROM t;
+ SELECT count() FROM t1;
+ ]], {
+ 1
+ })
+
+box.schema.create_space('TU')
+box.space.TU:format({{name = "I", type = "integer"},
+ {name = "U", type = "uuid"}})
+box.space.TU:create_index("primary")
+box.space.TU:insert({1, uuid})
+
+box.schema.create_space('TD')
+box.space.TD:format({{name = "I", type = "integer"},
+ {name = "D", type = "decimal"}})
+box.space.TD:create_index("primary")
+box.space.TD:insert({1, decimal})
+
+--
+-- Update of UUID or VARBINARY also does not lead to segfault, however throws an
+-- error since after changing value cannot be inserted into the field from SQL.
+--
+test:do_catchsql_test(
+ "gh-5913-5",
+ [[
+ UPDATE tu SET u = u;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to uuid"
+ })
+
+test:do_catchsql_test(
+ "gh-5913-6",
+ [[
+ UPDATE td SET d = d;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to decimal"
+ })
+
+test:finish_test()
New patch:
commit d41d8f756f8ab6bfae70ee5787bc4c509ddb844d
Author: Mergen Imeev <imeevma@gmail.com>
Date: Thu Mar 4 17:17:18 2021 +0300
sql: enhance vdbe_decode_msgpack_into_mem()
Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
properly initialized in case msgpack contains MP_EXT, MP_MAP, or
MP_ARRAY fields. This patch makes it so that after execution of this
function, all created MEMs are properly initialized.
Closes #5011
Closes #5704
Closes #5913
Needed for #5818
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 3b3b1f01d..9a4f38bb9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -846,16 +846,6 @@ vdbe_field_ref_fetch_data(struct vdbe_field_ref *field_ref, uint32_t fieldno)
return field_begin;
}
-static inline enum field_type
-vdbe_field_ref_fetch_type(struct vdbe_field_ref *field_ref, uint32_t fieldno)
-{
- const struct tuple_field *tf =
- vdbe_field_ref_fetch_field(field_ref, fieldno);
- if (tf == NULL || tf->type == FIELD_TYPE_ANY)
- return field_type_MAX;
- return tf->type;
-}
-
/**
* Fetch field by fieldno using vdbe_field_ref and store result
* in dest_mem.
@@ -879,17 +869,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
return -1;
- /*
- * MsgPack map, array or extension (unsupported in sql).
- * Wrap it in a blob verbatim.
- */
- if (dest_mem->flags == 0) {
- dest_mem->z = (char *) data;
- dest_mem->n = vdbe_field_ref_fetch_data(field_ref,
- fieldno + 1) - data;
- dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
- dest_mem->subtype = SQL_SUBTYPE_MSGPACK;
- }
/*
* Add 0 termination (at most for strings)
* Not sure why do we check MEM_Ephem
@@ -909,7 +888,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
dest_mem->flags |= MEM_Term;
}
UPDATE_MAX_BLOBSIZE(dest_mem);
- dest_mem->field_type = vdbe_field_ref_fetch_type(field_ref, fieldno);
return 0;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 91b64316e..772476377 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2793,38 +2793,62 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
{
const char *start_buf = buf;
switch (mp_typeof(*buf)) {
- case MP_ARRAY:
- case MP_MAP:
- case MP_EXT:
- default: {
- mem->flags = 0;
+ case MP_ARRAY: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_ARRAY;
+ break;
+ }
+ case MP_MAP: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_MAP;
+ break;
+ }
+ case MP_EXT: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
break;
}
case MP_NIL: {
mp_decode_nil(&buf);
mem->flags = MEM_Null;
+ mem->field_type = field_type_MAX;
break;
}
case MP_BOOL: {
mem->u.b = mp_decode_bool(&buf);
mem->flags = MEM_Bool;
+ mem->field_type = FIELD_TYPE_BOOLEAN;
break;
}
case MP_UINT: {
uint64_t v = mp_decode_uint(&buf);
mem->u.u = v;
mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_INTEGER;
break;
}
case MP_INT: {
mem->u.i = mp_decode_int(&buf);
mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
break;
}
case MP_STR: {
/* XXX u32->int */
mem->n = (int) mp_decode_strl(&buf);
mem->flags = MEM_Str | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_STRING;
install_blob:
mem->z = (char *)buf;
buf += mem->n;
@@ -2834,18 +2858,33 @@ install_blob:
/* XXX u32->int */
mem->n = (int) mp_decode_binl(&buf);
mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
goto install_blob;
}
case MP_FLOAT: {
mem->u.r = mp_decode_float(&buf);
- mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
break;
}
case MP_DOUBLE: {
mem->u.r = mp_decode_double(&buf);
- mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
break;
}
+ default:
+ unreachable();
}
*len = (uint32_t)(buf - start_buf);
return 0;
@@ -2868,15 +2907,8 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec
pMem->z = 0;
uint32_t sz = 0;
vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
- if (sz == 0) {
- /* MsgPack array, map or ext. Treat as blob. */
- pMem->z = (char *)zParse;
- mp_next(&zParse);
- pMem->n = zParse - pMem->z;
- pMem->flags = MEM_Blob | MEM_Ephem;
- } else {
- zParse += sz;
- }
+ assert(sz != 0);
+ zParse += sz;
pMem++;
}
}
diff --git a/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
new file mode 100755
index 000000000..60978c2b5
--- /dev/null
+++ b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
@@ -0,0 +1,99 @@
+#!/usr/bin/env tarantool
+local test = require("sqltester")
+test:plan(6)
+
+local uuid = require("uuid").fromstr("11111111-1111-1111-1111-111111111111")
+local decimal = require("decimal").new(111.111)
+
+box.schema.create_space('T')
+box.space.T:format({{name = "I", type = "integer"}, {name = "U", type = "uuid"},
+ {name = "D", type = "decimal"}})
+box.space.T:create_index("primary")
+box.space.T:insert({1, uuid, decimal})
+
+--
+-- Make sure that there is no segmentation fault on select from field that
+-- contains UUID or DECIMAL. Currently SQL does not support UUID and DECIMAL,
+-- so they treated as VARBINARY.
+--
+test:do_execsql_test(
+ "gh-5913-1",
+ [[
+ SELECT i, u, d FROM t;
+ SELECT i from t;
+ ]], {
+ 1
+ })
+
+box.schema.create_space('T1')
+box.space.T1:format({{name = "I", type = "integer"},
+ {name = "U", type = "uuid", is_nullable = true},
+ {name = "D", type = "decimal", is_nullable = true}})
+box.space.T1:create_index("primary")
+
+--
+-- Since SQL does not support UUID and DECIMAL and they treated as VARBINARY,
+-- they cannot be inserted from SQL.
+--
+test:do_catchsql_test(
+ "gh-5913-2",
+ [[
+ INSERT INTO t1 SELECT i, u, NULL FROM t;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to uuid"
+ })
+
+test:do_catchsql_test(
+ "gh-5913-3",
+ [[
+ INSERT INTO t1 SELECT i, NULL, d FROM t;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to decimal"
+ })
+
+--
+-- Still, if UUID or DECIMAL fields does not selected directly, insert is
+-- working properly.
+--
+test:do_execsql_test(
+ "gh-5913-4",
+ [[
+ INSERT INTO t1 SELECT * FROM t;
+ SELECT count() FROM t1;
+ ]], {
+ 1
+ })
+
+box.schema.create_space('TU')
+box.space.TU:format({{name = "I", type = "integer"},
+ {name = "U", type = "uuid"}})
+box.space.TU:create_index("primary")
+box.space.TU:insert({1, uuid})
+
+box.schema.create_space('TD')
+box.space.TD:format({{name = "I", type = "integer"},
+ {name = "D", type = "decimal"}})
+box.space.TD:create_index("primary")
+box.space.TD:insert({1, decimal})
+
+--
+-- Update of UUID or VARBINARY also does not lead to segfault, however throws an
+-- error since after changing value cannot be inserted into the field from SQL.
+--
+test:do_catchsql_test(
+ "gh-5913-5",
+ [[
+ UPDATE tu SET u = u;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to uuid"
+ })
+
+test:do_catchsql_test(
+ "gh-5913-6",
+ [[
+ UPDATE td SET d = d;
+ ]], {
+ 1, "Type mismatch: can not convert varbinary to decimal"
+ })
+
+test:finish_test()
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches
@ 2021-04-13 12:12 ` Mergen Imeev via Tarantool-patches
2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches
1 sibling, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 12:12 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Sorry, I forgot to add name of the new branch and link to issues that will be
closed by this patch. Here they are:
https://github.com/tarantool/tarantool/tree/imeevma/gh-5818-encapsulate-mem-type-checking-and-changing-reviewed
https://github.com/tarantool/tarantool/issues/5011
https://github.com/tarantool/tarantool/issues/5704
https://github.com/tarantool/tarantool/issues/5913
On Tue, Apr 13, 2021 at 03:01:18PM +0300, Mergen Imeev wrote:
> Thank you for the review! I'a sorry to ask you so late, but could you look at
> new version of the patch? It was decided that this patch should be pushed to
> 2.6 and 2.7 along with master. Only this patch of the patch-set.
>
> Diff and new version below. Please note, that I added additional SELECT to
> the first test since currently sql-tap test system does not support binary
> result. Also, in the last two tests I created new table for each since this way
> we could see proper error. If I used space T there then in the last test there
> would be the same error as in the test 5 due to UUID being before DECIMAL in
> space format. To remove dependence of their position, I added two new spaces.
>
> On Sun, Apr 11, 2021 at 07:42:22PM +0200, Vladislav Shpilevoy wrote:
> > Hi! Good job on the patch!
> >
> > On 09.04.2021 18:51, Mergen Imeev via Tarantool-patches wrote:
> > > Hi! Thank you for the review! My answer and new patch below. I didn't include
> > > diffs in answers since due to merge conflicts they are partly useless.
> > >
> > > On 30.03.2021 01:57, Vladislav Shpilevoy wrote:
> > >> Hi! I appreciate the work you did here!
> > >>
> > >> Truly huge patch. But it does the important thing which I think
> > >> should give a huge help in the task of SQL code rework.
> > >>
> > >> On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote:
> > >>> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
> > >>> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
> > >>> MP_ARRAY fields. Also, it doesn't set field_type.
> > >>
> > >> AFAIR, field type wasn't set deliberately. Because we use it only for
> > >> strictly typed values obtained from formatted space fields. It wasn't
> > >> applied to plain non-formatted values on purpose.
> > >>
> > >> Why do you change that?
> > >
> > > I didn't know about that. I thought that all MEMs that contains data should have
> > > field_type set. However, I tried to understand where did this field come from
> > > and found that it was added for two purposes: to show difference between NUMBER
> > > and INTEGER in MEM before DOUBLE was added and to show column name instead of
> > > type determined from MP-type in typeof(). I believe that both these purposes are
> > > not needed now and that this field should be removed from struct MEM. I created
> > > an issue for this: #5957. However, I was prohibited to remove this field for now
> > > by @tsafin, who believes that this field is actually important.
> >
> > I tend to agree, that it must be done separately. Although probably could be done
> > before this patchset to make it kind of simpler. Anyway, thanks for the
> > explanation.
>
>
> Diff:
>
>
> diff --git a/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
> new file mode 100755
> index 000000000..60978c2b5
> --- /dev/null
> +++ b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
> @@ -0,0 +1,99 @@
> +#!/usr/bin/env tarantool
> +local test = require("sqltester")
> +test:plan(6)
> +
> +local uuid = require("uuid").fromstr("11111111-1111-1111-1111-111111111111")
> +local decimal = require("decimal").new(111.111)
> +
> +box.schema.create_space('T')
> +box.space.T:format({{name = "I", type = "integer"}, {name = "U", type = "uuid"},
> + {name = "D", type = "decimal"}})
> +box.space.T:create_index("primary")
> +box.space.T:insert({1, uuid, decimal})
> +
> +--
> +-- Make sure that there is no segmentation fault on select from field that
> +-- contains UUID or DECIMAL. Currently SQL does not support UUID and DECIMAL,
> +-- so they treated as VARBINARY.
> +--
> +test:do_execsql_test(
> + "gh-5913-1",
> + [[
> + SELECT i, u, d FROM t;
> + SELECT i from t;
> + ]], {
> + 1
> + })
> +
> +box.schema.create_space('T1')
> +box.space.T1:format({{name = "I", type = "integer"},
> + {name = "U", type = "uuid", is_nullable = true},
> + {name = "D", type = "decimal", is_nullable = true}})
> +box.space.T1:create_index("primary")
> +
> +--
> +-- Since SQL does not support UUID and DECIMAL and they treated as VARBINARY,
> +-- they cannot be inserted from SQL.
> +--
> +test:do_catchsql_test(
> + "gh-5913-2",
> + [[
> + INSERT INTO t1 SELECT i, u, NULL FROM t;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to uuid"
> + })
> +
> +test:do_catchsql_test(
> + "gh-5913-3",
> + [[
> + INSERT INTO t1 SELECT i, NULL, d FROM t;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to decimal"
> + })
> +
> +--
> +-- Still, if UUID or DECIMAL fields does not selected directly, insert is
> +-- working properly.
> +--
> +test:do_execsql_test(
> + "gh-5913-4",
> + [[
> + INSERT INTO t1 SELECT * FROM t;
> + SELECT count() FROM t1;
> + ]], {
> + 1
> + })
> +
> +box.schema.create_space('TU')
> +box.space.TU:format({{name = "I", type = "integer"},
> + {name = "U", type = "uuid"}})
> +box.space.TU:create_index("primary")
> +box.space.TU:insert({1, uuid})
> +
> +box.schema.create_space('TD')
> +box.space.TD:format({{name = "I", type = "integer"},
> + {name = "D", type = "decimal"}})
> +box.space.TD:create_index("primary")
> +box.space.TD:insert({1, decimal})
> +
> +--
> +-- Update of UUID or VARBINARY also does not lead to segfault, however throws an
> +-- error since after changing value cannot be inserted into the field from SQL.
> +--
> +test:do_catchsql_test(
> + "gh-5913-5",
> + [[
> + UPDATE tu SET u = u;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to uuid"
> + })
> +
> +test:do_catchsql_test(
> + "gh-5913-6",
> + [[
> + UPDATE td SET d = d;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to decimal"
> + })
> +
> +test:finish_test()
>
>
>
> New patch:
>
>
> commit d41d8f756f8ab6bfae70ee5787bc4c509ddb844d
> Author: Mergen Imeev <imeevma@gmail.com>
> Date: Thu Mar 4 17:17:18 2021 +0300
>
> sql: enhance vdbe_decode_msgpack_into_mem()
>
> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
> MP_ARRAY fields. This patch makes it so that after execution of this
> function, all created MEMs are properly initialized.
>
> Closes #5011
> Closes #5704
> Closes #5913
> Needed for #5818
>
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 3b3b1f01d..9a4f38bb9 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -846,16 +846,6 @@ vdbe_field_ref_fetch_data(struct vdbe_field_ref *field_ref, uint32_t fieldno)
> return field_begin;
> }
>
> -static inline enum field_type
> -vdbe_field_ref_fetch_type(struct vdbe_field_ref *field_ref, uint32_t fieldno)
> -{
> - const struct tuple_field *tf =
> - vdbe_field_ref_fetch_field(field_ref, fieldno);
> - if (tf == NULL || tf->type == FIELD_TYPE_ANY)
> - return field_type_MAX;
> - return tf->type;
> -}
> -
> /**
> * Fetch field by fieldno using vdbe_field_ref and store result
> * in dest_mem.
> @@ -879,17 +869,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
> if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
> return -1;
>
> - /*
> - * MsgPack map, array or extension (unsupported in sql).
> - * Wrap it in a blob verbatim.
> - */
> - if (dest_mem->flags == 0) {
> - dest_mem->z = (char *) data;
> - dest_mem->n = vdbe_field_ref_fetch_data(field_ref,
> - fieldno + 1) - data;
> - dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
> - dest_mem->subtype = SQL_SUBTYPE_MSGPACK;
> - }
> /*
> * Add 0 termination (at most for strings)
> * Not sure why do we check MEM_Ephem
> @@ -909,7 +888,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
> dest_mem->flags |= MEM_Term;
> }
> UPDATE_MAX_BLOBSIZE(dest_mem);
> - dest_mem->field_type = vdbe_field_ref_fetch_type(field_ref, fieldno);
> return 0;
> }
>
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 91b64316e..772476377 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -2793,38 +2793,62 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
> {
> const char *start_buf = buf;
> switch (mp_typeof(*buf)) {
> - case MP_ARRAY:
> - case MP_MAP:
> - case MP_EXT:
> - default: {
> - mem->flags = 0;
> + case MP_ARRAY: {
> + mem->z = (char *)buf;
> + mp_next(&buf);
> + mem->n = buf - mem->z;
> + mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
> + mem->subtype = SQL_SUBTYPE_MSGPACK;
> + mem->field_type = FIELD_TYPE_ARRAY;
> + break;
> + }
> + case MP_MAP: {
> + mem->z = (char *)buf;
> + mp_next(&buf);
> + mem->n = buf - mem->z;
> + mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
> + mem->subtype = SQL_SUBTYPE_MSGPACK;
> + mem->field_type = FIELD_TYPE_MAP;
> + break;
> + }
> + case MP_EXT: {
> + mem->z = (char *)buf;
> + mp_next(&buf);
> + mem->n = buf - mem->z;
> + mem->flags = MEM_Blob | MEM_Ephem;
> + mem->field_type = FIELD_TYPE_VARBINARY;
> break;
> }
> case MP_NIL: {
> mp_decode_nil(&buf);
> mem->flags = MEM_Null;
> + mem->field_type = field_type_MAX;
> break;
> }
> case MP_BOOL: {
> mem->u.b = mp_decode_bool(&buf);
> mem->flags = MEM_Bool;
> + mem->field_type = FIELD_TYPE_BOOLEAN;
> break;
> }
> case MP_UINT: {
> uint64_t v = mp_decode_uint(&buf);
> mem->u.u = v;
> mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_INTEGER;
> break;
> }
> case MP_INT: {
> mem->u.i = mp_decode_int(&buf);
> mem->flags = MEM_Int;
> + mem->field_type = FIELD_TYPE_INTEGER;
> break;
> }
> case MP_STR: {
> /* XXX u32->int */
> mem->n = (int) mp_decode_strl(&buf);
> mem->flags = MEM_Str | MEM_Ephem;
> + mem->field_type = FIELD_TYPE_STRING;
> install_blob:
> mem->z = (char *)buf;
> buf += mem->n;
> @@ -2834,18 +2858,33 @@ install_blob:
> /* XXX u32->int */
> mem->n = (int) mp_decode_binl(&buf);
> mem->flags = MEM_Blob | MEM_Ephem;
> + mem->field_type = FIELD_TYPE_VARBINARY;
> goto install_blob;
> }
> case MP_FLOAT: {
> mem->u.r = mp_decode_float(&buf);
> - mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
> + if (sqlIsNaN(mem->u.r)) {
> + mem->flags = MEM_Null;
> + mem->field_type = FIELD_TYPE_DOUBLE;
> + } else {
> + mem->flags = MEM_Real;
> + mem->field_type = FIELD_TYPE_DOUBLE;
> + }
> break;
> }
> case MP_DOUBLE: {
> mem->u.r = mp_decode_double(&buf);
> - mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
> + if (sqlIsNaN(mem->u.r)) {
> + mem->flags = MEM_Null;
> + mem->field_type = FIELD_TYPE_DOUBLE;
> + } else {
> + mem->flags = MEM_Real;
> + mem->field_type = FIELD_TYPE_DOUBLE;
> + }
> break;
> }
> + default:
> + unreachable();
> }
> *len = (uint32_t)(buf - start_buf);
> return 0;
> @@ -2868,15 +2907,8 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec
> pMem->z = 0;
> uint32_t sz = 0;
> vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
> - if (sz == 0) {
> - /* MsgPack array, map or ext. Treat as blob. */
> - pMem->z = (char *)zParse;
> - mp_next(&zParse);
> - pMem->n = zParse - pMem->z;
> - pMem->flags = MEM_Blob | MEM_Ephem;
> - } else {
> - zParse += sz;
> - }
> + assert(sz != 0);
> + zParse += sz;
> pMem++;
> }
> }
> diff --git a/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
> new file mode 100755
> index 000000000..60978c2b5
> --- /dev/null
> +++ b/test/sql-tap/gh-5913-segfault-on-select-uuid.test.lua
> @@ -0,0 +1,99 @@
> +#!/usr/bin/env tarantool
> +local test = require("sqltester")
> +test:plan(6)
> +
> +local uuid = require("uuid").fromstr("11111111-1111-1111-1111-111111111111")
> +local decimal = require("decimal").new(111.111)
> +
> +box.schema.create_space('T')
> +box.space.T:format({{name = "I", type = "integer"}, {name = "U", type = "uuid"},
> + {name = "D", type = "decimal"}})
> +box.space.T:create_index("primary")
> +box.space.T:insert({1, uuid, decimal})
> +
> +--
> +-- Make sure that there is no segmentation fault on select from field that
> +-- contains UUID or DECIMAL. Currently SQL does not support UUID and DECIMAL,
> +-- so they treated as VARBINARY.
> +--
> +test:do_execsql_test(
> + "gh-5913-1",
> + [[
> + SELECT i, u, d FROM t;
> + SELECT i from t;
> + ]], {
> + 1
> + })
> +
> +box.schema.create_space('T1')
> +box.space.T1:format({{name = "I", type = "integer"},
> + {name = "U", type = "uuid", is_nullable = true},
> + {name = "D", type = "decimal", is_nullable = true}})
> +box.space.T1:create_index("primary")
> +
> +--
> +-- Since SQL does not support UUID and DECIMAL and they treated as VARBINARY,
> +-- they cannot be inserted from SQL.
> +--
> +test:do_catchsql_test(
> + "gh-5913-2",
> + [[
> + INSERT INTO t1 SELECT i, u, NULL FROM t;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to uuid"
> + })
> +
> +test:do_catchsql_test(
> + "gh-5913-3",
> + [[
> + INSERT INTO t1 SELECT i, NULL, d FROM t;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to decimal"
> + })
> +
> +--
> +-- Still, if UUID or DECIMAL fields does not selected directly, insert is
> +-- working properly.
> +--
> +test:do_execsql_test(
> + "gh-5913-4",
> + [[
> + INSERT INTO t1 SELECT * FROM t;
> + SELECT count() FROM t1;
> + ]], {
> + 1
> + })
> +
> +box.schema.create_space('TU')
> +box.space.TU:format({{name = "I", type = "integer"},
> + {name = "U", type = "uuid"}})
> +box.space.TU:create_index("primary")
> +box.space.TU:insert({1, uuid})
> +
> +box.schema.create_space('TD')
> +box.space.TD:format({{name = "I", type = "integer"},
> + {name = "D", type = "decimal"}})
> +box.space.TD:create_index("primary")
> +box.space.TD:insert({1, decimal})
> +
> +--
> +-- Update of UUID or VARBINARY also does not lead to segfault, however throws an
> +-- error since after changing value cannot be inserted into the field from SQL.
> +--
> +test:do_catchsql_test(
> + "gh-5913-5",
> + [[
> + UPDATE tu SET u = u;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to uuid"
> + })
> +
> +test:do_catchsql_test(
> + "gh-5913-6",
> + [[
> + UPDATE td SET d = d;
> + ]], {
> + 1, "Type mismatch: can not convert varbinary to decimal"
> + })
> +
> +test:finish_test()
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches
2021-04-13 12:12 ` Mergen Imeev via Tarantool-patches
@ 2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 23:34 ` Mergen Imeev via Tarantool-patches
1 sibling, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-13 23:22 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Good job on the patch!
On 13.04.2021 14:01, Mergen Imeev wrote:
> Thank you for the review! I'a sorry to ask you so late, but could you look at
> new version of the patch? It was decided that this patch should be pushed to
> 2.6 and 2.7 along with master. Only this patch of the patch-set.
I think the entire patchset must be pushed to all versions. Otherwise you are
going to have hard time to cherry-pick anything from the master branch.
Additionally, what about
sql: rework vdbe_decode_msgpack_into_mem()
? It also fixes a bug.
On the branch I see the entire patchset, not just this commit. Is it
intended?
This commit LGTM.
^ permalink raw reply [flat|nested] 107+ messages in thread
* Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()
2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 23:34 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 23:34 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review!
On Wed, Apr 14, 2021 at 01:22:34AM +0200, Vladislav Shpilevoy wrote:
> Good job on the patch!
>
> On 13.04.2021 14:01, Mergen Imeev wrote:
> > Thank you for the review! I'a sorry to ask you so late, but could you look at
> > new version of the patch? It was decided that this patch should be pushed to
> > 2.6 and 2.7 along with master. Only this patch of the patch-set.
>
> I think the entire patchset must be pushed to all versions. Otherwise you are
> going to have hard time to cherry-pick anything from the master branch.
>
I will ask about this tomorrow.
> Additionally, what about
>
> sql: rework vdbe_decode_msgpack_into_mem()
>
> ? It also fixes a bug.
>
True. I knew that it fixes a bug, but didn't thouth about the fact, that it
possibly won't be pushed to 2.6 and 2.7. For now I do not know what to do about
this patch. I will ask about this also.
> On the branch I see the entire patchset, not just this commit. Is it
> intended?
>
Yes, it is. I did not split this commit from patch-set, for now.
>
> This commit LGTM.
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 02/52] sql: disable unused code in sql/analyze.c
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 03/52] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches
` (39 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:51 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch disables unused code in sql/analyze.c. It will simplify
refactoring.
Needed for #5818
---
src/box/sql.c | 95 +++++++++++++++++++++++++++++++++++--------
src/box/sql.h | 8 ----
src/box/sql/analyze.c | 81 ++----------------------------------
src/box/sql/vdbe.c | 3 ++
4 files changed, 84 insertions(+), 103 deletions(-)
diff --git a/src/box/sql.c b/src/box/sql.c
index 3b53abcdb..aa91f003f 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -78,24 +78,6 @@ sql_init(void)
assert(db != NULL);
}
-void
-sql_load_schema(void)
-{
- assert(db->init.busy == 0);
- struct space *stat = space_by_name("_sql_stat1");
- assert(stat != NULL);
- if (stat->def->field_count == 0)
- return;
- db->init.busy = 1;
- if (sql_analysis_load(db) != 0) {
- if(!diag_is_empty(&fiber()->diag)) {
- diag_log();
- }
- panic("failed to initialize SQL subsystem");
- }
- db->init.busy = 0;
-}
-
sql *
sql_get(void)
{
@@ -1317,3 +1299,80 @@ vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref,
vdbe_field_ref_create(field_ref, tuple, tuple_data(tuple),
tuple->bsize);
}
+
+ssize_t
+sql_index_tuple_size(struct space *space, struct index *idx)
+{
+ assert(space != NULL);
+ assert(idx != NULL);
+ assert(idx->def->space_id == space->def->id);
+ ssize_t tuple_count = index_size(idx);
+ ssize_t space_size = space_bsize(space);
+ ssize_t avg_tuple_size = tuple_count != 0 ?
+ (space_size / tuple_count) : 0;
+ return avg_tuple_size;
+}
+
+/**
+ * default_tuple_est[] array contains default information
+ * which is used when we don't have real space, e.g. temporary
+ * objects representing result set of nested SELECT or VIEW.
+ *
+ * First number is supposed to contain the number of elements
+ * in the index. Since we do not know, guess 1 million.
+ * Second one is an estimate of the number of rows in the
+ * table that match any particular value of the first column of
+ * the index. Third one is an estimate of the number of
+ * rows that match any particular combination of the first 2
+ * columns of the index. And so on. It must always be true:
+ *
+ * default_tuple_est[N] <= default_tuple_est[N-1]
+ * default_tuple_est[N] >= 1
+ *
+ * Apart from that, we have little to go on besides intuition
+ * as to how default values should be initialized. The numbers
+ * generated here are based on typical values found in actual
+ * indices.
+ */
+const log_est_t default_tuple_est[] = {DEFAULT_TUPLE_LOG_COUNT,
+/** [10*log_{2}(x)]: 10, 9, 8, 7, 6, 5 */
+ 33, 32, 30, 28, 26, 23};
+
+LogEst
+sql_space_tuple_log_count(struct space *space)
+{
+ if (space == NULL || space->index_map == NULL)
+ return 0;
+
+ struct index *pk = space_index(space, 0);
+ assert(sqlLogEst(DEFAULT_TUPLE_COUNT) == DEFAULT_TUPLE_LOG_COUNT);
+ /* If space represents VIEW, return default number. */
+ if (pk == NULL)
+ return DEFAULT_TUPLE_LOG_COUNT;
+ return sqlLogEst(pk->vtab->size(pk));
+}
+
+log_est_t
+index_field_tuple_est(const struct index_def *idx_def, uint32_t field)
+{
+ assert(idx_def != NULL);
+ struct space *space = space_by_id(idx_def->space_id);
+ if (space == NULL || strcmp(idx_def->name, "fake_autoindex") == 0)
+ return idx_def->opts.stat->tuple_log_est[field];
+ assert(field <= idx_def->key_def->part_count);
+ /* Statistics is held only in real indexes. */
+ struct index *tnt_idx = space_index(space, idx_def->iid);
+ assert(tnt_idx != NULL);
+ if (tnt_idx->def->opts.stat == NULL) {
+ /*
+ * Last number for unique index is always 0:
+ * only one tuple exists with given full key
+ * in unique index and log(1) == 0.
+ */
+ if (field == idx_def->key_def->part_count &&
+ idx_def->opts.is_unique)
+ return 0;
+ return default_tuple_est[field + 1 >= 6 ? 6 : field];
+ }
+ return tnt_idx->def->opts.stat->tuple_log_est[field];
+}
diff --git a/src/box/sql.h b/src/box/sql.h
index f56f7a1f1..4c364306c 100644
--- a/src/box/sql.h
+++ b/src/box/sql.h
@@ -41,14 +41,6 @@ extern "C" {
void
sql_init(void);
-/**
- * Initialize SQL statistic system.
- *
- * Currently unused.
- */
-void
-sql_load_schema(void);
-
/**
* struct sql *
* sql_get();
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index f74f9b358..a015f70cb 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -116,6 +116,8 @@
#include "tarantoolInt.h"
#include "vdbeInt.h"
+#if 0
+
/**
* This routine generates code that opens the sql_stat1/4 tables.
* If the sql_statN tables do not previously exist, they are
@@ -1114,19 +1116,6 @@ sqlAnalyze(Parse * pParse, Token * pName)
sqlVdbeAddOp0(v, OP_Expire);
}
-ssize_t
-sql_index_tuple_size(struct space *space, struct index *idx)
-{
- assert(space != NULL);
- assert(idx != NULL);
- assert(idx->def->space_id == space->def->id);
- ssize_t tuple_count = index_size(idx);
- ssize_t space_size = space_bsize(space);
- ssize_t avg_tuple_size = tuple_count != 0 ?
- (space_size / tuple_count) : 0;
- return avg_tuple_size;
-}
-
/**
* Used to pass information from the analyzer reader through
* to the callback routine.
@@ -1538,70 +1527,6 @@ load_stat_to_index(const char *sql_select_load, struct index_stat **stats)
return 0;
}
-/**
- * default_tuple_est[] array contains default information
- * which is used when we don't have real space, e.g. temporary
- * objects representing result set of nested SELECT or VIEW.
- *
- * First number is supposed to contain the number of elements
- * in the index. Since we do not know, guess 1 million.
- * Second one is an estimate of the number of rows in the
- * table that match any particular value of the first column of
- * the index. Third one is an estimate of the number of
- * rows that match any particular combination of the first 2
- * columns of the index. And so on. It must always be true:
- *
- * default_tuple_est[N] <= default_tuple_est[N-1]
- * default_tuple_est[N] >= 1
- *
- * Apart from that, we have little to go on besides intuition
- * as to how default values should be initialized. The numbers
- * generated here are based on typical values found in actual
- * indices.
- */
-const log_est_t default_tuple_est[] = {DEFAULT_TUPLE_LOG_COUNT,
-/** [10*log_{2}(x)]: 10, 9, 8, 7, 6, 5 */
- 33, 32, 30, 28, 26, 23};
-
-LogEst
-sql_space_tuple_log_count(struct space *space)
-{
- if (space == NULL || space->index_map == NULL)
- return 0;
-
- struct index *pk = space_index(space, 0);
- assert(sqlLogEst(DEFAULT_TUPLE_COUNT) == DEFAULT_TUPLE_LOG_COUNT);
- /* If space represents VIEW, return default number. */
- if (pk == NULL)
- return DEFAULT_TUPLE_LOG_COUNT;
- return sqlLogEst(pk->vtab->size(pk));
-}
-
-log_est_t
-index_field_tuple_est(const struct index_def *idx_def, uint32_t field)
-{
- assert(idx_def != NULL);
- struct space *space = space_by_id(idx_def->space_id);
- if (space == NULL || strcmp(idx_def->name, "fake_autoindex") == 0)
- return idx_def->opts.stat->tuple_log_est[field];
- assert(field <= idx_def->key_def->part_count);
- /* Statistics is held only in real indexes. */
- struct index *tnt_idx = space_index(space, idx_def->iid);
- assert(tnt_idx != NULL);
- if (tnt_idx->def->opts.stat == NULL) {
- /*
- * Last number for unique index is always 0:
- * only one tuple exists with given full key
- * in unique index and log(1) == 0.
- */
- if (field == idx_def->key_def->part_count &&
- idx_def->opts.is_unique)
- return 0;
- return default_tuple_est[field + 1 >= 6 ? 6 : field];
- }
- return tnt_idx->def->opts.stat->tuple_log_est[field];
-}
-
/**
* This function performs copy of statistics.
* In contrast to index_stat_dup(), there is no assumption
@@ -1744,3 +1669,5 @@ fail:
box_txn_rollback();
return -1;
}
+
+#endif
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9a4f38bb9..4c1cd582b 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4825,8 +4825,11 @@ case OP_RenameTable: {
*/
case OP_LoadAnalysis: {
assert(pOp->p1==0 );
+ /* TODO: Enable analysis. */
+ /*
if (sql_analysis_load(db) != 0)
goto abort_due_to_error;
+ */
break;
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 03/52] sql: disable unused code in sql/legacy.c
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 02/52] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
` (38 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:51 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch disables unused code in sql/legacy.c. It will simplify
refactoring.
Needed for #5818
---
src/box/sql/legacy.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/box/sql/legacy.c b/src/box/sql/legacy.c
index 072954357..b1781b427 100644
--- a/src/box/sql/legacy.c
+++ b/src/box/sql/legacy.c
@@ -40,6 +40,8 @@
#include "box/execute.h"
#include "box/session.h"
+#if 0
+
/*
* Execute SQL code. Return one of the SQL_ success/failure
* codes.
@@ -162,3 +164,5 @@ sql_exec(sql * db, /* The database on which the SQL executes */
assert(rc == 0);
return rc;
}
+
+#endif
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (2 preceding siblings ...)
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 03/52] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches
2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 05/52] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (37 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:51 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
There is no need for NULL-termination, since MEMs converted to msgpack
and msgpack do not require NULL-terminated strings.
Needed for #5818
---
src/box/sql/vdbe.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4c1cd582b..18806b93f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1505,9 +1505,6 @@ case OP_SCopy: { /* out2 */
* the result row.
*/
case OP_ResultRow: {
- Mem *pMem;
- int i;
-
assert(p->nResColumn==pOp->p2);
assert(pOp->p1>0);
assert(pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1);
@@ -1516,20 +1513,14 @@ case OP_ResultRow: {
/* Invalidate all ephemeral cursor row caches */
p->cacheCtr = (p->cacheCtr + 2)|1;
- /* Make sure the results of the current row are \000 terminated
- * and have an assigned type. The results are de-ephemeralized as
- * a side effect.
- */
- pMem = p->pResultSet = &aMem[pOp->p1];
- for(i=0; i<pOp->p2; i++) {
+ p->pResultSet = &aMem[pOp->p1];
+#ifdef SQL_DEBUG
+ struct Mem *pMem = p->pResultSet;
+ for(int i = 0; i < pOp->p2; i++) {
assert(memIsValid(&pMem[i]));
- Deephemeralize(&pMem[i]);
- assert((pMem[i].flags & MEM_Ephem)==0
- || (pMem[i].flags & (MEM_Str|MEM_Blob))==0);
- sqlVdbeMemNulTerminate(&pMem[i]);
REGISTER_TRACE(p, pOp->p1+i, &pMem[i]);
}
- if (db->mallocFailed) goto no_mem;
+#endif
if (db->mTrace & SQL_TRACE_ROW) {
db->xTrace(SQL_TRACE_ROW, db->pTraceArg, p, 0);
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
@ 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 22:37 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 22:23 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for the patch!
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 4c1cd582b..18806b93f 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1516,20 +1513,14 @@ case OP_ResultRow: {
> /* Invalidate all ephemeral cursor row caches */
> p->cacheCtr = (p->cacheCtr + 2)|1;
>
> - /* Make sure the results of the current row are \000 terminated
> - * and have an assigned type. The results are de-ephemeralized as
> - * a side effect.
> - */
> - pMem = p->pResultSet = &aMem[pOp->p1];
> - for(i=0; i<pOp->p2; i++) {
> + p->pResultSet = &aMem[pOp->p1];
> +#ifdef SQL_DEBUG
> + struct Mem *pMem = p->pResultSet;
> + for(int i = 0; i < pOp->p2; i++) {
Please, add a whitespace after 'for'.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow
2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 22:37 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 22:37 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review. My answer below.
On Thu, Apr 15, 2021 at 12:23:07AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > index 4c1cd582b..18806b93f 100644
> > --- a/src/box/sql/vdbe.c
> > +++ b/src/box/sql/vdbe.c
> > @@ -1516,20 +1513,14 @@ case OP_ResultRow: {
> > /* Invalidate all ephemeral cursor row caches */
> > p->cacheCtr = (p->cacheCtr + 2)|1;
> >
> > - /* Make sure the results of the current row are \000 terminated
> > - * and have an assigned type. The results are de-ephemeralized as
> > - * a side effect.
> > - */
> > - pMem = p->pResultSet = &aMem[pOp->p1];
> > - for(i=0; i<pOp->p2; i++) {
> > + p->pResultSet = &aMem[pOp->p1];
> > +#ifdef SQL_DEBUG
> > + struct Mem *pMem = p->pResultSet;
> > + for(int i = 0; i < pOp->p2; i++) {
>
> Please, add a whitespace after 'for'.
Fixed:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 18806b93f..acbf72078 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1516,7 +1516,7 @@ case OP_ResultRow: {
p->pResultSet = &aMem[pOp->p1];
#ifdef SQL_DEBUG
struct Mem *pMem = p->pResultSet;
- for(int i = 0; i < pOp->p2; i++) {
+ for (int i = 0; i < pOp->p2; i++) {
assert(memIsValid(&pMem[i]));
REGISTER_TRACE(p, pOp->p1+i, &pMem[i]);
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 05/52] sql: move MEM-related functions to mem.c/mem.h
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (3 preceding siblings ...)
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 06/52] sql: refactor port_vdbemem_*() functions Mergen Imeev via Tarantool-patches
` (36 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:51 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thaks you for the review! My answers and new patch below.
On 30.03.2021 01:58, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> I didn't compare each line of each moved function with its
> original code, so I trust you didn't make any functional
> changes.
>
> See 2 comments below.
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> new file mode 100644
>> index 000000000..62338e1db
>> --- /dev/null
>> +++ b/src/box/sql/mem.c
>> @@ -0,0 +1,2376 @@
>
> <...>
>
>> +
>> +
>> +#ifdef SQL_DEBUG
>> +/*
>> + * This routine prepares a memory cell for modification by breaking
>> + * its link to a shallow copy and by marking any current shallow
>> + * copies of this cell as invalid.
>> + *
>> + * This is used for testing and debugging only - to make sure shallow
>> + * copies are not misused.
>> + */
>> +void
>> +sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
>
> 1. This is a vdbe method, not mem. It takes mem as an argument, but
> is a method of vdbe struct. Lets keep it in the old place.
>
Fixed. Moved it back and in patch "sql: disable unused code in sql/vdbemem.c"
moved it to vdbe.c.
>> +{
>> + int i;
>> + Mem *pX;
>> + for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
>> + if (pX->pScopyFrom == pMem) {
>> + pX->flags |= MEM_Undefined;
>> + pX->pScopyFrom = 0;
>> + }
>> + }
>> + pMem->pScopyFrom = 0;
>> +}
>
> <...>
>
>> +
>> +/*
>> + * Print the SQL that was used to generate a VDBE program.
>> + */
>> +void
>> +sqlVdbePrintSql(Vdbe * p)
>
> 2. Ditto. Such functions should be moved to vdbe.h/vdbe.c eventually.
> And ideally mem.h/mem.c should not know about vdbe anything (probably
> can't be done now).
>
Fixed. Moved it back to vdbe.c. I think mem.c need vdbeInt.h and sqlInt.h right
now, so we can't make it forget about struct Vdbe. I hope this will change in
future.
>> +{
>> + const char *z = 0;
>> + if (p->zSql) {
>> + z = p->zSql;
>> + } else if (p->nOp >= 1) {
>> + const VdbeOp *pOp = &p->aOp[0];
>> + if (pOp->opcode == OP_Init && pOp->p4.z != 0) {
>> + z = pOp->p4.z;
>> + while (sqlIsspace(*z))
>> + z++;
>> + }
>> + }
>> + if (z)
>> + printf("SQL: [%s]\n", z);
>> +}
New patch:
commit 16b3a31eb6de710a3512d38a40ca8fd52bb50bca
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 1 18:13:06 2021 +0300
sql: move MEM-related functions to mem.c/mem.h
This patch moves all MEM-related functions in mem.c/mem.h.
Part of #5818
diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 19203f770..41c32468f 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -42,6 +42,7 @@ set(sql_sources
sql/legacy.c
sql/main.c
sql/malloc.c
+ sql/mem.c
sql/os.c
sql/os_unix.c
sql/parse_def.c
diff --git a/src/box/sql.c b/src/box/sql.c
index aa91f003f..790ca7f70 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -34,6 +34,7 @@
#include "sql.h"
#include "sql/sqlInt.h"
#include "sql/tarantoolInt.h"
+#include "sql/mem.h"
#include "sql/vdbeInt.h"
#include "index.h"
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index a015f70cb..5a95caa39 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -114,6 +114,7 @@
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#if 0
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 521cc2622..6470e0300 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -45,6 +45,7 @@
*/
#include <ctype.h>
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "tarantoolInt.h"
#include "box/ck_constraint.h"
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 9b6179f3a..46814f341 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -35,6 +35,7 @@
* time functions, are implemented separately.)
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "version.h"
#include "coll/coll.h"
@@ -46,12 +47,6 @@
#include <unicode/ucol.h>
#include "box/coll_id_cache.h"
#include "box/schema.h"
-#include "box/func.h"
-#include "box/port.h"
-#include "box/tuple.h"
-#include "lua/msgpack.h"
-#include "lua/utils.h"
-#include "mpstream/mpstream.h"
/*
* Return the collating function associated with a function.
@@ -77,308 +72,6 @@ sqlSkipAccumulatorLoad(sql_context * context)
context->skipFlag = 1;
}
-/**
- * Allocate a sequence of initialized vdbe memory registers
- * on region.
- */
-static struct Mem *
-vdbemem_alloc_on_region(uint32_t count)
-{
- struct region *region = &fiber()->gc;
- size_t size;
- struct Mem *ret = region_alloc_array(region, typeof(*ret), count,
- &size);
- if (ret == NULL) {
- diag_set(OutOfMemory, size, "region_alloc_array", "ret");
- return NULL;
- }
- memset(ret, 0, count * sizeof(*ret));
- for (uint32_t i = 0; i < count; i++) {
- sqlVdbeMemInit(&ret[i], sql_get(), MEM_Null);
- assert(memIsValid(&ret[i]));
- }
- return ret;
-}
-
-static void
-port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
-{
- (void) is_flat;
- struct port_vdbemem *port = (struct port_vdbemem *) base;
- assert(is_flat == true);
- for (uint32_t i = 0; i < port->mem_count; i++) {
- sql_value *param =
- (sql_value *)((struct Mem *)port->mem + i);
- switch (sql_value_type(param)) {
- case MP_INT:
- luaL_pushint64(L, sql_value_int64(param));
- break;
- case MP_UINT:
- luaL_pushuint64(L, sql_value_uint64(param));
- break;
- case MP_DOUBLE:
- lua_pushnumber(L, sql_value_double(param));
- break;
- case MP_STR:
- lua_pushlstring(L, (const char *)sql_value_text(param),
- (size_t)sql_value_bytes(param));
- break;
- case MP_BIN:
- case MP_ARRAY:
- case MP_MAP:
- lua_pushlstring(L, sql_value_blob(param),
- (size_t) sql_value_bytes(param));
- break;
- case MP_NIL:
- lua_pushnil(L);
- break;
- case MP_BOOL:
- lua_pushboolean(L, sql_value_boolean(param));
- break;
- default:
- unreachable();
- }
- }
-}
-
-static const char *
-port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
-{
- struct port_vdbemem *port = (struct port_vdbemem *) base;
- struct region *region = &fiber()->gc;
- size_t region_svp = region_used(region);
- bool is_error = false;
- struct mpstream stream;
- mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
- set_encode_error, &is_error);
- mpstream_encode_array(&stream, port->mem_count);
- for (uint32_t i = 0; i < port->mem_count && !is_error; i++) {
- sql_value *param =
- (sql_value *)((struct Mem *)port->mem + i);
- switch (sql_value_type(param)) {
- case MP_INT: {
- sql_int64 val = sql_value_int64(param);
- if (val < 0) {
- mpstream_encode_int(&stream, val);
- break;
- }
- FALLTHROUGH;
- }
- case MP_UINT: {
- sql_uint64 val = sql_value_uint64(param);
- mpstream_encode_uint(&stream, val);
- break;
- }
- case MP_DOUBLE: {
- mpstream_encode_double(&stream,
- sql_value_double(param));
- break;
- }
- case MP_STR: {
- const char *str = (const char *) sql_value_text(param);
- mpstream_encode_strn(&stream, str,
- sql_value_bytes(param));
- break;
- }
- case MP_BIN:
- case MP_ARRAY:
- case MP_MAP: {
- mpstream_encode_binl(&stream, sql_value_bytes(param));
- mpstream_memcpy(&stream, sql_value_blob(param),
- sql_value_bytes(param));
- break;
- }
- case MP_NIL: {
- mpstream_encode_nil(&stream);
- break;
- }
- case MP_BOOL: {
- mpstream_encode_bool(&stream, sql_value_boolean(param));
- break;
- }
- default:
- unreachable();
- }
- }
- mpstream_flush(&stream);
- *size = region_used(region) - region_svp;
- if (is_error)
- goto error;
- const char *ret = (char *)region_join(region, *size);
- if (ret == NULL)
- goto error;
- return ret;
-error:
- diag_set(OutOfMemory, *size, "region", "ret");
- return NULL;
-}
-
-static const struct port_vtab port_vdbemem_vtab;
-
-void
-port_vdbemem_create(struct port *base, struct sql_value *mem,
- uint32_t mem_count)
-{
- struct port_vdbemem *port = (struct port_vdbemem *) base;
- port->vtab = &port_vdbemem_vtab;
- port->mem = mem;
- port->mem_count = mem_count;
-}
-
-static struct sql_value *
-port_vdbemem_get_vdbemem(struct port *base, uint32_t *mem_count)
-{
- struct port_vdbemem *port = (struct port_vdbemem *) base;
- assert(port->vtab == &port_vdbemem_vtab);
- *mem_count = port->mem_count;
- return port->mem;
-}
-
-static const struct port_vtab port_vdbemem_vtab = {
- .dump_msgpack = NULL,
- .dump_msgpack_16 = NULL,
- .dump_lua = port_vdbemem_dump_lua,
- .dump_plain = NULL,
- .get_msgpack = port_vdbemem_get_msgpack,
- .get_vdbemem = port_vdbemem_get_vdbemem,
- .destroy = NULL,
-};
-
-struct sql_value *
-port_lua_get_vdbemem(struct port *base, uint32_t *size)
-{
- struct port_lua *port = (struct port_lua *) base;
- struct lua_State *L = port->L;
- int argc = lua_gettop(L);
- if (argc == 0 || argc > 1) {
- diag_set(ClientError, ER_SQL_FUNC_WRONG_RET_COUNT, "Lua", argc);
- return NULL;
- }
- *size = argc;
- /** FIXME: Implement an ability to return a vector. */
- assert(*size == 1);
- struct region *region = &fiber()->gc;
- size_t region_svp = region_used(region);
- struct Mem *val = vdbemem_alloc_on_region(argc);
- if (val == NULL)
- return NULL;
- for (int i = 0; i < argc; i++) {
- struct luaL_field field;
- if (luaL_tofield(L, luaL_msgpack_default,
- NULL, -1 - i, &field) < 0) {
- goto error;
- }
- switch (field.type) {
- case MP_BOOL:
- mem_set_bool(&val[i], field.bval);
- break;
- case MP_FLOAT:
- mem_set_double(&val[i], field.fval);
- break;
- case MP_DOUBLE:
- mem_set_double(&val[i], field.dval);
- break;
- case MP_INT:
- mem_set_i64(&val[i], field.ival);
- break;
- case MP_UINT:
- mem_set_u64(&val[i], field.ival);
- break;
- case MP_STR:
- if (sqlVdbeMemSetStr(&val[i], field.sval.data,
- field.sval.len, 1,
- SQL_TRANSIENT) != 0)
- goto error;
- break;
- case MP_NIL:
- sqlVdbeMemSetNull(&val[i]);
- break;
- default:
- diag_set(ClientError, ER_SQL_EXECUTE,
- "Unsupported type passed from Lua");
- goto error;
- }
- }
- return (struct sql_value *)val;
-error:
- for (int i = 0; i < argc; i++)
- sqlVdbeMemRelease(&val[i]);
- region_truncate(region, region_svp);
- return NULL;
-}
-
-struct sql_value *
-port_c_get_vdbemem(struct port *base, uint32_t *size)
-{
- struct port_c *port = (struct port_c *)base;
- *size = port->size;
- if (*size == 0 || *size > 1) {
- diag_set(ClientError, ER_SQL_FUNC_WRONG_RET_COUNT, "C", *size);
- return NULL;
- }
- /** FIXME: Implement an ability to return a vector. */
- assert(*size == 1);
- struct region *region = &fiber()->gc;
- size_t region_svp = region_used(region);
- struct Mem *val = vdbemem_alloc_on_region(port->size);
- if (val == NULL)
- return NULL;
- int i = 0;
- const char *data;
- struct port_c_entry *pe;
- for (pe = port->first; pe != NULL; pe = pe->next) {
- if (pe->mp_size == 0) {
- data = tuple_data(pe->tuple);
- if (mp_decode_array(&data) != 1) {
- diag_set(ClientError, ER_SQL_EXECUTE,
- "Unsupported type passed from C");
- goto error;
- }
- } else {
- data = pe->mp;
- }
- uint32_t len;
- const char *str;
- switch (mp_typeof(*data)) {
- case MP_BOOL:
- mem_set_bool(&val[i], mp_decode_bool(&data));
- break;
- case MP_FLOAT:
- mem_set_double(&val[i], mp_decode_float(&data));
- break;
- case MP_DOUBLE:
- mem_set_double(&val[i], mp_decode_double(&data));
- break;
- case MP_INT:
- mem_set_i64(&val[i], mp_decode_int(&data));
- break;
- case MP_UINT:
- mem_set_u64(&val[i], mp_decode_uint(&data));
- break;
- case MP_STR:
- str = mp_decode_str(&data, &len);
- if (sqlVdbeMemSetStr(&val[i], str, len,
- 1, SQL_TRANSIENT) != 0)
- goto error;
- break;
- case MP_NIL:
- sqlVdbeMemSetNull(&val[i]);
- break;
- default:
- diag_set(ClientError, ER_SQL_EXECUTE,
- "Unsupported type passed from C");
- goto error;
- }
- i++;
- }
- return (struct sql_value *) val;
-error:
- for (int i = 0; i < port->size; i++)
- sqlVdbeMemRelease(&val[i]);
- region_truncate(region, region_svp);
- return NULL;
-}
-
/*
* Implementation of the non-aggregate min() and max() functions
*/
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index 588e142d2..02e9f9673 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -35,6 +35,7 @@
*/
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "box/ck_constraint.h"
#include "bit/bit.h"
diff --git a/src/box/sql/main.c b/src/box/sql/main.c
index 0b20f2132..b0d32ae32 100644
--- a/src/box/sql/main.c
+++ b/src/box/sql/main.c
@@ -36,6 +36,7 @@
* accessed by users of the library.
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "version.h"
#include "box/session.h"
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
new file mode 100644
index 000000000..d28b9a958
--- /dev/null
+++ b/src/box/sql/mem.c
@@ -0,0 +1,2638 @@
+/*
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "sqlInt.h"
+#include "mem.h"
+#include "vdbeInt.h"
+#include "coll/coll.h"
+#include "tarantoolInt.h"
+#include "box/schema.h"
+#include "box/tuple.h"
+#include "mpstream/mpstream.h"
+#include "box/port.h"
+#include "lua/utils.h"
+#include "lua/msgpack.h"
+
+static inline bool
+mem_has_msgpack_subtype(struct Mem *mem)
+{
+ return (mem->flags & MEM_Subtype) != 0 &&
+ mem->subtype == SQL_SUBTYPE_MSGPACK;
+}
+
+/*
+ * The pVal argument is known to be a value other than NULL.
+ * Convert it into a string with encoding enc and return a pointer
+ * to a zero-terminated version of that string.
+ */
+static SQL_NOINLINE const void *
+valueToText(sql_value * pVal)
+{
+ assert(pVal != 0);
+ assert((pVal->flags & (MEM_Null)) == 0);
+ if ((pVal->flags & (MEM_Blob | MEM_Str)) &&
+ !mem_has_msgpack_subtype(pVal)) {
+ if (ExpandBlob(pVal))
+ return 0;
+ pVal->flags |= MEM_Str;
+ sqlVdbeMemNulTerminate(pVal); /* IMP: R-31275-44060 */
+ } else {
+ sqlVdbeMemStringify(pVal);
+ assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
+ }
+ return pVal->z;
+}
+
+/**
+ * According to ANSI SQL string value can be converted to boolean
+ * type if string consists of literal "true" or "false" and
+ * number of leading and trailing spaces.
+ *
+ * For instance, " tRuE " can be successfully converted to
+ * boolean value true.
+ *
+ * @param str String to be converted to boolean. Assumed to be
+ * null terminated.
+ * @param[out] result Resulting value of cast.
+ * @retval 0 If string satisfies conditions above.
+ * @retval -1 Otherwise.
+ */
+static int
+str_cast_to_boolean(const char *str, bool *result)
+{
+ assert(str != NULL);
+ for (; *str == ' '; str++);
+ if (strncasecmp(str, SQL_TOKEN_TRUE, strlen(SQL_TOKEN_TRUE)) == 0) {
+ *result = true;
+ str += 4;
+ } else if (strncasecmp(str, SQL_TOKEN_FALSE,
+ strlen(SQL_TOKEN_FALSE)) == 0) {
+ *result = false;
+ str += 5;
+ } else {
+ return -1;
+ }
+ for (; *str != '\0'; ++str) {
+ if (*str != ' ')
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * Convert a 64-bit IEEE double into a 64-bit signed integer.
+ * If the double is out of range of a 64-bit signed integer then
+ * return the closest available 64-bit signed integer.
+ */
+static int
+doubleToInt64(double r, int64_t *i)
+{
+ /*
+ * Many compilers we encounter do not define constants for the
+ * minimum and maximum 64-bit integers, or they define them
+ * inconsistently. And many do not understand the "LL" notation.
+ * So we define our own static constants here using nothing
+ * larger than a 32-bit integer constant.
+ */
+ static const int64_t maxInt = LARGEST_INT64;
+ static const int64_t minInt = SMALLEST_INT64;
+ if (r <= (double)minInt) {
+ *i = minInt;
+ return -1;
+ } else if (r >= (double)maxInt) {
+ *i = maxInt;
+ return -1;
+ } else {
+ *i = (int64_t) r;
+ return *i != r;
+ }
+}
+
+/*
+ * It is already known that pMem contains an unterminated string.
+ * Add the zero terminator.
+ */
+static SQL_NOINLINE int
+vdbeMemAddTerminator(Mem * pMem)
+{
+ if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
+ return -1;
+ }
+ pMem->z[pMem->n] = 0;
+ pMem->z[pMem->n + 1] = 0;
+ pMem->flags |= MEM_Term;
+ return 0;
+}
+
+/*
+ * If the memory cell contains a value that must be freed by
+ * invoking the external callback in Mem.xDel, then this routine
+ * will free that value. It also sets Mem.flags to MEM_Null.
+ *
+ * This is a helper routine for sqlVdbeMemSetNull() and
+ * for sqlVdbeMemRelease(). Use those other routines as the
+ * entry point for releasing Mem resources.
+ */
+static SQL_NOINLINE void
+vdbeMemClearExternAndSetNull(Mem * p)
+{
+ assert(VdbeMemDynamic(p));
+ if (p->flags & MEM_Agg) {
+ sql_vdbemem_finalize(p, p->u.func);
+ assert((p->flags & MEM_Agg) == 0);
+ testcase(p->flags & MEM_Dyn);
+ }
+ if (p->flags & MEM_Dyn) {
+ assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
+ p->xDel((void *)p->z);
+ } else if (p->flags & MEM_Frame) {
+ VdbeFrame *pFrame = p->u.pFrame;
+ pFrame->pParent = pFrame->v->pDelFrame;
+ pFrame->v->pDelFrame = pFrame;
+ }
+ p->flags = MEM_Null;
+}
+
+/*
+ * Release memory held by the Mem p, both external memory cleared
+ * by p->xDel and memory in p->zMalloc.
+ *
+ * This is a helper routine invoked by sqlVdbeMemRelease() in
+ * the unusual case where there really is memory in p that needs
+ * to be freed.
+ */
+static SQL_NOINLINE void
+vdbeMemClear(Mem * p)
+{
+ if (VdbeMemDynamic(p)) {
+ vdbeMemClearExternAndSetNull(p);
+ }
+ if (p->szMalloc) {
+ sqlDbFree(p->db, p->zMalloc);
+ p->szMalloc = 0;
+ }
+ p->z = 0;
+}
+
+/*
+ * Make an shallow copy of pFrom into pTo. Prior contents of
+ * pTo are freed. The pFrom->z field is not duplicated. If
+ * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
+ * and flags gets srcType (either MEM_Ephem or MEM_Static).
+ */
+static SQL_NOINLINE void
+vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
+{
+ vdbeMemClearExternAndSetNull(pTo);
+ assert(!VdbeMemDynamic(pTo));
+ sqlVdbeMemShallowCopy(pTo, pFrom, eType);
+}
+
+/*
+ * Both *pMem1 and *pMem2 contain string values. Compare the two values
+ * using the collation sequence pColl. As usual, return a negative , zero
+ * or positive value if *pMem1 is less than, equal to or greater than
+ * *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
+ *
+ * Strungs assume to be UTF-8 encoded
+ */
+static int
+vdbeCompareMemString(const Mem * pMem1, const Mem * pMem2,
+ const struct coll * pColl)
+{
+ return pColl->cmp(pMem1->z, (size_t)pMem1->n,
+ pMem2->z, (size_t)pMem2->n, pColl);
+}
+
+/*
+ * The input pBlob is guaranteed to be a Blob that is not marked
+ * with MEM_Zero. Return true if it could be a zero-blob.
+ */
+static int
+isAllZero(const char *z, int n)
+{
+ int i;
+ for (i = 0; i < n; i++) {
+ if (z[i])
+ return 0;
+ }
+ return 1;
+}
+
+char *
+mem_type_to_str(const struct Mem *p)
+{
+ assert(p != NULL);
+ switch (p->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Null:
+ return "NULL";
+ case MEM_Str:
+ return "text";
+ case MEM_Int:
+ return "integer";
+ case MEM_UInt:
+ return "unsigned";
+ case MEM_Real:
+ return "real";
+ case MEM_Blob:
+ return "varbinary";
+ case MEM_Bool:
+ return "boolean";
+ default:
+ unreachable();
+ }
+}
+
+enum mp_type
+mem_mp_type(struct Mem *mem)
+{
+ switch (mem->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Int:
+ return MP_INT;
+ case MEM_UInt:
+ return MP_UINT;
+ case MEM_Real:
+ return MP_DOUBLE;
+ case MEM_Str:
+ return MP_STR;
+ case MEM_Blob:
+ if ((mem->flags & MEM_Subtype) == 0 ||
+ mem->subtype != SQL_SUBTYPE_MSGPACK)
+ return MP_BIN;
+ assert(mp_typeof(*mem->z) == MP_MAP ||
+ mp_typeof(*mem->z) == MP_ARRAY);
+ return mp_typeof(*mem->z);
+ case MEM_Bool:
+ return MP_BOOL;
+ case MEM_Null:
+ return MP_NIL;
+ default: unreachable();
+ }
+}
+
+/* EVIDENCE-OF: R-12793-43283 Every value in sql has one of five
+ * fundamental datatypes: 64-bit signed integer 64-bit IEEE floating
+ * point number string BLOB NULL
+ */
+enum mp_type
+sql_value_type(sql_value *pVal)
+{
+ struct Mem *mem = (struct Mem *) pVal;
+ return mem_mp_type(mem);
+}
+
+
+/*
+ * pMem currently only holds a string type (or maybe a BLOB that we can
+ * interpret as a string if we want to). Compute its corresponding
+ * numeric type, if has one. Set the pMem->u.r and pMem->u.i fields
+ * accordingly.
+ */
+static u16 SQL_NOINLINE
+computeNumericType(Mem *pMem)
+{
+ assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
+ assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
+ if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
+ return 0;
+ bool is_neg;
+ if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
+ return is_neg ? MEM_Int : MEM_UInt;
+ return MEM_Real;
+}
+
+/*
+ * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
+ * none.
+ *
+ * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
+ * But it does set pMem->u.r and pMem->u.i appropriately.
+ */
+u16
+numericType(Mem *pMem)
+{
+ if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+ return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
+ if (pMem->flags & (MEM_Str|MEM_Blob)) {
+ return computeNumericType(pMem);
+ }
+ return 0;
+}
+
+/*
+ * The sqlValueBytes() routine returns the number of bytes in the
+ * sql_value object assuming that it uses the encoding "enc".
+ * The valueBytes() routine is a helper function.
+ */
+static SQL_NOINLINE int
+valueBytes(sql_value * pVal)
+{
+ return valueToText(pVal) != 0 ? pVal->n : 0;
+}
+
+int
+sqlValueBytes(sql_value * pVal)
+{
+ Mem *p = (Mem *) pVal;
+ assert((p->flags & MEM_Null) == 0
+ || (p->flags & (MEM_Str | MEM_Blob)) == 0);
+ if ((p->flags & MEM_Str) != 0) {
+ return p->n;
+ }
+ if ((p->flags & MEM_Blob) != 0) {
+ if (p->flags & MEM_Zero) {
+ return p->n + p->u.nZero;
+ } else {
+ return p->n;
+ }
+ }
+ if (p->flags & MEM_Null)
+ return 0;
+ return valueBytes(pVal);
+}
+
+
+#ifdef SQL_DEBUG
+/*
+ * Check invariants on a Mem object.
+ *
+ * This routine is intended for use inside of assert() statements, like
+ * this: assert( sqlVdbeCheckMemInvariants(pMem) );
+ */
+int
+sqlVdbeCheckMemInvariants(Mem * p)
+{
+ /* If MEM_Dyn is set then Mem.xDel!=0.
+ * Mem.xDel is might not be initialized if MEM_Dyn is clear.
+ */
+ assert((p->flags & MEM_Dyn) == 0 || p->xDel != 0);
+
+ /* MEM_Dyn may only be set if Mem.szMalloc==0. In this way we
+ * ensure that if Mem.szMalloc>0 then it is safe to do
+ * Mem.z = Mem.zMalloc without having to check Mem.flags&MEM_Dyn.
+ * That saves a few cycles in inner loops.
+ */
+ assert((p->flags & MEM_Dyn) == 0 || p->szMalloc == 0);
+
+ /* Cannot be both MEM_Int and MEM_Real at the same time */
+ assert((p->flags & (MEM_Int | MEM_Real)) != (MEM_Int | MEM_Real));
+ /* Can't be both UInt and Int at the same time. */
+ assert((p->flags & (MEM_Int | MEM_UInt)) != (MEM_Int | MEM_UInt));
+
+ /* The szMalloc field holds the correct memory allocation size */
+ assert(p->szMalloc == 0 ||
+ p->szMalloc == sqlDbMallocSize(p->db, p->zMalloc));
+
+ /* If p holds a string or blob, the Mem.z must point to exactly
+ * one of the following:
+ *
+ * (1) Memory in Mem.zMalloc and managed by the Mem object
+ * (2) Memory to be freed using Mem.xDel
+ * (3) An ephemeral string or blob
+ * (4) A static string or blob
+ */
+ if ((p->flags & (MEM_Str | MEM_Blob)) && p->n > 0) {
+ assert(((p->szMalloc > 0 && p->z == p->zMalloc) ? 1 : 0) +
+ ((p->flags & MEM_Dyn) != 0 ? 1 : 0) +
+ ((p->flags & MEM_Ephem) != 0 ? 1 : 0) +
+ ((p->flags & MEM_Static) != 0 ? 1 : 0) == 1);
+ }
+ return 1;
+}
+
+/*
+ * Write a nice string representation of the contents of cell pMem
+ * into buffer zBuf, length nBuf.
+ */
+void
+sqlVdbeMemPrettyPrint(Mem *pMem, char *zBuf)
+{
+ char *zCsr = zBuf;
+ int f = pMem->flags;
+
+ if (f&MEM_Blob) {
+ int i;
+ char c;
+ if (f & MEM_Dyn) {
+ c = 'z';
+ assert((f & (MEM_Static|MEM_Ephem))==0);
+ } else if (f & MEM_Static) {
+ c = 't';
+ assert((f & (MEM_Dyn|MEM_Ephem))==0);
+ } else if (f & MEM_Ephem) {
+ c = 'e';
+ assert((f & (MEM_Static|MEM_Dyn))==0);
+ } else {
+ c = 's';
+ }
+
+ sql_snprintf(100, zCsr, "%c", c);
+ zCsr += sqlStrlen30(zCsr);
+ sql_snprintf(100, zCsr, "%d[", pMem->n);
+ zCsr += sqlStrlen30(zCsr);
+ for(i=0; i<16 && i<pMem->n; i++) {
+ sql_snprintf(100, zCsr, "%02X", ((int)pMem->z[i] & 0xFF));
+ zCsr += sqlStrlen30(zCsr);
+ }
+ for(i=0; i<16 && i<pMem->n; i++) {
+ char z = pMem->z[i];
+ if (z<32 || z>126) *zCsr++ = '.';
+ else *zCsr++ = z;
+ }
+ sql_snprintf(100, zCsr, "]%s", "(8)");
+ zCsr += sqlStrlen30(zCsr);
+ if (f & MEM_Zero) {
+ sql_snprintf(100, zCsr,"+%dz",pMem->u.nZero);
+ zCsr += sqlStrlen30(zCsr);
+ }
+ *zCsr = '\0';
+ } else if (f & MEM_Str) {
+ int j, k;
+ zBuf[0] = ' ';
+ if (f & MEM_Dyn) {
+ zBuf[1] = 'z';
+ assert((f & (MEM_Static|MEM_Ephem))==0);
+ } else if (f & MEM_Static) {
+ zBuf[1] = 't';
+ assert((f & (MEM_Dyn|MEM_Ephem))==0);
+ } else if (f & MEM_Ephem) {
+ zBuf[1] = 'e';
+ assert((f & (MEM_Static|MEM_Dyn))==0);
+ } else {
+ zBuf[1] = 's';
+ }
+ k = 2;
+ sql_snprintf(100, &zBuf[k], "%d", pMem->n);
+ k += sqlStrlen30(&zBuf[k]);
+ zBuf[k++] = '[';
+ for(j=0; j<15 && j<pMem->n; j++) {
+ u8 c = pMem->z[j];
+ if (c>=0x20 && c<0x7f) {
+ zBuf[k++] = c;
+ } else {
+ zBuf[k++] = '.';
+ }
+ }
+ zBuf[k++] = ']';
+ sql_snprintf(100,&zBuf[k],"(8)");
+ k += sqlStrlen30(&zBuf[k]);
+ zBuf[k++] = 0;
+ }
+}
+
+/*
+ * Print the value of a register for tracing purposes:
+ */
+static void
+memTracePrint(Mem *p)
+{
+ if (p->flags & MEM_Undefined) {
+ printf(" undefined");
+ } else if (p->flags & MEM_Null) {
+ printf(" NULL");
+ } else if ((p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str)) {
+ printf(" si:%lld", p->u.i);
+ } else if (p->flags & MEM_Int) {
+ printf(" i:%lld", p->u.i);
+ } else if (p->flags & MEM_UInt) {
+ printf(" u:%"PRIu64"", p->u.u);
+ } else if (p->flags & MEM_Real) {
+ printf(" r:%g", p->u.r);
+ } else if (p->flags & MEM_Bool) {
+ printf(" bool:%s", SQL_TOKEN_BOOLEAN(p->u.b));
+ } else {
+ char zBuf[200];
+ sqlVdbeMemPrettyPrint(p, zBuf);
+ printf(" %s", zBuf);
+ }
+ if (p->flags & MEM_Subtype) printf(" subtype=0x%02x", p->subtype);
+}
+
+void
+registerTrace(int iReg, Mem *p) {
+ printf("REG[%d] = ", iReg);
+ memTracePrint(p);
+ printf("\n");
+}
+#endif
+
+int
+mem_apply_numeric_type(struct Mem *record)
+{
+ if ((record->flags & MEM_Str) == 0)
+ return -1;
+ int64_t integer_value;
+ bool is_neg;
+ if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
+ mem_set_int(record, integer_value, is_neg);
+ return 0;
+ }
+ double float_value;
+ if (sqlAtoF(record->z, &float_value, record->n) == 0)
+ return -1;
+ mem_set_double(record, float_value);
+ return 0;
+}
+
+/*
+ * Convert pMem so that it is of type MEM_Real.
+ * Invalidate any prior representations.
+ */
+int
+sqlVdbeMemRealify(Mem * pMem)
+{
+ assert(EIGHT_BYTE_ALIGNMENT(pMem));
+ double v;
+ if (sqlVdbeRealValue(pMem, &v))
+ return -1;
+ mem_set_double(pMem, v);
+ return 0;
+}
+
+int
+vdbe_mem_numerify(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
+ return 0;
+ if ((mem->flags & MEM_Bool) != 0) {
+ mem->u.u = mem->u.b;
+ MemSetTypeFlag(mem, MEM_UInt);
+ return 0;
+ }
+ assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
+ bool is_neg;
+ int64_t i;
+ if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
+ mem_set_int(mem, i, is_neg);
+ } else {
+ double d;
+ if (sqlAtoF(mem->z, &d, mem->n) == 0)
+ return -1;
+ mem_set_double(mem, d);
+ }
+ return 0;
+}
+
+/*
+ * Cast the datatype of the value in pMem according to the type
+ * @type. Casting is different from applying type in that a cast
+ * is forced. In other words, the value is converted into the desired
+ * type even if that results in loss of data. This routine is
+ * used (for example) to implement the SQL "cast()" operator.
+ */
+int
+sqlVdbeMemCast(Mem * pMem, enum field_type type)
+{
+ assert(type < field_type_MAX);
+ if (pMem->flags & MEM_Null)
+ return 0;
+ switch (type) {
+ case FIELD_TYPE_SCALAR:
+ return 0;
+ case FIELD_TYPE_BOOLEAN:
+ if ((pMem->flags & MEM_Int) != 0) {
+ mem_set_bool(pMem, pMem->u.i);
+ return 0;
+ }
+ if ((pMem->flags & MEM_UInt) != 0) {
+ mem_set_bool(pMem, pMem->u.u);
+ return 0;
+ }
+ if ((pMem->flags & MEM_Real) != 0) {
+ mem_set_bool(pMem, pMem->u.r);
+ return 0;
+ }
+ if ((pMem->flags & MEM_Str) != 0) {
+ bool value;
+ if (str_cast_to_boolean(pMem->z, &value) != 0)
+ return -1;
+ mem_set_bool(pMem, value);
+ return 0;
+ }
+ if ((pMem->flags & MEM_Bool) != 0)
+ return 0;
+ return -1;
+ case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
+ if ((pMem->flags & (MEM_Blob | MEM_Str)) != 0) {
+ bool is_neg;
+ int64_t val;
+ if (sql_atoi64(pMem->z, &val, &is_neg, pMem->n) != 0)
+ return -1;
+ if (type == FIELD_TYPE_UNSIGNED && is_neg)
+ return -1;
+ mem_set_int(pMem, val, is_neg);
+ return 0;
+ }
+ if ((pMem->flags & MEM_Bool) != 0) {
+ pMem->u.u = pMem->u.b;
+ MemSetTypeFlag(pMem, MEM_UInt);
+ return 0;
+ }
+ if ((pMem->flags & MEM_Real) != 0) {
+ double d;
+ if (sqlVdbeRealValue(pMem, &d) != 0)
+ return -1;
+ if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
+ mem_set_int(pMem, d, d <= -1);
+ return 0;
+ }
+ if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
+ mem_set_u64(pMem, d);
+ return 0;
+ }
+ return -1;
+ }
+ if (type == FIELD_TYPE_UNSIGNED &&
+ (pMem->flags & MEM_UInt) == 0)
+ return -1;
+ return 0;
+ case FIELD_TYPE_DOUBLE:
+ return sqlVdbeMemRealify(pMem);
+ case FIELD_TYPE_NUMBER:
+ return vdbe_mem_numerify(pMem);
+ case FIELD_TYPE_VARBINARY:
+ if ((pMem->flags & MEM_Blob) != 0)
+ return 0;
+ if ((pMem->flags & MEM_Str) != 0) {
+ MemSetTypeFlag(pMem, MEM_Str);
+ return 0;
+ }
+ return -1;
+ default:
+ assert(type == FIELD_TYPE_STRING);
+ assert(MEM_Str == (MEM_Blob >> 3));
+ if ((pMem->flags & MEM_Bool) != 0) {
+ const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
+ sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
+ SQL_TRANSIENT);
+ return 0;
+ }
+ pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
+ sql_value_apply_type(pMem, FIELD_TYPE_STRING);
+ assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
+ pMem->flags &=
+ ~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
+ return 0;
+ }
+}
+
+/*
+ * The MEM structure is already a MEM_Real. Try to also make it a
+ * MEM_Int if we can.
+ */
+int
+mem_apply_integer_type(Mem *pMem)
+{
+ int rc;
+ i64 ix;
+ assert(pMem->flags & MEM_Real);
+ assert(EIGHT_BYTE_ALIGNMENT(pMem));
+
+ if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
+ mem_set_int(pMem, ix, pMem->u.r <= -1);
+ return rc;
+}
+
+/*
+ * Add MEM_Str to the set of representations for the given Mem. Numbers
+ * are converted using sql_snprintf(). Converting a BLOB to a string
+ * is a no-op.
+ *
+ * Existing representations MEM_Int and MEM_Real are invalidated if
+ * bForce is true but are retained if bForce is false.
+ *
+ * A MEM_Null value will never be passed to this function. This function is
+ * used for converting values to text for returning to the user (i.e. via
+ * sql_value_text()), or for ensuring that values to be used as btree
+ * keys are strings. In the former case a NULL pointer is returned the
+ * user and the latter is an internal programming error.
+ */
+int
+sqlVdbeMemStringify(Mem * pMem)
+{
+ int fg = pMem->flags;
+ int nByte = 32;
+
+ if ((fg & (MEM_Null | MEM_Str | MEM_Blob)) != 0 &&
+ !mem_has_msgpack_subtype(pMem))
+ return 0;
+
+ assert(!(fg & MEM_Zero));
+ assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool |
+ MEM_Blob)) != 0);
+ assert(EIGHT_BYTE_ALIGNMENT(pMem));
+
+ /*
+ * In case we have ARRAY/MAP we should save decoded value
+ * before clearing pMem->z.
+ */
+ char *value = NULL;
+ if (mem_has_msgpack_subtype(pMem)) {
+ const char *value_str = mp_str(pMem->z);
+ nByte = strlen(value_str) + 1;
+ value = region_alloc(&fiber()->gc, nByte);
+ memcpy(value, value_str, nByte);
+ }
+
+ if (sqlVdbeMemClearAndResize(pMem, nByte)) {
+ return -1;
+ }
+ if (fg & MEM_Int) {
+ sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
+ pMem->flags &= ~MEM_Int;
+ } else if ((fg & MEM_UInt) != 0) {
+ sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
+ pMem->flags &= ~MEM_UInt;
+ } else if ((fg & MEM_Bool) != 0) {
+ sql_snprintf(nByte, pMem->z, "%s",
+ SQL_TOKEN_BOOLEAN(pMem->u.b));
+ pMem->flags &= ~MEM_Bool;
+ } else if (mem_has_msgpack_subtype(pMem)) {
+ sql_snprintf(nByte, pMem->z, "%s", value);
+ pMem->flags &= ~MEM_Subtype;
+ pMem->subtype = SQL_SUBTYPE_NO;
+ } else {
+ assert(fg & MEM_Real);
+ sql_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
+ pMem->flags &= ~MEM_Real;
+ }
+ pMem->n = sqlStrlen30(pMem->z);
+ pMem->flags |= MEM_Str | MEM_Term;
+ return 0;
+}
+
+/*
+ * Make sure the given Mem is \u0000 terminated.
+ */
+int
+sqlVdbeMemNulTerminate(Mem * pMem)
+{
+ testcase((pMem->flags & (MEM_Term | MEM_Str)) == (MEM_Term | MEM_Str));
+ testcase((pMem->flags & (MEM_Term | MEM_Str)) == 0);
+ if ((pMem->flags & (MEM_Term | MEM_Str)) != MEM_Str) {
+ return 0; /* Nothing to do */
+ } else {
+ return vdbeMemAddTerminator(pMem);
+ }
+}
+
+/*
+ * If the given Mem* has a zero-filled tail, turn it into an ordinary
+ * blob stored in dynamically allocated space.
+ */
+int
+sqlVdbeMemExpandBlob(Mem * pMem)
+{
+ int nByte;
+ assert(pMem->flags & MEM_Zero);
+ assert(pMem->flags & MEM_Blob);
+
+ /* Set nByte to the number of bytes required to store the expanded blob. */
+ nByte = pMem->n + pMem->u.nZero;
+ if (nByte <= 0) {
+ nByte = 1;
+ }
+ if (sqlVdbeMemGrow(pMem, nByte, 1)) {
+ return -1;
+ }
+
+ memset(&pMem->z[pMem->n], 0, pMem->u.nZero);
+ pMem->n += pMem->u.nZero;
+ pMem->flags &= ~(MEM_Zero | MEM_Term);
+ return 0;
+}
+
+/*
+ * Exported version of mem_apply_type(). This one works on sql_value*,
+ * not the internal Mem* type.
+ */
+void
+sql_value_apply_type(
+ sql_value *pVal,
+ enum field_type type)
+{
+ mem_apply_type((Mem *) pVal, type);
+}
+
+int
+mem_apply_type(struct Mem *record, enum field_type type)
+{
+ if ((record->flags & MEM_Null) != 0)
+ return 0;
+ assert(type < field_type_MAX);
+ switch (type) {
+ case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
+ if ((record->flags & (MEM_Bool | MEM_Blob)) != 0)
+ return -1;
+ if ((record->flags & MEM_UInt) == MEM_UInt)
+ return 0;
+ if ((record->flags & MEM_Real) == MEM_Real) {
+ double d = record->u.r;
+ if (d >= 0) {
+ if (double_compare_uint64(d, UINT64_MAX,
+ 1) > 0)
+ return 0;
+ if ((double)(uint64_t)d == d)
+ mem_set_u64(record, (uint64_t)d);
+ } else {
+ if (double_compare_nint64(d, INT64_MIN, 1) < 0)
+ return 0;
+ if ((double)(int64_t)d == d)
+ mem_set_int(record, (int64_t)d, true);
+ }
+ return 0;
+ }
+ if ((record->flags & MEM_Str) != 0) {
+ bool is_neg;
+ int64_t i;
+ if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
+ return -1;
+ mem_set_int(record, i, is_neg);
+ }
+ if ((record->flags & MEM_Int) == MEM_Int) {
+ if (type == FIELD_TYPE_UNSIGNED)
+ return -1;
+ return 0;
+ }
+ return 0;
+ case FIELD_TYPE_BOOLEAN:
+ if ((record->flags & MEM_Bool) == MEM_Bool)
+ return 0;
+ return -1;
+ case FIELD_TYPE_NUMBER:
+ if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ return sqlVdbeMemRealify(record);
+ case FIELD_TYPE_DOUBLE:
+ if ((record->flags & MEM_Real) != 0)
+ return 0;
+ return sqlVdbeMemRealify(record);
+ case FIELD_TYPE_STRING:
+ /*
+ * Only attempt the conversion to TEXT if there is
+ * an integer or real representation (BLOB and
+ * NULL do not get converted).
+ */
+ if ((record->flags & MEM_Str) == 0 &&
+ (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
+ sqlVdbeMemStringify(record);
+ record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
+ return 0;
+ case FIELD_TYPE_VARBINARY:
+ if ((record->flags & MEM_Blob) == 0)
+ return -1;
+ return 0;
+ case FIELD_TYPE_SCALAR:
+ /* Can't cast MAP and ARRAY to scalar types. */
+ if ((record->flags & MEM_Subtype) != 0 &&
+ record->subtype == SQL_SUBTYPE_MSGPACK) {
+ assert(mp_typeof(*record->z) == MP_MAP ||
+ mp_typeof(*record->z) == MP_ARRAY);
+ return -1;
+ }
+ return 0;
+ case FIELD_TYPE_MAP:
+ if ((record->flags & MEM_Subtype) != 0 &&
+ record->subtype == SQL_SUBTYPE_MSGPACK &&
+ mp_typeof(*record->z) == MP_MAP)
+ return 0;
+ return -1;
+ case FIELD_TYPE_ARRAY:
+ if ((record->flags & MEM_Subtype) != 0 &&
+ record->subtype == SQL_SUBTYPE_MSGPACK &&
+ mp_typeof(*record->z) == MP_ARRAY)
+ return 0;
+ return -1;
+ case FIELD_TYPE_ANY:
+ return 0;
+ default:
+ return -1;
+ }
+}
+
+/**
+ * Convert the numeric value contained in MEM to double.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_double(struct Mem *mem)
+{
+ if ((mem->flags & MEM_Real) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
+ return -1;
+ double d;
+ if ((mem->flags & MEM_Int) != 0)
+ d = (double)mem->u.i;
+ else
+ d = (double)mem->u.u;
+ mem_set_double(mem, d);
+ return 0;
+}
+
+/**
+ * Convert the numeric value contained in MEM to unsigned.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_unsigned(struct Mem *mem)
+{
+ if ((mem->flags & MEM_UInt) != 0)
+ return 0;
+ if ((mem->flags & MEM_Int) != 0)
+ return -1;
+ if ((mem->flags & MEM_Real) == 0)
+ return -1;
+ double d = mem->u.r;
+ if (d < 0.0 || d >= (double)UINT64_MAX)
+ return -1;
+ mem_set_u64(mem, (uint64_t) d);
+ return 0;
+}
+
+/**
+ * Convert the numeric value contained in MEM to integer.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_integer(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
+ return 0;
+ if ((mem->flags & MEM_Real) == 0)
+ return -1;
+ double d = mem->u.r;
+ if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
+ return -1;
+ if (d < (double)INT64_MAX)
+ mem_set_int(mem, (int64_t) d, d < 0);
+ else
+ mem_set_int(mem, (uint64_t) d, false);
+ return 0;
+}
+
+int
+mem_convert_to_numeric(struct Mem *mem, enum field_type type)
+{
+ assert(mp_type_is_numeric(mem_mp_type(mem)) &&
+ sql_type_is_numeric(type));
+ assert(type != FIELD_TYPE_NUMBER);
+ if (type == FIELD_TYPE_DOUBLE)
+ return mem_convert_to_double(mem);
+ if (type == FIELD_TYPE_UNSIGNED)
+ return mem_convert_to_unsigned(mem);
+ assert(type == FIELD_TYPE_INTEGER);
+ return mem_convert_to_integer(mem);
+}
+
+/*
+ * Make sure pMem->z points to a writable allocation of at least
+ * min(n,32) bytes.
+ *
+ * If the bPreserve argument is true, then copy of the content of
+ * pMem->z into the new allocation. pMem must be either a string or
+ * blob if bPreserve is true. If bPreserve is false, any prior content
+ * in pMem->z is discarded.
+ */
+SQL_NOINLINE int
+sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
+{
+ assert(sqlVdbeCheckMemInvariants(pMem));
+ testcase(pMem->db == 0);
+
+ /* If the bPreserve flag is set to true, then the memory cell must already
+ * contain a valid string or blob value.
+ */
+ assert(bPreserve == 0 || pMem->flags & (MEM_Blob | MEM_Str));
+ testcase(bPreserve && pMem->z == 0);
+
+ assert(pMem->szMalloc == 0 ||
+ pMem->szMalloc == sqlDbMallocSize(pMem->db, pMem->zMalloc));
+ if (pMem->szMalloc < n) {
+ if (n < 32)
+ n = 32;
+ if (bPreserve && pMem->szMalloc > 0 && pMem->z == pMem->zMalloc) {
+ pMem->z = pMem->zMalloc =
+ sqlDbReallocOrFree(pMem->db, pMem->z, n);
+ bPreserve = 0;
+ } else {
+ if (pMem->szMalloc > 0)
+ sqlDbFree(pMem->db, pMem->zMalloc);
+ pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
+ }
+ if (pMem->zMalloc == 0) {
+ sqlVdbeMemSetNull(pMem);
+ pMem->z = 0;
+ pMem->szMalloc = 0;
+ return -1;
+ } else {
+ pMem->szMalloc = sqlDbMallocSize(pMem->db,
+ pMem->zMalloc);
+ }
+ }
+
+ if (bPreserve && pMem->z && pMem->z != pMem->zMalloc) {
+ memcpy(pMem->zMalloc, pMem->z, pMem->n);
+ }
+ if ((pMem->flags & MEM_Dyn) != 0) {
+ assert(pMem->xDel != 0 && pMem->xDel != SQL_DYNAMIC);
+ pMem->xDel((void *)(pMem->z));
+ }
+
+ pMem->z = pMem->zMalloc;
+ pMem->flags &= ~(MEM_Dyn | MEM_Ephem | MEM_Static);
+ return 0;
+}
+
+/*
+ * Change the pMem->zMalloc allocation to be at least szNew bytes.
+ * If pMem->zMalloc already meets or exceeds the requested size, this
+ * routine is a no-op.
+ *
+ * Any prior string or blob content in the pMem object may be discarded.
+ * The pMem->xDel destructor is called, if it exists. Though MEM_Str
+ * and MEM_Blob values may be discarded, MEM_Int, MEM_Real, and MEM_Null
+ * values are preserved.
+ *
+ * Return 0 on success or -1 if unable to complete the resizing.
+ */
+int
+sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
+{
+ assert(szNew > 0);
+ assert((pMem->flags & MEM_Dyn) == 0 || pMem->szMalloc == 0);
+ if (pMem->szMalloc < szNew) {
+ return sqlVdbeMemGrow(pMem, szNew, 0);
+ }
+ assert((pMem->flags & MEM_Dyn) == 0);
+ pMem->z = pMem->zMalloc;
+ pMem->flags &= (MEM_Null | MEM_Int | MEM_Real);
+ return 0;
+}
+
+void
+mem_set_bool(struct Mem *mem, bool value)
+{
+ sqlVdbeMemSetNull(mem);
+ mem->u.b = value;
+ mem->flags = MEM_Bool;
+ mem->field_type = FIELD_TYPE_BOOLEAN;
+}
+
+void
+mem_set_ptr(struct Mem *mem, void *ptr)
+{
+ sqlVdbeMemRelease(mem);
+ mem->flags = MEM_Ptr;
+ mem->u.p = ptr;
+}
+
+void
+mem_set_i64(struct Mem *mem, int64_t value)
+{
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ mem->u.i = value;
+ int flag = value < 0 ? MEM_Int : MEM_UInt;
+ MemSetTypeFlag(mem, flag);
+ mem->field_type = FIELD_TYPE_INTEGER;
+}
+
+void
+mem_set_u64(struct Mem *mem, uint64_t value)
+{
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ mem->u.u = value;
+ MemSetTypeFlag(mem, MEM_UInt);
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+}
+
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
+{
+ if (VdbeMemDynamic(mem))
+ sqlVdbeMemSetNull(mem);
+ if (is_neg) {
+ assert(value < 0);
+ mem->u.i = value;
+ MemSetTypeFlag(mem, MEM_Int);
+ } else {
+ mem->u.u = value;
+ MemSetTypeFlag(mem, MEM_UInt);
+ }
+ mem->field_type = FIELD_TYPE_INTEGER;
+}
+
+void
+mem_set_double(struct Mem *mem, double value)
+{
+ sqlVdbeMemSetNull(mem);
+ if (sqlIsNaN(value))
+ return;
+ mem->u.r = value;
+ MemSetTypeFlag(mem, MEM_Real);
+ mem->field_type = FIELD_TYPE_DOUBLE;
+}
+
+/*
+ * Change the value of a Mem to be a string or a BLOB.
+ *
+ * The memory management strategy depends on the value of the xDel
+ * parameter. If the value passed is SQL_TRANSIENT, then the
+ * string is copied into a (possibly existing) buffer managed by the
+ * Mem structure. Otherwise, any existing buffer is freed and the
+ * pointer copied.
+ *
+ * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
+ * size limit) then no memory allocation occurs. If the string can be
+ * stored without allocating memory, then it is. If a memory allocation
+ * is required to store the string, then value of pMem is unchanged. In
+ * either case, error is returned.
+ */
+int
+sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
+ const char *z, /* String pointer */
+ int n, /* Bytes in string, or negative */
+ u8 not_blob, /* Encoding of z. 0 for BLOBs */
+ void (*xDel) (void *) /* Destructor function */
+ )
+{
+ int nByte = n; /* New value for pMem->n */
+ int iLimit; /* Maximum allowed string or blob size */
+ u16 flags = 0; /* New value for pMem->flags */
+
+ /* If z is a NULL pointer, set pMem to contain an SQL NULL. */
+ if (!z) {
+ sqlVdbeMemSetNull(pMem);
+ return 0;
+ }
+
+ if (pMem->db) {
+ iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
+ } else {
+ iLimit = SQL_MAX_LENGTH;
+ }
+ flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
+ if (nByte < 0) {
+ assert(not_blob != 0);
+ nByte = sqlStrlen30(z);
+ if (nByte > iLimit)
+ nByte = iLimit + 1;
+ flags |= MEM_Term;
+ }
+
+ /* The following block sets the new values of Mem.z and Mem.xDel. It
+ * also sets a flag in local variable "flags" to indicate the memory
+ * management (one of MEM_Dyn or MEM_Static).
+ */
+ if (xDel == SQL_TRANSIENT) {
+ int nAlloc = nByte;
+ if (flags & MEM_Term) {
+ nAlloc += 1; //SQL_UTF8
+ }
+ if (nByte > iLimit) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
+ "string is too big");
+ return -1;
+ }
+ testcase(nAlloc == 0);
+ testcase(nAlloc == 31);
+ testcase(nAlloc == 32);
+ if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
+ return -1;
+ }
+ memcpy(pMem->z, z, nAlloc);
+ } else if (xDel == SQL_DYNAMIC) {
+ sqlVdbeMemRelease(pMem);
+ pMem->zMalloc = pMem->z = (char *)z;
+ pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
+ } else {
+ sqlVdbeMemRelease(pMem);
+ pMem->z = (char *)z;
+ pMem->xDel = xDel;
+ flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
+ }
+
+ pMem->n = nByte;
+ pMem->flags = flags;
+ assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
+ if ((pMem->flags & MEM_Str) != 0)
+ pMem->field_type = FIELD_TYPE_STRING;
+ else
+ pMem->field_type = FIELD_TYPE_VARBINARY;
+
+ if (nByte > iLimit) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
+ "is too big");
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * Initialize bulk memory to be a consistent Mem object.
+ *
+ * The minimum amount of initialization feasible is performed.
+ */
+void
+sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
+{
+ assert((flags & ~MEM_TypeMask) == 0);
+ pMem->flags = flags;
+ pMem->db = db;
+ pMem->szMalloc = 0;
+ pMem->field_type = field_type_MAX;
+}
+
+/*
+ * Delete any previous value and set the value stored in *pMem to NULL.
+ *
+ * This routine calls the Mem.xDel destructor to dispose of values that
+ * require the destructor. But it preserves the Mem.zMalloc memory allocation.
+ * To free all resources, use sqlVdbeMemRelease(), which both calls this
+ * routine to invoke the destructor and deallocates Mem.zMalloc.
+ *
+ * Use this routine to reset the Mem prior to insert a new value.
+ *
+ * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
+ */
+void
+sqlVdbeMemSetNull(Mem * pMem)
+{
+ if (VdbeMemDynamic(pMem)) {
+ vdbeMemClearExternAndSetNull(pMem);
+ } else {
+ pMem->flags = MEM_Null;
+ }
+}
+
+/*
+ * Delete any previous value and set the value to be a BLOB of length
+ * n containing all zeros.
+ */
+void
+sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
+{
+ sqlVdbeMemRelease(pMem);
+ pMem->flags = MEM_Blob | MEM_Zero;
+ pMem->n = 0;
+ if (n < 0)
+ n = 0;
+ pMem->u.nZero = n;
+ pMem->z = 0;
+}
+
+/*
+ * Change the string value of an sql_value object
+ */
+void
+sqlValueSetStr(sql_value * v, /* Value to be set */
+ int n, /* Length of string z */
+ const void *z, /* Text of the new string */
+ void (*xDel) (void *) /* Destructor for the string */
+ )
+{
+ if (v)
+ sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
+}
+
+void
+sqlValueSetNull(sql_value * p)
+{
+ sqlVdbeMemSetNull((Mem *) p);
+}
+
+/*
+ * Free an sql_value object
+ */
+void
+sqlValueFree(sql_value * v)
+{
+ if (!v)
+ return;
+ sqlVdbeMemRelease((Mem *) v);
+ sqlDbFree(((Mem *) v)->db, v);
+}
+
+/*
+ * Create a new sql_value object.
+ */
+sql_value *
+sqlValueNew(sql * db)
+{
+ Mem *p = sqlDbMallocZero(db, sizeof(*p));
+ if (p) {
+ p->flags = MEM_Null;
+ p->db = db;
+ }
+ return p;
+}
+
+void
+initMemArray(Mem * p, int N, sql * db, u32 flags)
+{
+ while ((N--) > 0) {
+ p->db = db;
+ p->flags = flags;
+ p->szMalloc = 0;
+ p->field_type = field_type_MAX;
+#ifdef SQL_DEBUG
+ p->pScopyFrom = 0;
+#endif
+ p++;
+ }
+}
+
+void
+releaseMemArray(Mem * p, int N)
+{
+ if (p && N) {
+ Mem *pEnd = &p[N];
+ sql *db = p->db;
+ do {
+ assert((&p[1]) == pEnd || p[0].db == p[1].db);
+ assert(sqlVdbeCheckMemInvariants(p));
+
+ /* This block is really an inlined version of sqlVdbeMemRelease()
+ * that takes advantage of the fact that the memory cell value is
+ * being set to NULL after releasing any dynamic resources.
+ *
+ * The justification for duplicating code is that according to
+ * callgrind, this causes a certain test case to hit the CPU 4.7
+ * percent less (x86 linux, gcc version 4.1.2, -O6) than if
+ * sqlMemRelease() were called from here. With -O2, this jumps
+ * to 6.6 percent. The test case is inserting 1000 rows into a table
+ * with no indexes using a single prepared INSERT statement, bind()
+ * and reset(). Inserts are grouped into a transaction.
+ */
+ testcase(p->flags & MEM_Agg);
+ testcase(p->flags & MEM_Dyn);
+ testcase(p->flags & MEM_Frame);
+ if (p->
+ flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
+ sqlVdbeMemRelease(p);
+ } else if (p->szMalloc) {
+ sqlDbFree(db, p->zMalloc);
+ p->szMalloc = 0;
+ }
+
+ p->flags = MEM_Undefined;
+ } while ((++p) < pEnd);
+ }
+}
+
+int
+mem_value_bool(const struct Mem *mem, bool *b)
+{
+ if ((mem->flags & MEM_Bool) != 0) {
+ *b = mem->u.b;
+ return 0;
+ }
+ return -1;
+}
+
+/*
+ * Return some kind of integer value which is the best we can do
+ * at representing the value that *pMem describes as an integer.
+ * If pMem is an integer, then the value is exact. If pMem is
+ * a floating-point then the value returned is the integer part.
+ * If pMem is a string or blob, then we make an attempt to convert
+ * it into an integer and return that. If pMem represents an
+ * an SQL-NULL value, return 0.
+ *
+ * If pMem represents a string value, its encoding might be changed.
+ */
+int
+sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
+{
+ int flags;
+ assert(EIGHT_BYTE_ALIGNMENT(pMem));
+ flags = pMem->flags;
+ if (flags & MEM_Int) {
+ *i = pMem->u.i;
+ *is_neg = true;
+ return 0;
+ } else if (flags & MEM_UInt) {
+ *i = pMem->u.u;
+ *is_neg = false;
+ return 0;
+ } else if (flags & MEM_Real) {
+ *is_neg = pMem->u.r < 0;
+ return doubleToInt64(pMem->u.r, i);
+ } else if (flags & (MEM_Str)) {
+ assert(pMem->z || pMem->n == 0);
+ if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
+ return 0;
+ }
+ return -1;
+}
+
+/*
+ * Return the best representation of pMem that we can get into a
+ * double. If pMem is already a double or an integer, return its
+ * value. If it is a string or blob, try to convert it to a double.
+ * If it is a NULL, return 0.0.
+ */
+int
+sqlVdbeRealValue(Mem * pMem, double *v)
+{
+ assert(EIGHT_BYTE_ALIGNMENT(pMem));
+ if (pMem->flags & MEM_Real) {
+ *v = pMem->u.r;
+ return 0;
+ } else if (pMem->flags & MEM_Int) {
+ *v = (double)pMem->u.i;
+ return 0;
+ } else if ((pMem->flags & MEM_UInt) != 0) {
+ *v = (double)pMem->u.u;
+ return 0;
+ } else if (pMem->flags & MEM_Str) {
+ if (sqlAtoF(pMem->z, v, pMem->n))
+ return 0;
+ }
+ return -1;
+}
+
+/**************************** sql_value_ ******************************
+ * The following routines extract information from a Mem or sql_value
+ * structure.
+ */
+const void *
+sql_value_blob(sql_value * pVal)
+{
+ Mem *p = (Mem *) pVal;
+ if (p->flags & (MEM_Blob | MEM_Str)) {
+ if (ExpandBlob(p) != 0) {
+ assert(p->flags == MEM_Null && p->z == 0);
+ return 0;
+ }
+ p->flags |= MEM_Blob;
+ return p->n ? p->z : 0;
+ } else {
+ return sql_value_text(pVal);
+ }
+}
+
+int
+sql_value_bytes(sql_value * pVal)
+{
+ return sqlValueBytes(pVal);
+}
+
+double
+sql_value_double(sql_value * pVal)
+{
+ double v = 0.0;
+ sqlVdbeRealValue((Mem *) pVal, &v);
+ return v;
+}
+
+bool
+sql_value_boolean(sql_value *val)
+{
+ bool b = false;
+ int rc = mem_value_bool((struct Mem *) val, &b);
+ assert(rc == 0);
+ (void) rc;
+ return b;
+}
+
+int
+sql_value_int(sql_value * pVal)
+{
+ int64_t i = 0;
+ bool is_neg;
+ sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
+ return (int)i;
+}
+
+sql_int64
+sql_value_int64(sql_value * pVal)
+{
+ int64_t i = 0;
+ bool unused;
+ sqlVdbeIntValue((Mem *) pVal, &i, &unused);
+ return i;
+}
+
+uint64_t
+sql_value_uint64(sql_value *val)
+{
+ int64_t i = 0;
+ bool is_neg;
+ sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
+ assert(!is_neg);
+ return i;
+}
+
+const unsigned char *
+sql_value_text(sql_value * pVal)
+{
+ return (const unsigned char *)sqlValueText(pVal);
+}
+
+/* This function is only available internally, it is not part of the
+ * external API. It works in a similar way to sql_value_text(),
+ * except the data returned is in the encoding specified by the second
+ * parameter, which must be one of SQL_UTF16BE, SQL_UTF16LE or
+ * SQL_UTF8.
+ *
+ * (2006-02-16:) The enc value can be or-ed with SQL_UTF16_ALIGNED.
+ * If that is the case, then the result must be aligned on an even byte
+ * boundary.
+ */
+const void *
+sqlValueText(sql_value * pVal)
+{
+ if (!pVal)
+ return 0;
+ if ((pVal->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)) {
+ return pVal->z;
+ }
+ if (pVal->flags & MEM_Null) {
+ return 0;
+ }
+ return valueToText(pVal);
+}
+
+const char *
+sql_value_to_diag_str(sql_value *value)
+{
+ enum mp_type mp_type = sql_value_type(value);
+ if (mp_type_is_bloblike(mp_type)) {
+ if (mem_has_msgpack_subtype(value))
+ return sqlValueText(value);
+ return "varbinary";
+ }
+ return sqlValueText(value);
+}
+
+enum sql_subtype
+sql_value_subtype(sql_value * pVal)
+{
+ return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
+}
+
+/*
+ * Return a pointer to static memory containing an SQL NULL value.
+ */
+const Mem *
+columnNullValue(void)
+{
+ /* Even though the Mem structure contains an element
+ * of type i64, on certain architectures (x86) with certain compiler
+ * switches (-Os), gcc may align this Mem object on a 4-byte boundary
+ * instead of an 8-byte one. This all works fine, except that when
+ * running with SQL_DEBUG defined the sql code sometimes assert()s
+ * that a Mem structure is located on an 8-byte boundary. To prevent
+ * these assert()s from failing, when building with SQL_DEBUG defined
+ * using gcc, we force nullMem to be 8-byte aligned using the magical
+ * __attribute__((aligned(8))) macro.
+ */
+ static const Mem nullMem
+#if defined(SQL_DEBUG) && defined(__GNUC__)
+ __attribute__ ((aligned(8)))
+#endif
+ = {
+ /* .u = */ {
+ 0},
+ /* .flags = */ (u16) MEM_Null,
+ /* .eSubtype = */ (u8) 0,
+ /* .field_type = */ field_type_MAX,
+ /* .n = */ (int)0,
+ /* .z = */ (char *)0,
+ /* .zMalloc = */ (char *)0,
+ /* .szMalloc = */ (int)0,
+ /* .uTemp = */ (u32) 0,
+ /* .db = */ (sql *) 0,
+ /* .xDel = */ (void (*)(void *))0,
+#ifdef SQL_DEBUG
+ /* .pScopyFrom = */ (Mem *) 0,
+ /* .pFiller = */ (void *)0,
+#endif
+ };
+ return &nullMem;
+}
+
+/*
+ * Return true if the Mem object contains a TEXT or BLOB that is
+ * too large - whose size exceeds SQL_MAX_LENGTH.
+ */
+int
+sqlVdbeMemTooBig(Mem * p)
+{
+ assert(p->db != 0);
+ if (p->flags & (MEM_Str | MEM_Blob)) {
+ int n = p->n;
+ if (p->flags & MEM_Zero) {
+ n += p->u.nZero;
+ }
+ return n > p->db->aLimit[SQL_LIMIT_LENGTH];
+ }
+ return 0;
+}
+
+/*
+ * Compare two blobs. Return negative, zero, or positive if the first
+ * is less than, equal to, or greater than the second, respectively.
+ * If one blob is a prefix of the other, then the shorter is the lessor.
+ */
+static SQL_NOINLINE int
+sqlBlobCompare(const Mem * pB1, const Mem * pB2)
+{
+ int c;
+ int n1 = pB1->n;
+ int n2 = pB2->n;
+
+ /* It is possible to have a Blob value that has some non-zero content
+ * followed by zero content. But that only comes up for Blobs formed
+ * by the OP_MakeRecord opcode, and such Blobs never get passed into
+ * sqlMemCompare().
+ */
+ assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
+ assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
+
+ if ((pB1->flags | pB2->flags) & MEM_Zero) {
+ if (pB1->flags & pB2->flags & MEM_Zero) {
+ return pB1->u.nZero - pB2->u.nZero;
+ } else if (pB1->flags & MEM_Zero) {
+ if (!isAllZero(pB2->z, pB2->n))
+ return -1;
+ return pB1->u.nZero - n2;
+ } else {
+ if (!isAllZero(pB1->z, pB1->n))
+ return +1;
+ return n1 - pB2->u.nZero;
+ }
+ }
+ c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
+ if (c)
+ return c;
+ return n1 - n2;
+}
+
+/*
+ * Compare the values contained by the two memory cells, returning
+ * negative, zero or positive if pMem1 is less than, equal to, or greater
+ * than pMem2. Sorting order is NULL's first, followed by numbers (integers
+ * and reals) sorted numerically, followed by text ordered by the collating
+ * sequence pColl and finally blob's ordered by memcmp().
+ *
+ * Two NULL values are considered equal by this function.
+ */
+int
+sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
+{
+ int f1, f2;
+ int combined_flags;
+
+ f1 = pMem1->flags;
+ f2 = pMem2->flags;
+ combined_flags = f1 | f2;
+
+ /* If one value is NULL, it is less than the other. If both values
+ * are NULL, return 0.
+ */
+ if (combined_flags & MEM_Null) {
+ return (f2 & MEM_Null) - (f1 & MEM_Null);
+ }
+
+ if ((combined_flags & MEM_Bool) != 0) {
+ if ((f1 & f2 & MEM_Bool) != 0) {
+ if (pMem1->u.b == pMem2->u.b)
+ return 0;
+ if (pMem1->u.b)
+ return 1;
+ return -1;
+ }
+ if ((f2 & MEM_Bool) != 0)
+ return +1;
+ return -1;
+ }
+
+ /* At least one of the two values is a number
+ */
+ if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
+ if ((f1 & f2 & MEM_Int) != 0) {
+ if (pMem1->u.i < pMem2->u.i)
+ return -1;
+ if (pMem1->u.i > pMem2->u.i)
+ return +1;
+ return 0;
+ }
+ if ((f1 & f2 & MEM_UInt) != 0) {
+ if (pMem1->u.u < pMem2->u.u)
+ return -1;
+ if (pMem1->u.u > pMem2->u.u)
+ return +1;
+ return 0;
+ }
+ if ((f1 & f2 & MEM_Real) != 0) {
+ if (pMem1->u.r < pMem2->u.r)
+ return -1;
+ if (pMem1->u.r > pMem2->u.r)
+ return +1;
+ return 0;
+ }
+ if ((f1 & MEM_Int) != 0) {
+ if ((f2 & MEM_Real) != 0) {
+ return double_compare_nint64(pMem2->u.r,
+ pMem1->u.i, -1);
+ } else {
+ return -1;
+ }
+ }
+ if ((f1 & MEM_UInt) != 0) {
+ if ((f2 & MEM_Real) != 0) {
+ return double_compare_uint64(pMem2->u.r,
+ pMem1->u.u, -1);
+ } else if ((f2 & MEM_Int) != 0) {
+ return +1;
+ } else {
+ return -1;
+ }
+ }
+ if ((f1 & MEM_Real) != 0) {
+ if ((f2 & MEM_Int) != 0) {
+ return double_compare_nint64(pMem1->u.r,
+ pMem2->u.i, 1);
+ } else if ((f2 & MEM_UInt) != 0) {
+ return double_compare_uint64(pMem1->u.r,
+ pMem2->u.u, 1);
+ } else {
+ return -1;
+ }
+ }
+ return +1;
+ }
+
+ /* If one value is a string and the other is a blob, the string is less.
+ * If both are strings, compare using the collating functions.
+ */
+ if (combined_flags & MEM_Str) {
+ if ((f1 & MEM_Str) == 0) {
+ return 1;
+ }
+ if ((f2 & MEM_Str) == 0) {
+ return -1;
+ }
+ /* The collation sequence must be defined at this point, even if
+ * the user deletes the collation sequence after the vdbe program is
+ * compiled (this was not always the case).
+ */
+ if (pColl) {
+ return vdbeCompareMemString(pMem1, pMem2, pColl);
+ } else {
+ size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
+ int res;
+ res = memcmp(pMem1->z, pMem2->z, n);
+ if (res == 0)
+ res = (int)pMem1->n - (int)pMem2->n;
+ return res;
+ }
+ /* If a NULL pointer was passed as the collate function, fall through
+ * to the blob case and use memcmp().
+ */
+ }
+
+ /* Both values must be blobs. Compare using memcmp(). */
+ return sqlBlobCompare(pMem1, pMem2);
+}
+
+bool
+mem_is_type_compatible(struct Mem *mem, enum field_type type)
+{
+ enum mp_type mp_type = mem_mp_type(mem);
+ assert(mp_type < MP_EXT);
+ return field_mp_plain_type_is_compatible(type, mp_type, true);
+}
+
+/* Allocate memory for internal VDBE structure on region. */
+int
+vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
+{
+ vdbe_mem->n = size;
+ vdbe_mem->z = region_alloc(&fiber()->gc, size);
+ if (vdbe_mem->z == NULL)
+ return -1;
+ vdbe_mem->flags = MEM_Ephem | MEM_Blob;
+ assert(sqlVdbeCheckMemInvariants(vdbe_mem));
+ return 0;
+}
+
+/*
+ * Make a full copy of pFrom into pTo. Prior contents of pTo are
+ * freed before the copy is made.
+ */
+int
+sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
+{
+ int rc = 0;
+
+ if (VdbeMemDynamic(pTo))
+ vdbeMemClearExternAndSetNull(pTo);
+ memcpy(pTo, pFrom, MEMCELLSIZE);
+ pTo->flags &= ~MEM_Dyn;
+ if (pTo->flags & (MEM_Str | MEM_Blob)) {
+ if (0 == (pFrom->flags & MEM_Static)) {
+ pTo->flags |= MEM_Ephem;
+ rc = sqlVdbeMemMakeWriteable(pTo);
+ }
+ }
+
+ return rc;
+}
+
+void
+sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
+{
+ assert(pTo->db == pFrom->db);
+ if (VdbeMemDynamic(pTo)) {
+ vdbeClrCopy(pTo, pFrom, srcType);
+ return;
+ }
+ memcpy(pTo, pFrom, MEMCELLSIZE);
+ if ((pFrom->flags & MEM_Static) == 0) {
+ pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
+ assert(srcType == MEM_Ephem || srcType == MEM_Static);
+ pTo->flags |= srcType;
+ }
+}
+
+/*
+ * Transfer the contents of pFrom to pTo. Any existing value in pTo is
+ * freed. If pFrom contains ephemeral data, a copy is made.
+ *
+ * pFrom contains an SQL NULL when this routine returns.
+ */
+void
+sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
+{
+ assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
+
+ sqlVdbeMemRelease(pTo);
+ memcpy(pTo, pFrom, sizeof(Mem));
+ pFrom->flags = MEM_Null;
+ pFrom->szMalloc = 0;
+}
+
+/*
+ * Change pMem so that its MEM_Str or MEM_Blob value is stored in
+ * MEM.zMalloc, where it can be safely written.
+ *
+ * Return 0 on success or -1 if malloc fails.
+ */
+int
+sqlVdbeMemMakeWriteable(Mem * pMem)
+{
+ if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
+ if (ExpandBlob(pMem))
+ return -1;
+ if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
+ if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
+ return -1;
+ }
+ pMem->z[pMem->n] = 0;
+ pMem->z[pMem->n + 1] = 0;
+ pMem->flags |= MEM_Term;
+ }
+ }
+ pMem->flags &= ~MEM_Ephem;
+#ifdef SQL_DEBUG
+ pMem->pScopyFrom = 0;
+#endif
+
+ return 0;
+}
+
+/*
+ * Release any memory resources held by the Mem. Both the memory that is
+ * free by Mem.xDel and the Mem.zMalloc allocation are freed.
+ *
+ * Use this routine prior to clean up prior to abandoning a Mem, or to
+ * reset a Mem back to its minimum memory utilization.
+ *
+ * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
+ * prior to inserting new content into the Mem.
+ */
+void
+sqlVdbeMemRelease(Mem * p)
+{
+ assert(sqlVdbeCheckMemInvariants(p));
+ if (VdbeMemDynamic(p) || p->szMalloc) {
+ vdbeMemClear(p);
+ }
+}
+
+int
+sql_vdbemem_finalize(struct Mem *mem, struct func *func)
+{
+ assert(func != NULL);
+ assert(func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
+ assert(func->def->aggregate == FUNC_AGGREGATE_GROUP);
+ assert((mem->flags & MEM_Null) != 0 || func == mem->u.func);
+ sql_context ctx;
+ memset(&ctx, 0, sizeof(ctx));
+ Mem t;
+ memset(&t, 0, sizeof(t));
+ t.flags = MEM_Null;
+ t.db = mem->db;
+ t.field_type = field_type_MAX;
+ ctx.pOut = &t;
+ ctx.pMem = mem;
+ ctx.func = func;
+ ((struct func_sql_builtin *)func)->finalize(&ctx);
+ assert((mem->flags & MEM_Dyn) == 0);
+ if (mem->szMalloc > 0)
+ sqlDbFree(mem->db, mem->zMalloc);
+ memcpy(mem, &t, sizeof(t));
+ return ctx.is_aborted ? -1 : 0;
+}
+
+int
+sqlVdbeCompareMsgpack(const char **key1,
+ struct UnpackedRecord *unpacked, int key2_idx)
+{
+ const char *aKey1 = *key1;
+ Mem *pKey2 = unpacked->aMem + key2_idx;
+ Mem mem1;
+ int rc = 0;
+ switch (mp_typeof(*aKey1)) {
+ default:{
+ /* FIXME */
+ rc = -1;
+ break;
+ }
+ case MP_NIL:{
+ rc = -((pKey2->flags & MEM_Null) == 0);
+ mp_decode_nil(&aKey1);
+ break;
+ }
+ case MP_BOOL:{
+ mem1.u.b = mp_decode_bool(&aKey1);
+ if ((pKey2->flags & MEM_Bool) != 0) {
+ if (mem1.u.b != pKey2->u.b)
+ rc = mem1.u.b ? 1 : -1;
+ } else {
+ rc = (pKey2->flags & MEM_Null) != 0 ? 1 : -1;
+ }
+ break;
+ }
+ case MP_UINT:{
+ mem1.u.u = mp_decode_uint(&aKey1);
+ if ((pKey2->flags & MEM_Int) != 0) {
+ rc = +1;
+ } else if ((pKey2->flags & MEM_UInt) != 0) {
+ if (mem1.u.u < pKey2->u.u)
+ rc = -1;
+ else if (mem1.u.u > pKey2->u.u)
+ rc = +1;
+ } else if ((pKey2->flags & MEM_Real) != 0) {
+ rc = double_compare_uint64(pKey2->u.r,
+ mem1.u.u, -1);
+ } else if ((pKey2->flags & MEM_Null) != 0) {
+ rc = 1;
+ } else if ((pKey2->flags & MEM_Bool) != 0) {
+ rc = 1;
+ } else {
+ rc = -1;
+ }
+ break;
+ }
+ case MP_INT:{
+ mem1.u.i = mp_decode_int(&aKey1);
+ if ((pKey2->flags & MEM_UInt) != 0) {
+ rc = -1;
+ } else if ((pKey2->flags & MEM_Int) != 0) {
+ if (mem1.u.i < pKey2->u.i) {
+ rc = -1;
+ } else if (mem1.u.i > pKey2->u.i) {
+ rc = +1;
+ }
+ } else if (pKey2->flags & MEM_Real) {
+ rc = double_compare_nint64(pKey2->u.r, mem1.u.i,
+ -1);
+ } else if ((pKey2->flags & MEM_Null) != 0) {
+ rc = 1;
+ } else if ((pKey2->flags & MEM_Bool) != 0) {
+ rc = 1;
+ } else {
+ rc = -1;
+ }
+ break;
+ }
+ case MP_FLOAT:{
+ mem1.u.r = mp_decode_float(&aKey1);
+ goto do_float;
+ }
+ case MP_DOUBLE:{
+ mem1.u.r = mp_decode_double(&aKey1);
+ do_float:
+ if ((pKey2->flags & MEM_Int) != 0) {
+ rc = double_compare_nint64(mem1.u.r, pKey2->u.i,
+ 1);
+ } else if (pKey2->flags & MEM_UInt) {
+ rc = double_compare_uint64(mem1.u.r,
+ pKey2->u.u, 1);
+ } else if (pKey2->flags & MEM_Real) {
+ if (mem1.u.r < pKey2->u.r) {
+ rc = -1;
+ } else if (mem1.u.r > pKey2->u.r) {
+ rc = +1;
+ }
+ } else if ((pKey2->flags & MEM_Null) != 0) {
+ rc = 1;
+ } else if ((pKey2->flags & MEM_Bool) != 0) {
+ rc = 1;
+ } else {
+ rc = -1;
+ }
+ break;
+ }
+ case MP_STR:{
+ if (pKey2->flags & MEM_Str) {
+ struct key_def *key_def = unpacked->key_def;
+ mem1.n = mp_decode_strl(&aKey1);
+ mem1.z = (char *)aKey1;
+ aKey1 += mem1.n;
+ struct coll *coll =
+ key_def->parts[key2_idx].coll;
+ if (coll != NULL) {
+ mem1.flags = MEM_Str;
+ rc = vdbeCompareMemString(&mem1, pKey2,
+ coll);
+ } else {
+ goto do_bin_cmp;
+ }
+ } else {
+ rc = (pKey2->flags & MEM_Blob) ? -1 : +1;
+ }
+ break;
+ }
+ case MP_BIN:{
+ mem1.n = mp_decode_binl(&aKey1);
+ mem1.z = (char *)aKey1;
+ aKey1 += mem1.n;
+ do_blob:
+ if (pKey2->flags & MEM_Blob) {
+ if (pKey2->flags & MEM_Zero) {
+ if (!isAllZero
+ ((const char *)mem1.z, mem1.n)) {
+ rc = 1;
+ } else {
+ rc = mem1.n - pKey2->u.nZero;
+ }
+ } else {
+ int nCmp;
+ do_bin_cmp:
+ nCmp = MIN(mem1.n, pKey2->n);
+ rc = memcmp(mem1.z, pKey2->z, nCmp);
+ if (rc == 0)
+ rc = mem1.n - pKey2->n;
+ }
+ } else {
+ rc = 1;
+ }
+ break;
+ }
+ case MP_ARRAY:
+ case MP_MAP:
+ case MP_EXT:{
+ mem1.z = (char *)aKey1;
+ mp_next(&aKey1);
+ mem1.n = aKey1 - (char *)mem1.z;
+ goto do_blob;
+ }
+ }
+ *key1 = aKey1;
+ return rc;
+}
+
+int
+sqlVdbeRecordCompareMsgpack(const void *key1,
+ struct UnpackedRecord *key2)
+{
+ int rc = 0;
+ u32 i, n = mp_decode_array((const char**)&key1);
+
+ n = MIN(n, key2->nField);
+
+ for (i = 0; i != n; i++) {
+ rc = sqlVdbeCompareMsgpack((const char**)&key1, key2, i);
+ if (rc != 0) {
+ if (key2->key_def->parts[i].sort_order !=
+ SORT_ORDER_ASC) {
+ rc = -rc;
+ }
+ return rc;
+ }
+ }
+
+ key2->eqSeen = 1;
+ return key2->default_rc;
+}
+
+int
+vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
+{
+ const char *start_buf = buf;
+ switch (mp_typeof(*buf)) {
+ case MP_ARRAY: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_ARRAY;
+ break;
+ }
+ case MP_MAP: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = FIELD_TYPE_MAP;
+ break;
+ }
+ case MP_EXT: {
+ mem->z = (char *)buf;
+ mp_next(&buf);
+ mem->n = buf - mem->z;
+ mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+ break;
+ }
+ case MP_NIL: {
+ mp_decode_nil(&buf);
+ mem->flags = MEM_Null;
+ mem->field_type = field_type_MAX;
+ break;
+ }
+ case MP_BOOL: {
+ mem->u.b = mp_decode_bool(&buf);
+ mem->flags = MEM_Bool;
+ mem->field_type = FIELD_TYPE_BOOLEAN;
+ break;
+ }
+ case MP_UINT: {
+ uint64_t v = mp_decode_uint(&buf);
+ mem->u.u = v;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ break;
+ }
+ case MP_INT: {
+ mem->u.i = mp_decode_int(&buf);
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ break;
+ }
+ case MP_STR: {
+ /* XXX u32->int */
+ mem->n = (int) mp_decode_strl(&buf);
+ mem->flags = MEM_Str | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_STRING;
+install_blob:
+ mem->z = (char *)buf;
+ buf += mem->n;
+ break;
+ }
+ case MP_BIN: {
+ /* XXX u32->int */
+ mem->n = (int) mp_decode_binl(&buf);
+ mem->flags = MEM_Blob | MEM_Ephem;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+ goto install_blob;
+ }
+ case MP_FLOAT: {
+ mem->u.r = mp_decode_float(&buf);
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
+ break;
+ }
+ case MP_DOUBLE: {
+ mem->u.r = mp_decode_double(&buf);
+ if (sqlIsNaN(mem->u.r)) {
+ mem->flags = MEM_Null;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ } else {
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ }
+ break;
+ }
+ default:
+ unreachable();
+ }
+ *len = (uint32_t)(buf - start_buf);
+ return 0;
+}
+
+void
+mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
+{
+ assert(memIsValid(var));
+ int64_t i;
+ if (var->flags & MEM_Null) {
+ mpstream_encode_nil(stream);
+ } else if (var->flags & MEM_Real) {
+ mpstream_encode_double(stream, var->u.r);
+ } else if (var->flags & MEM_Int) {
+ i = var->u.i;
+ mpstream_encode_int(stream, i);
+ } else if (var->flags & MEM_UInt) {
+ i = var->u.u;
+ mpstream_encode_uint(stream, i);
+ } else if (var->flags & MEM_Str) {
+ mpstream_encode_strn(stream, var->z, var->n);
+ } else if (var->flags & MEM_Bool) {
+ mpstream_encode_bool(stream, var->u.b);
+ } else {
+ /*
+ * Emit BIN header iff the BLOB doesn't store
+ * MsgPack content.
+ */
+ if (!mem_has_msgpack_subtype(var)) {
+ uint32_t binl = var->n +
+ ((var->flags & MEM_Zero) ?
+ var->u.nZero : 0);
+ mpstream_encode_binl(stream, binl);
+ }
+ mpstream_memcpy(stream, var->z, var->n);
+ if (var->flags & MEM_Zero)
+ mpstream_memset(stream, 0, var->u.nZero);
+ }
+}
+
+char *
+sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
+ uint32_t *tuple_size, struct region *region)
+{
+ size_t used = region_used(region);
+ bool is_error = false;
+ struct mpstream stream;
+ mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
+ set_encode_error, &is_error);
+ mpstream_encode_array(&stream, field_count);
+ for (struct Mem *field = fields; field < fields + field_count; field++)
+ mpstream_encode_vdbe_mem(&stream, field);
+ mpstream_flush(&stream);
+ if (is_error) {
+ diag_set(OutOfMemory, stream.pos - stream.buf,
+ "mpstream_flush", "stream");
+ return NULL;
+ }
+ *tuple_size = region_used(region) - used;
+ char *tuple = region_join(region, *tuple_size);
+ if (tuple == NULL) {
+ diag_set(OutOfMemory, *tuple_size, "region_join", "tuple");
+ return NULL;
+ }
+ mp_tuple_assert(tuple, tuple + *tuple_size);
+ return tuple;
+}
+
+/**
+ * Allocate a sequence of initialized vdbe memory registers
+ * on region.
+ */
+static struct Mem *
+vdbemem_alloc_on_region(uint32_t count)
+{
+ struct region *region = &fiber()->gc;
+ size_t size;
+ struct Mem *ret = region_alloc_array(region, typeof(*ret), count,
+ &size);
+ if (ret == NULL) {
+ diag_set(OutOfMemory, size, "region_alloc_array", "ret");
+ return NULL;
+ }
+ memset(ret, 0, count * sizeof(*ret));
+ for (uint32_t i = 0; i < count; i++) {
+ sqlVdbeMemInit(&ret[i], sql_get(), MEM_Null);
+ assert(memIsValid(&ret[i]));
+ }
+ return ret;
+}
+
+static void
+port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
+{
+ (void) is_flat;
+ struct port_vdbemem *port = (struct port_vdbemem *) base;
+ assert(is_flat == true);
+ for (uint32_t i = 0; i < port->mem_count; i++) {
+ sql_value *param =
+ (sql_value *)((struct Mem *)port->mem + i);
+ switch (sql_value_type(param)) {
+ case MP_INT:
+ luaL_pushint64(L, sql_value_int64(param));
+ break;
+ case MP_UINT:
+ luaL_pushuint64(L, sql_value_uint64(param));
+ break;
+ case MP_DOUBLE:
+ lua_pushnumber(L, sql_value_double(param));
+ break;
+ case MP_STR:
+ lua_pushlstring(L, (const char *)sql_value_text(param),
+ (size_t)sql_value_bytes(param));
+ break;
+ case MP_BIN:
+ case MP_ARRAY:
+ case MP_MAP:
+ lua_pushlstring(L, sql_value_blob(param),
+ (size_t) sql_value_bytes(param));
+ break;
+ case MP_NIL:
+ lua_pushnil(L);
+ break;
+ case MP_BOOL:
+ lua_pushboolean(L, sql_value_boolean(param));
+ break;
+ default:
+ unreachable();
+ }
+ }
+}
+
+static const char *
+port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
+{
+ struct port_vdbemem *port = (struct port_vdbemem *) base;
+ struct region *region = &fiber()->gc;
+ size_t region_svp = region_used(region);
+ bool is_error = false;
+ struct mpstream stream;
+ mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
+ set_encode_error, &is_error);
+ mpstream_encode_array(&stream, port->mem_count);
+ for (uint32_t i = 0; i < port->mem_count && !is_error; i++) {
+ sql_value *param =
+ (sql_value *)((struct Mem *)port->mem + i);
+ switch (sql_value_type(param)) {
+ case MP_INT: {
+ sql_int64 val = sql_value_int64(param);
+ if (val < 0) {
+ mpstream_encode_int(&stream, val);
+ break;
+ }
+ FALLTHROUGH;
+ }
+ case MP_UINT: {
+ sql_uint64 val = sql_value_uint64(param);
+ mpstream_encode_uint(&stream, val);
+ break;
+ }
+ case MP_DOUBLE: {
+ mpstream_encode_double(&stream,
+ sql_value_double(param));
+ break;
+ }
+ case MP_STR: {
+ const char *str = (const char *) sql_value_text(param);
+ mpstream_encode_strn(&stream, str,
+ sql_value_bytes(param));
+ break;
+ }
+ case MP_BIN:
+ case MP_ARRAY:
+ case MP_MAP: {
+ mpstream_encode_binl(&stream, sql_value_bytes(param));
+ mpstream_memcpy(&stream, sql_value_blob(param),
+ sql_value_bytes(param));
+ break;
+ }
+ case MP_NIL: {
+ mpstream_encode_nil(&stream);
+ break;
+ }
+ case MP_BOOL: {
+ mpstream_encode_bool(&stream, sql_value_boolean(param));
+ break;
+ }
+ default:
+ unreachable();
+ }
+ }
+ mpstream_flush(&stream);
+ *size = region_used(region) - region_svp;
+ if (is_error)
+ goto error;
+ const char *ret = (char *)region_join(region, *size);
+ if (ret == NULL)
+ goto error;
+ return ret;
+error:
+ diag_set(OutOfMemory, *size, "region", "ret");
+ return NULL;
+}
+
+static const struct port_vtab port_vdbemem_vtab;
+
+void
+port_vdbemem_create(struct port *base, struct sql_value *mem,
+ uint32_t mem_count)
+{
+ struct port_vdbemem *port = (struct port_vdbemem *) base;
+ port->vtab = &port_vdbemem_vtab;
+ port->mem = mem;
+ port->mem_count = mem_count;
+}
+
+static struct sql_value *
+port_vdbemem_get_vdbemem(struct port *base, uint32_t *mem_count)
+{
+ struct port_vdbemem *port = (struct port_vdbemem *) base;
+ assert(port->vtab == &port_vdbemem_vtab);
+ *mem_count = port->mem_count;
+ return port->mem;
+}
+
+static const struct port_vtab port_vdbemem_vtab = {
+ .dump_msgpack = NULL,
+ .dump_msgpack_16 = NULL,
+ .dump_lua = port_vdbemem_dump_lua,
+ .dump_plain = NULL,
+ .get_msgpack = port_vdbemem_get_msgpack,
+ .get_vdbemem = port_vdbemem_get_vdbemem,
+ .destroy = NULL,
+};
+
+struct sql_value *
+port_lua_get_vdbemem(struct port *base, uint32_t *size)
+{
+ struct port_lua *port = (struct port_lua *) base;
+ struct lua_State *L = port->L;
+ int argc = lua_gettop(L);
+ if (argc == 0 || argc > 1) {
+ diag_set(ClientError, ER_SQL_FUNC_WRONG_RET_COUNT, "Lua", argc);
+ return NULL;
+ }
+ *size = argc;
+ /** FIXME: Implement an ability to return a vector. */
+ assert(*size == 1);
+ struct region *region = &fiber()->gc;
+ size_t region_svp = region_used(region);
+ struct Mem *val = vdbemem_alloc_on_region(argc);
+ if (val == NULL)
+ return NULL;
+ for (int i = 0; i < argc; i++) {
+ struct luaL_field field;
+ if (luaL_tofield(L, luaL_msgpack_default,
+ NULL, -1 - i, &field) < 0) {
+ goto error;
+ }
+ switch (field.type) {
+ case MP_BOOL:
+ mem_set_bool(&val[i], field.bval);
+ break;
+ case MP_FLOAT:
+ mem_set_double(&val[i], field.fval);
+ break;
+ case MP_DOUBLE:
+ mem_set_double(&val[i], field.dval);
+ break;
+ case MP_INT:
+ mem_set_i64(&val[i], field.ival);
+ break;
+ case MP_UINT:
+ mem_set_u64(&val[i], field.ival);
+ break;
+ case MP_STR:
+ if (sqlVdbeMemSetStr(&val[i], field.sval.data,
+ field.sval.len, 1,
+ SQL_TRANSIENT) != 0)
+ goto error;
+ break;
+ case MP_NIL:
+ sqlVdbeMemSetNull(&val[i]);
+ break;
+ default:
+ diag_set(ClientError, ER_SQL_EXECUTE,
+ "Unsupported type passed from Lua");
+ goto error;
+ }
+ }
+ return (struct sql_value *)val;
+error:
+ for (int i = 0; i < argc; i++)
+ sqlVdbeMemRelease(&val[i]);
+ region_truncate(region, region_svp);
+ return NULL;
+}
+
+struct sql_value *
+port_c_get_vdbemem(struct port *base, uint32_t *size)
+{
+ struct port_c *port = (struct port_c *)base;
+ *size = port->size;
+ if (*size == 0 || *size > 1) {
+ diag_set(ClientError, ER_SQL_FUNC_WRONG_RET_COUNT, "C", *size);
+ return NULL;
+ }
+ /** FIXME: Implement an ability to return a vector. */
+ assert(*size == 1);
+ struct region *region = &fiber()->gc;
+ size_t region_svp = region_used(region);
+ struct Mem *val = vdbemem_alloc_on_region(port->size);
+ if (val == NULL)
+ return NULL;
+ int i = 0;
+ const char *data;
+ struct port_c_entry *pe;
+ for (pe = port->first; pe != NULL; pe = pe->next) {
+ if (pe->mp_size == 0) {
+ data = tuple_data(pe->tuple);
+ if (mp_decode_array(&data) != 1) {
+ diag_set(ClientError, ER_SQL_EXECUTE,
+ "Unsupported type passed from C");
+ goto error;
+ }
+ } else {
+ data = pe->mp;
+ }
+ uint32_t len;
+ const char *str;
+ switch (mp_typeof(*data)) {
+ case MP_BOOL:
+ mem_set_bool(&val[i], mp_decode_bool(&data));
+ break;
+ case MP_FLOAT:
+ mem_set_double(&val[i], mp_decode_float(&data));
+ break;
+ case MP_DOUBLE:
+ mem_set_double(&val[i], mp_decode_double(&data));
+ break;
+ case MP_INT:
+ mem_set_i64(&val[i], mp_decode_int(&data));
+ break;
+ case MP_UINT:
+ mem_set_u64(&val[i], mp_decode_uint(&data));
+ break;
+ case MP_STR:
+ str = mp_decode_str(&data, &len);
+ if (sqlVdbeMemSetStr(&val[i], str, len,
+ 1, SQL_TRANSIENT) != 0)
+ goto error;
+ break;
+ case MP_NIL:
+ sqlVdbeMemSetNull(&val[i]);
+ break;
+ default:
+ diag_set(ClientError, ER_SQL_EXECUTE,
+ "Unsupported type passed from C");
+ goto error;
+ }
+ i++;
+ }
+ return (struct sql_value *) val;
+error:
+ for (int i = 0; i < port->size; i++)
+ sqlVdbeMemRelease(&val[i]);
+ region_truncate(region, region_svp);
+ return NULL;
+}
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
new file mode 100644
index 000000000..acc8ce054
--- /dev/null
+++ b/src/box/sql/mem.h
@@ -0,0 +1,485 @@
+#pragma once
+/*
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "box/field_def.h"
+
+struct sql;
+struct Vdbe;
+struct region;
+struct mpstream;
+struct VdbeFrame;
+
+/*
+ * Internally, the vdbe manipulates nearly all SQL values as Mem
+ * structures. Each Mem struct may cache multiple representations (string,
+ * integer etc.) of the same value.
+ */
+struct Mem {
+ union MemValue {
+ double r; /* Real value used when MEM_Real is set in flags */
+ i64 i; /* Integer value used when MEM_Int is set in flags */
+ uint64_t u; /* Unsigned integer used when MEM_UInt is set. */
+ bool b; /* Boolean value used when MEM_Bool is set in flags */
+ int nZero; /* Used when bit MEM_Zero is set in flags */
+ void *p; /* Generic pointer */
+ /**
+ * A pointer to function implementation.
+ * Used only when flags==MEM_Agg.
+ */
+ struct func *func;
+ struct VdbeFrame *pFrame; /* Used when flags==MEM_Frame */
+ } u;
+ u32 flags; /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
+ /** Subtype for this value. */
+ enum sql_subtype subtype;
+ /**
+ * If value is fetched from tuple, then this property
+ * contains type of corresponding space's field. If it's
+ * value field_type_MAX then we can rely on on format
+ * (msgpack) type which is represented by 'flags'.
+ */
+ enum field_type field_type;
+ int n; /* size (in bytes) of string value, excluding trailing '\0' */
+ char *z; /* String or BLOB value */
+ /* ShallowCopy only needs to copy the information above */
+ char *zMalloc; /* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
+ int szMalloc; /* Size of the zMalloc allocation */
+ u32 uTemp; /* Transient storage for serial_type in OP_MakeRecord */
+ sql *db; /* The associated database connection */
+ void (*xDel) (void *); /* Destructor for Mem.z - only valid if MEM_Dyn */
+#ifdef SQL_DEBUG
+ Mem *pScopyFrom; /* This Mem is a shallow copy of pScopyFrom */
+ void *pFiller; /* So that sizeof(Mem) is a multiple of 8 */
+#endif
+};
+
+/*
+ * Size of struct Mem not including the Mem.zMalloc member or anything that
+ * follows.
+ */
+#define MEMCELLSIZE offsetof(Mem,zMalloc)
+
+/* One or more of the following flags are set to indicate the validOK
+ * representations of the value stored in the Mem struct.
+ *
+ * If the MEM_Null flag is set, then the value is an SQL NULL value.
+ * No other flags may be set in this case.
+ *
+ * If the MEM_Str flag is set then Mem.z points at a string representation.
+ * Usually this is encoded in the same unicode encoding as the main
+ * database (see below for exceptions). If the MEM_Term flag is also
+ * set, then the string is nul terminated. The MEM_Int and MEM_Real
+ * flags may coexist with the MEM_Str flag.
+ */
+#define MEM_Null 0x0001 /* Value is NULL */
+#define MEM_Str 0x0002 /* Value is a string */
+#define MEM_Int 0x0004 /* Value is an integer */
+#define MEM_Real 0x0008 /* Value is a real number */
+#define MEM_Blob 0x0010 /* Value is a BLOB */
+#define MEM_Bool 0x0020 /* Value is a bool */
+#define MEM_UInt 0x0040 /* Value is an unsigned integer */
+#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
+#define MEM_Undefined 0x0100 /* Value is undefined */
+#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
+#define MEM_TypeMask 0x83ff /* Mask of type bits */
+
+/* Whenever Mem contains a valid string or blob representation, one of
+ * the following flags must be set to determine the memory management
+ * policy for Mem.z. The MEM_Term flag tells us whether or not the
+ * string is \000 or \u0000 terminated
+ */
+#define MEM_Term 0x0400 /* String rep is nul terminated */
+#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
+#define MEM_Static 0x1000 /* Mem.z points to a static string */
+#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
+#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
+#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
+#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
+#define MEM_Ptr 0x20000 /* Value is a generic pointer */
+
+/**
+ * In contrast to Mem_TypeMask, this one allows to get
+ * pure type of memory cell, i.e. without _Dyn/_Zero and other
+ * auxiliary flags.
+ */
+enum {
+ MEM_PURE_TYPE_MASK = 0x7f
+};
+
+static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
+ MEM_Blob | MEM_Bool | MEM_UInt),
+ "value of type mask must consist of corresponding to memory "\
+ "type bits");
+
+/**
+ * Simple type to str convertor. It is used to simplify
+ * error reporting.
+ */
+char *
+mem_type_to_str(const struct Mem *p);
+
+/*
+ * Return the MP_type of the value of the MEM.
+ * Analogue of sql_value_type() but operates directly on
+ * transparent memory cell.
+ */
+enum mp_type
+mem_mp_type(struct Mem *mem);
+
+enum mp_type
+sql_value_type(struct Mem *);
+u16
+numericType(Mem *pMem);
+
+int sqlValueBytes(struct Mem *);
+
+#ifdef SQL_DEBUG
+int sqlVdbeCheckMemInvariants(struct Mem *);
+void sqlVdbeMemPrettyPrint(Mem * pMem, char *zBuf);
+void
+registerTrace(int iReg, Mem *p);
+
+/*
+ * Return true if a memory cell is not marked as invalid. This macro
+ * is for use inside assert() statements only.
+ */
+#define memIsValid(M) ((M)->flags & MEM_Undefined)==0
+#endif
+
+/**
+ * Try to convert a string value into a numeric representation
+ * if we can do so without loss of information. Firstly, value
+ * is attempted to be converted to integer, and in case of fail -
+ * to floating point number. Note that function is assumed to be
+ * called on memory cell containing string, i.e. mem->type == MEM_Str.
+ *
+ * @param record Memory cell containing value to be converted.
+ * @retval 0 If value can be converted to integer or number.
+ * @retval -1 Otherwise.
+ */
+int
+mem_apply_numeric_type(struct Mem *record);
+int sqlVdbeMemRealify(struct Mem *);
+
+/**
+ * Convert @a mem to NUMBER type, so that after conversion it has
+ * one of types MEM_Real, MEM_Int or MEM_UInt. If conversion is
+ * not possible, function returns -1.
+ *
+ * Beware - this function changes value and type of @a mem
+ * argument.
+ */
+int
+vdbe_mem_numerify(struct Mem *mem);
+
+int sqlVdbeMemCast(struct Mem *, enum field_type type);
+int mem_apply_integer_type(struct Mem *);
+int sqlVdbeMemStringify(struct Mem *);
+int sqlVdbeMemNulTerminate(struct Mem *);
+int sqlVdbeMemExpandBlob(struct Mem *);
+#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
+void sql_value_apply_type(struct Mem *val, enum field_type type);
+
+
+/**
+ * Processing is determined by the field type parameter:
+ *
+ * INTEGER:
+ * If memory holds floating point value and it can be
+ * converted without loss (2.0 - > 2), it's type is
+ * changed to INT. Otherwise, simply return success status.
+ *
+ * NUMBER:
+ * If memory holds INT or floating point value,
+ * no actions take place.
+ *
+ * STRING:
+ * Convert mem to a string representation.
+ *
+ * SCALAR:
+ * Mem is unchanged, but flag is set to BLOB in case of
+ * scalar-like type. Otherwise, (MAP, ARRAY) conversion
+ * is impossible.
+ *
+ * BOOLEAN:
+ * If memory holds BOOLEAN no actions take place.
+ *
+ * ANY:
+ * Mem is unchanged, no actions take place.
+ *
+ * MAP/ARRAY:
+ * These types can't be casted to scalar ones, or to each
+ * other. So the only valid conversion is to type itself.
+ *
+ * @param record The value to apply type to.
+ * @param type The type to be applied.
+ */
+int
+mem_apply_type(struct Mem *record, enum field_type type);
+
+/**
+ * Convert the numeric value contained in MEM to another numeric
+ * type.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @param type The type to convert to.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+int
+mem_convert_to_numeric(struct Mem *mem, enum field_type type);
+
+/** Setters = Change MEM value. */
+
+int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
+int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
+
+void
+mem_set_bool(struct Mem *mem, bool value);
+
+/**
+ * Set VDBE memory register with given pointer as a data.
+ * @param mem VDBE memory register to update.
+ * @param ptr Pointer to use.
+ */
+void
+mem_set_ptr(struct Mem *mem, void *ptr);
+
+/**
+ * Set integer value. Depending on its sign MEM_Int (in case
+ * of negative value) or MEM_UInt flag is set.
+ */
+void
+mem_set_i64(struct Mem *mem, int64_t value);
+
+/** Set unsigned value and MEM_UInt flag. */
+void
+mem_set_u64(struct Mem *mem, uint64_t value);
+
+/**
+ * Set integer value. According to is_neg flag value is considered
+ * to be signed or unsigned.
+ */
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
+
+/** Set double value and MEM_Real flag. */
+void
+mem_set_double(struct Mem *mem, double value);
+
+int
+sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
+void
+sqlVdbeMemInit(struct Mem *, sql *, u32);
+void
+sqlVdbeMemSetNull(struct Mem *);
+void
+sqlVdbeMemSetZeroBlob(struct Mem *, int);
+void sqlValueSetStr(struct Mem *, int, const void *,
+ void (*)(void *));
+void sqlValueSetNull(struct Mem *);
+void sqlValueFree(struct Mem *);
+struct Mem *sqlValueNew(struct sql *);
+
+/*
+ * Initialize an array of N Mem element.
+ */
+void
+initMemArray(Mem * p, int N, sql * db, u32 flags);
+
+/*
+ * Release an array of N Mem elements
+ */
+void
+releaseMemArray(Mem * p, int N);
+
+/*
+ * Clear any existing type flags from a Mem and replace them with f
+ */
+#define MemSetTypeFlag(p, f) \
+ ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f)
+
+/** Getters. */
+
+int
+mem_value_bool(const struct Mem *mem, bool *b);
+int sqlVdbeIntValue(struct Mem *, int64_t *, bool *is_neg);
+int sqlVdbeRealValue(struct Mem *, double *);
+const void *
+sql_value_blob(struct Mem *);
+
+int
+sql_value_bytes(struct Mem *);
+
+double
+sql_value_double(struct Mem *);
+
+bool
+sql_value_boolean(struct Mem *val);
+
+int
+sql_value_int(struct Mem *);
+
+sql_int64
+sql_value_int64(struct Mem *);
+
+uint64_t
+sql_value_uint64(struct Mem *val);
+
+const unsigned char *
+sql_value_text(struct Mem *);
+
+const void *sqlValueText(struct Mem *);
+
+/**
+ * Return pointer to a string with the data type in the case of
+ * binary data stored in @a value. Otherwise, return the result
+ * of sql_value_text(). It is used due to the fact that not all
+ * binary strings can be displayed correctly (e.g. contain
+ * unprintable symbols).
+ */
+const char *
+sql_value_to_diag_str(struct Mem *value);
+#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
+
+enum sql_subtype
+sql_value_subtype(sql_value * pVal);
+
+const Mem *
+columnNullValue(void);
+
+/** Checkers. */
+
+static inline bool
+sql_value_is_null(struct Mem *value)
+{
+ return sql_value_type(value) == MP_NIL;
+}
+
+int sqlVdbeMemTooBig(Mem *);
+
+/* Return TRUE if Mem X contains dynamically allocated content - anything
+ * that needs to be deallocated to avoid a leak.
+ */
+#define VdbeMemDynamic(X) \
+ (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
+
+
+int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
+
+/**
+ * Check that MEM_type of the mem is compatible with given type.
+ *
+ * @param mem The MEM that contains the value to check.
+ * @param type The type to check.
+ * @retval TRUE if the MEM_type of the value and the given type
+ * are compatible, FALSE otherwise.
+ */
+bool
+mem_is_type_compatible(struct Mem *mem, enum field_type type);
+
+/** MEM manipulate functions. */
+
+int
+vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
+int sqlVdbeMemCopy(Mem *, const Mem *);
+void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
+void sqlVdbeMemMove(Mem *, Mem *);
+int sqlVdbeMemMakeWriteable(Mem *);
+void sqlVdbeMemRelease(Mem * p);
+
+/**
+ * Memory cell mem contains the context of an aggregate function.
+ * This routine calls the finalize method for that function. The
+ * result of the aggregate is stored back into mem.
+ *
+ * Returns -1 if the finalizer reports an error. 0 otherwise.
+ */
+int
+sql_vdbemem_finalize(struct Mem *mem, struct func *func);
+
+/** MEM and msgpack functions. */
+
+/**
+ * Perform comparison of two keys: one is packed and one is not.
+ *
+ * @param key1 Pointer to pointer to first key.
+ * @param unpacked Pointer to unpacked tuple.
+ * @param key2_idx index of key in umpacked record to compare.
+ *
+ * @retval +1 if key1 > pUnpacked[iKey2], -1 ptherwise.
+ */
+int sqlVdbeCompareMsgpack(const char **key1,
+ struct UnpackedRecord *unpacked, int key2_idx);
+
+/**
+ * Perform comparison of two tuples: unpacked (key1) and packed (key2)
+ *
+ * @param key1 Packed key.
+ * @param unpacked Unpacked key.
+ *
+ * @retval +1 if key1 > unpacked, -1 otherwise.
+ */
+int sqlVdbeRecordCompareMsgpack(const void *key1,
+ struct UnpackedRecord *key2);
+
+/**
+ * Decode msgpack and save value into VDBE memory cell.
+ *
+ * @param buf Buffer to deserialize msgpack from.
+ * @param mem Memory cell to write value into.
+ * @param len[out] Length of decoded part.
+ * @retval Return code: < 0 in case of error.
+ * @retval 0 on success.
+ */
+int
+vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len);
+
+/**
+ * Perform encoding memory variable to stream.
+ * @param stream Initialized mpstream encoder object.
+ * @param var Vdbe memory variable to encode with stream.
+ */
+void
+mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var);
+
+/**
+ * Perform encoding field_count Vdbe memory fields on region as
+ * msgpack array.
+ * @param fields The first Vdbe memory field to encode.
+ * @param field_count Count of fields to encode.
+ * @param[out] tuple_size Size of encoded tuple.
+ * @param region Region to use.
+ * @retval NULL on error, diag message is set.
+ * @retval Pointer to valid tuple on success.
+ */
+char *
+sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
+ uint32_t *tuple_size, struct region *region);
diff --git a/src/box/sql/pragma.c b/src/box/sql/pragma.c
index c15f2e0d1..b820b492e 100644
--- a/src/box/sql/pragma.c
+++ b/src/box/sql/pragma.c
@@ -39,6 +39,7 @@
#include "box/coll_id_cache.h"
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "box/schema.h"
#include "box/session.h"
diff --git a/src/box/sql/printf.c b/src/box/sql/printf.c
index a3ff5bb09..cf32ba3f3 100644
--- a/src/box/sql/printf.c
+++ b/src/box/sql/printf.c
@@ -10,6 +10,7 @@
* sql.
*/
#include "sqlInt.h"
+#include "mem.h"
/*
* Conversion types fall into various categories as defined by the
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index fb2795103..0fa388ae9 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -36,6 +36,7 @@
#include "coll/coll.h"
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "box/box.h"
#include "box/coll_id_cache.h"
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 95a6fedc9..c1a42fc2f 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -351,20 +351,6 @@ sql_stricmp(const char *, const char *);
int
sql_strnicmp(const char *, const char *, int);
- const void *
-sql_value_blob(sql_value *);
-
-int
-sql_value_bytes(sql_value *);
-
-double
-sql_value_double(sql_value *);
-
-bool
-sql_value_boolean(sql_value *val);
-
-int
-sql_value_int(sql_value *);
/**
* Get row column subtype.
@@ -375,34 +361,6 @@ sql_value_int(sql_value *);
enum sql_subtype
sql_column_subtype(struct sql_stmt *stmt, int i);
-sql_int64
-sql_value_int64(sql_value *);
-
-uint64_t
-sql_value_uint64(sql_value *val);
-
-const unsigned char *
-sql_value_text(sql_value *);
-
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(sql_value *value);
-
-enum mp_type
-sql_value_type(sql_value *);
-
-static inline bool
-sql_value_is_null(sql_value *value)
-{
- return sql_value_type(value) == MP_NIL;
-}
-
sql *
sql_context_db_handle(sql_context *);
@@ -4000,16 +3958,8 @@ int
sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
int64_t *res, bool *is_res_neg);
-const void *sqlValueText(sql_value *);
-int sqlValueBytes(sql_value *);
-void sqlValueSetStr(sql_value *, int, const void *,
- void (*)(void *));
-void sqlValueSetNull(sql_value *);
-void sqlValueFree(sql_value *);
-sql_value *sqlValueNew(sql *);
int sqlValueFromExpr(sql *, Expr *, enum field_type type,
sql_value **);
-void sql_value_apply_type(sql_value *val, enum field_type type);
extern const unsigned char sqlOpcodeProperty[];
extern const unsigned char sqlUpperToLower[];
diff --git a/src/box/sql/trigger.c b/src/box/sql/trigger.c
index 0c387bc3b..7c983fea5 100644
--- a/src/box/sql/trigger.c
+++ b/src/box/sql/trigger.c
@@ -37,6 +37,7 @@
#include "box/schema.h"
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
/* See comment in sqlInt.h */
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 18806b93f..775bce96f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -46,6 +46,7 @@
#include "box/tuple.h"
#include "box/port.h"
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "tarantoolInt.h"
@@ -257,424 +258,6 @@ allocateCursor(
return pCx;
}
-int
-mem_apply_numeric_type(struct Mem *record)
-{
- if ((record->flags & MEM_Str) == 0)
- return -1;
- int64_t integer_value;
- bool is_neg;
- if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
- mem_set_int(record, integer_value, is_neg);
- return 0;
- }
- double float_value;
- if (sqlAtoF(record->z, &float_value, record->n) == 0)
- return -1;
- mem_set_double(record, float_value);
- return 0;
-}
-
-/**
- * Processing is determined by the field type parameter:
- *
- * INTEGER:
- * If memory holds floating point value and it can be
- * converted without loss (2.0 - > 2), it's type is
- * changed to INT. Otherwise, simply return success status.
- *
- * NUMBER:
- * If memory holds INT or floating point value,
- * no actions take place.
- *
- * STRING:
- * Convert mem to a string representation.
- *
- * SCALAR:
- * Mem is unchanged, but flag is set to BLOB in case of
- * scalar-like type. Otherwise, (MAP, ARRAY) conversion
- * is impossible.
- *
- * BOOLEAN:
- * If memory holds BOOLEAN no actions take place.
- *
- * ANY:
- * Mem is unchanged, no actions take place.
- *
- * MAP/ARRAY:
- * These types can't be casted to scalar ones, or to each
- * other. So the only valid conversion is to type itself.
- *
- * @param record The value to apply type to.
- * @param type The type to be applied.
- */
-static int
-mem_apply_type(struct Mem *record, enum field_type type)
-{
- if ((record->flags & MEM_Null) != 0)
- return 0;
- assert(type < field_type_MAX);
- switch (type) {
- case FIELD_TYPE_INTEGER:
- case FIELD_TYPE_UNSIGNED:
- if ((record->flags & (MEM_Bool | MEM_Blob)) != 0)
- return -1;
- if ((record->flags & MEM_UInt) == MEM_UInt)
- return 0;
- if ((record->flags & MEM_Real) == MEM_Real) {
- double d = record->u.r;
- if (d >= 0) {
- if (double_compare_uint64(d, UINT64_MAX,
- 1) > 0)
- return 0;
- if ((double)(uint64_t)d == d)
- mem_set_u64(record, (uint64_t)d);
- } else {
- if (double_compare_nint64(d, INT64_MIN, 1) < 0)
- return 0;
- if ((double)(int64_t)d == d)
- mem_set_int(record, (int64_t)d, true);
- }
- return 0;
- }
- if ((record->flags & MEM_Str) != 0) {
- bool is_neg;
- int64_t i;
- if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
- return -1;
- mem_set_int(record, i, is_neg);
- }
- if ((record->flags & MEM_Int) == MEM_Int) {
- if (type == FIELD_TYPE_UNSIGNED)
- return -1;
- return 0;
- }
- return 0;
- case FIELD_TYPE_BOOLEAN:
- if ((record->flags & MEM_Bool) == MEM_Bool)
- return 0;
- return -1;
- case FIELD_TYPE_NUMBER:
- if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
- return 0;
- return sqlVdbeMemRealify(record);
- case FIELD_TYPE_DOUBLE:
- if ((record->flags & MEM_Real) != 0)
- return 0;
- return sqlVdbeMemRealify(record);
- case FIELD_TYPE_STRING:
- /*
- * Only attempt the conversion to TEXT if there is
- * an integer or real representation (BLOB and
- * NULL do not get converted).
- */
- if ((record->flags & MEM_Str) == 0 &&
- (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
- sqlVdbeMemStringify(record);
- record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
- return 0;
- case FIELD_TYPE_VARBINARY:
- if ((record->flags & MEM_Blob) == 0)
- return -1;
- return 0;
- case FIELD_TYPE_SCALAR:
- /* Can't cast MAP and ARRAY to scalar types. */
- if ((record->flags & MEM_Subtype) != 0 &&
- record->subtype == SQL_SUBTYPE_MSGPACK) {
- assert(mp_typeof(*record->z) == MP_MAP ||
- mp_typeof(*record->z) == MP_ARRAY);
- return -1;
- }
- return 0;
- case FIELD_TYPE_MAP:
- if ((record->flags & MEM_Subtype) != 0 &&
- record->subtype == SQL_SUBTYPE_MSGPACK &&
- mp_typeof(*record->z) == MP_MAP)
- return 0;
- return -1;
- case FIELD_TYPE_ARRAY:
- if ((record->flags & MEM_Subtype) != 0 &&
- record->subtype == SQL_SUBTYPE_MSGPACK &&
- mp_typeof(*record->z) == MP_ARRAY)
- return 0;
- return -1;
- case FIELD_TYPE_ANY:
- return 0;
- default:
- return -1;
- }
-}
-
-/*
- * Exported version of mem_apply_type(). This one works on sql_value*,
- * not the internal Mem* type.
- */
-void
-sql_value_apply_type(
- sql_value *pVal,
- enum field_type type)
-{
- mem_apply_type((Mem *) pVal, type);
-}
-
-/**
- * Check that MEM_type of the mem is compatible with given type.
- *
- * @param mem The MEM that contains the value to check.
- * @param type The type to check.
- * @retval TRUE if the MEM_type of the value and the given type
- * are compatible, FALSE otherwise.
- */
-static bool
-mem_is_type_compatible(struct Mem *mem, enum field_type type)
-{
- enum mp_type mp_type = mem_mp_type(mem);
- assert(mp_type < MP_EXT);
- return field_mp_plain_type_is_compatible(type, mp_type, true);
-}
-
-/**
- * Convert the numeric value contained in MEM to double.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_double(struct Mem *mem)
-{
- if ((mem->flags & MEM_Real) != 0)
- return 0;
- if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
- return -1;
- double d;
- if ((mem->flags & MEM_Int) != 0)
- d = (double)mem->u.i;
- else
- d = (double)mem->u.u;
- mem_set_double(mem, d);
- return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to unsigned.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_unsigned(struct Mem *mem)
-{
- if ((mem->flags & MEM_UInt) != 0)
- return 0;
- if ((mem->flags & MEM_Int) != 0)
- return -1;
- if ((mem->flags & MEM_Real) == 0)
- return -1;
- double d = mem->u.r;
- if (d < 0.0 || d >= (double)UINT64_MAX)
- return -1;
- mem_set_u64(mem, (uint64_t) d);
- return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
- if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
- return 0;
- if ((mem->flags & MEM_Real) == 0)
- return -1;
- double d = mem->u.r;
- if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
- return -1;
- if (d < (double)INT64_MAX)
- mem_set_int(mem, (int64_t) d, d < 0);
- else
- mem_set_int(mem, (uint64_t) d, false);
- return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to another numeric
- * type.
- *
- * @param mem The MEM that contains the numeric value.
- * @param type The type to convert to.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_numeric(struct Mem *mem, enum field_type type)
-{
- assert(mp_type_is_numeric(mem_mp_type(mem)) &&
- sql_type_is_numeric(type));
- assert(type != FIELD_TYPE_NUMBER);
- if (type == FIELD_TYPE_DOUBLE)
- return mem_convert_to_double(mem);
- if (type == FIELD_TYPE_UNSIGNED)
- return mem_convert_to_unsigned(mem);
- assert(type == FIELD_TYPE_INTEGER);
- return mem_convert_to_integer(mem);
-}
-
-/*
- * pMem currently only holds a string type (or maybe a BLOB that we can
- * interpret as a string if we want to). Compute its corresponding
- * numeric type, if has one. Set the pMem->u.r and pMem->u.i fields
- * accordingly.
- */
-static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
-{
- assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
- assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
- if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
- return 0;
- bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
- return is_neg ? MEM_Int : MEM_UInt;
- return MEM_Real;
-}
-
-/*
- * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
- * none.
- *
- * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
- * But it does set pMem->u.r and pMem->u.i appropriately.
- */
-static u16 numericType(Mem *pMem)
-{
- if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
- return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
- if (pMem->flags & (MEM_Str|MEM_Blob)) {
- return computeNumericType(pMem);
- }
- return 0;
-}
-
-#ifdef SQL_DEBUG
-/*
- * Write a nice string representation of the contents of cell pMem
- * into buffer zBuf, length nBuf.
- */
-void
-sqlVdbeMemPrettyPrint(Mem *pMem, char *zBuf)
-{
- char *zCsr = zBuf;
- int f = pMem->flags;
-
- if (f&MEM_Blob) {
- int i;
- char c;
- if (f & MEM_Dyn) {
- c = 'z';
- assert((f & (MEM_Static|MEM_Ephem))==0);
- } else if (f & MEM_Static) {
- c = 't';
- assert((f & (MEM_Dyn|MEM_Ephem))==0);
- } else if (f & MEM_Ephem) {
- c = 'e';
- assert((f & (MEM_Static|MEM_Dyn))==0);
- } else {
- c = 's';
- }
-
- sql_snprintf(100, zCsr, "%c", c);
- zCsr += sqlStrlen30(zCsr);
- sql_snprintf(100, zCsr, "%d[", pMem->n);
- zCsr += sqlStrlen30(zCsr);
- for(i=0; i<16 && i<pMem->n; i++) {
- sql_snprintf(100, zCsr, "%02X", ((int)pMem->z[i] & 0xFF));
- zCsr += sqlStrlen30(zCsr);
- }
- for(i=0; i<16 && i<pMem->n; i++) {
- char z = pMem->z[i];
- if (z<32 || z>126) *zCsr++ = '.';
- else *zCsr++ = z;
- }
- sql_snprintf(100, zCsr, "]%s", "(8)");
- zCsr += sqlStrlen30(zCsr);
- if (f & MEM_Zero) {
- sql_snprintf(100, zCsr,"+%dz",pMem->u.nZero);
- zCsr += sqlStrlen30(zCsr);
- }
- *zCsr = '\0';
- } else if (f & MEM_Str) {
- int j, k;
- zBuf[0] = ' ';
- if (f & MEM_Dyn) {
- zBuf[1] = 'z';
- assert((f & (MEM_Static|MEM_Ephem))==0);
- } else if (f & MEM_Static) {
- zBuf[1] = 't';
- assert((f & (MEM_Dyn|MEM_Ephem))==0);
- } else if (f & MEM_Ephem) {
- zBuf[1] = 'e';
- assert((f & (MEM_Static|MEM_Dyn))==0);
- } else {
- zBuf[1] = 's';
- }
- k = 2;
- sql_snprintf(100, &zBuf[k], "%d", pMem->n);
- k += sqlStrlen30(&zBuf[k]);
- zBuf[k++] = '[';
- for(j=0; j<15 && j<pMem->n; j++) {
- u8 c = pMem->z[j];
- if (c>=0x20 && c<0x7f) {
- zBuf[k++] = c;
- } else {
- zBuf[k++] = '.';
- }
- }
- zBuf[k++] = ']';
- sql_snprintf(100,&zBuf[k],"(8)");
- k += sqlStrlen30(&zBuf[k]);
- zBuf[k++] = 0;
- }
-}
-#endif
-
-#ifdef SQL_DEBUG
-/*
- * Print the value of a register for tracing purposes:
- */
-static void
-memTracePrint(Mem *p)
-{
- if (p->flags & MEM_Undefined) {
- printf(" undefined");
- } else if (p->flags & MEM_Null) {
- printf(" NULL");
- } else if ((p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str)) {
- printf(" si:%lld", p->u.i);
- } else if (p->flags & MEM_Int) {
- printf(" i:%lld", p->u.i);
- } else if (p->flags & MEM_UInt) {
- printf(" u:%"PRIu64"", p->u.u);
- } else if (p->flags & MEM_Real) {
- printf(" r:%g", p->u.r);
- } else if (p->flags & MEM_Bool) {
- printf(" bool:%s", SQL_TOKEN_BOOLEAN(p->u.b));
- } else {
- char zBuf[200];
- sqlVdbeMemPrettyPrint(p, zBuf);
- printf(" %s", zBuf);
- }
- if (p->flags & MEM_Subtype) printf(" subtype=0x%02x", p->subtype);
-}
-static void
-registerTrace(int iReg, Mem *p) {
- printf("REG[%d] = ", iReg);
- memTracePrint(p);
- printf("\n");
-}
-#endif
-
#ifdef SQL_DEBUG
# define REGISTER_TRACE(P,R,M) \
if(P->sql_flags&SQL_VdbeTrace) registerTrace(R,M);
@@ -727,43 +310,6 @@ vdbe_add_new_autoinc_id(struct Vdbe *vdbe, int64_t id)
return 0;
}
-char *
-mem_type_to_str(const struct Mem *p)
-{
- assert(p != NULL);
- switch (p->flags & MEM_PURE_TYPE_MASK) {
- case MEM_Null:
- return "NULL";
- case MEM_Str:
- return "text";
- case MEM_Int:
- return "integer";
- case MEM_UInt:
- return "unsigned";
- case MEM_Real:
- return "real";
- case MEM_Blob:
- return "varbinary";
- case MEM_Bool:
- return "boolean";
- default:
- unreachable();
- }
-}
-
-/* Allocate memory for internal VDBE structure on region. */
-static int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
-{
- vdbe_mem->n = size;
- vdbe_mem->z = region_alloc(&fiber()->gc, size);
- if (vdbe_mem->z == NULL)
- return -1;
- vdbe_mem->flags = MEM_Ephem | MEM_Blob;
- assert(sqlVdbeCheckMemInvariants(vdbe_mem));
- return 0;
-}
-
static inline const struct tuple_field *
vdbe_field_ref_fetch_field(struct vdbe_field_ref *field_ref, uint32_t fieldno)
{
diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h
index 48815d4ec..118f1cd83 100644
--- a/src/box/sql/vdbe.h
+++ b/src/box/sql/vdbe.h
@@ -268,7 +268,6 @@ void sqlVdbeSwap(Vdbe *, Vdbe *);
VdbeOp *sqlVdbeTakeOpArray(Vdbe *, int *, int *);
sql_value *sqlVdbeGetBoundValue(Vdbe *, int, u8);
char *sqlVdbeExpandSql(Vdbe *, const char *);
-int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
/**
* Perform unpacking of provided message pack.
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 7205f1af3..b2d4f27c1 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -155,151 +155,6 @@ struct VdbeFrame {
int nDbChange; /* Value of db->nChange */
};
-#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
-
-/*
- * Internally, the vdbe manipulates nearly all SQL values as Mem
- * structures. Each Mem struct may cache multiple representations (string,
- * integer etc.) of the same value.
- */
-struct Mem {
- union MemValue {
- double r; /* Real value used when MEM_Real is set in flags */
- i64 i; /* Integer value used when MEM_Int is set in flags */
- uint64_t u; /* Unsigned integer used when MEM_UInt is set. */
- bool b; /* Boolean value used when MEM_Bool is set in flags */
- int nZero; /* Used when bit MEM_Zero is set in flags */
- void *p; /* Generic pointer */
- /**
- * A pointer to function implementation.
- * Used only when flags==MEM_Agg.
- */
- struct func *func;
- VdbeFrame *pFrame; /* Used when flags==MEM_Frame */
- } u;
- u32 flags; /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
- /** Subtype for this value. */
- enum sql_subtype subtype;
- /**
- * If value is fetched from tuple, then this property
- * contains type of corresponding space's field. If it's
- * value field_type_MAX then we can rely on on format
- * (msgpack) type which is represented by 'flags'.
- */
- enum field_type field_type;
- int n; /* size (in bytes) of string value, excluding trailing '\0' */
- char *z; /* String or BLOB value */
- /* ShallowCopy only needs to copy the information above */
- char *zMalloc; /* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
- int szMalloc; /* Size of the zMalloc allocation */
- u32 uTemp; /* Transient storage for serial_type in OP_MakeRecord */
- sql *db; /* The associated database connection */
- void (*xDel) (void *); /* Destructor for Mem.z - only valid if MEM_Dyn */
-#ifdef SQL_DEBUG
- Mem *pScopyFrom; /* This Mem is a shallow copy of pScopyFrom */
- void *pFiller; /* So that sizeof(Mem) is a multiple of 8 */
-#endif
-};
-
-/*
- * Size of struct Mem not including the Mem.zMalloc member or anything that
- * follows.
- */
-#define MEMCELLSIZE offsetof(Mem,zMalloc)
-
-/* One or more of the following flags are set to indicate the validOK
- * representations of the value stored in the Mem struct.
- *
- * If the MEM_Null flag is set, then the value is an SQL NULL value.
- * No other flags may be set in this case.
- *
- * If the MEM_Str flag is set then Mem.z points at a string representation.
- * Usually this is encoded in the same unicode encoding as the main
- * database (see below for exceptions). If the MEM_Term flag is also
- * set, then the string is nul terminated. The MEM_Int and MEM_Real
- * flags may coexist with the MEM_Str flag.
- */
-#define MEM_Null 0x0001 /* Value is NULL */
-#define MEM_Str 0x0002 /* Value is a string */
-#define MEM_Int 0x0004 /* Value is an integer */
-#define MEM_Real 0x0008 /* Value is a real number */
-#define MEM_Blob 0x0010 /* Value is a BLOB */
-#define MEM_Bool 0x0020 /* Value is a bool */
-#define MEM_UInt 0x0040 /* Value is an unsigned integer */
-#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
-#define MEM_Undefined 0x0100 /* Value is undefined */
-#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
-#define MEM_TypeMask 0x83ff /* Mask of type bits */
-
-/* Whenever Mem contains a valid string or blob representation, one of
- * the following flags must be set to determine the memory management
- * policy for Mem.z. The MEM_Term flag tells us whether or not the
- * string is \000 or \u0000 terminated
- */
-#define MEM_Term 0x0400 /* String rep is nul terminated */
-#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
-#define MEM_Static 0x1000 /* Mem.z points to a static string */
-#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
-#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
-#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
-#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
-#define MEM_Ptr 0x20000 /* Value is a generic pointer */
-
-/**
- * In contrast to Mem_TypeMask, this one allows to get
- * pure type of memory cell, i.e. without _Dyn/_Zero and other
- * auxiliary flags.
- */
-enum {
- MEM_PURE_TYPE_MASK = 0x7f
-};
-
-static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
- MEM_Blob | MEM_Bool | MEM_UInt),
- "value of type mask must consist of corresponding to memory "\
- "type bits");
-
-/**
- * Simple type to str convertor. It is used to simplify
- * error reporting.
- */
-char *
-mem_type_to_str(const struct Mem *p);
-
-/**
- * Try to convert a string value into a numeric representation
- * if we can do so without loss of information. Firstly, value
- * is attempted to be converted to integer, and in case of fail -
- * to floating point number. Note that function is assumed to be
- * called on memory cell containing string, i.e. mem->type == MEM_Str.
- *
- * @param record Memory cell containing value to be converted.
- * @retval 0 If value can be converted to integer or number.
- * @retval -1 Otherwise.
- */
-int
-mem_apply_numeric_type(struct Mem *record);
-
-/* Return TRUE if Mem X contains dynamically allocated content - anything
- * that needs to be deallocated to avoid a leak.
- */
-#define VdbeMemDynamic(X) \
- (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
-
-/*
- * Clear any existing type flags from a Mem and replace them with f
- */
-#define MemSetTypeFlag(p, f) \
- ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f)
-
-/*
- * Return true if a memory cell is not marked as invalid. This macro
- * is for use inside assert() statements only.
- */
-#ifdef SQL_DEBUG
-#define memIsValid(M) ((M)->flags & MEM_Undefined)==0
-#endif
-
/*
* The "context" argument for an installable function. A pointer to an
* instance of this structure is the first argument to the routines used
@@ -477,83 +332,8 @@ int sqlVdbeExec(Vdbe *);
int sqlVdbeList(Vdbe *);
int sqlVdbeHalt(Vdbe *);
-int sqlVdbeMemTooBig(Mem *);
-int sqlVdbeMemCopy(Mem *, const Mem *);
-void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
-void sqlVdbeMemMove(Mem *, Mem *);
-int sqlVdbeMemNulTerminate(Mem *);
-int sqlVdbeMemSetStr(Mem *, const char *, int, u8, void (*)(void *));
-void
-mem_set_bool(struct Mem *mem, bool value);
-
-/**
- * Set VDBE memory register with given pointer as a data.
- * @param mem VDBE memory register to update.
- * @param ptr Pointer to use.
- */
-void
-mem_set_ptr(struct Mem *mem, void *ptr);
-
-/**
- * Set integer value. Depending on its sign MEM_Int (in case
- * of negative value) or MEM_UInt flag is set.
- */
-void
-mem_set_i64(struct Mem *mem, int64_t value);
-
-/** Set unsigned value and MEM_UInt flag. */
-void
-mem_set_u64(struct Mem *mem, uint64_t value);
-
-/**
- * Set integer value. According to is_neg flag value is considered
- * to be signed or unsigned.
- */
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
-
-/** Set double value and MEM_Real flag. */
-void
-mem_set_double(struct Mem *mem, double value);
-
-void sqlVdbeMemInit(Mem *, sql *, u32);
-void sqlVdbeMemSetNull(Mem *);
-void sqlVdbeMemSetZeroBlob(Mem *, int);
-int sqlVdbeMemMakeWriteable(Mem *);
-int sqlVdbeMemStringify(Mem *);
-int sqlVdbeIntValue(Mem *, int64_t *, bool *is_neg);
-
-int sqlVdbeRealValue(Mem *, double *);
-
-int
-mem_value_bool(const struct Mem *mem, bool *b);
-
-int mem_apply_integer_type(Mem *);
-int sqlVdbeMemRealify(Mem *);
-
-/**
- * Convert @a mem to NUMBER type, so that after conversion it has
- * one of types MEM_Real, MEM_Int or MEM_UInt. If conversion is
- * not possible, function returns -1.
- *
- * Beware - this function changes value and type of @a mem
- * argument.
- */
-int
-vdbe_mem_numerify(struct Mem *mem);
-
-int sqlVdbeMemCast(Mem *, enum field_type type);
int sqlVdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
-void sqlVdbeMemRelease(Mem * p);
-
-/*
- * Return the MP_type of the value of the MEM.
- * Analogue of sql_value_type() but operates directly on
- * transparent memory cell.
- */
-enum mp_type
-mem_mp_type(struct Mem *mem);
/**
* In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
@@ -566,19 +346,7 @@ mem_mp_type(struct Mem *mem);
#define mp_type_is_numeric(X) ((X) == MP_INT || (X) == MP_UINT ||\
(X) == MP_DOUBLE)
-/**
- * Memory cell mem contains the context of an aggregate function.
- * This routine calls the finalize method for that function. The
- * result of the aggregate is stored back into mem.
- *
- * Returns -1 if the finalizer reports an error. 0 otherwise.
- */
-int
-sql_vdbemem_finalize(struct Mem *mem, struct func *func);
-
const char *sqlOpcodeName(int);
-int sqlVdbeMemGrow(Mem * pMem, int n, int preserve);
-int sqlVdbeMemClearAndResize(Mem * pMem, int n);
int sqlVdbeCloseStatement(Vdbe *, int);
void sqlVdbeFrameDelete(VdbeFrame *);
int sqlVdbeFrameRestore(VdbeFrame *);
@@ -598,7 +366,6 @@ int sqlVdbeSorterCompare(const VdbeCursor *, Mem *, int, int *);
#ifdef SQL_DEBUG
void sqlVdbeMemAboutToChange(Vdbe *, Mem *);
-int sqlVdbeCheckMemInvariants(Mem *);
#endif
int sqlVdbeCheckFk(Vdbe *, int);
@@ -606,48 +373,9 @@ int sqlVdbeCheckFk(Vdbe *, int);
int sqlVdbeMemTranslate(Mem *, u8);
#ifdef SQL_DEBUG
void sqlVdbePrintSql(Vdbe *);
-void sqlVdbeMemPrettyPrint(Mem * pMem, char *zBuf);
#endif
int sqlVdbeMemHandleBom(Mem * pMem);
-int sqlVdbeMemExpandBlob(Mem *);
-#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
-
-/**
- * Perform comparison of two keys: one is packed and one is not.
- *
- * @param key1 Pointer to pointer to first key.
- * @param unpacked Pointer to unpacked tuple.
- * @param key2_idx index of key in umpacked record to compare.
- *
- * @retval +1 if key1 > pUnpacked[iKey2], -1 ptherwise.
- */
-int sqlVdbeCompareMsgpack(const char **key1,
- struct UnpackedRecord *unpacked, int key2_idx);
-
-/**
- * Perform comparison of two tuples: unpacked (key1) and packed (key2)
- *
- * @param key1 Packed key.
- * @param unpacked Unpacked key.
- *
- * @retval +1 if key1 > unpacked, -1 otherwise.
- */
-int sqlVdbeRecordCompareMsgpack(const void *key1,
- struct UnpackedRecord *key2);
-
-/**
- * Decode msgpack and save value into VDBE memory cell.
- *
- * @param buf Buffer to deserialize msgpack from.
- * @param mem Memory cell to write value into.
- * @param len[out] Length of decoded part.
- * @retval Return code: < 0 in case of error.
- * @retval 0 on success.
- */
-int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len);
-
struct mpstream;
struct region;
@@ -658,26 +386,4 @@ set_encode_error(void *error_ctx)
*(bool *)error_ctx = true;
}
-/**
- * Perform encoding memory variable to stream.
- * @param stream Initialized mpstream encoder object.
- * @param var Vdbe memory variable to encode with stream.
- */
-void
-mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var);
-
-/**
- * Perform encoding field_count Vdbe memory fields on region as
- * msgpack array.
- * @param fields The first Vdbe memory field to encode.
- * @param field_count Count of fields to encode.
- * @param[out] tuple_size Size of encoded tuple.
- * @param region Region to use.
- * @retval NULL on error, diag message is set.
- * @retval Pointer to valid tuple on success.
- */
-char *
-sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
- uint32_t *tuple_size, struct region *region);
-
#endif /* !defined(SQL_VDBEINT_H) */
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7c59ef83f..24fa99f17 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -35,6 +35,7 @@
* VDBE.
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "box/session.h"
@@ -122,101 +123,6 @@ sql_metadata_is_full()
return current_session()->sql_flags & SQL_FullMetadata;
}
-/**************************** sql_value_ ******************************
- * The following routines extract information from a Mem or sql_value
- * structure.
- */
-const void *
-sql_value_blob(sql_value * pVal)
-{
- Mem *p = (Mem *) pVal;
- if (p->flags & (MEM_Blob | MEM_Str)) {
- if (ExpandBlob(p) != 0) {
- assert(p->flags == MEM_Null && p->z == 0);
- return 0;
- }
- p->flags |= MEM_Blob;
- return p->n ? p->z : 0;
- } else {
- return sql_value_text(pVal);
- }
-}
-
-int
-sql_value_bytes(sql_value * pVal)
-{
- return sqlValueBytes(pVal);
-}
-
-double
-sql_value_double(sql_value * pVal)
-{
- double v = 0.0;
- sqlVdbeRealValue((Mem *) pVal, &v);
- return v;
-}
-
-bool
-sql_value_boolean(sql_value *val)
-{
- bool b = false;
- int rc = mem_value_bool((struct Mem *) val, &b);
- assert(rc == 0);
- (void) rc;
- return b;
-}
-
-int
-sql_value_int(sql_value * pVal)
-{
- int64_t i = 0;
- bool is_neg;
- sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
- return (int)i;
-}
-
-sql_int64
-sql_value_int64(sql_value * pVal)
-{
- int64_t i = 0;
- bool unused;
- sqlVdbeIntValue((Mem *) pVal, &i, &unused);
- return i;
-}
-
-uint64_t
-sql_value_uint64(sql_value *val)
-{
- int64_t i = 0;
- bool is_neg;
- sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
- assert(!is_neg);
- return i;
-}
-
-enum sql_subtype
-sql_value_subtype(sql_value * pVal)
-{
- return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
-}
-
-const unsigned char *
-sql_value_text(sql_value * pVal)
-{
- return (const unsigned char *)sqlValueText(pVal);
-}
-
-/* EVIDENCE-OF: R-12793-43283 Every value in sql has one of five
- * fundamental datatypes: 64-bit signed integer 64-bit IEEE floating
- * point number string BLOB NULL
- */
-enum mp_type
-sql_value_type(sql_value *pVal)
-{
- struct Mem *mem = (struct Mem *) pVal;
- return mem_mp_type(mem);
-}
-
/* Make a copy of an sql_value object
*/
sql_value *
@@ -582,47 +488,6 @@ sql_data_count(sql_stmt * pStmt)
return pVm->nResColumn;
}
-/*
- * Return a pointer to static memory containing an SQL NULL value.
- */
-static const Mem *
-columnNullValue(void)
-{
- /* Even though the Mem structure contains an element
- * of type i64, on certain architectures (x86) with certain compiler
- * switches (-Os), gcc may align this Mem object on a 4-byte boundary
- * instead of an 8-byte one. This all works fine, except that when
- * running with SQL_DEBUG defined the sql code sometimes assert()s
- * that a Mem structure is located on an 8-byte boundary. To prevent
- * these assert()s from failing, when building with SQL_DEBUG defined
- * using gcc, we force nullMem to be 8-byte aligned using the magical
- * __attribute__((aligned(8))) macro.
- */
- static const Mem nullMem
-#if defined(SQL_DEBUG) && defined(__GNUC__)
- __attribute__ ((aligned(8)))
-#endif
- = {
- /* .u = */ {
- 0},
- /* .flags = */ (u16) MEM_Null,
- /* .eSubtype = */ (u8) 0,
- /* .field_type = */ field_type_MAX,
- /* .n = */ (int)0,
- /* .z = */ (char *)0,
- /* .zMalloc = */ (char *)0,
- /* .szMalloc = */ (int)0,
- /* .uTemp = */ (u32) 0,
- /* .db = */ (sql *) 0,
- /* .xDel = */ (void (*)(void *))0,
-#ifdef SQL_DEBUG
- /* .pScopyFrom = */ (Mem *) 0,
- /* .pFiller = */ (void *)0,
-#endif
- };
- return &nullMem;
-}
-
/*
* Check to see if column iCol of the given statement is valid. If
* it is, return a pointer to the Mem for the value of that column.
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 772476377..79c3d60e0 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -41,6 +41,7 @@
#include "box/txn.h"
#include "msgpuck/msgpuck.h"
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "tarantoolInt.h"
#include "box/execute.h"
@@ -1194,65 +1195,6 @@ sqlVdbePrintOp(FILE * pOut, int pc, Op * pOp)
}
#endif
-/*
- * Initialize an array of N Mem element.
- */
-static void
-initMemArray(Mem * p, int N, sql * db, u32 flags)
-{
- while ((N--) > 0) {
- p->db = db;
- p->flags = flags;
- p->szMalloc = 0;
- p->field_type = field_type_MAX;
-#ifdef SQL_DEBUG
- p->pScopyFrom = 0;
-#endif
- p++;
- }
-}
-
-/*
- * Release an array of N Mem elements
- */
-static void
-releaseMemArray(Mem * p, int N)
-{
- if (p && N) {
- Mem *pEnd = &p[N];
- sql *db = p->db;
- do {
- assert((&p[1]) == pEnd || p[0].db == p[1].db);
- assert(sqlVdbeCheckMemInvariants(p));
-
- /* This block is really an inlined version of sqlVdbeMemRelease()
- * that takes advantage of the fact that the memory cell value is
- * being set to NULL after releasing any dynamic resources.
- *
- * The justification for duplicating code is that according to
- * callgrind, this causes a certain test case to hit the CPU 4.7
- * percent less (x86 linux, gcc version 4.1.2, -O6) than if
- * sqlMemRelease() were called from here. With -O2, this jumps
- * to 6.6 percent. The test case is inserting 1000 rows into a table
- * with no indexes using a single prepared INSERT statement, bind()
- * and reset(). Inserts are grouped into a transaction.
- */
- testcase(p->flags & MEM_Agg);
- testcase(p->flags & MEM_Dyn);
- testcase(p->flags & MEM_Frame);
- if (p->
- flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
- sqlVdbeMemRelease(p);
- } else if (p->szMalloc) {
- sqlDbFree(db, p->zMalloc);
- p->szMalloc = 0;
- }
-
- p->flags = MEM_Undefined;
- } while ((++p) < pEnd);
- }
-}
-
/*
* Delete a VdbeFrame object and its contents. VdbeFrame objects are
* allocated by the OP_Program opcode in sqlVdbeExec().
@@ -2334,204 +2276,6 @@ sqlVdbeAllocUnpackedRecord(struct sql *db, struct key_def *key_def)
return p;
}
-/*
- * Both *pMem1 and *pMem2 contain string values. Compare the two values
- * using the collation sequence pColl. As usual, return a negative , zero
- * or positive value if *pMem1 is less than, equal to or greater than
- * *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
- *
- * Strungs assume to be UTF-8 encoded
- */
-static int
-vdbeCompareMemString(const Mem * pMem1, const Mem * pMem2,
- const struct coll * pColl)
-{
- return pColl->cmp(pMem1->z, (size_t)pMem1->n,
- pMem2->z, (size_t)pMem2->n, pColl);
-}
-
-/*
- * The input pBlob is guaranteed to be a Blob that is not marked
- * with MEM_Zero. Return true if it could be a zero-blob.
- */
-static int
-isAllZero(const char *z, int n)
-{
- int i;
- for (i = 0; i < n; i++) {
- if (z[i])
- return 0;
- }
- return 1;
-}
-
-/*
- * Compare two blobs. Return negative, zero, or positive if the first
- * is less than, equal to, or greater than the second, respectively.
- * If one blob is a prefix of the other, then the shorter is the lessor.
- */
-static SQL_NOINLINE int
-sqlBlobCompare(const Mem * pB1, const Mem * pB2)
-{
- int c;
- int n1 = pB1->n;
- int n2 = pB2->n;
-
- /* It is possible to have a Blob value that has some non-zero content
- * followed by zero content. But that only comes up for Blobs formed
- * by the OP_MakeRecord opcode, and such Blobs never get passed into
- * sqlMemCompare().
- */
- assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
- assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
-
- if ((pB1->flags | pB2->flags) & MEM_Zero) {
- if (pB1->flags & pB2->flags & MEM_Zero) {
- return pB1->u.nZero - pB2->u.nZero;
- } else if (pB1->flags & MEM_Zero) {
- if (!isAllZero(pB2->z, pB2->n))
- return -1;
- return pB1->u.nZero - n2;
- } else {
- if (!isAllZero(pB1->z, pB1->n))
- return +1;
- return n1 - pB2->u.nZero;
- }
- }
- c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
- if (c)
- return c;
- return n1 - n2;
-}
-
-/*
- * Compare the values contained by the two memory cells, returning
- * negative, zero or positive if pMem1 is less than, equal to, or greater
- * than pMem2. Sorting order is NULL's first, followed by numbers (integers
- * and reals) sorted numerically, followed by text ordered by the collating
- * sequence pColl and finally blob's ordered by memcmp().
- *
- * Two NULL values are considered equal by this function.
- */
-int
-sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
-{
- int f1, f2;
- int combined_flags;
-
- f1 = pMem1->flags;
- f2 = pMem2->flags;
- combined_flags = f1 | f2;
-
- /* If one value is NULL, it is less than the other. If both values
- * are NULL, return 0.
- */
- if (combined_flags & MEM_Null) {
- return (f2 & MEM_Null) - (f1 & MEM_Null);
- }
-
- if ((combined_flags & MEM_Bool) != 0) {
- if ((f1 & f2 & MEM_Bool) != 0) {
- if (pMem1->u.b == pMem2->u.b)
- return 0;
- if (pMem1->u.b)
- return 1;
- return -1;
- }
- if ((f2 & MEM_Bool) != 0)
- return +1;
- return -1;
- }
-
- /* At least one of the two values is a number
- */
- if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- if ((f1 & f2 & MEM_Int) != 0) {
- if (pMem1->u.i < pMem2->u.i)
- return -1;
- if (pMem1->u.i > pMem2->u.i)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_UInt) != 0) {
- if (pMem1->u.u < pMem2->u.u)
- return -1;
- if (pMem1->u.u > pMem2->u.u)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_Real) != 0) {
- if (pMem1->u.r < pMem2->u.r)
- return -1;
- if (pMem1->u.r > pMem2->u.r)
- return +1;
- return 0;
- }
- if ((f1 & MEM_Int) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_nint64(pMem2->u.r,
- pMem1->u.i, -1);
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_UInt) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_uint64(pMem2->u.r,
- pMem1->u.u, -1);
- } else if ((f2 & MEM_Int) != 0) {
- return +1;
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_Real) != 0) {
- if ((f2 & MEM_Int) != 0) {
- return double_compare_nint64(pMem1->u.r,
- pMem2->u.i, 1);
- } else if ((f2 & MEM_UInt) != 0) {
- return double_compare_uint64(pMem1->u.r,
- pMem2->u.u, 1);
- } else {
- return -1;
- }
- }
- return +1;
- }
-
- /* If one value is a string and the other is a blob, the string is less.
- * If both are strings, compare using the collating functions.
- */
- if (combined_flags & MEM_Str) {
- if ((f1 & MEM_Str) == 0) {
- return 1;
- }
- if ((f2 & MEM_Str) == 0) {
- return -1;
- }
- /* The collation sequence must be defined at this point, even if
- * the user deletes the collation sequence after the vdbe program is
- * compiled (this was not always the case).
- */
- if (pColl) {
- return vdbeCompareMemString(pMem1, pMem2, pColl);
- } else {
- size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
- int res;
- res = memcmp(pMem1->z, pMem2->z, n);
- if (res == 0)
- res = (int)pMem1->n - (int)pMem2->n;
- return res;
- }
- /* If a NULL pointer was passed as the collate function, fall through
- * to the blob case and use memcmp().
- */
- }
-
- /* Both values must be blobs. Compare using memcmp(). */
- return sqlBlobCompare(pMem1, pMem2);
-}
-
/*
* This routine sets the value to be returned by subsequent calls to
* sql_changes() on the database handle 'db'.
@@ -2605,291 +2349,6 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
return 0;
}
-int
-sqlVdbeCompareMsgpack(const char **key1,
- struct UnpackedRecord *unpacked, int key2_idx)
-{
- const char *aKey1 = *key1;
- Mem *pKey2 = unpacked->aMem + key2_idx;
- Mem mem1;
- int rc = 0;
- switch (mp_typeof(*aKey1)) {
- default:{
- /* FIXME */
- rc = -1;
- break;
- }
- case MP_NIL:{
- rc = -((pKey2->flags & MEM_Null) == 0);
- mp_decode_nil(&aKey1);
- break;
- }
- case MP_BOOL:{
- mem1.u.b = mp_decode_bool(&aKey1);
- if ((pKey2->flags & MEM_Bool) != 0) {
- if (mem1.u.b != pKey2->u.b)
- rc = mem1.u.b ? 1 : -1;
- } else {
- rc = (pKey2->flags & MEM_Null) != 0 ? 1 : -1;
- }
- break;
- }
- case MP_UINT:{
- mem1.u.u = mp_decode_uint(&aKey1);
- if ((pKey2->flags & MEM_Int) != 0) {
- rc = +1;
- } else if ((pKey2->flags & MEM_UInt) != 0) {
- if (mem1.u.u < pKey2->u.u)
- rc = -1;
- else if (mem1.u.u > pKey2->u.u)
- rc = +1;
- } else if ((pKey2->flags & MEM_Real) != 0) {
- rc = double_compare_uint64(pKey2->u.r,
- mem1.u.u, -1);
- } else if ((pKey2->flags & MEM_Null) != 0) {
- rc = 1;
- } else if ((pKey2->flags & MEM_Bool) != 0) {
- rc = 1;
- } else {
- rc = -1;
- }
- break;
- }
- case MP_INT:{
- mem1.u.i = mp_decode_int(&aKey1);
- if ((pKey2->flags & MEM_UInt) != 0) {
- rc = -1;
- } else if ((pKey2->flags & MEM_Int) != 0) {
- if (mem1.u.i < pKey2->u.i) {
- rc = -1;
- } else if (mem1.u.i > pKey2->u.i) {
- rc = +1;
- }
- } else if (pKey2->flags & MEM_Real) {
- rc = double_compare_nint64(pKey2->u.r, mem1.u.i,
- -1);
- } else if ((pKey2->flags & MEM_Null) != 0) {
- rc = 1;
- } else if ((pKey2->flags & MEM_Bool) != 0) {
- rc = 1;
- } else {
- rc = -1;
- }
- break;
- }
- case MP_FLOAT:{
- mem1.u.r = mp_decode_float(&aKey1);
- goto do_float;
- }
- case MP_DOUBLE:{
- mem1.u.r = mp_decode_double(&aKey1);
- do_float:
- if ((pKey2->flags & MEM_Int) != 0) {
- rc = double_compare_nint64(mem1.u.r, pKey2->u.i,
- 1);
- } else if (pKey2->flags & MEM_UInt) {
- rc = double_compare_uint64(mem1.u.r,
- pKey2->u.u, 1);
- } else if (pKey2->flags & MEM_Real) {
- if (mem1.u.r < pKey2->u.r) {
- rc = -1;
- } else if (mem1.u.r > pKey2->u.r) {
- rc = +1;
- }
- } else if ((pKey2->flags & MEM_Null) != 0) {
- rc = 1;
- } else if ((pKey2->flags & MEM_Bool) != 0) {
- rc = 1;
- } else {
- rc = -1;
- }
- break;
- }
- case MP_STR:{
- if (pKey2->flags & MEM_Str) {
- struct key_def *key_def = unpacked->key_def;
- mem1.n = mp_decode_strl(&aKey1);
- mem1.z = (char *)aKey1;
- aKey1 += mem1.n;
- struct coll *coll =
- key_def->parts[key2_idx].coll;
- if (coll != NULL) {
- mem1.flags = MEM_Str;
- rc = vdbeCompareMemString(&mem1, pKey2,
- coll);
- } else {
- goto do_bin_cmp;
- }
- } else {
- rc = (pKey2->flags & MEM_Blob) ? -1 : +1;
- }
- break;
- }
- case MP_BIN:{
- mem1.n = mp_decode_binl(&aKey1);
- mem1.z = (char *)aKey1;
- aKey1 += mem1.n;
- do_blob:
- if (pKey2->flags & MEM_Blob) {
- if (pKey2->flags & MEM_Zero) {
- if (!isAllZero
- ((const char *)mem1.z, mem1.n)) {
- rc = 1;
- } else {
- rc = mem1.n - pKey2->u.nZero;
- }
- } else {
- int nCmp;
- do_bin_cmp:
- nCmp = MIN(mem1.n, pKey2->n);
- rc = memcmp(mem1.z, pKey2->z, nCmp);
- if (rc == 0)
- rc = mem1.n - pKey2->n;
- }
- } else {
- rc = 1;
- }
- break;
- }
- case MP_ARRAY:
- case MP_MAP:
- case MP_EXT:{
- mem1.z = (char *)aKey1;
- mp_next(&aKey1);
- mem1.n = aKey1 - (char *)mem1.z;
- goto do_blob;
- }
- }
- *key1 = aKey1;
- return rc;
-}
-
-int
-sqlVdbeRecordCompareMsgpack(const void *key1,
- struct UnpackedRecord *key2)
-{
- int rc = 0;
- u32 i, n = mp_decode_array((const char**)&key1);
-
- n = MIN(n, key2->nField);
-
- for (i = 0; i != n; i++) {
- rc = sqlVdbeCompareMsgpack((const char**)&key1, key2, i);
- if (rc != 0) {
- if (key2->key_def->parts[i].sort_order !=
- SORT_ORDER_ASC) {
- rc = -rc;
- }
- return rc;
- }
- }
-
- key2->eqSeen = 1;
- return key2->default_rc;
-}
-
-int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
-{
- const char *start_buf = buf;
- switch (mp_typeof(*buf)) {
- case MP_ARRAY: {
- mem->z = (char *)buf;
- mp_next(&buf);
- mem->n = buf - mem->z;
- mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
- mem->subtype = SQL_SUBTYPE_MSGPACK;
- mem->field_type = FIELD_TYPE_ARRAY;
- break;
- }
- case MP_MAP: {
- mem->z = (char *)buf;
- mp_next(&buf);
- mem->n = buf - mem->z;
- mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
- mem->subtype = SQL_SUBTYPE_MSGPACK;
- mem->field_type = FIELD_TYPE_MAP;
- break;
- }
- case MP_EXT: {
- mem->z = (char *)buf;
- mp_next(&buf);
- mem->n = buf - mem->z;
- mem->flags = MEM_Blob | MEM_Ephem;
- mem->field_type = FIELD_TYPE_VARBINARY;
- break;
- }
- case MP_NIL: {
- mp_decode_nil(&buf);
- mem->flags = MEM_Null;
- mem->field_type = field_type_MAX;
- break;
- }
- case MP_BOOL: {
- mem->u.b = mp_decode_bool(&buf);
- mem->flags = MEM_Bool;
- mem->field_type = FIELD_TYPE_BOOLEAN;
- break;
- }
- case MP_UINT: {
- uint64_t v = mp_decode_uint(&buf);
- mem->u.u = v;
- mem->flags = MEM_UInt;
- mem->field_type = FIELD_TYPE_INTEGER;
- break;
- }
- case MP_INT: {
- mem->u.i = mp_decode_int(&buf);
- mem->flags = MEM_Int;
- mem->field_type = FIELD_TYPE_INTEGER;
- break;
- }
- case MP_STR: {
- /* XXX u32->int */
- mem->n = (int) mp_decode_strl(&buf);
- mem->flags = MEM_Str | MEM_Ephem;
- mem->field_type = FIELD_TYPE_STRING;
-install_blob:
- mem->z = (char *)buf;
- buf += mem->n;
- break;
- }
- case MP_BIN: {
- /* XXX u32->int */
- mem->n = (int) mp_decode_binl(&buf);
- mem->flags = MEM_Blob | MEM_Ephem;
- mem->field_type = FIELD_TYPE_VARBINARY;
- goto install_blob;
- }
- case MP_FLOAT: {
- mem->u.r = mp_decode_float(&buf);
- if (sqlIsNaN(mem->u.r)) {
- mem->flags = MEM_Null;
- mem->field_type = FIELD_TYPE_DOUBLE;
- } else {
- mem->flags = MEM_Real;
- mem->field_type = FIELD_TYPE_DOUBLE;
- }
- break;
- }
- case MP_DOUBLE: {
- mem->u.r = mp_decode_double(&buf);
- if (sqlIsNaN(mem->u.r)) {
- mem->flags = MEM_Null;
- mem->field_type = FIELD_TYPE_DOUBLE;
- } else {
- mem->flags = MEM_Real;
- mem->field_type = FIELD_TYPE_DOUBLE;
- }
- break;
- }
- default:
- unreachable();
- }
- *len = (uint32_t)(buf - start_buf);
- return 0;
-}
-
void
sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the record format */
const void *pKey, /* The binary record */
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 1101f7205..092350caa 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -37,867 +37,13 @@
* name sql_value
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "tarantoolInt.h"
#include "box/schema.h"
#include "box/tuple.h"
#include "mpstream/mpstream.h"
-#ifdef SQL_DEBUG
-/*
- * Check invariants on a Mem object.
- *
- * This routine is intended for use inside of assert() statements, like
- * this: assert( sqlVdbeCheckMemInvariants(pMem) );
- */
-int
-sqlVdbeCheckMemInvariants(Mem * p)
-{
- /* If MEM_Dyn is set then Mem.xDel!=0.
- * Mem.xDel is might not be initialized if MEM_Dyn is clear.
- */
- assert((p->flags & MEM_Dyn) == 0 || p->xDel != 0);
-
- /* MEM_Dyn may only be set if Mem.szMalloc==0. In this way we
- * ensure that if Mem.szMalloc>0 then it is safe to do
- * Mem.z = Mem.zMalloc without having to check Mem.flags&MEM_Dyn.
- * That saves a few cycles in inner loops.
- */
- assert((p->flags & MEM_Dyn) == 0 || p->szMalloc == 0);
-
- /* Cannot be both MEM_Int and MEM_Real at the same time */
- assert((p->flags & (MEM_Int | MEM_Real)) != (MEM_Int | MEM_Real));
- /* Can't be both UInt and Int at the same time. */
- assert((p->flags & (MEM_Int | MEM_UInt)) != (MEM_Int | MEM_UInt));
-
- /* The szMalloc field holds the correct memory allocation size */
- assert(p->szMalloc == 0 ||
- p->szMalloc == sqlDbMallocSize(p->db, p->zMalloc));
-
- /* If p holds a string or blob, the Mem.z must point to exactly
- * one of the following:
- *
- * (1) Memory in Mem.zMalloc and managed by the Mem object
- * (2) Memory to be freed using Mem.xDel
- * (3) An ephemeral string or blob
- * (4) A static string or blob
- */
- if ((p->flags & (MEM_Str | MEM_Blob)) && p->n > 0) {
- assert(((p->szMalloc > 0 && p->z == p->zMalloc) ? 1 : 0) +
- ((p->flags & MEM_Dyn) != 0 ? 1 : 0) +
- ((p->flags & MEM_Ephem) != 0 ? 1 : 0) +
- ((p->flags & MEM_Static) != 0 ? 1 : 0) == 1);
- }
- return 1;
-}
-#endif
-
-/*
- * Make sure pMem->z points to a writable allocation of at least
- * min(n,32) bytes.
- *
- * If the bPreserve argument is true, then copy of the content of
- * pMem->z into the new allocation. pMem must be either a string or
- * blob if bPreserve is true. If bPreserve is false, any prior content
- * in pMem->z is discarded.
- */
-SQL_NOINLINE int
-sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
-{
- assert(sqlVdbeCheckMemInvariants(pMem));
- testcase(pMem->db == 0);
-
- /* If the bPreserve flag is set to true, then the memory cell must already
- * contain a valid string or blob value.
- */
- assert(bPreserve == 0 || pMem->flags & (MEM_Blob | MEM_Str));
- testcase(bPreserve && pMem->z == 0);
-
- assert(pMem->szMalloc == 0 ||
- pMem->szMalloc == sqlDbMallocSize(pMem->db, pMem->zMalloc));
- if (pMem->szMalloc < n) {
- if (n < 32)
- n = 32;
- if (bPreserve && pMem->szMalloc > 0 && pMem->z == pMem->zMalloc) {
- pMem->z = pMem->zMalloc =
- sqlDbReallocOrFree(pMem->db, pMem->z, n);
- bPreserve = 0;
- } else {
- if (pMem->szMalloc > 0)
- sqlDbFree(pMem->db, pMem->zMalloc);
- pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
- }
- if (pMem->zMalloc == 0) {
- sqlVdbeMemSetNull(pMem);
- pMem->z = 0;
- pMem->szMalloc = 0;
- return -1;
- } else {
- pMem->szMalloc = sqlDbMallocSize(pMem->db,
- pMem->zMalloc);
- }
- }
-
- if (bPreserve && pMem->z && pMem->z != pMem->zMalloc) {
- memcpy(pMem->zMalloc, pMem->z, pMem->n);
- }
- if ((pMem->flags & MEM_Dyn) != 0) {
- assert(pMem->xDel != 0 && pMem->xDel != SQL_DYNAMIC);
- pMem->xDel((void *)(pMem->z));
- }
-
- pMem->z = pMem->zMalloc;
- pMem->flags &= ~(MEM_Dyn | MEM_Ephem | MEM_Static);
- return 0;
-}
-
-/*
- * Change the pMem->zMalloc allocation to be at least szNew bytes.
- * If pMem->zMalloc already meets or exceeds the requested size, this
- * routine is a no-op.
- *
- * Any prior string or blob content in the pMem object may be discarded.
- * The pMem->xDel destructor is called, if it exists. Though MEM_Str
- * and MEM_Blob values may be discarded, MEM_Int, MEM_Real, and MEM_Null
- * values are preserved.
- *
- * Return 0 on success or -1 if unable to complete the resizing.
- */
-int
-sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
-{
- assert(szNew > 0);
- assert((pMem->flags & MEM_Dyn) == 0 || pMem->szMalloc == 0);
- if (pMem->szMalloc < szNew) {
- return sqlVdbeMemGrow(pMem, szNew, 0);
- }
- assert((pMem->flags & MEM_Dyn) == 0);
- pMem->z = pMem->zMalloc;
- pMem->flags &= (MEM_Null | MEM_Int | MEM_Real);
- return 0;
-}
-
-/*
- * Change pMem so that its MEM_Str or MEM_Blob value is stored in
- * MEM.zMalloc, where it can be safely written.
- *
- * Return 0 on success or -1 if malloc fails.
- */
-int
-sqlVdbeMemMakeWriteable(Mem * pMem)
-{
- if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
- if (ExpandBlob(pMem))
- return -1;
- if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
- if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
- return -1;
- }
- pMem->z[pMem->n] = 0;
- pMem->z[pMem->n + 1] = 0;
- pMem->flags |= MEM_Term;
- }
- }
- pMem->flags &= ~MEM_Ephem;
-#ifdef SQL_DEBUG
- pMem->pScopyFrom = 0;
-#endif
-
- return 0;
-}
-
-/*
- * If the given Mem* has a zero-filled tail, turn it into an ordinary
- * blob stored in dynamically allocated space.
- */
-int
-sqlVdbeMemExpandBlob(Mem * pMem)
-{
- int nByte;
- assert(pMem->flags & MEM_Zero);
- assert(pMem->flags & MEM_Blob);
-
- /* Set nByte to the number of bytes required to store the expanded blob. */
- nByte = pMem->n + pMem->u.nZero;
- if (nByte <= 0) {
- nByte = 1;
- }
- if (sqlVdbeMemGrow(pMem, nByte, 1)) {
- return -1;
- }
-
- memset(&pMem->z[pMem->n], 0, pMem->u.nZero);
- pMem->n += pMem->u.nZero;
- pMem->flags &= ~(MEM_Zero | MEM_Term);
- return 0;
-}
-
-/*
- * It is already known that pMem contains an unterminated string.
- * Add the zero terminator.
- */
-static SQL_NOINLINE int
-vdbeMemAddTerminator(Mem * pMem)
-{
- if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
- return -1;
- }
- pMem->z[pMem->n] = 0;
- pMem->z[pMem->n + 1] = 0;
- pMem->flags |= MEM_Term;
- return 0;
-}
-
-/*
- * Make sure the given Mem is \u0000 terminated.
- */
-int
-sqlVdbeMemNulTerminate(Mem * pMem)
-{
- testcase((pMem->flags & (MEM_Term | MEM_Str)) == (MEM_Term | MEM_Str));
- testcase((pMem->flags & (MEM_Term | MEM_Str)) == 0);
- if ((pMem->flags & (MEM_Term | MEM_Str)) != MEM_Str) {
- return 0; /* Nothing to do */
- } else {
- return vdbeMemAddTerminator(pMem);
- }
-}
-
-static inline bool
-mem_has_msgpack_subtype(struct Mem *mem)
-{
- return (mem->flags & MEM_Subtype) != 0 &&
- mem->subtype == SQL_SUBTYPE_MSGPACK;
-}
-
-/*
- * Add MEM_Str to the set of representations for the given Mem. Numbers
- * are converted using sql_snprintf(). Converting a BLOB to a string
- * is a no-op.
- *
- * Existing representations MEM_Int and MEM_Real are invalidated if
- * bForce is true but are retained if bForce is false.
- *
- * A MEM_Null value will never be passed to this function. This function is
- * used for converting values to text for returning to the user (i.e. via
- * sql_value_text()), or for ensuring that values to be used as btree
- * keys are strings. In the former case a NULL pointer is returned the
- * user and the latter is an internal programming error.
- */
-int
-sqlVdbeMemStringify(Mem * pMem)
-{
- int fg = pMem->flags;
- int nByte = 32;
-
- if ((fg & (MEM_Null | MEM_Str | MEM_Blob)) != 0 &&
- !mem_has_msgpack_subtype(pMem))
- return 0;
-
- assert(!(fg & MEM_Zero));
- assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool |
- MEM_Blob)) != 0);
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
- /*
- * In case we have ARRAY/MAP we should save decoded value
- * before clearing pMem->z.
- */
- char *value = NULL;
- if (mem_has_msgpack_subtype(pMem)) {
- const char *value_str = mp_str(pMem->z);
- nByte = strlen(value_str) + 1;
- value = region_alloc(&fiber()->gc, nByte);
- memcpy(value, value_str, nByte);
- }
-
- if (sqlVdbeMemClearAndResize(pMem, nByte)) {
- return -1;
- }
- if (fg & MEM_Int) {
- sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
- pMem->flags &= ~MEM_Int;
- } else if ((fg & MEM_UInt) != 0) {
- sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
- pMem->flags &= ~MEM_UInt;
- } else if ((fg & MEM_Bool) != 0) {
- sql_snprintf(nByte, pMem->z, "%s",
- SQL_TOKEN_BOOLEAN(pMem->u.b));
- pMem->flags &= ~MEM_Bool;
- } else if (mem_has_msgpack_subtype(pMem)) {
- sql_snprintf(nByte, pMem->z, "%s", value);
- pMem->flags &= ~MEM_Subtype;
- pMem->subtype = SQL_SUBTYPE_NO;
- } else {
- assert(fg & MEM_Real);
- sql_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
- pMem->flags &= ~MEM_Real;
- }
- pMem->n = sqlStrlen30(pMem->z);
- pMem->flags |= MEM_Str | MEM_Term;
- return 0;
-}
-
-int
-sql_vdbemem_finalize(struct Mem *mem, struct func *func)
-{
- assert(func != NULL);
- assert(func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
- assert(func->def->aggregate == FUNC_AGGREGATE_GROUP);
- assert((mem->flags & MEM_Null) != 0 || func == mem->u.func);
- sql_context ctx;
- memset(&ctx, 0, sizeof(ctx));
- Mem t;
- memset(&t, 0, sizeof(t));
- t.flags = MEM_Null;
- t.db = mem->db;
- t.field_type = field_type_MAX;
- ctx.pOut = &t;
- ctx.pMem = mem;
- ctx.func = func;
- ((struct func_sql_builtin *)func)->finalize(&ctx);
- assert((mem->flags & MEM_Dyn) == 0);
- if (mem->szMalloc > 0)
- sqlDbFree(mem->db, mem->zMalloc);
- memcpy(mem, &t, sizeof(t));
- return ctx.is_aborted ? -1 : 0;
-}
-
-/*
- * If the memory cell contains a value that must be freed by
- * invoking the external callback in Mem.xDel, then this routine
- * will free that value. It also sets Mem.flags to MEM_Null.
- *
- * This is a helper routine for sqlVdbeMemSetNull() and
- * for sqlVdbeMemRelease(). Use those other routines as the
- * entry point for releasing Mem resources.
- */
-static SQL_NOINLINE void
-vdbeMemClearExternAndSetNull(Mem * p)
-{
- assert(VdbeMemDynamic(p));
- if (p->flags & MEM_Agg) {
- sql_vdbemem_finalize(p, p->u.func);
- assert((p->flags & MEM_Agg) == 0);
- testcase(p->flags & MEM_Dyn);
- }
- if (p->flags & MEM_Dyn) {
- assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
- p->xDel((void *)p->z);
- } else if (p->flags & MEM_Frame) {
- VdbeFrame *pFrame = p->u.pFrame;
- pFrame->pParent = pFrame->v->pDelFrame;
- pFrame->v->pDelFrame = pFrame;
- }
- p->flags = MEM_Null;
-}
-
-/*
- * Release memory held by the Mem p, both external memory cleared
- * by p->xDel and memory in p->zMalloc.
- *
- * This is a helper routine invoked by sqlVdbeMemRelease() in
- * the unusual case where there really is memory in p that needs
- * to be freed.
- */
-static SQL_NOINLINE void
-vdbeMemClear(Mem * p)
-{
- if (VdbeMemDynamic(p)) {
- vdbeMemClearExternAndSetNull(p);
- }
- if (p->szMalloc) {
- sqlDbFree(p->db, p->zMalloc);
- p->szMalloc = 0;
- }
- p->z = 0;
-}
-
-/*
- * Release any memory resources held by the Mem. Both the memory that is
- * free by Mem.xDel and the Mem.zMalloc allocation are freed.
- *
- * Use this routine prior to clean up prior to abandoning a Mem, or to
- * reset a Mem back to its minimum memory utilization.
- *
- * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
- * prior to inserting new content into the Mem.
- */
-void
-sqlVdbeMemRelease(Mem * p)
-{
- assert(sqlVdbeCheckMemInvariants(p));
- if (VdbeMemDynamic(p) || p->szMalloc) {
- vdbeMemClear(p);
- }
-}
-
-enum mp_type
-mem_mp_type(struct Mem *mem)
-{
- switch (mem->flags & MEM_PURE_TYPE_MASK) {
- case MEM_Int:
- return MP_INT;
- case MEM_UInt:
- return MP_UINT;
- case MEM_Real:
- return MP_DOUBLE;
- case MEM_Str:
- return MP_STR;
- case MEM_Blob:
- if ((mem->flags & MEM_Subtype) == 0 ||
- mem->subtype != SQL_SUBTYPE_MSGPACK)
- return MP_BIN;
- assert(mp_typeof(*mem->z) == MP_MAP ||
- mp_typeof(*mem->z) == MP_ARRAY);
- return mp_typeof(*mem->z);
- case MEM_Bool:
- return MP_BOOL;
- case MEM_Null:
- return MP_NIL;
- default: unreachable();
- }
-}
-
-/*
- * Convert a 64-bit IEEE double into a 64-bit signed integer.
- * If the double is out of range of a 64-bit signed integer then
- * return the closest available 64-bit signed integer.
- */
-static int
-doubleToInt64(double r, int64_t *i)
-{
- /*
- * Many compilers we encounter do not define constants for the
- * minimum and maximum 64-bit integers, or they define them
- * inconsistently. And many do not understand the "LL" notation.
- * So we define our own static constants here using nothing
- * larger than a 32-bit integer constant.
- */
- static const int64_t maxInt = LARGEST_INT64;
- static const int64_t minInt = SMALLEST_INT64;
- if (r <= (double)minInt) {
- *i = minInt;
- return -1;
- } else if (r >= (double)maxInt) {
- *i = maxInt;
- return -1;
- } else {
- *i = (int64_t) r;
- return *i != r;
- }
-}
-
-/*
- * Return some kind of integer value which is the best we can do
- * at representing the value that *pMem describes as an integer.
- * If pMem is an integer, then the value is exact. If pMem is
- * a floating-point then the value returned is the integer part.
- * If pMem is a string or blob, then we make an attempt to convert
- * it into an integer and return that. If pMem represents an
- * an SQL-NULL value, return 0.
- *
- * If pMem represents a string value, its encoding might be changed.
- */
-int
-sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
-{
- int flags;
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
- flags = pMem->flags;
- if (flags & MEM_Int) {
- *i = pMem->u.i;
- *is_neg = true;
- return 0;
- } else if (flags & MEM_UInt) {
- *i = pMem->u.u;
- *is_neg = false;
- return 0;
- } else if (flags & MEM_Real) {
- *is_neg = pMem->u.r < 0;
- return doubleToInt64(pMem->u.r, i);
- } else if (flags & (MEM_Str)) {
- assert(pMem->z || pMem->n == 0);
- if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
- return 0;
- }
- return -1;
-}
-
-/*
- * Return the best representation of pMem that we can get into a
- * double. If pMem is already a double or an integer, return its
- * value. If it is a string or blob, try to convert it to a double.
- * If it is a NULL, return 0.0.
- */
-int
-sqlVdbeRealValue(Mem * pMem, double *v)
-{
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
- if (pMem->flags & MEM_Real) {
- *v = pMem->u.r;
- return 0;
- } else if (pMem->flags & MEM_Int) {
- *v = (double)pMem->u.i;
- return 0;
- } else if ((pMem->flags & MEM_UInt) != 0) {
- *v = (double)pMem->u.u;
- return 0;
- } else if (pMem->flags & MEM_Str) {
- if (sqlAtoF(pMem->z, v, pMem->n))
- return 0;
- }
- return -1;
-}
-
-int
-mem_value_bool(const struct Mem *mem, bool *b)
-{
- if ((mem->flags & MEM_Bool) != 0) {
- *b = mem->u.b;
- return 0;
- }
- return -1;
-}
-
-/*
- * The MEM structure is already a MEM_Real. Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
- int rc;
- i64 ix;
- assert(pMem->flags & MEM_Real);
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
- mem_set_int(pMem, ix, pMem->u.r <= -1);
- return rc;
-}
-
-/*
- * Convert pMem so that it is of type MEM_Real.
- * Invalidate any prior representations.
- */
-int
-sqlVdbeMemRealify(Mem * pMem)
-{
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
- double v;
- if (sqlVdbeRealValue(pMem, &v))
- return -1;
- mem_set_double(pMem, v);
- return 0;
-}
-
-int
-vdbe_mem_numerify(struct Mem *mem)
-{
- if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
- return 0;
- if ((mem->flags & MEM_Bool) != 0) {
- mem->u.u = mem->u.b;
- MemSetTypeFlag(mem, MEM_UInt);
- return 0;
- }
- assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
- bool is_neg;
- int64_t i;
- if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
- mem_set_int(mem, i, is_neg);
- } else {
- double d;
- if (sqlAtoF(mem->z, &d, mem->n) == 0)
- return -1;
- mem_set_double(mem, d);
- }
- return 0;
-}
-
-/**
- * According to ANSI SQL string value can be converted to boolean
- * type if string consists of literal "true" or "false" and
- * number of leading and trailing spaces.
- *
- * For instance, " tRuE " can be successfully converted to
- * boolean value true.
- *
- * @param str String to be converted to boolean. Assumed to be
- * null terminated.
- * @param[out] result Resulting value of cast.
- * @retval 0 If string satisfies conditions above.
- * @retval -1 Otherwise.
- */
-static int
-str_cast_to_boolean(const char *str, bool *result)
-{
- assert(str != NULL);
- for (; *str == ' '; str++);
- if (strncasecmp(str, SQL_TOKEN_TRUE, strlen(SQL_TOKEN_TRUE)) == 0) {
- *result = true;
- str += 4;
- } else if (strncasecmp(str, SQL_TOKEN_FALSE,
- strlen(SQL_TOKEN_FALSE)) == 0) {
- *result = false;
- str += 5;
- } else {
- return -1;
- }
- for (; *str != '\0'; ++str) {
- if (*str != ' ')
- return -1;
- }
- return 0;
-}
-
-/*
- * Cast the datatype of the value in pMem according to the type
- * @type. Casting is different from applying type in that a cast
- * is forced. In other words, the value is converted into the desired
- * type even if that results in loss of data. This routine is
- * used (for example) to implement the SQL "cast()" operator.
- */
-int
-sqlVdbeMemCast(Mem * pMem, enum field_type type)
-{
- assert(type < field_type_MAX);
- if (pMem->flags & MEM_Null)
- return 0;
- switch (type) {
- case FIELD_TYPE_SCALAR:
- return 0;
- case FIELD_TYPE_BOOLEAN:
- if ((pMem->flags & MEM_Int) != 0) {
- mem_set_bool(pMem, pMem->u.i);
- return 0;
- }
- if ((pMem->flags & MEM_UInt) != 0) {
- mem_set_bool(pMem, pMem->u.u);
- return 0;
- }
- if ((pMem->flags & MEM_Real) != 0) {
- mem_set_bool(pMem, pMem->u.r);
- return 0;
- }
- if ((pMem->flags & MEM_Str) != 0) {
- bool value;
- if (str_cast_to_boolean(pMem->z, &value) != 0)
- return -1;
- mem_set_bool(pMem, value);
- return 0;
- }
- if ((pMem->flags & MEM_Bool) != 0)
- return 0;
- return -1;
- case FIELD_TYPE_INTEGER:
- case FIELD_TYPE_UNSIGNED:
- if ((pMem->flags & (MEM_Blob | MEM_Str)) != 0) {
- bool is_neg;
- int64_t val;
- if (sql_atoi64(pMem->z, &val, &is_neg, pMem->n) != 0)
- return -1;
- if (type == FIELD_TYPE_UNSIGNED && is_neg)
- return -1;
- mem_set_int(pMem, val, is_neg);
- return 0;
- }
- if ((pMem->flags & MEM_Bool) != 0) {
- pMem->u.u = pMem->u.b;
- MemSetTypeFlag(pMem, MEM_UInt);
- return 0;
- }
- if ((pMem->flags & MEM_Real) != 0) {
- double d;
- if (sqlVdbeRealValue(pMem, &d) != 0)
- return -1;
- if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
- mem_set_int(pMem, d, d <= -1);
- return 0;
- }
- if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
- mem_set_u64(pMem, d);
- return 0;
- }
- return -1;
- }
- if (type == FIELD_TYPE_UNSIGNED &&
- (pMem->flags & MEM_UInt) == 0)
- return -1;
- return 0;
- case FIELD_TYPE_DOUBLE:
- return sqlVdbeMemRealify(pMem);
- case FIELD_TYPE_NUMBER:
- return vdbe_mem_numerify(pMem);
- case FIELD_TYPE_VARBINARY:
- if ((pMem->flags & MEM_Blob) != 0)
- return 0;
- if ((pMem->flags & MEM_Str) != 0) {
- MemSetTypeFlag(pMem, MEM_Str);
- return 0;
- }
- return -1;
- default:
- assert(type == FIELD_TYPE_STRING);
- assert(MEM_Str == (MEM_Blob >> 3));
- if ((pMem->flags & MEM_Bool) != 0) {
- const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
- sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
- SQL_TRANSIENT);
- return 0;
- }
- pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
- sql_value_apply_type(pMem, FIELD_TYPE_STRING);
- assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
- pMem->flags &=
- ~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
- return 0;
- }
-}
-
-/*
- * Initialize bulk memory to be a consistent Mem object.
- *
- * The minimum amount of initialization feasible is performed.
- */
-void
-sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
-{
- assert((flags & ~MEM_TypeMask) == 0);
- pMem->flags = flags;
- pMem->db = db;
- pMem->szMalloc = 0;
- pMem->field_type = field_type_MAX;
-}
-
-/*
- * Delete any previous value and set the value stored in *pMem to NULL.
- *
- * This routine calls the Mem.xDel destructor to dispose of values that
- * require the destructor. But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use sqlVdbeMemRelease(), which both calls this
- * routine to invoke the destructor and deallocates Mem.zMalloc.
- *
- * Use this routine to reset the Mem prior to insert a new value.
- *
- * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
- */
-void
-sqlVdbeMemSetNull(Mem * pMem)
-{
- if (VdbeMemDynamic(pMem)) {
- vdbeMemClearExternAndSetNull(pMem);
- } else {
- pMem->flags = MEM_Null;
- }
-}
-
-void
-sqlValueSetNull(sql_value * p)
-{
- sqlVdbeMemSetNull((Mem *) p);
-}
-
-void
-mem_set_ptr(struct Mem *mem, void *ptr)
-{
- sqlVdbeMemRelease(mem);
- mem->flags = MEM_Ptr;
- mem->u.p = ptr;
-}
-
-/*
- * Delete any previous value and set the value to be a BLOB of length
- * n containing all zeros.
- */
-void
-sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
-{
- sqlVdbeMemRelease(pMem);
- pMem->flags = MEM_Blob | MEM_Zero;
- pMem->n = 0;
- if (n < 0)
- n = 0;
- pMem->u.nZero = n;
- pMem->z = 0;
-}
-
-void
-mem_set_bool(struct Mem *mem, bool value)
-{
- sqlVdbeMemSetNull(mem);
- mem->u.b = value;
- mem->flags = MEM_Bool;
- mem->field_type = FIELD_TYPE_BOOLEAN;
-}
-
-void
-mem_set_i64(struct Mem *mem, int64_t value)
-{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
- mem->u.i = value;
- int flag = value < 0 ? MEM_Int : MEM_UInt;
- MemSetTypeFlag(mem, flag);
- mem->field_type = FIELD_TYPE_INTEGER;
-}
-
-void
-mem_set_u64(struct Mem *mem, uint64_t value)
-{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
- mem->u.u = value;
- MemSetTypeFlag(mem, MEM_UInt);
- mem->field_type = FIELD_TYPE_UNSIGNED;
-}
-
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
-{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
- if (is_neg) {
- assert(value < 0);
- mem->u.i = value;
- MemSetTypeFlag(mem, MEM_Int);
- } else {
- mem->u.u = value;
- MemSetTypeFlag(mem, MEM_UInt);
- }
- mem->field_type = FIELD_TYPE_INTEGER;
-}
-
-void
-mem_set_double(struct Mem *mem, double value)
-{
- sqlVdbeMemSetNull(mem);
- if (sqlIsNaN(value))
- return;
- mem->u.r = value;
- MemSetTypeFlag(mem, MEM_Real);
- mem->field_type = FIELD_TYPE_DOUBLE;
-}
-
-/*
- * Return true if the Mem object contains a TEXT or BLOB that is
- * too large - whose size exceeds SQL_MAX_LENGTH.
- */
-int
-sqlVdbeMemTooBig(Mem * p)
-{
- assert(p->db != 0);
- if (p->flags & (MEM_Str | MEM_Blob)) {
- int n = p->n;
- if (p->flags & MEM_Zero) {
- n += p->u.nZero;
- }
- return n > p->db->aLimit[SQL_LIMIT_LENGTH];
- }
- return 0;
-}
-
#ifdef SQL_DEBUG
/*
* This routine prepares a memory cell for modification by breaking
@@ -921,173 +67,6 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
pMem->pScopyFrom = 0;
}
#endif /* SQL_DEBUG */
-
-/*
- * Make an shallow copy of pFrom into pTo. Prior contents of
- * pTo are freed. The pFrom->z field is not duplicated. If
- * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
- * and flags gets srcType (either MEM_Ephem or MEM_Static).
- */
-static SQL_NOINLINE void
-vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
-{
- vdbeMemClearExternAndSetNull(pTo);
- assert(!VdbeMemDynamic(pTo));
- sqlVdbeMemShallowCopy(pTo, pFrom, eType);
-}
-
-void
-sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
-{
- assert(pTo->db == pFrom->db);
- if (VdbeMemDynamic(pTo)) {
- vdbeClrCopy(pTo, pFrom, srcType);
- return;
- }
- memcpy(pTo, pFrom, MEMCELLSIZE);
- if ((pFrom->flags & MEM_Static) == 0) {
- pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
- assert(srcType == MEM_Ephem || srcType == MEM_Static);
- pTo->flags |= srcType;
- }
-}
-
-/*
- * Make a full copy of pFrom into pTo. Prior contents of pTo are
- * freed before the copy is made.
- */
-int
-sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
-{
- int rc = 0;
-
- if (VdbeMemDynamic(pTo))
- vdbeMemClearExternAndSetNull(pTo);
- memcpy(pTo, pFrom, MEMCELLSIZE);
- pTo->flags &= ~MEM_Dyn;
- if (pTo->flags & (MEM_Str | MEM_Blob)) {
- if (0 == (pFrom->flags & MEM_Static)) {
- pTo->flags |= MEM_Ephem;
- rc = sqlVdbeMemMakeWriteable(pTo);
- }
- }
-
- return rc;
-}
-
-/*
- * Transfer the contents of pFrom to pTo. Any existing value in pTo is
- * freed. If pFrom contains ephemeral data, a copy is made.
- *
- * pFrom contains an SQL NULL when this routine returns.
- */
-void
-sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
-{
- assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
-
- sqlVdbeMemRelease(pTo);
- memcpy(pTo, pFrom, sizeof(Mem));
- pFrom->flags = MEM_Null;
- pFrom->szMalloc = 0;
-}
-
-/*
- * Change the value of a Mem to be a string or a BLOB.
- *
- * The memory management strategy depends on the value of the xDel
- * parameter. If the value passed is SQL_TRANSIENT, then the
- * string is copied into a (possibly existing) buffer managed by the
- * Mem structure. Otherwise, any existing buffer is freed and the
- * pointer copied.
- *
- * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
- * size limit) then no memory allocation occurs. If the string can be
- * stored without allocating memory, then it is. If a memory allocation
- * is required to store the string, then value of pMem is unchanged. In
- * either case, error is returned.
- */
-int
-sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
- const char *z, /* String pointer */
- int n, /* Bytes in string, or negative */
- u8 not_blob, /* Encoding of z. 0 for BLOBs */
- void (*xDel) (void *) /* Destructor function */
- )
-{
- int nByte = n; /* New value for pMem->n */
- int iLimit; /* Maximum allowed string or blob size */
- u16 flags = 0; /* New value for pMem->flags */
-
- /* If z is a NULL pointer, set pMem to contain an SQL NULL. */
- if (!z) {
- sqlVdbeMemSetNull(pMem);
- return 0;
- }
-
- if (pMem->db) {
- iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
- } else {
- iLimit = SQL_MAX_LENGTH;
- }
- flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
- if (nByte < 0) {
- assert(not_blob != 0);
- nByte = sqlStrlen30(z);
- if (nByte > iLimit)
- nByte = iLimit + 1;
- flags |= MEM_Term;
- }
-
- /* The following block sets the new values of Mem.z and Mem.xDel. It
- * also sets a flag in local variable "flags" to indicate the memory
- * management (one of MEM_Dyn or MEM_Static).
- */
- if (xDel == SQL_TRANSIENT) {
- int nAlloc = nByte;
- if (flags & MEM_Term) {
- nAlloc += 1; //SQL_UTF8
- }
- if (nByte > iLimit) {
- diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
- "string is too big");
- return -1;
- }
- testcase(nAlloc == 0);
- testcase(nAlloc == 31);
- testcase(nAlloc == 32);
- if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
- return -1;
- }
- memcpy(pMem->z, z, nAlloc);
- } else if (xDel == SQL_DYNAMIC) {
- sqlVdbeMemRelease(pMem);
- pMem->zMalloc = pMem->z = (char *)z;
- pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
- } else {
- sqlVdbeMemRelease(pMem);
- pMem->z = (char *)z;
- pMem->xDel = xDel;
- flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
- }
-
- pMem->n = nByte;
- pMem->flags = flags;
- assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
- if ((pMem->flags & MEM_Str) != 0)
- pMem->field_type = FIELD_TYPE_STRING;
- else
- pMem->field_type = FIELD_TYPE_VARBINARY;
-
- if (nByte > iLimit) {
- diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
- "is too big");
- return -1;
- }
-
- return 0;
-}
-
/*
* Move data out of a btree key or data field and into a Mem structure.
* The data is payload from the entry that pCur is currently pointing
@@ -1153,79 +132,6 @@ sqlVdbeMemFromBtree(BtCursor * pCur, /* Cursor pointing at record to retrieve. *
return rc;
}
-/*
- * The pVal argument is known to be a value other than NULL.
- * Convert it into a string with encoding enc and return a pointer
- * to a zero-terminated version of that string.
- */
-static SQL_NOINLINE const void *
-valueToText(sql_value * pVal)
-{
- assert(pVal != 0);
- assert((pVal->flags & (MEM_Null)) == 0);
- if ((pVal->flags & (MEM_Blob | MEM_Str)) &&
- !mem_has_msgpack_subtype(pVal)) {
- if (ExpandBlob(pVal))
- return 0;
- pVal->flags |= MEM_Str;
- sqlVdbeMemNulTerminate(pVal); /* IMP: R-31275-44060 */
- } else {
- sqlVdbeMemStringify(pVal);
- assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
- }
- return pVal->z;
-}
-
-/* This function is only available internally, it is not part of the
- * external API. It works in a similar way to sql_value_text(),
- * except the data returned is in the encoding specified by the second
- * parameter, which must be one of SQL_UTF16BE, SQL_UTF16LE or
- * SQL_UTF8.
- *
- * (2006-02-16:) The enc value can be or-ed with SQL_UTF16_ALIGNED.
- * If that is the case, then the result must be aligned on an even byte
- * boundary.
- */
-const void *
-sqlValueText(sql_value * pVal)
-{
- if (!pVal)
- return 0;
- if ((pVal->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)) {
- return pVal->z;
- }
- if (pVal->flags & MEM_Null) {
- return 0;
- }
- return valueToText(pVal);
-}
-
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
- enum mp_type mp_type = sql_value_type(value);
- if (mp_type_is_bloblike(mp_type)) {
- if (mem_has_msgpack_subtype(value))
- return sqlValueText(value);
- return "varbinary";
- }
- return sqlValueText(value);
-}
-
-/*
- * Create a new sql_value object.
- */
-sql_value *
-sqlValueNew(sql * db)
-{
- Mem *p = sqlDbMallocZero(db, sizeof(*p));
- if (p) {
- p->flags = MEM_Null;
- p->db = db;
- }
- return p;
-}
-
/*
* Context object passed by sqlStat4ProbeSetValue() through to
* valueNew(). See comments above valueNew() for details.
@@ -1747,125 +653,3 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
sqlDbFree(aMem[0].db, pRec);
}
}
-
-/*
- * Change the string value of an sql_value object
- */
-void
-sqlValueSetStr(sql_value * v, /* Value to be set */
- int n, /* Length of string z */
- const void *z, /* Text of the new string */
- void (*xDel) (void *) /* Destructor for the string */
- )
-{
- if (v)
- sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
-}
-
-/*
- * Free an sql_value object
- */
-void
-sqlValueFree(sql_value * v)
-{
- if (!v)
- return;
- sqlVdbeMemRelease((Mem *) v);
- sqlDbFree(((Mem *) v)->db, v);
-}
-
-/*
- * The sqlValueBytes() routine returns the number of bytes in the
- * sql_value object assuming that it uses the encoding "enc".
- * The valueBytes() routine is a helper function.
- */
-static SQL_NOINLINE int
-valueBytes(sql_value * pVal)
-{
- return valueToText(pVal) != 0 ? pVal->n : 0;
-}
-
-int
-sqlValueBytes(sql_value * pVal)
-{
- Mem *p = (Mem *) pVal;
- assert((p->flags & MEM_Null) == 0
- || (p->flags & (MEM_Str | MEM_Blob)) == 0);
- if ((p->flags & MEM_Str) != 0) {
- return p->n;
- }
- if ((p->flags & MEM_Blob) != 0) {
- if (p->flags & MEM_Zero) {
- return p->n + p->u.nZero;
- } else {
- return p->n;
- }
- }
- if (p->flags & MEM_Null)
- return 0;
- return valueBytes(pVal);
-}
-
-void
-mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
-{
- assert(memIsValid(var));
- int64_t i;
- if (var->flags & MEM_Null) {
- mpstream_encode_nil(stream);
- } else if (var->flags & MEM_Real) {
- mpstream_encode_double(stream, var->u.r);
- } else if (var->flags & MEM_Int) {
- i = var->u.i;
- mpstream_encode_int(stream, i);
- } else if (var->flags & MEM_UInt) {
- i = var->u.u;
- mpstream_encode_uint(stream, i);
- } else if (var->flags & MEM_Str) {
- mpstream_encode_strn(stream, var->z, var->n);
- } else if (var->flags & MEM_Bool) {
- mpstream_encode_bool(stream, var->u.b);
- } else {
- /*
- * Emit BIN header iff the BLOB doesn't store
- * MsgPack content.
- */
- if (!mem_has_msgpack_subtype(var)) {
- uint32_t binl = var->n +
- ((var->flags & MEM_Zero) ?
- var->u.nZero : 0);
- mpstream_encode_binl(stream, binl);
- }
- mpstream_memcpy(stream, var->z, var->n);
- if (var->flags & MEM_Zero)
- mpstream_memset(stream, 0, var->u.nZero);
- }
-}
-
-char *
-sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
- uint32_t *tuple_size, struct region *region)
-{
- size_t used = region_used(region);
- bool is_error = false;
- struct mpstream stream;
- mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
- set_encode_error, &is_error);
- mpstream_encode_array(&stream, field_count);
- for (struct Mem *field = fields; field < fields + field_count; field++)
- mpstream_encode_vdbe_mem(&stream, field);
- mpstream_flush(&stream);
- if (is_error) {
- diag_set(OutOfMemory, stream.pos - stream.buf,
- "mpstream_flush", "stream");
- return NULL;
- }
- *tuple_size = region_used(region) - used;
- char *tuple = region_join(region, *tuple_size);
- if (tuple == NULL) {
- diag_set(OutOfMemory, *tuple_size, "region_join", "tuple");
- return NULL;
- }
- mp_tuple_assert(tuple, tuple + *tuple_size);
- return tuple;
-}
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index a2d681255..927f85559 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -152,6 +152,7 @@
* the main thread to read from.
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
/*
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index 2ee9f668c..e84bb3192 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -37,6 +37,7 @@
* The Vdbe parse-tree explainer is also found here.
*/
#include "sqlInt.h"
+#include "mem.h"
#include "vdbeInt.h"
/*
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 9c4f8b917..e5f35fbf8 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -40,6 +40,7 @@
#include "coll/coll.h"
#include "sqlInt.h"
#include "tarantoolInt.h"
+#include "mem.h"
#include "vdbeInt.h"
#include "whereInt.h"
#include "box/coll_id_cache.h"
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 3811ef3cf..0c002dbee 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -40,6 +40,7 @@
#include "box/coll_id_cache.h"
#include "coll/coll.h"
#include "sqlInt.h"
+#include "mem.h"
#include "whereInt.h"
/* Forward declarations */
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 06/52] sql: refactor port_vdbemem_*() functions
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (4 preceding siblings ...)
2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 05/52] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:59 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 07/52] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
` (35 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:59 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch simplifies port_vdbemem_dump_lua() and
port_vdbemem_get_msgpack() functions.
Part of #5818
---
src/box/sql/mem.c | 85 +++++++++--------------------------------------
1 file changed, 16 insertions(+), 69 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index d28b9a958..25f5f2f2d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -2365,33 +2365,26 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
struct port_vdbemem *port = (struct port_vdbemem *) base;
assert(is_flat == true);
for (uint32_t i = 0; i < port->mem_count; i++) {
- sql_value *param =
- (sql_value *)((struct Mem *)port->mem + i);
- switch (sql_value_type(param)) {
- case MP_INT:
- luaL_pushint64(L, sql_value_int64(param));
+ struct Mem *mem = (struct Mem *)port->mem + i;
+ switch (mem->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Int:
+ luaL_pushint64(L, mem->u.i);
break;
- case MP_UINT:
- luaL_pushuint64(L, sql_value_uint64(param));
+ case MEM_UInt:
+ luaL_pushuint64(L, mem->u.u);
break;
- case MP_DOUBLE:
- lua_pushnumber(L, sql_value_double(param));
+ case MEM_Real:
+ lua_pushnumber(L, mem->u.r);
break;
- case MP_STR:
- lua_pushlstring(L, (const char *)sql_value_text(param),
- (size_t)sql_value_bytes(param));
+ case MEM_Str:
+ case MEM_Blob:
+ lua_pushlstring(L, mem->z, mem->n);
break;
- case MP_BIN:
- case MP_ARRAY:
- case MP_MAP:
- lua_pushlstring(L, sql_value_blob(param),
- (size_t) sql_value_bytes(param));
- break;
- case MP_NIL:
+ case MEM_Null:
lua_pushnil(L);
break;
- case MP_BOOL:
- lua_pushboolean(L, sql_value_boolean(param));
+ case MEM_Bool:
+ lua_pushboolean(L, mem->u.b);
break;
default:
unreachable();
@@ -2410,54 +2403,8 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
set_encode_error, &is_error);
mpstream_encode_array(&stream, port->mem_count);
- for (uint32_t i = 0; i < port->mem_count && !is_error; i++) {
- sql_value *param =
- (sql_value *)((struct Mem *)port->mem + i);
- switch (sql_value_type(param)) {
- case MP_INT: {
- sql_int64 val = sql_value_int64(param);
- if (val < 0) {
- mpstream_encode_int(&stream, val);
- break;
- }
- FALLTHROUGH;
- }
- case MP_UINT: {
- sql_uint64 val = sql_value_uint64(param);
- mpstream_encode_uint(&stream, val);
- break;
- }
- case MP_DOUBLE: {
- mpstream_encode_double(&stream,
- sql_value_double(param));
- break;
- }
- case MP_STR: {
- const char *str = (const char *) sql_value_text(param);
- mpstream_encode_strn(&stream, str,
- sql_value_bytes(param));
- break;
- }
- case MP_BIN:
- case MP_ARRAY:
- case MP_MAP: {
- mpstream_encode_binl(&stream, sql_value_bytes(param));
- mpstream_memcpy(&stream, sql_value_blob(param),
- sql_value_bytes(param));
- break;
- }
- case MP_NIL: {
- mpstream_encode_nil(&stream);
- break;
- }
- case MP_BOOL: {
- mpstream_encode_bool(&stream, sql_value_boolean(param));
- break;
- }
- default:
- unreachable();
- }
- }
+ for (uint32_t i = 0; i < port->mem_count && !is_error; i++)
+ mpstream_encode_vdbe_mem(&stream, (struct Mem *)port->mem + i);
mpstream_flush(&stream);
*size = region_used(region) - region_svp;
if (is_error)
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 07/52] sql: remove unused MEM-related functions
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (5 preceding siblings ...)
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 06/52] sql: refactor port_vdbemem_*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:59 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 08/52] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
` (34 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:59 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answer and new patch below.
On 30.03.2021 01:58, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch removes remove unused MEM-related functions.
>
> You can simply omit commit message when it is the same as the
> title.
Fixed, removed the commit message.
New patch:
commit a3cd1dea89dae108586c1cf3cd0a3cbc9badfbc4
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 10:58:15 2021 +0300
sql: remove unused MEM-related functions
Part of #5818
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index c1a42fc2f..dd8163f5e 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -474,10 +474,6 @@ sql_column_text(sql_stmt *,
enum mp_type
sql_column_type(sql_stmt *stmt, int field_no);
-sql_value *
-sql_column_value(sql_stmt *,
- int iCol);
-
/*
* Terminate the current execution of an SQL statement and reset
* it back to its starting state so that it can be reused.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 775bce96f..a4a8c45f2 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -181,14 +181,6 @@ vdbeTakeBranch(int iSrcLine, u8 I, u8 M)
}
#endif
-/*
- * Convert the given register into a string if it isn't one
- * already. Return non-zero if a malloc() fails.
- */
-#define Stringify(P) \
- if(((P)->flags&(MEM_Str|MEM_Blob))==0 && sqlVdbeMemStringify(P)) \
- { goto no_mem; }
-
/*
* An ephemeral string value (signified by the MEM_Ephem flag) contains
* a pointer to a dynamically allocated string where some other entity
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index b2d4f27c1..541b93d30 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -333,8 +333,6 @@ int sqlVdbeList(Vdbe *);
int sqlVdbeHalt(Vdbe *);
-int sqlVdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
-
/**
* In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
* messagepacks are stored as binary string (i.e. featuring
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 24fa99f17..671338361 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -101,63 +101,12 @@ sql_stmt_reset(sql_stmt *pStmt)
return rc;
}
-/*
- * Set all the parameters in the compiled SQL statement to NULL.
- */
-int
-sql_clear_bindings(sql_stmt * pStmt)
-{
- int i;
- int rc = 0;
- Vdbe *p = (Vdbe *) pStmt;
- for (i = 0; i < p->nVar; i++) {
- sqlVdbeMemRelease(&p->aVar[i]);
- p->aVar[i].flags = MEM_Null;
- }
- return rc;
-}
-
bool
sql_metadata_is_full()
{
return current_session()->sql_flags & SQL_FullMetadata;
}
-/* Make a copy of an sql_value object
- */
-sql_value *
-sql_value_dup(const sql_value * pOrig)
-{
- sql_value *pNew;
- if (pOrig == 0)
- return 0;
- pNew = sql_malloc(sizeof(*pNew));
- if (pNew == 0)
- return 0;
- memset(pNew, 0, sizeof(*pNew));
- memcpy(pNew, pOrig, MEMCELLSIZE);
- pNew->flags &= ~MEM_Dyn;
- pNew->db = 0;
- if (pNew->flags & (MEM_Str | MEM_Blob)) {
- pNew->flags &= ~(MEM_Static | MEM_Dyn);
- pNew->flags |= MEM_Ephem;
- if (sqlVdbeMemMakeWriteable(pNew) != 0) {
- sqlValueFree(pNew);
- pNew = 0;
- }
- }
- return pNew;
-}
-
-/* Destroy an sql_value object previously obtained from
- * sql_value_dup().
- */
-void
-sql_value_free(sql_value * pOld)
-{
- sqlValueFree(pOld);
-}
-
/**************************** sql_result_ ******************************
* The following routines are used by user-defined functions to specify
* the function result.
@@ -566,17 +515,6 @@ sql_column_text(sql_stmt * pStmt, int i)
return sql_value_text(columnMem(pStmt, i));
}
-sql_value *
-sql_column_value(sql_stmt * pStmt, int i)
-{
- Mem *pOut = columnMem(pStmt, i);
- if (pOut->flags & MEM_Static) {
- pOut->flags &= ~MEM_Static;
- pOut->flags |= MEM_Ephem;
- }
- return (sql_value *) pOut;
-}
-
enum mp_type
sql_column_type(sql_stmt * pStmt, int i)
{
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 092350caa..d977cbac8 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -67,71 +67,6 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
pMem->pScopyFrom = 0;
}
#endif /* SQL_DEBUG */
-/*
- * Move data out of a btree key or data field and into a Mem structure.
- * The data is payload from the entry that pCur is currently pointing
- * to. offset and amt determine what portion of the data or key to retrieve.
- * The result is written into the pMem element.
- *
- * The pMem object must have been initialized. This routine will use
- * pMem->zMalloc to hold the content from the btree, if possible. New
- * pMem->zMalloc space will be allocated if necessary. The calling routine
- * is responsible for making sure that the pMem object is eventually
- * destroyed.
- *
- * If this routine fails for any reason (malloc returns NULL or unable
- * to read from the disk) then the pMem is left in an inconsistent state.
- */
-static SQL_NOINLINE int
-vdbeMemFromBtreeResize(BtCursor * pCur, /* Cursor pointing at record to retrieve. */
- u32 offset, /* Offset from the start of data to return bytes from. */
- u32 amt, /* Number of bytes to return. */
- Mem * pMem /* OUT: Return data in this Mem structure. */
- )
-{
- int rc;
- pMem->flags = MEM_Null;
- if (0 == (rc = sqlVdbeMemClearAndResize(pMem, amt + 2))) {
- sqlCursorPayload(pCur, offset, amt, pMem->z);
- pMem->z[amt] = 0;
- pMem->z[amt + 1] = 0;
- pMem->flags = MEM_Blob | MEM_Term;
- pMem->n = (int) amt;
- }
- return rc;
-}
-
-int
-sqlVdbeMemFromBtree(BtCursor * pCur, /* Cursor pointing at record to retrieve. */
- u32 offset, /* Offset from the start of data to return bytes from. */
- u32 amt, /* Number of bytes to return. */
- Mem * pMem /* OUT: Return data in this Mem structure. */
- )
-{
- char *zData; /* Data from the btree layer */
- u32 available = 0; /* Number of bytes available on the local btree page */
- int rc = 0; /* Return code */
-
- assert(sqlCursorIsValid(pCur));
- assert(!VdbeMemDynamic(pMem));
- assert(pCur->curFlags & BTCF_TaCursor ||
- pCur->curFlags & BTCF_TEphemCursor);
-
-
- zData = (char *)tarantoolsqlPayloadFetch(pCur, &available);
- assert(zData != 0);
-
- if (offset + amt <= available) {
- pMem->z = &zData[offset];
- pMem->flags = MEM_Blob | MEM_Ephem;
- pMem->n = (int)amt;
- } else {
- rc = vdbeMemFromBtreeResize(pCur, offset, amt, pMem);
- }
-
- return rc;
-}
-
/*
* Context object passed by sqlStat4ProbeSetValue() through to
* valueNew(). See comments above valueNew() for details.
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 08/52] sql: disable unused code in sql/vdbemem.c
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (6 preceding siblings ...)
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 07/52] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:59 ` Mergen Imeev via Tarantool-patches
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
` (33 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:59 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answer and new patch below.
On 30.03.2021 01:58, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> I didn't even know we have vdbemem.c. It should be deleted someday
> when we have mem.h/mem.c instead.
>
>> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
>> index dd8163f5e..b4293d961 100644
>> --- a/src/box/sql/sqlInt.h
>> +++ b/src/box/sql/sqlInt.h
>> @@ -4151,31 +4151,54 @@ sql_expr_new_column(struct sql *db, struct SrcList *src_list, int src_idx,
>>
>> int sqlExprCheckIN(Parse *, Expr *);
>>
>> -int sqlStat4ProbeSetValue(Parse *, struct index_def *, UnpackedRecord **, Expr *, int,
>> - int, int *);
>> -int sqlStat4ValueFromExpr(Parse *, Expr *, enum field_type type,
>> - sql_value **);
>> -void sqlStat4ProbeFree(UnpackedRecord *);
>> +/* TODO: Enable this function when stat-tables will be revived. */
>> +static inline int
>> +sqlStat4ProbeSetValue(struct Parse *parse, struct index_def *def,
>> + struct UnpackedRecord **rec, struct Expr *expr, int n,
>> + int i, int *out)
>
> You can also make it take '...' instead of all of these arguments individually.
> The same for the other functions.
>
Thank you! Dixed.
>> +{
>> + (void)parse;
>> + (void)def;
>> + (void)rec;
>> + (void)expr;
>> + (void)n;
>> + (void)i;
>> + (void)out;
>> + unreachable();
>> + return 0;
>> +}
New patch:
commit 152edbab6445a2b77f06c2164310a00a335b6df4
Author: Mergen Imeev <imeevma@gmail.com>
Date: Thu Apr 1 15:09:15 2021 +0300
sql: disable unused code in sql/vdbemem.c
This patch disables unused code in sql/vdbemem.c. It will simplify
refactoring.
Part of #5818
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index dd8163f5e..e075224c6 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -4151,31 +4151,38 @@ sql_expr_new_column(struct sql *db, struct SrcList *src_list, int src_idx,
int sqlExprCheckIN(Parse *, Expr *);
-int sqlStat4ProbeSetValue(Parse *, struct index_def *, UnpackedRecord **, Expr *, int,
- int, int *);
-int sqlStat4ValueFromExpr(Parse *, Expr *, enum field_type type,
- sql_value **);
-void sqlStat4ProbeFree(UnpackedRecord *);
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
+sqlStat4ProbeSetValue(struct Parse *parse, ...)
+{
+ (void)parse;
+ unreachable();
+ return 0;
+}
-/**
- * Extract the col_num-th column from the record. Write
- * the column value into *res. If *res is initially NULL
- * then a new sql_value object is allocated.
- *
- * If *res is initially NULL then the caller is responsible for
- * ensuring that the value written into *res is eventually
- * freed.
- *
- * @param db Database handle.
- * @param record Pointer to buffer containing record.
- * @param col_num Column to extract.
- * @param[out] res Extracted value.
- *
- * @retval -1 on error or 0.
- */
-int
-sql_stat4_column(struct sql *db, const char *record, uint32_t col_num,
- sql_value **res);
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
+sqlStat4ValueFromExpr(struct Parse *parse, ...)
+{
+ (void)parse;
+ unreachable();
+ return 0;
+}
+
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline void
+sqlStat4ProbeFree(struct UnpackedRecord *rec)
+{
+ (void)rec;
+}
+
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
+sql_stat4_column(struct sql *db, ...)
+{
+ (void)db;
+ return 0;
+}
/*
* The interface to the LEMON-generated parser
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index a4a8c45f2..e269857ea 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -58,16 +58,30 @@
#include "box/sequence.h"
#include "box/session_settings.h"
+#ifdef SQL_DEBUG
+
/*
- * Invoke this macro on memory cells just prior to changing the
- * value of the cell. This macro verifies that shallow copies are
- * not misused. A shallow copy of a string or blob just copies a
- * pointer to the string or blob, not the content. If the original
- * is changed while the copy is still in use, the string or blob might
- * be changed out from under the copy. This macro verifies that nothing
- * like that ever happens.
+ * This routine prepares a memory cell for modification by breaking
+ * its link to a shallow copy and by marking any current shallow
+ * copies of this cell as invalid.
+ *
+ * This is used for testing and debugging only - to make sure shallow
+ * copies are not misused.
*/
-#ifdef SQL_DEBUG
+static void
+sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
+{
+ int i;
+ Mem *pX;
+ for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
+ if (pX->pScopyFrom == pMem) {
+ pX->flags |= MEM_Undefined;
+ pX->pScopyFrom = 0;
+ }
+ }
+ pMem->pScopyFrom = 0;
+}
+
# define memAboutToChange(P,M) sqlVdbeMemAboutToChange(P,M)
#else
# define memAboutToChange(P,M)
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 541b93d30..b48e45770 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -362,10 +362,6 @@ int sqlVdbeSorterRewind(const VdbeCursor *, int *);
int sqlVdbeSorterWrite(const VdbeCursor *, Mem *);
int sqlVdbeSorterCompare(const VdbeCursor *, Mem *, int, int *);
-#ifdef SQL_DEBUG
-void sqlVdbeMemAboutToChange(Vdbe *, Mem *);
-#endif
-
int sqlVdbeCheckFk(Vdbe *, int);
int sqlVdbeMemTranslate(Mem *, u8);
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index d977cbac8..263fe5b00 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -44,29 +44,8 @@
#include "box/tuple.h"
#include "mpstream/mpstream.h"
-#ifdef SQL_DEBUG
-/*
- * This routine prepares a memory cell for modification by breaking
- * its link to a shallow copy and by marking any current shallow
- * copies of this cell as invalid.
- *
- * This is used for testing and debugging only - to make sure shallow
- * copies are not misused.
- */
-void
-sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
-{
- int i;
- Mem *pX;
- for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
- if (pX->pScopyFrom == pMem) {
- pX->flags |= MEM_Undefined;
- pX->pScopyFrom = 0;
- }
- }
- pMem->pScopyFrom = 0;
-}
-#endif /* SQL_DEBUG */
+#if 0
+
/*
* Context object passed by sqlStat4ProbeSetValue() through to
* valueNew(). See comments above valueNew() for details.
@@ -546,6 +525,22 @@ sqlStat4ValueFromExpr(Parse * pParse, /* Parse context */
return stat4ValueFromExpr(pParse, pExpr, type, 0, ppVal);
}
+/**
+ * Extract the col_num-th column from the record. Write
+ * the column value into *res. If *res is initially NULL
+ * then a new sql_value object is allocated.
+ *
+ * If *res is initially NULL then the caller is responsible for
+ * ensuring that the value written into *res is eventually
+ * freed.
+ *
+ * @param db Database handle.
+ * @param record Pointer to buffer containing record.
+ * @param col_num Column to extract.
+ * @param[out] res Extracted value.
+ *
+ * @retval -1 on error or 0.
+ */
int
sql_stat4_column(struct sql *db, const char *record, uint32_t col_num,
sql_value **res)
@@ -588,3 +583,5 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
sqlDbFree(aMem[0].db, pRec);
}
}
+
+#endif
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (7 preceding siblings ...)
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 08/52] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:59 ` Mergen Imeev via Tarantool-patches
2021-04-11 17:44 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 10/52] sql: introduce mem_create() Mergen Imeev via Tarantool-patches
` (32 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:59 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 01:58, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 62338e1db..0ed5e38d4 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -37,6 +37,39 @@
>> #include "box/tuple.h"
>> #include "mpstream/mpstream.h"
>>
>> +enum {
>> + BUF_SIZE = 32,
>> +};
>> +
>> +const char *
>> +mem_str(const struct Mem *mem)
>> +{
>> + char buf[BUF_SIZE];
>> + switch (mem->flags & MEM_PURE_TYPE_MASK) {
>> + case MEM_Null:
>> + return "NULL";
>> + case MEM_Str:
>> + return tt_sprintf("%.*s", mem->n, mem->z);
>> + case MEM_Int:
>> + return tt_sprintf("%lld", mem->u.i);
>> + case MEM_UInt:
>> + return tt_sprintf("%llu", mem->u.u);
>> + case MEM_Real:
>> + sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
>> + return tt_sprintf("%s", buf);
>> + case MEM_Blob:
>> + if ((mem->flags & MEM_Subtype) == 0)
>> + return "varbinary";
>> + assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
>> + return mp_str(mem->z);
>> + case MEM_Bool:
>> + return mem->u.b ? "TRUE" : "FALSE";
>
> 1. Why are they capital?
>
We already have macro SQL_TOKEN_BOOLEAN which returns capital version of these
two words. I didn't want to use this macro, however decided that in this
function these words also should be capital.
>> + default:
>> + break;
>> + }
>> + return "unknown";
>> +}
>> diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
>> index e84bb3192..eceaa953b 100644
>> --- a/src/box/sql/vdbetrace.c
>> +++ b/src/box/sql/vdbetrace.c
>> @@ -147,33 +145,10 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
>> zRawSql += nToken;
>> nextIndex = idx + 1;
>> assert(idx > 0 && idx <= p->nVar);
>> - pVar = &p->aVar[idx - 1];
>> - if (pVar->flags & MEM_Null) {
>> - sqlStrAccumAppend(&out, "NULL", 4);
>> - } else if (pVar->flags & MEM_Int) {
>> - sqlXPrintf(&out, "%lld", pVar->u.i);
>> - } else if (pVar->flags & MEM_UInt) {
>> - sqlXPrintf(&out, "%llu", pVar->u.u);
>> - } else if (pVar->flags & MEM_Real) {
>> - sqlXPrintf(&out, "%!.15g", pVar->u.r);
>> - } else if (pVar->flags & MEM_Str) {
>> - int nOut; /* Number of bytes of the string text to include in output */
>> - nOut = pVar->n;
>> - sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
>> - } else if (pVar->flags & MEM_Zero) {
>> - sqlXPrintf(&out, "zeroblob(%d)",
>> - pVar->u.nZero);
>> - } else {
>> - int nOut; /* Number of bytes of the blob to include in output */
>> - assert(pVar->flags & MEM_Blob);
>> - sqlStrAccumAppend(&out, "x'", 2);
>> - nOut = pVar->n;
>> - for (i = 0; i < nOut; i++) {
>> - sqlXPrintf(&out, "%02x",
>> - pVar->z[i] & 0xff);
>> - }
>> - sqlStrAccumAppend(&out, "'", 1);
>> - }
>> + const char *value = mem_str(&p->aVar[idx - 1]);
>> + uint32_t size = MIN(strlen(value), sizeof(zBase) - 1);
>
> 2. strlen() might be called twice because MIN is a macro.
>
Fixed.
>> + memcpy(zBase, value, size);
>> + zBase[size] = '\0';
>> }
>> }
>> if (out.accError)
New patch:
commit 2ce8b15b5c1869dec995b4a9f348fe7649f6af47
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 13:52:11 2021 +0300
sql: introduce mem_str()
This patch introduces mem_str() which allows to receive value of MEM as
a string. Due to the limitations of static_alloc(), this function cannot
be used to safely retrieve a value of MEM converted to string. This
function is suitable for debugging, displaying the value in an error
message, etc.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 46814f341..99ce938d5 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -543,7 +543,7 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -685,7 +685,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
UNUSED_PARAMETER(argc);
if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -1577,7 +1577,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "text");
+ mem_str(argv[0]), "text");
context->is_aborted = true;
return;
}
@@ -1650,7 +1650,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
if (mem_apply_numeric_type(argv[0]) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "number");
+ mem_str(argv[0]), "number");
context->is_aborted = true;
return;
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 25f5f2f2d..a2a0fc33e 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,39 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+enum {
+ BUF_SIZE = 32,
+};
+
+const char *
+mem_str(const struct Mem *mem)
+{
+ char buf[BUF_SIZE];
+ switch (mem->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Null:
+ return "NULL";
+ case MEM_Str:
+ return tt_sprintf("%.*s", mem->n, mem->z);
+ case MEM_Int:
+ return tt_sprintf("%lld", mem->u.i);
+ case MEM_UInt:
+ return tt_sprintf("%llu", mem->u.u);
+ case MEM_Real:
+ sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
+ return tt_sprintf("%s", buf);
+ case MEM_Blob:
+ if ((mem->flags & MEM_Subtype) == 0)
+ return "varbinary";
+ assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
+ return mp_str(mem->z);
+ case MEM_Bool:
+ return mem->u.b ? "TRUE" : "FALSE";
+ default:
+ break;
+ }
+ return "unknown";
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1592,18 +1625,6 @@ sqlValueText(sql_value * pVal)
return valueToText(pVal);
}
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
- enum mp_type mp_type = sql_value_type(value);
- if (mp_type_is_bloblike(mp_type)) {
- if (mem_has_msgpack_subtype(value))
- return sqlValueText(value);
- return "varbinary";
- }
- return sqlValueText(value);
-}
-
enum sql_subtype
sql_value_subtype(sql_value * pVal)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index acc8ce054..7b9456426 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,13 @@ struct Mem {
*/
#define MEMCELLSIZE offsetof(Mem,zMalloc)
+/**
+ * Return a string that represent content of MEM. String is either allocated
+ * using static_alloc() of just a static variable.
+ */
+const char *
+mem_str(const struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -358,15 +365,6 @@ sql_value_text(struct Mem *);
const void *sqlValueText(struct Mem *);
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(struct Mem *value);
#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
enum sql_subtype
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e269857ea..ec3d23cb2 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1265,12 +1265,12 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
} else {
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "numeric");
+ mem_str(pIn1), "numeric");
goto abort_due_to_error;
}
if (sqlVdbeRealValue(pIn2, &rB) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "numeric");
+ mem_str(pIn2), "numeric");
goto abort_due_to_error;
}
assert(((type1 | type2) & MEM_Real) != 0);
@@ -1546,12 +1546,12 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
bool unused;
if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "integer");
+ mem_str(pIn2), "integer");
goto abort_due_to_error;
}
if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
op = pOp->opcode;
@@ -1616,7 +1616,7 @@ case OP_MustBeInt: { /* jump, in1 */
if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
if (pOp->p2==0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
} else {
goto jump_to_p2;
@@ -1673,7 +1673,7 @@ case OP_Cast: { /* in1 */
UPDATE_MAX_BLOBSIZE(pIn1);
if (rc == 0)
break;
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sql_value_to_diag_str(pIn1),
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(pIn1),
field_type_strs[pOp->p2]);
goto abort_due_to_error;
}
@@ -1846,7 +1846,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
if (mem_apply_numeric_type(pIn3) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3),
+ mem_str(pIn3),
"numeric");
goto abort_due_to_error;
}
@@ -2112,7 +2112,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v1 = pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
pIn2 = &aMem[pOp->p2];
@@ -2122,7 +2122,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v2 = pIn2->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "boolean");
+ mem_str(pIn2), "boolean");
goto abort_due_to_error;
}
if (pOp->opcode==OP_And) {
@@ -2152,7 +2152,7 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
if ((pIn1->flags & MEM_Null)==0) {
if ((pIn1->flags & MEM_Bool) == 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
mem_set_bool(pOut, ! pIn1->u.b);
@@ -2177,7 +2177,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
bool is_neg;
if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
mem_set_i64(pOut, ~i);
@@ -2223,7 +2223,7 @@ case OP_IfNot: { /* jump, in1 */
c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
VdbeBranchTaken(c!=0, 2);
@@ -2403,7 +2403,7 @@ case OP_ApplyType: {
continue;
type_mismatch:
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), field_type_strs[type]);
+ mem_str(pIn1), field_type_strs[type]);
goto abort_due_to_error;
}
break;
@@ -3032,7 +3032,7 @@ case OP_SeekGT: { /* jump, in3 */
is_neg = i < 0;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3), "integer");
+ mem_str(pIn3), "integer");
goto abort_due_to_error;
}
iKey = i;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 79c3d60e0..b7e148422 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
break;
}
case P4_MEM:{
- Mem *pMem = pOp->p4.pMem;
- if (pMem->flags & MEM_Str) {
- zP4 = pMem->z;
- } else if (pMem->flags & MEM_Int) {
- sqlXPrintf(&x, "%lld", pMem->u.i);
- } else if (pMem->flags & MEM_UInt) {
- sqlXPrintf(&x, "%llu", pMem->u.u);
- } else if (pMem->flags & MEM_Real) {
- sqlXPrintf(&x, "%.16g", pMem->u.r);
- } else if (pMem->flags & MEM_Null) {
- zP4 = "NULL";
- } else {
- assert(pMem->flags & MEM_Blob);
- zP4 = "(binary string)";
- }
+ const char *value = mem_str(pOp->p4.pMem);
+ uint32_t size = MIN((int)strlen(value), nTemp - 1);
+ memcpy(zP4, value, size);
+ zP4[size] = '\0';
break;
}
case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index e84bb3192..4ca56865d 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -97,8 +97,6 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
int nextIndex = 1; /* Index of next ? host parameter */
int n; /* Length of a token prefix */
int nToken; /* Length of the parameter token */
- int i; /* Loop counter */
- Mem *pVar; /* Value of a host parameter */
StrAccum out; /* Accumulate the output here */
char zBase[100]; /* Initial working space */
@@ -147,33 +145,11 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
zRawSql += nToken;
nextIndex = idx + 1;
assert(idx > 0 && idx <= p->nVar);
- pVar = &p->aVar[idx - 1];
- if (pVar->flags & MEM_Null) {
- sqlStrAccumAppend(&out, "NULL", 4);
- } else if (pVar->flags & MEM_Int) {
- sqlXPrintf(&out, "%lld", pVar->u.i);
- } else if (pVar->flags & MEM_UInt) {
- sqlXPrintf(&out, "%llu", pVar->u.u);
- } else if (pVar->flags & MEM_Real) {
- sqlXPrintf(&out, "%!.15g", pVar->u.r);
- } else if (pVar->flags & MEM_Str) {
- int nOut; /* Number of bytes of the string text to include in output */
- nOut = pVar->n;
- sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
- } else if (pVar->flags & MEM_Zero) {
- sqlXPrintf(&out, "zeroblob(%d)",
- pVar->u.nZero);
- } else {
- int nOut; /* Number of bytes of the blob to include in output */
- assert(pVar->flags & MEM_Blob);
- sqlStrAccumAppend(&out, "x'", 2);
- nOut = pVar->n;
- for (i = 0; i < nOut; i++) {
- sqlXPrintf(&out, "%02x",
- pVar->z[i] & 0xff);
- }
- sqlStrAccumAppend(&out, "'", 1);
- }
+ const char *value = mem_str(&p->aVar[idx - 1]);
+ uint32_t len = strlen(value);
+ uint32_t size = MIN(len, sizeof(zBase) - 1);
+ memcpy(zBase, value, size);
+ zBase[size] = '\0';
}
}
if (out.accError)
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str()
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
@ 2021-04-11 17:44 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 12:36 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 17:44 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
I appreciate the work you did here!
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 79c3d60e0..b7e148422 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
> break;
> }
> case P4_MEM:{
> - Mem *pMem = pOp->p4.pMem;
> - if (pMem->flags & MEM_Str) {
> - zP4 = pMem->z;
> - } else if (pMem->flags & MEM_Int) {
> - sqlXPrintf(&x, "%lld", pMem->u.i);
> - } else if (pMem->flags & MEM_UInt) {
> - sqlXPrintf(&x, "%llu", pMem->u.u);
> - } else if (pMem->flags & MEM_Real) {
> - sqlXPrintf(&x, "%.16g", pMem->u.r);
> - } else if (pMem->flags & MEM_Null) {
> - zP4 = "NULL";
> - } else {
> - assert(pMem->flags & MEM_Blob);
> - zP4 = "(binary string)";
> - }
> + const char *value = mem_str(pOp->p4.pMem);
> + uint32_t size = MIN((int)strlen(value), nTemp - 1);
> + memcpy(zP4, value, size);
> + zP4[size] = '\0';
'x' stays in an invalid state now, because its counter nChar is
not updated. I would propose to use sqlStrAccumAppend/sqlStrAccumAppendAll
instead of memcpy and manual 0 termination.
The same below in sqlVdbeExpandSql().
> break;
> }
> case P4_INTARRAY:{
> diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
> index e84bb3192..4ca56865d 100644
> --- a/src/box/sql/vdbetrace.c
> +++ b/src/box/sql/vdbetrace.c
> @@ -147,33 +145,11 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
> zRawSql += nToken;
> nextIndex = idx + 1;
> assert(idx > 0 && idx <= p->nVar);
> - pVar = &p->aVar[idx - 1];
> - if (pVar->flags & MEM_Null) {
> - sqlStrAccumAppend(&out, "NULL", 4);
> - } else if (pVar->flags & MEM_Int) {
> - sqlXPrintf(&out, "%lld", pVar->u.i);
> - } else if (pVar->flags & MEM_UInt) {
> - sqlXPrintf(&out, "%llu", pVar->u.u);
> - } else if (pVar->flags & MEM_Real) {
> - sqlXPrintf(&out, "%!.15g", pVar->u.r);
> - } else if (pVar->flags & MEM_Str) {
> - int nOut; /* Number of bytes of the string text to include in output */
> - nOut = pVar->n;
> - sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
> - } else if (pVar->flags & MEM_Zero) {
> - sqlXPrintf(&out, "zeroblob(%d)",
> - pVar->u.nZero);
> - } else {
> - int nOut; /* Number of bytes of the blob to include in output */
> - assert(pVar->flags & MEM_Blob);
> - sqlStrAccumAppend(&out, "x'", 2);
> - nOut = pVar->n;
> - for (i = 0; i < nOut; i++) {
> - sqlXPrintf(&out, "%02x",
> - pVar->z[i] & 0xff);
> - }
> - sqlStrAccumAppend(&out, "'", 1);
> - }
> + const char *value = mem_str(&p->aVar[idx - 1]);
> + uint32_t len = strlen(value);
> + uint32_t size = MIN(len, sizeof(zBase) - 1);
> + memcpy(zBase, value, size);
> + zBase[size] = '\0';
> }
> }
> if (out.accError)
>
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str()
2021-04-11 17:44 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 12:36 ` Mergen Imeev via Tarantool-patches
2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 12:36 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Sun, Apr 11, 2021 at 07:44:53PM +0200, Vladislav Shpilevoy wrote:
> I appreciate the work you did here!
>
> > diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> > index 79c3d60e0..b7e148422 100644
> > --- a/src/box/sql/vdbeaux.c
> > +++ b/src/box/sql/vdbeaux.c
> > @@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
> > break;
> > }
> > case P4_MEM:{
> > - Mem *pMem = pOp->p4.pMem;
> > - if (pMem->flags & MEM_Str) {
> > - zP4 = pMem->z;
> > - } else if (pMem->flags & MEM_Int) {
> > - sqlXPrintf(&x, "%lld", pMem->u.i);
> > - } else if (pMem->flags & MEM_UInt) {
> > - sqlXPrintf(&x, "%llu", pMem->u.u);
> > - } else if (pMem->flags & MEM_Real) {
> > - sqlXPrintf(&x, "%.16g", pMem->u.r);
> > - } else if (pMem->flags & MEM_Null) {
> > - zP4 = "NULL";
> > - } else {
> > - assert(pMem->flags & MEM_Blob);
> > - zP4 = "(binary string)";
> > - }
> > + const char *value = mem_str(pOp->p4.pMem);
> > + uint32_t size = MIN((int)strlen(value), nTemp - 1);
> > + memcpy(zP4, value, size);
> > + zP4[size] = '\0';
>
> 'x' stays in an invalid state now, because its counter nChar is
> not updated. I would propose to use sqlStrAccumAppend/sqlStrAccumAppendAll
> instead of memcpy and manual 0 termination.
>
> The same below in sqlVdbeExpandSql().
>
Fixed.
> > break;
> > }
> > case P4_INTARRAY:{
> > diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
> > index e84bb3192..4ca56865d 100644
> > --- a/src/box/sql/vdbetrace.c
> > +++ b/src/box/sql/vdbetrace.c
> > @@ -147,33 +145,11 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
> > zRawSql += nToken;
> > nextIndex = idx + 1;
> > assert(idx > 0 && idx <= p->nVar);
> > - pVar = &p->aVar[idx - 1];
> > - if (pVar->flags & MEM_Null) {
> > - sqlStrAccumAppend(&out, "NULL", 4);
> > - } else if (pVar->flags & MEM_Int) {
> > - sqlXPrintf(&out, "%lld", pVar->u.i);
> > - } else if (pVar->flags & MEM_UInt) {
> > - sqlXPrintf(&out, "%llu", pVar->u.u);
> > - } else if (pVar->flags & MEM_Real) {
> > - sqlXPrintf(&out, "%!.15g", pVar->u.r);
> > - } else if (pVar->flags & MEM_Str) {
> > - int nOut; /* Number of bytes of the string text to include in output */
> > - nOut = pVar->n;
> > - sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
> > - } else if (pVar->flags & MEM_Zero) {
> > - sqlXPrintf(&out, "zeroblob(%d)",
> > - pVar->u.nZero);
> > - } else {
> > - int nOut; /* Number of bytes of the blob to include in output */
> > - assert(pVar->flags & MEM_Blob);
> > - sqlStrAccumAppend(&out, "x'", 2);
> > - nOut = pVar->n;
> > - for (i = 0; i < nOut; i++) {
> > - sqlXPrintf(&out, "%02x",
> > - pVar->z[i] & 0xff);
> > - }
> > - sqlStrAccumAppend(&out, "'", 1);
> > - }
> > + const char *value = mem_str(&p->aVar[idx - 1]);
> > + uint32_t len = strlen(value);
> > + uint32_t size = MIN(len, sizeof(zBase) - 1);
> > + memcpy(zBase, value, size);
> > + zBase[size] = '\0';
> > }
> > }
> > if (out.accError)
> >
Diff:
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index b7e148422..907c9f5c6 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1109,9 +1109,9 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
}
case P4_MEM:{
const char *value = mem_str(pOp->p4.pMem);
- uint32_t size = MIN((int)strlen(value), nTemp - 1);
- memcpy(zP4, value, size);
- zP4[size] = '\0';
+ int len = strlen(value);
+ uint32_t size = MIN(len, nTemp - 1);
+ sqlStrAccumAppend(&x, value, size);
break;
}
case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index 4ca56865d..677de65e2 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -148,8 +148,7 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
const char *value = mem_str(&p->aVar[idx - 1]);
uint32_t len = strlen(value);
uint32_t size = MIN(len, sizeof(zBase) - 1);
- memcpy(zBase, value, size);
- zBase[size] = '\0';
+ sqlStrAccumAppend(&out, value, size);
}
}
if (out.accError)
New patch
commit 3ef9df0c7c1a9947a63fd024b966690cc065d2b0
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 13:52:11 2021 +0300
sql: introduce mem_str()
This patch introduces mem_str() which allows to receive value of MEM as
a string. Due to the limitations of static_alloc(), this function cannot
be used to safely retrieve a value of MEM converted to string. This
function is suitable for debugging, displaying the value in an error
message, etc.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 46814f341..99ce938d5 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -543,7 +543,7 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -685,7 +685,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
UNUSED_PARAMETER(argc);
if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -1577,7 +1577,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "text");
+ mem_str(argv[0]), "text");
context->is_aborted = true;
return;
}
@@ -1650,7 +1650,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
if (mem_apply_numeric_type(argv[0]) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "number");
+ mem_str(argv[0]), "number");
context->is_aborted = true;
return;
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 25f5f2f2d..a2a0fc33e 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,39 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+enum {
+ BUF_SIZE = 32,
+};
+
+const char *
+mem_str(const struct Mem *mem)
+{
+ char buf[BUF_SIZE];
+ switch (mem->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Null:
+ return "NULL";
+ case MEM_Str:
+ return tt_sprintf("%.*s", mem->n, mem->z);
+ case MEM_Int:
+ return tt_sprintf("%lld", mem->u.i);
+ case MEM_UInt:
+ return tt_sprintf("%llu", mem->u.u);
+ case MEM_Real:
+ sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
+ return tt_sprintf("%s", buf);
+ case MEM_Blob:
+ if ((mem->flags & MEM_Subtype) == 0)
+ return "varbinary";
+ assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
+ return mp_str(mem->z);
+ case MEM_Bool:
+ return mem->u.b ? "TRUE" : "FALSE";
+ default:
+ break;
+ }
+ return "unknown";
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1592,18 +1625,6 @@ sqlValueText(sql_value * pVal)
return valueToText(pVal);
}
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
- enum mp_type mp_type = sql_value_type(value);
- if (mp_type_is_bloblike(mp_type)) {
- if (mem_has_msgpack_subtype(value))
- return sqlValueText(value);
- return "varbinary";
- }
- return sqlValueText(value);
-}
-
enum sql_subtype
sql_value_subtype(sql_value * pVal)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index acc8ce054..7b9456426 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,13 @@ struct Mem {
*/
#define MEMCELLSIZE offsetof(Mem,zMalloc)
+/**
+ * Return a string that represent content of MEM. String is either allocated
+ * using static_alloc() of just a static variable.
+ */
+const char *
+mem_str(const struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -358,15 +365,6 @@ sql_value_text(struct Mem *);
const void *sqlValueText(struct Mem *);
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(struct Mem *value);
#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
enum sql_subtype
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e269857ea..ec3d23cb2 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1265,12 +1265,12 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
} else {
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "numeric");
+ mem_str(pIn1), "numeric");
goto abort_due_to_error;
}
if (sqlVdbeRealValue(pIn2, &rB) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "numeric");
+ mem_str(pIn2), "numeric");
goto abort_due_to_error;
}
assert(((type1 | type2) & MEM_Real) != 0);
@@ -1546,12 +1546,12 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
bool unused;
if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "integer");
+ mem_str(pIn2), "integer");
goto abort_due_to_error;
}
if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
op = pOp->opcode;
@@ -1616,7 +1616,7 @@ case OP_MustBeInt: { /* jump, in1 */
if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
if (pOp->p2==0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
} else {
goto jump_to_p2;
@@ -1673,7 +1673,7 @@ case OP_Cast: { /* in1 */
UPDATE_MAX_BLOBSIZE(pIn1);
if (rc == 0)
break;
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sql_value_to_diag_str(pIn1),
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(pIn1),
field_type_strs[pOp->p2]);
goto abort_due_to_error;
}
@@ -1846,7 +1846,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
if (mem_apply_numeric_type(pIn3) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3),
+ mem_str(pIn3),
"numeric");
goto abort_due_to_error;
}
@@ -2112,7 +2112,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v1 = pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
pIn2 = &aMem[pOp->p2];
@@ -2122,7 +2122,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v2 = pIn2->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "boolean");
+ mem_str(pIn2), "boolean");
goto abort_due_to_error;
}
if (pOp->opcode==OP_And) {
@@ -2152,7 +2152,7 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
if ((pIn1->flags & MEM_Null)==0) {
if ((pIn1->flags & MEM_Bool) == 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
mem_set_bool(pOut, ! pIn1->u.b);
@@ -2177,7 +2177,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
bool is_neg;
if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
mem_set_i64(pOut, ~i);
@@ -2223,7 +2223,7 @@ case OP_IfNot: { /* jump, in1 */
c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
VdbeBranchTaken(c!=0, 2);
@@ -2403,7 +2403,7 @@ case OP_ApplyType: {
continue;
type_mismatch:
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), field_type_strs[type]);
+ mem_str(pIn1), field_type_strs[type]);
goto abort_due_to_error;
}
break;
@@ -3032,7 +3032,7 @@ case OP_SeekGT: { /* jump, in3 */
is_neg = i < 0;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3), "integer");
+ mem_str(pIn3), "integer");
goto abort_due_to_error;
}
iKey = i;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 79c3d60e0..907c9f5c6 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
break;
}
case P4_MEM:{
- Mem *pMem = pOp->p4.pMem;
- if (pMem->flags & MEM_Str) {
- zP4 = pMem->z;
- } else if (pMem->flags & MEM_Int) {
- sqlXPrintf(&x, "%lld", pMem->u.i);
- } else if (pMem->flags & MEM_UInt) {
- sqlXPrintf(&x, "%llu", pMem->u.u);
- } else if (pMem->flags & MEM_Real) {
- sqlXPrintf(&x, "%.16g", pMem->u.r);
- } else if (pMem->flags & MEM_Null) {
- zP4 = "NULL";
- } else {
- assert(pMem->flags & MEM_Blob);
- zP4 = "(binary string)";
- }
+ const char *value = mem_str(pOp->p4.pMem);
+ int len = strlen(value);
+ uint32_t size = MIN(len, nTemp - 1);
+ sqlStrAccumAppend(&x, value, size);
break;
}
case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index e84bb3192..677de65e2 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -97,8 +97,6 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
int nextIndex = 1; /* Index of next ? host parameter */
int n; /* Length of a token prefix */
int nToken; /* Length of the parameter token */
- int i; /* Loop counter */
- Mem *pVar; /* Value of a host parameter */
StrAccum out; /* Accumulate the output here */
char zBase[100]; /* Initial working space */
@@ -147,33 +145,10 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
zRawSql += nToken;
nextIndex = idx + 1;
assert(idx > 0 && idx <= p->nVar);
- pVar = &p->aVar[idx - 1];
- if (pVar->flags & MEM_Null) {
- sqlStrAccumAppend(&out, "NULL", 4);
- } else if (pVar->flags & MEM_Int) {
- sqlXPrintf(&out, "%lld", pVar->u.i);
- } else if (pVar->flags & MEM_UInt) {
- sqlXPrintf(&out, "%llu", pVar->u.u);
- } else if (pVar->flags & MEM_Real) {
- sqlXPrintf(&out, "%!.15g", pVar->u.r);
- } else if (pVar->flags & MEM_Str) {
- int nOut; /* Number of bytes of the string text to include in output */
- nOut = pVar->n;
- sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
- } else if (pVar->flags & MEM_Zero) {
- sqlXPrintf(&out, "zeroblob(%d)",
- pVar->u.nZero);
- } else {
- int nOut; /* Number of bytes of the blob to include in output */
- assert(pVar->flags & MEM_Blob);
- sqlStrAccumAppend(&out, "x'", 2);
- nOut = pVar->n;
- for (i = 0; i < nOut; i++) {
- sqlXPrintf(&out, "%02x",
- pVar->z[i] & 0xff);
- }
- sqlStrAccumAppend(&out, "'", 1);
- }
+ const char *value = mem_str(&p->aVar[idx - 1]);
+ uint32_t len = strlen(value);
+ uint32_t size = MIN(len, sizeof(zBase) - 1);
+ sqlStrAccumAppend(&out, value, size);
}
}
if (out.accError)
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str()
2021-04-13 12:36 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 22:42 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 22:23 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Good job on the fixes!
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 79c3d60e0..907c9f5c6 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
> break;
> }
> case P4_MEM:{
> - Mem *pMem = pOp->p4.pMem;
> - if (pMem->flags & MEM_Str) {
> - zP4 = pMem->z;
> - } else if (pMem->flags & MEM_Int) {
> - sqlXPrintf(&x, "%lld", pMem->u.i);
> - } else if (pMem->flags & MEM_UInt) {
> - sqlXPrintf(&x, "%llu", pMem->u.u);
> - } else if (pMem->flags & MEM_Real) {
> - sqlXPrintf(&x, "%.16g", pMem->u.r);
> - } else if (pMem->flags & MEM_Null) {
> - zP4 = "NULL";
> - } else {
> - assert(pMem->flags & MEM_Blob);
> - zP4 = "(binary string)";
> - }
> + const char *value = mem_str(pOp->p4.pMem);
> + int len = strlen(value);
> + uint32_t size = MIN(len, nTemp - 1);
Why do you need that limit? The accum is created as
sqlStrAccumInit(&x, 0, zTemp, nTemp, 0). The last argument 0 means
it won't realloc anything, so it is safe to append. The same in the
next diff hunk.
Consider this diff:
====================
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index a2a0fc33e..95a035fd4 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -52,7 +52,9 @@ mem_str(const struct Mem *mem)
case MEM_Null:
return "NULL";
case MEM_Str:
- return tt_sprintf("%.*s", mem->n, mem->z);
+ if ((mem->flags & MEM_Term) != 0)
+ return mem->z;
+ return tt_cstr(mem->z, mem->n);
case MEM_Int:
return tt_sprintf("%lld", mem->u.i);
case MEM_UInt:
@@ -68,9 +70,8 @@ mem_str(const struct Mem *mem)
case MEM_Bool:
return mem->u.b ? "TRUE" : "FALSE";
default:
- break;
+ return "unknown";
}
- return "unknown";
}
static inline bool
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 907c9f5c6..9ef0445bd 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1109,9 +1109,7 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
}
case P4_MEM:{
const char *value = mem_str(pOp->p4.pMem);
- int len = strlen(value);
- uint32_t size = MIN(len, nTemp - 1);
- sqlStrAccumAppend(&x, value, size);
+ sqlStrAccumAppend(&x, value, strlen(value));
break;
}
case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index 677de65e2..a26880041 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -146,9 +146,7 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
nextIndex = idx + 1;
assert(idx > 0 && idx <= p->nVar);
const char *value = mem_str(&p->aVar[idx - 1]);
- uint32_t len = strlen(value);
- uint32_t size = MIN(len, sizeof(zBase) - 1);
- sqlStrAccumAppend(&out, value, size);
+ sqlStrAccumAppend(&out, value, strlen(value));
}
}
if (out.accError)
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str()
2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 22:42 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 22:42 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review and diff! My answer below. Also, I forgot to mention
in the last letter that I pushed rebased and fixed patches onto old branch,
imeevma/gh-5818-encapsulate-mem-type-checking-and-changing
New patch below.
On Thu, Apr 15, 2021 at 12:23:36AM +0200, Vladislav Shpilevoy wrote:
> Good job on the fixes!
>
> > diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> > index 79c3d60e0..907c9f5c6 100644
> > --- a/src/box/sql/vdbeaux.c
> > +++ b/src/box/sql/vdbeaux.c
> > @@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
> > break;
> > }
> > case P4_MEM:{
> > - Mem *pMem = pOp->p4.pMem;
> > - if (pMem->flags & MEM_Str) {
> > - zP4 = pMem->z;
> > - } else if (pMem->flags & MEM_Int) {
> > - sqlXPrintf(&x, "%lld", pMem->u.i);
> > - } else if (pMem->flags & MEM_UInt) {
> > - sqlXPrintf(&x, "%llu", pMem->u.u);
> > - } else if (pMem->flags & MEM_Real) {
> > - sqlXPrintf(&x, "%.16g", pMem->u.r);
> > - } else if (pMem->flags & MEM_Null) {
> > - zP4 = "NULL";
> > - } else {
> > - assert(pMem->flags & MEM_Blob);
> > - zP4 = "(binary string)";
> > - }
> > + const char *value = mem_str(pOp->p4.pMem);
> > + int len = strlen(value);
> > + uint32_t size = MIN(len, nTemp - 1);
>
> Why do you need that limit? The accum is created as
> sqlStrAccumInit(&x, 0, zTemp, nTemp, 0). The last argument 0 means
> it won't realloc anything, so it is safe to append. The same in the
> next diff hunk.
>
> Consider this diff:
>
> ====================
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index a2a0fc33e..95a035fd4 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -52,7 +52,9 @@ mem_str(const struct Mem *mem)
> case MEM_Null:
> return "NULL";
> case MEM_Str:
> - return tt_sprintf("%.*s", mem->n, mem->z);
> + if ((mem->flags & MEM_Term) != 0)
> + return mem->z;
> + return tt_cstr(mem->z, mem->n);
> case MEM_Int:
> return tt_sprintf("%lld", mem->u.i);
> case MEM_UInt:
> @@ -68,9 +70,8 @@ mem_str(const struct Mem *mem)
> case MEM_Bool:
> return mem->u.b ? "TRUE" : "FALSE";
> default:
> - break;
> + return "unknown";
> }
> - return "unknown";
> }
>
> static inline bool
> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 907c9f5c6..9ef0445bd 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1109,9 +1109,7 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
> }
> case P4_MEM:{
> const char *value = mem_str(pOp->p4.pMem);
> - int len = strlen(value);
> - uint32_t size = MIN(len, nTemp - 1);
> - sqlStrAccumAppend(&x, value, size);
> + sqlStrAccumAppend(&x, value, strlen(value));
> break;
> }
> case P4_INTARRAY:{
> diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
> index 677de65e2..a26880041 100644
> --- a/src/box/sql/vdbetrace.c
> +++ b/src/box/sql/vdbetrace.c
> @@ -146,9 +146,7 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
> nextIndex = idx + 1;
> assert(idx > 0 && idx <= p->nVar);
> const char *value = mem_str(&p->aVar[idx - 1]);
> - uint32_t len = strlen(value);
> - uint32_t size = MIN(len, sizeof(zBase) - 1);
> - sqlStrAccumAppend(&out, value, size);
> + sqlStrAccumAppend(&out, value, strlen(value));
> }
> }
> if (out.accError)
Thank you again! I applied this patch and checked that tests are fine.
New patch:
commit cd39e0296477effaa054a139604ee52a382dd7be
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 13:52:11 2021 +0300
sql: introduce mem_str()
This patch introduces mem_str() which allows to receive value of MEM as
a string. Due to the limitations of static_alloc(), this function cannot
be used to safely retrieve a value of MEM converted to string. This
function is suitable for debugging, displaying the value in an error
message, etc.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 46814f341..99ce938d5 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -543,7 +543,7 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -685,7 +685,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
UNUSED_PARAMETER(argc);
if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "numeric");
+ mem_str(argv[0]), "numeric");
context->is_aborted = true;
return;
}
@@ -1577,7 +1577,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
enum mp_type mp_type = sql_value_type(argv[0]);
if (mp_type_is_bloblike(mp_type)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "text");
+ mem_str(argv[0]), "text");
context->is_aborted = true;
return;
}
@@ -1650,7 +1650,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
if (mem_apply_numeric_type(argv[0]) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(argv[0]), "number");
+ mem_str(argv[0]), "number");
context->is_aborted = true;
return;
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 25f5f2f2d..95a035fd4 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,40 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+enum {
+ BUF_SIZE = 32,
+};
+
+const char *
+mem_str(const struct Mem *mem)
+{
+ char buf[BUF_SIZE];
+ switch (mem->flags & MEM_PURE_TYPE_MASK) {
+ case MEM_Null:
+ return "NULL";
+ case MEM_Str:
+ if ((mem->flags & MEM_Term) != 0)
+ return mem->z;
+ return tt_cstr(mem->z, mem->n);
+ case MEM_Int:
+ return tt_sprintf("%lld", mem->u.i);
+ case MEM_UInt:
+ return tt_sprintf("%llu", mem->u.u);
+ case MEM_Real:
+ sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
+ return tt_sprintf("%s", buf);
+ case MEM_Blob:
+ if ((mem->flags & MEM_Subtype) == 0)
+ return "varbinary";
+ assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
+ return mp_str(mem->z);
+ case MEM_Bool:
+ return mem->u.b ? "TRUE" : "FALSE";
+ default:
+ return "unknown";
+ }
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1592,18 +1626,6 @@ sqlValueText(sql_value * pVal)
return valueToText(pVal);
}
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
- enum mp_type mp_type = sql_value_type(value);
- if (mp_type_is_bloblike(mp_type)) {
- if (mem_has_msgpack_subtype(value))
- return sqlValueText(value);
- return "varbinary";
- }
- return sqlValueText(value);
-}
-
enum sql_subtype
sql_value_subtype(sql_value * pVal)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index acc8ce054..7b9456426 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,13 @@ struct Mem {
*/
#define MEMCELLSIZE offsetof(Mem,zMalloc)
+/**
+ * Return a string that represent content of MEM. String is either allocated
+ * using static_alloc() of just a static variable.
+ */
+const char *
+mem_str(const struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -358,15 +365,6 @@ sql_value_text(struct Mem *);
const void *sqlValueText(struct Mem *);
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(struct Mem *value);
#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
enum sql_subtype
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index d03725106..cfd694c2a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1265,12 +1265,12 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
} else {
if (sqlVdbeRealValue(pIn1, &rA) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "numeric");
+ mem_str(pIn1), "numeric");
goto abort_due_to_error;
}
if (sqlVdbeRealValue(pIn2, &rB) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "numeric");
+ mem_str(pIn2), "numeric");
goto abort_due_to_error;
}
assert(((type1 | type2) & MEM_Real) != 0);
@@ -1546,12 +1546,12 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
bool unused;
if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "integer");
+ mem_str(pIn2), "integer");
goto abort_due_to_error;
}
if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
op = pOp->opcode;
@@ -1616,7 +1616,7 @@ case OP_MustBeInt: { /* jump, in1 */
if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
if (pOp->p2==0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
} else {
goto jump_to_p2;
@@ -1673,7 +1673,7 @@ case OP_Cast: { /* in1 */
UPDATE_MAX_BLOBSIZE(pIn1);
if (rc == 0)
break;
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sql_value_to_diag_str(pIn1),
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(pIn1),
field_type_strs[pOp->p2]);
goto abort_due_to_error;
}
@@ -1846,7 +1846,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
if (mem_apply_numeric_type(pIn3) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3),
+ mem_str(pIn3),
"numeric");
goto abort_due_to_error;
}
@@ -2112,7 +2112,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v1 = pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
pIn2 = &aMem[pOp->p2];
@@ -2122,7 +2122,7 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
v2 = pIn2->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn2), "boolean");
+ mem_str(pIn2), "boolean");
goto abort_due_to_error;
}
if (pOp->opcode==OP_And) {
@@ -2152,7 +2152,7 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
if ((pIn1->flags & MEM_Null)==0) {
if ((pIn1->flags & MEM_Bool) == 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
mem_set_bool(pOut, ! pIn1->u.b);
@@ -2177,7 +2177,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
bool is_neg;
if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "integer");
+ mem_str(pIn1), "integer");
goto abort_due_to_error;
}
mem_set_i64(pOut, ~i);
@@ -2223,7 +2223,7 @@ case OP_IfNot: { /* jump, in1 */
c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), "boolean");
+ mem_str(pIn1), "boolean");
goto abort_due_to_error;
}
VdbeBranchTaken(c!=0, 2);
@@ -2403,7 +2403,7 @@ case OP_ApplyType: {
continue;
type_mismatch:
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn1), field_type_strs[type]);
+ mem_str(pIn1), field_type_strs[type]);
goto abort_due_to_error;
}
break;
@@ -3032,7 +3032,7 @@ case OP_SeekGT: { /* jump, in3 */
is_neg = i < 0;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- sql_value_to_diag_str(pIn3), "integer");
+ mem_str(pIn3), "integer");
goto abort_due_to_error;
}
iKey = i;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 79c3d60e0..9ef0445bd 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1108,21 +1108,8 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
break;
}
case P4_MEM:{
- Mem *pMem = pOp->p4.pMem;
- if (pMem->flags & MEM_Str) {
- zP4 = pMem->z;
- } else if (pMem->flags & MEM_Int) {
- sqlXPrintf(&x, "%lld", pMem->u.i);
- } else if (pMem->flags & MEM_UInt) {
- sqlXPrintf(&x, "%llu", pMem->u.u);
- } else if (pMem->flags & MEM_Real) {
- sqlXPrintf(&x, "%.16g", pMem->u.r);
- } else if (pMem->flags & MEM_Null) {
- zP4 = "NULL";
- } else {
- assert(pMem->flags & MEM_Blob);
- zP4 = "(binary string)";
- }
+ const char *value = mem_str(pOp->p4.pMem);
+ sqlStrAccumAppend(&x, value, strlen(value));
break;
}
case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index e84bb3192..a26880041 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -97,8 +97,6 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
int nextIndex = 1; /* Index of next ? host parameter */
int n; /* Length of a token prefix */
int nToken; /* Length of the parameter token */
- int i; /* Loop counter */
- Mem *pVar; /* Value of a host parameter */
StrAccum out; /* Accumulate the output here */
char zBase[100]; /* Initial working space */
@@ -147,33 +145,8 @@ sqlVdbeExpandSql(Vdbe * p, /* The prepared statement being evaluated */
zRawSql += nToken;
nextIndex = idx + 1;
assert(idx > 0 && idx <= p->nVar);
- pVar = &p->aVar[idx - 1];
- if (pVar->flags & MEM_Null) {
- sqlStrAccumAppend(&out, "NULL", 4);
- } else if (pVar->flags & MEM_Int) {
- sqlXPrintf(&out, "%lld", pVar->u.i);
- } else if (pVar->flags & MEM_UInt) {
- sqlXPrintf(&out, "%llu", pVar->u.u);
- } else if (pVar->flags & MEM_Real) {
- sqlXPrintf(&out, "%!.15g", pVar->u.r);
- } else if (pVar->flags & MEM_Str) {
- int nOut; /* Number of bytes of the string text to include in output */
- nOut = pVar->n;
- sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
- } else if (pVar->flags & MEM_Zero) {
- sqlXPrintf(&out, "zeroblob(%d)",
- pVar->u.nZero);
- } else {
- int nOut; /* Number of bytes of the blob to include in output */
- assert(pVar->flags & MEM_Blob);
- sqlStrAccumAppend(&out, "x'", 2);
- nOut = pVar->n;
- for (i = 0; i < nOut; i++) {
- sqlXPrintf(&out, "%02x",
- pVar->z[i] & 0xff);
- }
- sqlStrAccumAppend(&out, "'", 1);
- }
+ const char *value = mem_str(&p->aVar[idx - 1]);
+ sqlStrAccumAppend(&out, value, strlen(value));
}
}
if (out.accError)
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 10/52] sql: introduce mem_create()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (8 preceding siblings ...)
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
@ 2021-04-09 16:59 ` Mergen Imeev via Tarantool-patches
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
` (31 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 16:59 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_create(). This function should be used to
initialize a MEM. MEM should be initialized before usage.
Part of #5818
---
src/box/sql/mem.c | 51 +++++++++++++++++--------------------------
src/box/sql/mem.h | 12 ++++------
src/box/sql/vdbe.c | 2 +-
src/box/sql/vdbeaux.c | 8 +++++--
4 files changed, 31 insertions(+), 42 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index a2a0fc33e..5135637d9 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -73,6 +73,25 @@ mem_str(const struct Mem *mem)
return "unknown";
}
+void
+mem_create(struct Mem *mem)
+{
+ mem->flags = MEM_Null;
+ mem->subtype = SQL_SUBTYPE_NO;
+ mem->field_type = field_type_MAX;
+ mem->n = 0;
+ mem->z = NULL;
+ mem->zMalloc = NULL;
+ mem->szMalloc = 0;
+ mem->uTemp = 0;
+ mem->db = sql_get();
+ mem->xDel = NULL;
+#ifdef SQL_DEBUG
+ mem->pScopyFrom = NULL;
+ mem->pFiller = NULL;
+#endif
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1299,21 +1318,6 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
return 0;
}
-/*
- * Initialize bulk memory to be a consistent Mem object.
- *
- * The minimum amount of initialization feasible is performed.
- */
-void
-sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
-{
- assert((flags & ~MEM_TypeMask) == 0);
- pMem->flags = flags;
- pMem->db = db;
- pMem->szMalloc = 0;
- pMem->field_type = field_type_MAX;
-}
-
/*
* Delete any previous value and set the value stored in *pMem to NULL.
*
@@ -1398,21 +1402,6 @@ sqlValueNew(sql * db)
return p;
}
-void
-initMemArray(Mem * p, int N, sql * db, u32 flags)
-{
- while ((N--) > 0) {
- p->db = db;
- p->flags = flags;
- p->szMalloc = 0;
- p->field_type = field_type_MAX;
-#ifdef SQL_DEBUG
- p->pScopyFrom = 0;
-#endif
- p++;
- }
-}
-
void
releaseMemArray(Mem * p, int N)
{
@@ -2373,7 +2362,7 @@ vdbemem_alloc_on_region(uint32_t count)
}
memset(ret, 0, count * sizeof(*ret));
for (uint32_t i = 0; i < count; i++) {
- sqlVdbeMemInit(&ret[i], sql_get(), MEM_Null);
+ mem_create(&ret[i]);
assert(memIsValid(&ret[i]));
}
return ret;
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 7b9456426..2a3d1078e 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -94,6 +94,10 @@ struct Mem {
const char *
mem_str(const struct Mem *mem);
+/** Initialize MEM and set NULL. */
+void
+mem_create(struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -304,8 +308,6 @@ mem_set_double(struct Mem *mem, double value);
int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void
-sqlVdbeMemInit(struct Mem *, sql *, u32);
-void
sqlVdbeMemSetNull(struct Mem *);
void
sqlVdbeMemSetZeroBlob(struct Mem *, int);
@@ -315,12 +317,6 @@ void sqlValueSetNull(struct Mem *);
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
-/*
- * Initialize an array of N Mem element.
- */
-void
-initMemArray(Mem * p, int N, sql * db, u32 flags);
-
/*
* Release an array of N Mem elements
*/
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index ec3d23cb2..74bf7f903 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4739,7 +4739,7 @@ case OP_AggStep: {
#endif
pMem->n++;
- sqlVdbeMemInit(&t, db, MEM_Null);
+ mem_create(&t);
pCtx->pOut = &t;
pCtx->is_aborted = false;
pCtx->skipFlag = 0;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index b7e148422..850158572 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1589,9 +1589,13 @@ sqlVdbeMakeReady(Vdbe * p, /* The VDBE */
} else {
p->nCursor = nCursor;
p->nVar = (ynVar) nVar;
- initMemArray(p->aVar, nVar, db, MEM_Null);
+ for (int i = 0; i < nVar; ++i)
+ mem_create(&p->aVar[i]);
p->nMem = nMem;
- initMemArray(p->aMem, nMem, db, MEM_Undefined);
+ for (int i = 0; i < nMem; ++i) {
+ mem_create(&p->aMem[i]);
+ p->aMem[i].flags = MEM_Undefined;
+ }
memset(p->apCsr, 0, nCursor * sizeof(VdbeCursor *));
}
sqlVdbeRewind(p);
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (9 preceding siblings ...)
2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 10/52] sql: introduce mem_create() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:36 ` Mergen Imeev via Tarantool-patches
2021-04-11 17:46 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
` (30 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:36 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_destroy(). This function should be used to
free and destroing all objects owned by MEM, if necessary.
Part of #5818
---
src/box/sql/func.c | 2 +-
src/box/sql/mem.c | 175 +++++++++++++-----------------------------
src/box/sql/mem.h | 5 +-
src/box/sql/vdbe.c | 6 +-
src/box/sql/vdbeInt.h | 2 +-
src/box/sql/vdbeapi.c | 4 +-
src/box/sql/vdbemem.c | 2 +-
7 files changed, 64 insertions(+), 132 deletions(-)
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 99ce938d5..13d8dd32c 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1796,7 +1796,7 @@ minMaxFinalize(sql_context * context)
if (pRes->flags) {
sql_result_value(context, pRes);
}
- sqlVdbeMemRelease(pRes);
+ mem_destroy(pRes);
}
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5135637d9..805dc7054 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -92,6 +92,38 @@ mem_create(struct Mem *mem)
#endif
}
+static inline void
+mem_clear(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) != 0) {
+ if ((mem->flags & MEM_Agg) != 0)
+ sql_vdbemem_finalize(mem, mem->u.func);
+ assert((mem->flags & MEM_Agg) == 0);
+ if ((mem->flags & MEM_Dyn) != 0) {
+ assert(mem->xDel != SQL_DYNAMIC && mem->xDel != NULL);
+ mem->xDel((void *)mem->z);
+ } else if ((mem->flags & MEM_Frame) != 0) {
+ struct VdbeFrame *frame = mem->u.pFrame;
+ frame->pParent = frame->v->pDelFrame;
+ frame->v->pDelFrame = frame;
+ }
+ }
+ mem->flags = MEM_Null;
+ mem->field_type = field_type_MAX;
+}
+
+void
+mem_destroy(struct Mem *mem)
+{
+ mem_clear(mem);
+ if (mem->szMalloc > 0)
+ sqlDbFree(mem->db, mem->zMalloc);
+ mem->n = 0;
+ mem->z = NULL;
+ mem->szMalloc = 0;
+ mem->zMalloc = NULL;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -203,56 +235,6 @@ vdbeMemAddTerminator(Mem * pMem)
return 0;
}
-/*
- * If the memory cell contains a value that must be freed by
- * invoking the external callback in Mem.xDel, then this routine
- * will free that value. It also sets Mem.flags to MEM_Null.
- *
- * This is a helper routine for sqlVdbeMemSetNull() and
- * for sqlVdbeMemRelease(). Use those other routines as the
- * entry point for releasing Mem resources.
- */
-static SQL_NOINLINE void
-vdbeMemClearExternAndSetNull(Mem * p)
-{
- assert(VdbeMemDynamic(p));
- if (p->flags & MEM_Agg) {
- sql_vdbemem_finalize(p, p->u.func);
- assert((p->flags & MEM_Agg) == 0);
- testcase(p->flags & MEM_Dyn);
- }
- if (p->flags & MEM_Dyn) {
- assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
- p->xDel((void *)p->z);
- } else if (p->flags & MEM_Frame) {
- VdbeFrame *pFrame = p->u.pFrame;
- pFrame->pParent = pFrame->v->pDelFrame;
- pFrame->v->pDelFrame = pFrame;
- }
- p->flags = MEM_Null;
-}
-
-/*
- * Release memory held by the Mem p, both external memory cleared
- * by p->xDel and memory in p->zMalloc.
- *
- * This is a helper routine invoked by sqlVdbeMemRelease() in
- * the unusual case where there really is memory in p that needs
- * to be freed.
- */
-static SQL_NOINLINE void
-vdbeMemClear(Mem * p)
-{
- if (VdbeMemDynamic(p)) {
- vdbeMemClearExternAndSetNull(p);
- }
- if (p->szMalloc) {
- sqlDbFree(p->db, p->zMalloc);
- p->szMalloc = 0;
- }
- p->z = 0;
-}
-
/*
* Make an shallow copy of pFrom into pTo. Prior contents of
* pTo are freed. The pFrom->z field is not duplicated. If
@@ -262,7 +244,7 @@ vdbeMemClear(Mem * p)
static SQL_NOINLINE void
vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
{
- vdbeMemClearExternAndSetNull(pTo);
+ mem_clear(pTo);
assert(!VdbeMemDynamic(pTo));
sqlVdbeMemShallowCopy(pTo, pFrom, eType);
}
@@ -1108,7 +1090,7 @@ sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
}
if (pMem->zMalloc == 0) {
- sqlVdbeMemSetNull(pMem);
+ mem_clear(pMem);
pMem->z = 0;
pMem->szMalloc = 0;
return -1;
@@ -1160,7 +1142,7 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
void
mem_set_bool(struct Mem *mem, bool value)
{
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.b = value;
mem->flags = MEM_Bool;
mem->field_type = FIELD_TYPE_BOOLEAN;
@@ -1169,7 +1151,7 @@ mem_set_bool(struct Mem *mem, bool value)
void
mem_set_ptr(struct Mem *mem, void *ptr)
{
- sqlVdbeMemRelease(mem);
+ mem_destroy(mem);
mem->flags = MEM_Ptr;
mem->u.p = ptr;
}
@@ -1177,8 +1159,7 @@ mem_set_ptr(struct Mem *mem, void *ptr)
void
mem_set_i64(struct Mem *mem, int64_t value)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.i = value;
int flag = value < 0 ? MEM_Int : MEM_UInt;
MemSetTypeFlag(mem, flag);
@@ -1188,8 +1169,7 @@ mem_set_i64(struct Mem *mem, int64_t value)
void
mem_set_u64(struct Mem *mem, uint64_t value)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.u = value;
MemSetTypeFlag(mem, MEM_UInt);
mem->field_type = FIELD_TYPE_UNSIGNED;
@@ -1198,8 +1178,7 @@ mem_set_u64(struct Mem *mem, uint64_t value)
void
mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
if (is_neg) {
assert(value < 0);
mem->u.i = value;
@@ -1214,7 +1193,7 @@ mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
void
mem_set_double(struct Mem *mem, double value)
{
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
if (sqlIsNaN(value))
return;
mem->u.r = value;
@@ -1251,7 +1230,7 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
if (!z) {
- sqlVdbeMemSetNull(pMem);
+ mem_clear(pMem);
return 0;
}
@@ -1291,11 +1270,11 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
}
memcpy(pMem->z, z, nAlloc);
} else if (xDel == SQL_DYNAMIC) {
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->zMalloc = pMem->z = (char *)z;
pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
} else {
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->z = (char *)z;
pMem->xDel = xDel;
flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
@@ -1323,21 +1302,17 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
*
* This routine calls the Mem.xDel destructor to dispose of values that
* require the destructor. But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use sqlVdbeMemRelease(), which both calls this
+ * To free all resources, use mem_destroy(), which both calls this
* routine to invoke the destructor and deallocates Mem.zMalloc.
*
* Use this routine to reset the Mem prior to insert a new value.
*
- * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
+ * Use mem_destroy() to complete erase the Mem prior to abandoning it.
*/
void
sqlVdbeMemSetNull(Mem * pMem)
{
- if (VdbeMemDynamic(pMem)) {
- vdbeMemClearExternAndSetNull(pMem);
- } else {
- pMem->flags = MEM_Null;
- }
+ mem_clear(pMem);
}
/*
@@ -1347,7 +1322,7 @@ sqlVdbeMemSetNull(Mem * pMem)
void
sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
{
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->flags = MEM_Blob | MEM_Zero;
pMem->n = 0;
if (n < 0)
@@ -1384,7 +1359,7 @@ sqlValueFree(sql_value * v)
{
if (!v)
return;
- sqlVdbeMemRelease((Mem *) v);
+ mem_destroy((Mem *) v);
sqlDbFree(((Mem *) v)->db, v);
}
@@ -1407,34 +1382,10 @@ releaseMemArray(Mem * p, int N)
{
if (p && N) {
Mem *pEnd = &p[N];
- sql *db = p->db;
do {
assert((&p[1]) == pEnd || p[0].db == p[1].db);
assert(sqlVdbeCheckMemInvariants(p));
-
- /* This block is really an inlined version of sqlVdbeMemRelease()
- * that takes advantage of the fact that the memory cell value is
- * being set to NULL after releasing any dynamic resources.
- *
- * The justification for duplicating code is that according to
- * callgrind, this causes a certain test case to hit the CPU 4.7
- * percent less (x86 linux, gcc version 4.1.2, -O6) than if
- * sqlMemRelease() were called from here. With -O2, this jumps
- * to 6.6 percent. The test case is inserting 1000 rows into a table
- * with no indexes using a single prepared INSERT statement, bind()
- * and reset(). Inserts are grouped into a transaction.
- */
- testcase(p->flags & MEM_Agg);
- testcase(p->flags & MEM_Dyn);
- testcase(p->flags & MEM_Frame);
- if (p->
- flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
- sqlVdbeMemRelease(p);
- } else if (p->szMalloc) {
- sqlDbFree(db, p->zMalloc);
- p->szMalloc = 0;
- }
-
+ mem_destroy(p);
p->flags = MEM_Undefined;
} while ((++p) < pEnd);
}
@@ -1876,8 +1827,7 @@ sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
{
int rc = 0;
- if (VdbeMemDynamic(pTo))
- vdbeMemClearExternAndSetNull(pTo);
+ mem_clear(pTo);
memcpy(pTo, pFrom, MEMCELLSIZE);
pTo->flags &= ~MEM_Dyn;
if (pTo->flags & (MEM_Str | MEM_Blob)) {
@@ -1917,7 +1867,7 @@ sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
{
assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
- sqlVdbeMemRelease(pTo);
+ mem_destroy(pTo);
memcpy(pTo, pFrom, sizeof(Mem));
pFrom->flags = MEM_Null;
pFrom->szMalloc = 0;
@@ -1952,25 +1902,6 @@ sqlVdbeMemMakeWriteable(Mem * pMem)
return 0;
}
-/*
- * Release any memory resources held by the Mem. Both the memory that is
- * free by Mem.xDel and the Mem.zMalloc allocation are freed.
- *
- * Use this routine prior to clean up prior to abandoning a Mem, or to
- * reset a Mem back to its minimum memory utilization.
- *
- * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
- * prior to inserting new content into the Mem.
- */
-void
-sqlVdbeMemRelease(Mem * p)
-{
- assert(sqlVdbeCheckMemInvariants(p));
- if (VdbeMemDynamic(p) || p->szMalloc) {
- vdbeMemClear(p);
- }
-}
-
int
sql_vdbemem_finalize(struct Mem *mem, struct func *func)
{
@@ -2517,7 +2448,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
return (struct sql_value *)val;
error:
for (int i = 0; i < argc; i++)
- sqlVdbeMemRelease(&val[i]);
+ mem_destroy(&val[i]);
region_truncate(region, region_svp);
return NULL;
}
@@ -2589,7 +2520,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
return (struct sql_value *) val;
error:
for (int i = 0; i < port->size; i++)
- sqlVdbeMemRelease(&val[i]);
+ mem_destroy(&val[i]);
region_truncate(region, region_svp);
return NULL;
}
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 2a3d1078e..82b3084fb 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -98,6 +98,10 @@ mem_str(const struct Mem *mem);
void
mem_create(struct Mem *mem);
+/** Free all allocated memory in MEM and set MEM to NULL. */
+void
+mem_destroy(struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -407,7 +411,6 @@ int sqlVdbeMemCopy(Mem *, const Mem *);
void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemMakeWriteable(Mem *);
-void sqlVdbeMemRelease(Mem * p);
/**
* Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 74bf7f903..7cc72dc38 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2500,7 +2500,7 @@ case OP_MakeRecord: {
* to be passed to Tarantool. Before that, make
* sure previously allocated memory has gone.
*/
- sqlVdbeMemRelease(pOut);
+ mem_destroy(pOut);
pOut->flags = MEM_Blob | MEM_Ephem;
pOut->n = tuple_size;
pOut->z = tuple;
@@ -4454,7 +4454,7 @@ case OP_Program: { /* jump */
if (!pFrame) {
goto no_mem;
}
- sqlVdbeMemRelease(pRt);
+ mem_destroy(pRt);
pRt->flags = MEM_Frame;
pRt->u.pFrame = pFrame;
@@ -4747,7 +4747,7 @@ case OP_AggStep: {
struct func_sql_builtin *func = (struct func_sql_builtin *)pCtx->func;
func->call(pCtx, pCtx->argc, pCtx->argv);
if (pCtx->is_aborted) {
- sqlVdbeMemRelease(&t);
+ mem_destroy(&t);
goto abort_due_to_error;
}
assert(t.flags==MEM_Null);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index b48e45770..cbf32ccdf 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -130,7 +130,7 @@ struct VdbeCursor {
* is linked into the Vdbe.pDelFrame list. The contents of the Vdbe.pDelFrame
* list is deleted when the VM is reset in VdbeHalt(). The reason for doing
* this instead of deleting the VdbeFrame immediately is to avoid recursive
- * calls to sqlVdbeMemRelease() when the memory cells belonging to the
+ * calls to mem_destroy() when the memory cells belonging to the
* child frame are released.
*
* The currently executing frame is stored in Vdbe.pFrame. Vdbe.pFrame is
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 671338361..a195f8dfd 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -681,9 +681,7 @@ vdbeUnbind(Vdbe * p, int i)
}
i--;
pVar = &p->aVar[i];
- sqlVdbeMemRelease(pVar);
- pVar->flags = MEM_Null;
- pVar->field_type = field_type_MAX;
+ mem_destroy(pVar);
return 0;
}
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 263fe5b00..bb87bb902 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -579,7 +579,7 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
int part_count = pRec->key_def->part_count;
struct Mem *aMem = pRec->aMem;
for (int i = 0; i < part_count; i++)
- sqlVdbeMemRelease(&aMem[i]);
+ mem_destroy(&aMem[i]);
sqlDbFree(aMem[0].db, pRec);
}
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy()
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
@ 2021-04-11 17:46 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 12:42 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 17:46 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for working on this!
See 3 comments below.
On 09.04.2021 19:36, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_destroy(). This function should be used to
> free and destroing all objects owned by MEM, if necessary.
1. "to free and destroy".
> Part of #5818
> ---
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 5135637d9..805dc7054 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -92,6 +92,38 @@ mem_create(struct Mem *mem)
> +
> +void
> +mem_destroy(struct Mem *mem)
> +{
> + mem_clear(mem);
> + if (mem->szMalloc > 0)
> + sqlDbFree(mem->db, mem->zMalloc);
> + mem->n = 0;
> + mem->z = NULL;
> + mem->szMalloc = 0;
> + mem->zMalloc = NULL;
2. You could move szMalloc and zMalloc nullification under the 'if'
above.
> +}
> @@ -1384,7 +1359,7 @@ sqlValueFree(sql_value * v)
> {
> if (!v)
> return;
> - sqlVdbeMemRelease((Mem *) v);
> + mem_destroy((Mem *) v);
3. No need for whitespace after unary operators.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy()
2021-04-11 17:46 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 12:42 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 12:42 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers, diff and new patch below.
On Sun, Apr 11, 2021 at 07:46:05PM +0200, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> See 3 comments below.
>
> On 09.04.2021 19:36, Mergen Imeev via Tarantool-patches wrote:
> > This patch introduces mem_destroy(). This function should be used to
> > free and destroing all objects owned by MEM, if necessary.
>
> 1. "to free and destroy".
>
Fixed.
> > Part of #5818
> > ---
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 5135637d9..805dc7054 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -92,6 +92,38 @@ mem_create(struct Mem *mem)
> > +
> > +void
> > +mem_destroy(struct Mem *mem)
> > +{
> > + mem_clear(mem);
> > + if (mem->szMalloc > 0)
> > + sqlDbFree(mem->db, mem->zMalloc);
> > + mem->n = 0;
> > + mem->z = NULL;
> > + mem->szMalloc = 0;
> > + mem->zMalloc = NULL;
>
> 2. You could move szMalloc and zMalloc nullification under the 'if'
> above.
>
Done.
> > +}
> > @@ -1384,7 +1359,7 @@ sqlValueFree(sql_value * v)
> > {
> > if (!v)
> > return;
> > - sqlVdbeMemRelease((Mem *) v);
> > + mem_destroy((Mem *) v);
>
> 3. No need for whitespace after unary operators.
Fixed.
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 805dc7054..e9062b602 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -116,12 +116,13 @@ void
mem_destroy(struct Mem *mem)
{
mem_clear(mem);
- if (mem->szMalloc > 0)
+ if (mem->szMalloc > 0) {
sqlDbFree(mem->db, mem->zMalloc);
+ mem->szMalloc = 0;
+ mem->zMalloc = NULL;
+ }
mem->n = 0;
mem->z = NULL;
- mem->szMalloc = 0;
- mem->zMalloc = NULL;
}
static inline bool
@@ -1359,7 +1360,7 @@ sqlValueFree(sql_value * v)
{
if (!v)
return;
- mem_destroy((Mem *) v);
+ mem_destroy(v);
sqlDbFree(((Mem *) v)->db, v);
}
New patch:
commit 4efcd4a2b2a6654448a33affd6aa43410938f3cf
Author: Mergen Imeev <imeevma@gmail.com>
Date: Thu Mar 4 14:07:35 2021 +0300
sql: introduce mem_destroy()
This patch introduces mem_destroy(). This function should be used to
free and destroy all objects owned by MEM, if necessary.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 99ce938d5..13d8dd32c 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1796,7 +1796,7 @@ minMaxFinalize(sql_context * context)
if (pRes->flags) {
sql_result_value(context, pRes);
}
- sqlVdbeMemRelease(pRes);
+ mem_destroy(pRes);
}
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5135637d9..e9062b602 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -92,6 +92,39 @@ mem_create(struct Mem *mem)
#endif
}
+static inline void
+mem_clear(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) != 0) {
+ if ((mem->flags & MEM_Agg) != 0)
+ sql_vdbemem_finalize(mem, mem->u.func);
+ assert((mem->flags & MEM_Agg) == 0);
+ if ((mem->flags & MEM_Dyn) != 0) {
+ assert(mem->xDel != SQL_DYNAMIC && mem->xDel != NULL);
+ mem->xDel((void *)mem->z);
+ } else if ((mem->flags & MEM_Frame) != 0) {
+ struct VdbeFrame *frame = mem->u.pFrame;
+ frame->pParent = frame->v->pDelFrame;
+ frame->v->pDelFrame = frame;
+ }
+ }
+ mem->flags = MEM_Null;
+ mem->field_type = field_type_MAX;
+}
+
+void
+mem_destroy(struct Mem *mem)
+{
+ mem_clear(mem);
+ if (mem->szMalloc > 0) {
+ sqlDbFree(mem->db, mem->zMalloc);
+ mem->szMalloc = 0;
+ mem->zMalloc = NULL;
+ }
+ mem->n = 0;
+ mem->z = NULL;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -203,56 +236,6 @@ vdbeMemAddTerminator(Mem * pMem)
return 0;
}
-/*
- * If the memory cell contains a value that must be freed by
- * invoking the external callback in Mem.xDel, then this routine
- * will free that value. It also sets Mem.flags to MEM_Null.
- *
- * This is a helper routine for sqlVdbeMemSetNull() and
- * for sqlVdbeMemRelease(). Use those other routines as the
- * entry point for releasing Mem resources.
- */
-static SQL_NOINLINE void
-vdbeMemClearExternAndSetNull(Mem * p)
-{
- assert(VdbeMemDynamic(p));
- if (p->flags & MEM_Agg) {
- sql_vdbemem_finalize(p, p->u.func);
- assert((p->flags & MEM_Agg) == 0);
- testcase(p->flags & MEM_Dyn);
- }
- if (p->flags & MEM_Dyn) {
- assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
- p->xDel((void *)p->z);
- } else if (p->flags & MEM_Frame) {
- VdbeFrame *pFrame = p->u.pFrame;
- pFrame->pParent = pFrame->v->pDelFrame;
- pFrame->v->pDelFrame = pFrame;
- }
- p->flags = MEM_Null;
-}
-
-/*
- * Release memory held by the Mem p, both external memory cleared
- * by p->xDel and memory in p->zMalloc.
- *
- * This is a helper routine invoked by sqlVdbeMemRelease() in
- * the unusual case where there really is memory in p that needs
- * to be freed.
- */
-static SQL_NOINLINE void
-vdbeMemClear(Mem * p)
-{
- if (VdbeMemDynamic(p)) {
- vdbeMemClearExternAndSetNull(p);
- }
- if (p->szMalloc) {
- sqlDbFree(p->db, p->zMalloc);
- p->szMalloc = 0;
- }
- p->z = 0;
-}
-
/*
* Make an shallow copy of pFrom into pTo. Prior contents of
* pTo are freed. The pFrom->z field is not duplicated. If
@@ -262,7 +245,7 @@ vdbeMemClear(Mem * p)
static SQL_NOINLINE void
vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
{
- vdbeMemClearExternAndSetNull(pTo);
+ mem_clear(pTo);
assert(!VdbeMemDynamic(pTo));
sqlVdbeMemShallowCopy(pTo, pFrom, eType);
}
@@ -1108,7 +1091,7 @@ sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
}
if (pMem->zMalloc == 0) {
- sqlVdbeMemSetNull(pMem);
+ mem_clear(pMem);
pMem->z = 0;
pMem->szMalloc = 0;
return -1;
@@ -1160,7 +1143,7 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
void
mem_set_bool(struct Mem *mem, bool value)
{
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.b = value;
mem->flags = MEM_Bool;
mem->field_type = FIELD_TYPE_BOOLEAN;
@@ -1169,7 +1152,7 @@ mem_set_bool(struct Mem *mem, bool value)
void
mem_set_ptr(struct Mem *mem, void *ptr)
{
- sqlVdbeMemRelease(mem);
+ mem_destroy(mem);
mem->flags = MEM_Ptr;
mem->u.p = ptr;
}
@@ -1177,8 +1160,7 @@ mem_set_ptr(struct Mem *mem, void *ptr)
void
mem_set_i64(struct Mem *mem, int64_t value)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.i = value;
int flag = value < 0 ? MEM_Int : MEM_UInt;
MemSetTypeFlag(mem, flag);
@@ -1188,8 +1170,7 @@ mem_set_i64(struct Mem *mem, int64_t value)
void
mem_set_u64(struct Mem *mem, uint64_t value)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
mem->u.u = value;
MemSetTypeFlag(mem, MEM_UInt);
mem->field_type = FIELD_TYPE_UNSIGNED;
@@ -1198,8 +1179,7 @@ mem_set_u64(struct Mem *mem, uint64_t value)
void
mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
{
- if (VdbeMemDynamic(mem))
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
if (is_neg) {
assert(value < 0);
mem->u.i = value;
@@ -1214,7 +1194,7 @@ mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
void
mem_set_double(struct Mem *mem, double value)
{
- sqlVdbeMemSetNull(mem);
+ mem_clear(mem);
if (sqlIsNaN(value))
return;
mem->u.r = value;
@@ -1251,7 +1231,7 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
if (!z) {
- sqlVdbeMemSetNull(pMem);
+ mem_clear(pMem);
return 0;
}
@@ -1291,11 +1271,11 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
}
memcpy(pMem->z, z, nAlloc);
} else if (xDel == SQL_DYNAMIC) {
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->zMalloc = pMem->z = (char *)z;
pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
} else {
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->z = (char *)z;
pMem->xDel = xDel;
flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
@@ -1323,21 +1303,17 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
*
* This routine calls the Mem.xDel destructor to dispose of values that
* require the destructor. But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use sqlVdbeMemRelease(), which both calls this
+ * To free all resources, use mem_destroy(), which both calls this
* routine to invoke the destructor and deallocates Mem.zMalloc.
*
* Use this routine to reset the Mem prior to insert a new value.
*
- * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
+ * Use mem_destroy() to complete erase the Mem prior to abandoning it.
*/
void
sqlVdbeMemSetNull(Mem * pMem)
{
- if (VdbeMemDynamic(pMem)) {
- vdbeMemClearExternAndSetNull(pMem);
- } else {
- pMem->flags = MEM_Null;
- }
+ mem_clear(pMem);
}
/*
@@ -1347,7 +1323,7 @@ sqlVdbeMemSetNull(Mem * pMem)
void
sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
{
- sqlVdbeMemRelease(pMem);
+ mem_destroy(pMem);
pMem->flags = MEM_Blob | MEM_Zero;
pMem->n = 0;
if (n < 0)
@@ -1384,7 +1360,7 @@ sqlValueFree(sql_value * v)
{
if (!v)
return;
- sqlVdbeMemRelease((Mem *) v);
+ mem_destroy(v);
sqlDbFree(((Mem *) v)->db, v);
}
@@ -1407,34 +1383,10 @@ releaseMemArray(Mem * p, int N)
{
if (p && N) {
Mem *pEnd = &p[N];
- sql *db = p->db;
do {
assert((&p[1]) == pEnd || p[0].db == p[1].db);
assert(sqlVdbeCheckMemInvariants(p));
-
- /* This block is really an inlined version of sqlVdbeMemRelease()
- * that takes advantage of the fact that the memory cell value is
- * being set to NULL after releasing any dynamic resources.
- *
- * The justification for duplicating code is that according to
- * callgrind, this causes a certain test case to hit the CPU 4.7
- * percent less (x86 linux, gcc version 4.1.2, -O6) than if
- * sqlMemRelease() were called from here. With -O2, this jumps
- * to 6.6 percent. The test case is inserting 1000 rows into a table
- * with no indexes using a single prepared INSERT statement, bind()
- * and reset(). Inserts are grouped into a transaction.
- */
- testcase(p->flags & MEM_Agg);
- testcase(p->flags & MEM_Dyn);
- testcase(p->flags & MEM_Frame);
- if (p->
- flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
- sqlVdbeMemRelease(p);
- } else if (p->szMalloc) {
- sqlDbFree(db, p->zMalloc);
- p->szMalloc = 0;
- }
-
+ mem_destroy(p);
p->flags = MEM_Undefined;
} while ((++p) < pEnd);
}
@@ -1876,8 +1828,7 @@ sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
{
int rc = 0;
- if (VdbeMemDynamic(pTo))
- vdbeMemClearExternAndSetNull(pTo);
+ mem_clear(pTo);
memcpy(pTo, pFrom, MEMCELLSIZE);
pTo->flags &= ~MEM_Dyn;
if (pTo->flags & (MEM_Str | MEM_Blob)) {
@@ -1917,7 +1868,7 @@ sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
{
assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
- sqlVdbeMemRelease(pTo);
+ mem_destroy(pTo);
memcpy(pTo, pFrom, sizeof(Mem));
pFrom->flags = MEM_Null;
pFrom->szMalloc = 0;
@@ -1952,25 +1903,6 @@ sqlVdbeMemMakeWriteable(Mem * pMem)
return 0;
}
-/*
- * Release any memory resources held by the Mem. Both the memory that is
- * free by Mem.xDel and the Mem.zMalloc allocation are freed.
- *
- * Use this routine prior to clean up prior to abandoning a Mem, or to
- * reset a Mem back to its minimum memory utilization.
- *
- * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
- * prior to inserting new content into the Mem.
- */
-void
-sqlVdbeMemRelease(Mem * p)
-{
- assert(sqlVdbeCheckMemInvariants(p));
- if (VdbeMemDynamic(p) || p->szMalloc) {
- vdbeMemClear(p);
- }
-}
-
int
sql_vdbemem_finalize(struct Mem *mem, struct func *func)
{
@@ -2517,7 +2449,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
return (struct sql_value *)val;
error:
for (int i = 0; i < argc; i++)
- sqlVdbeMemRelease(&val[i]);
+ mem_destroy(&val[i]);
region_truncate(region, region_svp);
return NULL;
}
@@ -2589,7 +2521,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
return (struct sql_value *) val;
error:
for (int i = 0; i < port->size; i++)
- sqlVdbeMemRelease(&val[i]);
+ mem_destroy(&val[i]);
region_truncate(region, region_svp);
return NULL;
}
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 2a3d1078e..82b3084fb 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -98,6 +98,10 @@ mem_str(const struct Mem *mem);
void
mem_create(struct Mem *mem);
+/** Free all allocated memory in MEM and set MEM to NULL. */
+void
+mem_destroy(struct Mem *mem);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -407,7 +411,6 @@ int sqlVdbeMemCopy(Mem *, const Mem *);
void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemMakeWriteable(Mem *);
-void sqlVdbeMemRelease(Mem * p);
/**
* Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 74bf7f903..7cc72dc38 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2500,7 +2500,7 @@ case OP_MakeRecord: {
* to be passed to Tarantool. Before that, make
* sure previously allocated memory has gone.
*/
- sqlVdbeMemRelease(pOut);
+ mem_destroy(pOut);
pOut->flags = MEM_Blob | MEM_Ephem;
pOut->n = tuple_size;
pOut->z = tuple;
@@ -4454,7 +4454,7 @@ case OP_Program: { /* jump */
if (!pFrame) {
goto no_mem;
}
- sqlVdbeMemRelease(pRt);
+ mem_destroy(pRt);
pRt->flags = MEM_Frame;
pRt->u.pFrame = pFrame;
@@ -4747,7 +4747,7 @@ case OP_AggStep: {
struct func_sql_builtin *func = (struct func_sql_builtin *)pCtx->func;
func->call(pCtx, pCtx->argc, pCtx->argv);
if (pCtx->is_aborted) {
- sqlVdbeMemRelease(&t);
+ mem_destroy(&t);
goto abort_due_to_error;
}
assert(t.flags==MEM_Null);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index b48e45770..cbf32ccdf 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -130,7 +130,7 @@ struct VdbeCursor {
* is linked into the Vdbe.pDelFrame list. The contents of the Vdbe.pDelFrame
* list is deleted when the VM is reset in VdbeHalt(). The reason for doing
* this instead of deleting the VdbeFrame immediately is to avoid recursive
- * calls to sqlVdbeMemRelease() when the memory cells belonging to the
+ * calls to mem_destroy() when the memory cells belonging to the
* child frame are released.
*
* The currently executing frame is stored in Vdbe.pFrame. Vdbe.pFrame is
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 671338361..a195f8dfd 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -681,9 +681,7 @@ vdbeUnbind(Vdbe * p, int i)
}
i--;
pVar = &p->aVar[i];
- sqlVdbeMemRelease(pVar);
- pVar->flags = MEM_Null;
- pVar->field_type = field_type_MAX;
+ mem_destroy(pVar);
return 0;
}
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 263fe5b00..bb87bb902 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -579,7 +579,7 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
int part_count = pRec->key_def->part_count;
struct Mem *aMem = pRec->aMem;
for (int i = 0; i < part_count; i++)
- sqlVdbeMemRelease(&aMem[i]);
+ mem_destroy(&aMem[i]);
sqlDbFree(aMem[0].db, pRec);
}
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (10 preceding siblings ...)
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:36 ` Mergen Imeev via Tarantool-patches
2021-04-11 17:59 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
` (29 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:36 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:01, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> I have a general comment affecting this entire naming schema.
> The names seem too long. Besides, we already had some mem_is_*()
> and mem_set_*() functions before this patch, which used short
> names for the types.
>
> What I mean is 'integer' -> 'int', 'boolean' -> 'bool',
> 'string' -> 'str', 'binary' -> 'bin', and so on.
>
I believe I fixed for most of mem_is_*(), mem_set_*() and mem_get_*() functions.
Some functions still have long names, for example mem_is_ephemeral(). I didn't
come to proper short name for these functions.
> For the integers we have several functions because we split
> unsigned, signed, and always negative integers. So we would
> need more int-like names. For instance,
>
> mem_set_uint(uint64_t) - for MEM_UInt.
> mem_set_nint(int64_t) - for MEM_Int.
> mem_set_int(int64_t) - for both, checks the sign inside.
> mem_set_sint(int64_t, bool) - for both, takes the sign flag
> in the second argument
>
> This can be discussed. The main point - shorter is better IMO.
>
I do not hink that splitting is needed. I see it more like field_type -> name of
function + some functions for internal use.
> See 14 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_is_*() functions that allows to check current
>> MEM state.
>>
>> Part of #5818
>> ---
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index e600a9800..81b537d9b 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -736,8 +736,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
>> context->is_aborted = true;
>> return;
>> }
>> - if (sql_value_is_null(argv[1])
>> - || (argc == 3 && sql_value_is_null(argv[2]))
>> + if (mem_is_null(argv[1])
>> + || (argc == 3 && mem_is_null(argv[2]))
>
> 1. This is not movement of huge code blocks, it is rather new code
> now. And in the new code better use our code style. Such as
>
> - || goes in the end of line, not in the beginning of a next line;
> - Unary operators don't have a whitespace after them;
> - Comparison with NULL should be explicit, no implicit boot casts.
>
> I mark the places below which I was able to find after a swift look.
>
Thanks, fixed.
>> ) {
>> return;
>> }
>> @@ -1578,13 +1576,13 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
>> assert(zStr == sql_value_text(argv[0])); /* No encoding change */
>> zPattern = sql_value_text(argv[1]);
>> if (zPattern == 0) {
>> - assert(sql_value_is_null(argv[1])
>> + assert(mem_is_null(argv[1])
>> || sql_context_db_handle(context)->mallocFailed);
>> return;
>> }
>> nPattern = sql_value_bytes(argv[1]);
>> if (nPattern == 0) {
>> - assert(! sql_value_is_null(argv[1]));
>> + assert(! mem_is_null(argv[1]));
>
> 2. Whitespace after unary operator.
>
Fixed.
>> sql_result_value(context, argv[0]);
>> return;
>> }
>> @@ -2039,7 +2035,7 @@ countStep(sql_context * context, int argc, sql_value ** argv)
>> return;
>> }
>> p = sql_aggregate_context(context, sizeof(*p));
>> - if ((argc == 0 || ! sql_value_is_null(argv[0])) && p) {
>> + if ((argc == 0 || ! mem_is_null(argv[0])) && p) {
>
> 3. Ditto.
>
Fixed.
>> p->n++;
>> }
>> }
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index ec6aaab64..abc9291ef 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -37,6 +37,142 @@
>> #include "box/tuple.h"
>> #include "mpstream/mpstream.h"
>>
>> +bool
>> +mem_is_null(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Null) != 0;
>> +}
>
> 4. Maybe better move them all to mem.h. These one-liners easily
> can be inlined (the ones which are <= 3 lines long could be moved).
>
In one of the patches I move MEM types to mem.c so they are not visible from
outside anymore. I think it is right way, at least for now. We may return
MEM types back after we convert them to enum, so there won't be a possiblity
of setting two or more MEM types at the same moment. If we do so, then these
mem_is_*() functions won't be as importans as now and we may make them inlined
or even get rid of them.
>> +
>> +bool
>> +mem_is_unsigned(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_UInt) != 0;
>> +}
>> +
>> +bool
>> +mem_is_string(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Str) != 0;
>> +}
>> +
>> +bool
>> +mem_is_number(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Real | MEM_Int |MEM_UInt)) != 0;
>
> 5. Missed whitespace after the last '|'.
>
Fixed.
>> +}
>> +
>> +bool
>> +mem_is_double(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Real) != 0;
>> +}
>> +
>> +bool
>> +mem_is_integer(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Int | MEM_UInt)) != 0;
>> +}
>> +
>> +bool
>> +mem_is_boolean(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Bool) != 0;
>> +}
>> +
>> +bool
>> +mem_is_binary(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Blob) != 0;
>> +}
>> +
>> +bool
>> +mem_is_map(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Blob) != 0 &&
>> + (mem->flags & MEM_Subtype) != 0 &&
>> + mem->subtype == SQL_SUBTYPE_MSGPACK &&
>> + mp_typeof(*mem->z) == MP_MAP;
>> +}
>> +
>> +bool
>> +mem_is_array(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Blob) != 0 &&
>> + (mem->flags & MEM_Subtype) != 0 &&
>> + mem->subtype == SQL_SUBTYPE_MSGPACK &&
>> + mp_typeof(*mem->z) == MP_ARRAY;
>> +}
>> +
>> +bool
>> +mem_is_aggregate(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Agg) != 0;
>> +}
>> +
>> +bool
>> +mem_is_varstring(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Blob | MEM_Str)) != 0;
>
> 6. It does not look right to call it varstring if it includes
> binary. A string is always binary, but not each binary object
> is a string.
>
> Maybe mem_is_bytes()? mem_is_bytearray()?
>
Thank you. I changed 'varstring' to 'bytes' everywhere.
>> +}
>> +
>> +bool
>> +mem_is_frame(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Frame) != 0;
>> +}
>> +
>> +bool
>> +mem_is_undefined(const struct Mem *mem)
>> +{
>> + return (mem->flags & MEM_Undefined) != 0;
>> +}
>> +
>> +bool
>> +mem_is_static(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
>> + (mem->flags & MEM_Static) != 0;
>> +}
>> +
>> +bool
>> +mem_is_ephemeral(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
>> + (mem->flags & MEM_Ephem) != 0;
>
> 7. How can it be that MEM_Ephem is set, but Str/Blob are not?
>
There is actually a possiblity. After sqlVdbeMemAboutToChange() is called MEM
may become invalid after which MEM_Undefined is changed to MEM_Ephem and then
MEM become valid again. For now I disable this SCopyFrom mechanism, but did
not remove it completely. May be we will enable it later.
>> +}
>> +
>> +bool
>> +mem_is_dynamic(const struct Mem *mem)
>> +{
>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
>> + (mem->flags & MEM_Dyn) != 0;
>> +}
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 12712efb4..05e0f78c1 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -1088,10 +1086,10 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
>> * Concatenation operation can be applied only to
>> * strings and blobs.
>> */
>> - uint32_t str_type_p1 = pIn1->flags & (MEM_Blob | MEM_Str);
>> - uint32_t str_type_p2 = pIn2->flags & (MEM_Blob | MEM_Str);
>> - if (str_type_p1 == 0 || str_type_p2 == 0) {
>> - char *inconsistent_type = str_type_p1 == 0 ?
>> + bool str_type_p1 = mem_is_varstring(pIn1);
>> + bool str_type_p2 = mem_is_varstring(pIn2);
>
> 8. They are not types now. Only flags. Should be renamed to something
> more appropriate.
>
I removed them since they were used in two times only. And the second time is in
case of an error.
Fixed, now they are proper types.
>> + if (!str_type_p1 || !str_type_p2) {
>> + char *inconsistent_type = !str_type_p1 ?
>> mem_type_to_str(pIn1) :
>> mem_type_to_str(pIn2);
>> diag_set(ClientError, ER_INCONSISTENT_TYPES,
>> @@ -1100,7 +1098,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
>> }
>>
>> /* Moreover, both operands must be of the same type. */
>> - if (str_type_p1 != str_type_p2) {
>> + if (mem_is_string(pIn1) != mem_is_string(pIn2)) {
>
> 9. I would recommend mem_is_same_type(). Up to you.
>
Thanks, fixed by using mem_is_same_type().
>> diag_set(ClientError, ER_INCONSISTENT_TYPES,
>> mem_type_to_str(pIn2), mem_type_to_str(pIn1));
>> goto abort_due_to_error;
>> @@ -1186,14 +1183,16 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
>> pIn2 = &aMem[pOp->p2];
>> type2 = numericType(pIn2);
>> pOut = vdbe_prepare_null_out(p, pOp->p3);
>> - flags = pIn1->flags | pIn2->flags;
>> - if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
>> + if (mem_is_null(pIn1) || mem_is_null(pIn2))
>> + goto arithmetic_result_is_null;
>> if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
>> (type2 & (MEM_Int | MEM_UInt)) != 0) {
>> iA = pIn1->u.i;
>> iB = pIn2->u.i;
>> - bool is_lhs_neg = pIn1->flags & MEM_Int;
>> - bool is_rhs_neg = pIn2->flags & MEM_Int;
>> + bool is_lhs_neg = mem_is_integer(pIn1) &&
>> + !mem_is_unsigned(pIn1);
>> + bool is_rhs_neg = mem_is_integer(pIn2) &&
>> + !mem_is_unsigned(pIn2);
>
> 10. The checks look overcomplicated. Worth adding mem_is_nint()?
>
Still not sure. Also, this part will be moved to mem.c in a few patches where we
do not have any problems with determining sign of INTEGER.
>> bool is_res_neg;
>> switch( pOp->opcode) {
>> case OP_Add: {
>> @@ -1509,7 +1508,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
>> pIn1 = &aMem[pOp->p1];
>> pIn2 = &aMem[pOp->p2];
>> pOut = vdbe_prepare_null_out(p, pOp->p3);
>> - if ((pIn1->flags | pIn2->flags) & MEM_Null) {
>> + if (mem_is_null(pIn1) || mem_is_null(pIn2)) {
>
> 11. This is at least third time I see the check of kind "one of them is null".
> And I see more below. Probably worth adding a function which would do it more
> efficient, in one check: mem_is_any_null(mem1, mem2) or something. Up to you.
>
Thanks, added mem_is_any_null().
>> @@ -1757,11 +1756,10 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
>> * or not both operands are null.
>> */
>> assert(pOp->opcode==OP_Eq || pOp->opcode==OP_Ne);
>> - assert((flags1 & MEM_Cleared)==0);
>> + assert(!mem_is_cleared(pIn1));
>> assert((pOp->p5 & SQL_JUMPIFNULL)==0);
>> - if ((flags1&flags3&MEM_Null)!=0
>> - && (flags3&MEM_Cleared)==0
>> - ) {
>> + if (mem_is_null(pIn1) && mem_is_null(pIn3) &&
>
> 12. You already know pIn1 or pIn3 is NULL from the 'if' above. So it
> would be just a bit faster and easier to do mem_is_same_type(). Only
> one branch.
>
Thanks, fixed.
>> + !mem_is_cleared(pIn3)) {
>> res = 0; /* Operands are equal */
>> } else {
>> res = 1; /* Operands are not equal */
>> @@ -2982,18 +2965,18 @@ case OP_SeekGT: { /* jump, in3 */
>> * the seek, so convert it.
>> */
>> pIn3 = &aMem[int_field];
>> - if ((pIn3->flags & MEM_Null) != 0)
>> + if (mem_is_null(pIn3))
>> goto skip_truncate;
>> - if ((pIn3->flags & MEM_Str) != 0)
>> + if (mem_is_string(pIn3))
>> mem_apply_numeric_type(pIn3);
>> int64_t i;
>> - if ((pIn3->flags & MEM_Int) == MEM_Int) {
>> + if (mem_is_integer(pIn3) && !mem_is_unsigned(pIn3)) {
>
> 13. Better be mem_is_nint(), mentioned in one of the previous
> comments.
>
Again, not sure about this. Also, this part too will be moved to mem.c in a few
patches.
>> @@ -3352,11 +3336,11 @@ case OP_FCopy: { /* out2 */
>> pIn1 = &aMem[pOp->p1];
>> }
>>
>> - if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && (pIn1->flags & MEM_Null)) {
>> + if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && mem_is_null(pIn1)) {
>
> 14. Should be explicit != 0.
>
Fixed.
>> pOut = vdbe_prepare_null_out(p, pOp->p2);
>> } else {
>> assert(memIsValid(pIn1));
>> - assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
>> + assert(mem_is_integer(pIn1));
New patch:
commit 6724ff5d04df88fc611d4f921e6bb0b541cd5863
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 11:27:48 2021 +0300
sql: introduce mem_is_*() functions()
This patch introduces mem_is_*() functions that allows to check current
MEM state.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 13d8dd32c..a0108220f 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -92,10 +92,10 @@ minmaxFunc(sql_context * context, int argc, sql_value ** argv)
pColl = sqlGetFuncCollSeq(context);
assert(mask == -1 || mask == 0);
iBest = 0;
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
for (i = 1; i < argc; i++) {
- if (sql_value_is_null(argv[i]))
+ if (mem_is_null(argv[i]))
return;
if ((sqlMemCompare(argv[iBest], argv[i], pColl) ^ mask) >=
0) {
@@ -430,11 +430,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
context->is_aborted = true;
return;
}
- if (sql_value_is_null(argv[1])
- || (argc == 3 && sql_value_is_null(argv[2]))
- ) {
+ if (mem_is_null(argv[1]) || (argc == 3 && mem_is_null(argv[2])))
return;
- }
p0type = sql_value_type(argv[0]);
p1 = sql_value_int(argv[1]);
if (p0type == MP_BIN) {
@@ -532,16 +529,15 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
return;
}
if (argc == 2) {
- if (sql_value_is_null(argv[1]))
+ if (mem_is_null(argv[1]))
return;
n = sql_value_int(argv[1]);
if (n < 0)
n = 0;
}
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
- enum mp_type mp_type = sql_value_type(argv[0]);
- if (mp_type_is_bloblike(mp_type)) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "numeric");
context->is_aborted = true;
@@ -601,8 +597,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv) \
const char *z2; \
int n; \
UNUSED_PARAMETER(argc); \
- int arg_type = sql_value_type(argv[0]); \
- if (mp_type_is_bloblike(arg_type)) { \
+ if (mem_is_bin(argv[0])) { \
diag_set(ClientError, ER_INCONSISTENT_TYPES, "text", \
"varbinary"); \
context->is_aborted = true; \
@@ -683,7 +678,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
unsigned char *p;
assert(argc == 1);
UNUSED_PARAMETER(argc);
- if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "numeric");
context->is_aborted = true;
@@ -1133,7 +1128,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
break;
}
default:{
- assert(sql_value_is_null(argv[0]));
+ assert(mem_is_null(argv[0]));
sql_result_text(context, "NULL", 4, SQL_STATIC);
break;
}
@@ -1272,13 +1267,13 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
assert(zStr == sql_value_text(argv[0])); /* No encoding change */
zPattern = sql_value_text(argv[1]);
if (zPattern == 0) {
- assert(sql_value_is_null(argv[1])
+ assert(mem_is_null(argv[1])
|| sql_context_db_handle(context)->mallocFailed);
return;
}
nPattern = sql_value_bytes(argv[1]);
if (nPattern == 0) {
- assert(! sql_value_is_null(argv[1]));
+ assert(!mem_is_null(argv[1]));
sql_result_value(context, argv[0]);
return;
}
@@ -1442,10 +1437,9 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg)
{
/* In case of VARBINARY type default trim octet is X'00'. */
const unsigned char *default_trim;
- enum mp_type val_type = sql_value_type(arg);
- if (val_type == MP_NIL)
+ if (mem_is_null(arg))
return;
- if (mp_type_is_bloblike(val_type))
+ if (mem_is_bin(arg))
default_trim = (const unsigned char *) "\0";
else
default_trim = (const unsigned char *) " ";
@@ -1574,8 +1568,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
1, 2, 6, 2, 3, 0, 1, 0, 2, 0, 2, 0, 0, 0, 0, 0,
};
assert(argc == 1);
- enum mp_type mp_type = sql_value_type(argv[0]);
- if (mp_type_is_bloblike(mp_type)) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "text");
context->is_aborted = true;
@@ -1733,9 +1726,8 @@ countStep(sql_context * context, int argc, sql_value ** argv)
return;
}
p = sql_aggregate_context(context, sizeof(*p));
- if ((argc == 0 || ! sql_value_is_null(argv[0])) && p) {
+ if ((argc == 0 || !mem_is_null(argv[0])) && p != NULL)
p->n++;
- }
}
static void
@@ -1762,7 +1754,7 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
if (!pBest)
return;
- if (sql_value_is_null(argv[0])) {
+ if (mem_is_null(argv[0])) {
if (pBest->flags)
sqlSkipAccumulatorLoad(context);
} else if (pBest->flags) {
@@ -1816,7 +1808,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
context->is_aborted = true;
return;
}
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
pAccum =
(StrAccum *) sql_aggregate_context(context, sizeof(*pAccum));
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 805dc7054..25b2e75ee 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,149 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+bool
+mem_is_null(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Null) != 0;
+}
+
+bool
+mem_is_uint(const struct Mem *mem)
+{
+ return (mem->flags & MEM_UInt) != 0;
+}
+
+bool
+mem_is_str(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Str) != 0;
+}
+
+bool
+mem_is_num(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0;
+}
+
+bool
+mem_is_double(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Real) != 0;
+}
+
+bool
+mem_is_int(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Int | MEM_UInt)) != 0;
+}
+
+bool
+mem_is_bool(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Bool) != 0;
+}
+
+bool
+mem_is_bin(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Blob) != 0;
+}
+
+bool
+mem_is_map(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Blob) != 0 &&
+ (mem->flags & MEM_Subtype) != 0 &&
+ mem->subtype == SQL_SUBTYPE_MSGPACK &&
+ mp_typeof(*mem->z) == MP_MAP;
+}
+
+bool
+mem_is_array(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Blob) != 0 &&
+ (mem->flags & MEM_Subtype) != 0 &&
+ mem->subtype == SQL_SUBTYPE_MSGPACK &&
+ mp_typeof(*mem->z) == MP_ARRAY;
+}
+
+bool
+mem_is_agg(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Agg) != 0;
+}
+
+bool
+mem_is_bytes(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Blob | MEM_Str)) != 0;
+}
+
+bool
+mem_is_frame(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Frame) != 0;
+}
+
+bool
+mem_is_invalid(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Undefined) != 0;
+}
+
+bool
+mem_is_static(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Static) != 0;
+}
+
+bool
+mem_is_ephemeral(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Ephem) != 0;
+}
+
+bool
+mem_is_dynamic(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Dyn) != 0;
+}
+
+bool
+mem_is_allocated(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == mem->zMalloc;
+}
+
+bool
+mem_is_cleared(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Null) != 0 && (mem->flags & MEM_Cleared) != 0;
+}
+
+bool
+mem_is_zerobin(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Blob) != 0 && (mem->flags & MEM_Zero) != 0;
+}
+
+bool
+mem_is_same_type(const struct Mem *mem1, const struct Mem *mem2)
+{
+ return (mem1->flags & MEM_PURE_TYPE_MASK) ==
+ (mem2->flags & MEM_PURE_TYPE_MASK);
+}
+
+bool
+mem_is_any_null(const struct Mem *mem1, const struct Mem *mem2)
+{
+ return ((mem1->flags | mem2->flags) & MEM_Null) != 0;
+}
+
enum {
BUF_SIZE = 32,
};
@@ -1043,8 +1186,7 @@ mem_convert_to_integer(struct Mem *mem)
int
mem_convert_to_numeric(struct Mem *mem, enum field_type type)
{
- assert(mp_type_is_numeric(mem_mp_type(mem)) &&
- sql_type_is_numeric(type));
+ assert(mem_is_num(mem) && sql_type_is_numeric(type));
assert(type != FIELD_TYPE_NUMBER);
if (type == FIELD_TYPE_DOUBLE)
return mem_convert_to_double(mem);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 82b3084fb..e4e586d4d 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,72 @@ struct Mem {
*/
#define MEMCELLSIZE offsetof(Mem,zMalloc)
+bool
+mem_is_null(const struct Mem *mem);
+
+bool
+mem_is_uint(const struct Mem *mem);
+
+bool
+mem_is_str(const struct Mem *mem);
+
+bool
+mem_is_num(const struct Mem *mem);
+
+bool
+mem_is_double(const struct Mem *mem);
+
+bool
+mem_is_int(const struct Mem *mem);
+
+bool
+mem_is_bool(const struct Mem *mem);
+
+bool
+mem_is_bin(const struct Mem *mem);
+
+bool
+mem_is_map(const struct Mem *mem);
+
+bool
+mem_is_array(const struct Mem *mem);
+
+bool
+mem_is_agg(const struct Mem *mem);
+
+bool
+mem_is_bytes(const struct Mem *mem);
+
+bool
+mem_is_frame(const struct Mem *mem);
+
+bool
+mem_is_invalid(const struct Mem *mem);
+
+bool
+mem_is_static(const struct Mem *mem);
+
+bool
+mem_is_ephemeral(const struct Mem *mem);
+
+bool
+mem_is_dynamic(const struct Mem *mem);
+
+bool
+mem_is_allocated(const struct Mem *mem);
+
+bool
+mem_is_cleared(const struct Mem *mem);
+
+bool
+mem_is_zerobin(const struct Mem *mem);
+
+bool
+mem_is_same_type(const struct Mem *mem1, const struct Mem *mem2);
+
+bool
+mem_is_any_null(const struct Mem *mem1, const struct Mem *mem2);
+
/**
* Return a string that represent content of MEM. String is either allocated
* using static_alloc() of just a static variable.
@@ -375,12 +441,6 @@ columnNullValue(void);
/** Checkers. */
-static inline bool
-sql_value_is_null(struct Mem *value)
-{
- return sql_value_type(value) == MP_NIL;
-}
-
int sqlVdbeMemTooBig(Mem *);
/* Return TRUE if Mem X contains dynamically allocated content - anything
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 0fa388ae9..b9107fccc 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1092,7 +1092,7 @@ selectInnerLoop(Parse * pParse, /* The parser context */
* re-use second for Null op-code.
*
* Change to an OP_Null sets the
- * MEM_Cleared bit on the first
+ * Cleared flag on the first
* register of the previous value.
* This will cause the OP_Ne below
* to always fail on the first
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7cc72dc38..f054a0f43 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -120,8 +120,8 @@ int sql_sort_count = 0;
#endif
/*
- * The next global variable records the size of the largest MEM_Blob
- * or MEM_Str that has been used by a VDBE opcode. The test procedures
+ * The next global variable records the size of the largest varbinary
+ * or string that has been used by a VDBE opcode. The test procedures
* use this information to make sure that the zero-blob functionality
* is working correctly. This variable has no function other than to
* help verify the correct operation of the library.
@@ -131,9 +131,8 @@ int sql_max_blobsize = 0;
static void
updateMaxBlobsize(Mem *p)
{
- if ((p->flags & (MEM_Str|MEM_Blob))!=0 && p->n>sql_max_blobsize) {
+ if (mem_is_bytes(p) && p->n > sql_max_blobsize)
sql_max_blobsize = p->n;
- }
}
#endif
@@ -425,8 +424,7 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
* Add 0 termination (at most for strings)
* Not sure why do we check MEM_Ephem
*/
- if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) ==
- (MEM_Ephem | MEM_Str)) {
+ if (mem_is_str(dest_mem) && mem_is_ephemeral(dest_mem)) {
int len = dest_mem->n;
if (dest_mem->szMalloc < len + 1) {
if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0)
@@ -659,7 +657,7 @@ case OP_Gosub: { /* jump */
*/
case OP_Return: { /* in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags==MEM_UInt);
+ assert(mem_is_uint(pIn1));
pOp = &aOp[pIn1->u.u];
pIn1->flags = MEM_Undefined;
break;
@@ -698,7 +696,7 @@ case OP_InitCoroutine: { /* jump */
case OP_EndCoroutine: { /* in1 */
VdbeOp *pCaller;
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags == MEM_UInt);
+ assert(mem_is_uint(pIn1));
assert(pIn1->u.u < (uint64_t) p->nOp);
pCaller = &aOp[pIn1->u.u];
assert(pCaller->opcode==OP_Yield);
@@ -1108,7 +1106,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
pIn2 = &aMem[pOp->p2];
pOut = vdbe_prepare_null_out(p, pOp->p3);
assert(pIn1!=pOut);
- if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn2)) {
/* Force NULL be of type STRING. */
pOut->field_type = FIELD_TYPE_STRING;
break;
@@ -1117,10 +1115,8 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* Concatenation operation can be applied only to
* strings and blobs.
*/
- uint32_t str_type_p1 = pIn1->flags & (MEM_Blob | MEM_Str);
- uint32_t str_type_p2 = pIn2->flags & (MEM_Blob | MEM_Str);
- if (str_type_p1 == 0 || str_type_p2 == 0) {
- char *inconsistent_type = str_type_p1 == 0 ?
+ if (!mem_is_bytes(pIn1) || !mem_is_bytes(pIn2)) {
+ char *inconsistent_type = !mem_is_bytes(pIn1) ?
mem_type_to_str(pIn1) :
mem_type_to_str(pIn2);
diag_set(ClientError, ER_INCONSISTENT_TYPES,
@@ -1129,7 +1125,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
}
/* Moreover, both operands must be of the same type. */
- if (str_type_p1 != str_type_p2) {
+ if (!mem_is_same_type(pIn1, pIn2)) {
diag_set(ClientError, ER_INCONSISTENT_TYPES,
mem_type_to_str(pIn2), mem_type_to_str(pIn1));
goto abort_due_to_error;
@@ -1143,7 +1139,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
goto no_mem;
}
- if (pIn1->flags & MEM_Str)
+ if (mem_is_str(pIn1))
MemSetTypeFlag(pOut, MEM_Str);
else
MemSetTypeFlag(pOut, MEM_Blob);
@@ -1202,7 +1198,6 @@ case OP_Subtract: /* same as TK_MINUS, in1, in2, out3 */
case OP_Multiply: /* same as TK_STAR, in1, in2, out3 */
case OP_Divide: /* same as TK_SLASH, in1, in2, out3 */
case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
- u32 flags; /* Combined MEM_* flags from both inputs */
u16 type1; /* Numeric type of left operand */
u16 type2; /* Numeric type of right operand */
i64 iA; /* Integer value of left operand */
@@ -1215,14 +1210,14 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
pIn2 = &aMem[pOp->p2];
type2 = numericType(pIn2);
pOut = vdbe_prepare_null_out(p, pOp->p3);
- flags = pIn1->flags | pIn2->flags;
- if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
+ if (mem_is_any_null(pIn1, pIn2))
+ goto arithmetic_result_is_null;
if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
(type2 & (MEM_Int | MEM_UInt)) != 0) {
iA = pIn1->u.i;
iB = pIn2->u.i;
- bool is_lhs_neg = pIn1->flags & MEM_Int;
- bool is_rhs_neg = pIn2->flags & MEM_Int;
+ bool is_lhs_neg = mem_is_int(pIn1) && !mem_is_uint(pIn1);
+ bool is_rhs_neg = mem_is_int(pIn2) && !mem_is_uint(pIn2);
bool is_res_neg;
switch( pOp->opcode) {
case OP_Add: {
@@ -1429,7 +1424,7 @@ case OP_BuiltinFunction: {
goto abort_due_to_error;
/* Copy the result of the function into register P3 */
- if (pOut->flags & (MEM_Str|MEM_Blob)) {
+ if (mem_is_bytes(pOut)) {
if (sqlVdbeMemTooBig(pCtx->pOut)) goto too_big;
}
@@ -1488,7 +1483,7 @@ case OP_FunctionByName: {
* Copy the result of the function invocation into
* register P3.
*/
- if ((pOut->flags & (MEM_Str | MEM_Blob)) != 0)
+ if (mem_is_bytes(pOut))
if (sqlVdbeMemTooBig(pOut)) goto too_big;
REGISTER_TRACE(p, pOp->p3, pOut);
@@ -1538,7 +1533,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
pIn1 = &aMem[pOp->p1];
pIn2 = &aMem[pOp->p2];
pOut = vdbe_prepare_null_out(p, pOp->p3);
- if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn2)) {
/* Force NULL be of type INTEGER. */
pOut->field_type = FIELD_TYPE_INTEGER;
break;
@@ -1597,7 +1592,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
case OP_AddImm: { /* in1 */
pIn1 = &aMem[pOp->p1];
memAboutToChange(p, pIn1);
- assert((pIn1->flags & MEM_UInt) != 0 && pOp->p2 >= 0);
+ assert(mem_is_uint(pIn1) && pOp->p2 >= 0);
pIn1->u.u += pOp->p2;
break;
}
@@ -1611,9 +1606,9 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+ if (!mem_is_int(pIn1)) {
mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+ if (!mem_is_int(pIn1)) {
if (pOp->p2==0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(pIn1), "integer");
@@ -1637,7 +1632,7 @@ case OP_MustBeInt: { /* jump, in1 */
*/
case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) != 0) {
+ if (mem_is_int(pIn1)) {
sqlVdbeMemRealify(pIn1);
}
break;
@@ -1778,7 +1773,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
flags3 = pIn3->flags;
enum field_type ft_p1 = pIn1->field_type;
enum field_type ft_p3 = pIn3->field_type;
- if ((flags1 | flags3)&MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
/* If SQL_NULLEQ is set (which will only happen if the operator is
@@ -1786,11 +1781,10 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
* or not both operands are null.
*/
assert(pOp->opcode==OP_Eq || pOp->opcode==OP_Ne);
- assert((flags1 & MEM_Cleared)==0);
+ assert(!mem_is_cleared(pIn1));
assert((pOp->p5 & SQL_JUMPIFNULL)==0);
- if ((flags1&flags3&MEM_Null)!=0
- && (flags3&MEM_Cleared)==0
- ) {
+ if (mem_is_same_type(pIn1, pIn3) &&
+ !mem_is_cleared(pIn3)) {
res = 0; /* Operands are equal */
} else {
res = 1; /* Operands are not equal */
@@ -1812,22 +1806,17 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
- } else if (((flags1 | flags3) & MEM_Bool) != 0 ||
- ((flags1 | flags3) & MEM_Blob) != 0) {
- /*
- * If one of values is of type BOOLEAN or VARBINARY,
- * then the second one must be of the same type as
- * well. Otherwise an error is raised.
- */
- int type_arg1 = flags1 & (MEM_Bool | MEM_Blob);
- int type_arg3 = flags3 & (MEM_Bool | MEM_Blob);
- if (type_arg1 != type_arg3) {
- char *inconsistent_type = type_arg1 != 0 ?
+ } else if (mem_is_bool(pIn1) || mem_is_bool(pIn3) ||
+ mem_is_bin(pIn1) || mem_is_bin(pIn3)) {
+ if (!mem_is_same_type(pIn1, pIn3)) {
+ char *inconsistent_type = mem_is_bool(pIn1) ||
+ mem_is_bin(pIn1) ?
mem_type_to_str(pIn3) :
mem_type_to_str(pIn1);
- char *expected_type = type_arg1 != 0 ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn3);
+ char *expected_type = mem_is_bool(pIn1) ||
+ mem_is_bin(pIn1) ?
+ mem_type_to_str(pIn1) :
+ mem_type_to_str(pIn3);
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
inconsistent_type, expected_type);
goto abort_due_to_error;
@@ -1836,28 +1825,24 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
} else {
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (sql_type_is_numeric(type)) {
- if ((flags1 | flags3)&MEM_Str) {
- if ((flags1 & MEM_Str) == MEM_Str) {
- mem_apply_numeric_type(pIn1);
- testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
- flags3 = pIn3->flags;
- }
- if ((flags3 & MEM_Str) == MEM_Str) {
- if (mem_apply_numeric_type(pIn3) != 0) {
- diag_set(ClientError,
- ER_SQL_TYPE_MISMATCH,
- mem_str(pIn3),
- "numeric");
- goto abort_due_to_error;
- }
-
+ if (mem_is_str(pIn1)) {
+ mem_apply_numeric_type(pIn1);
+ flags3 = pIn3->flags;
+ }
+ if (mem_is_str(pIn3)) {
+ if (mem_apply_numeric_type(pIn3) != 0) {
+ diag_set(ClientError,
+ ER_SQL_TYPE_MISMATCH,
+ mem_str(pIn3),
+ "numeric");
+ goto abort_due_to_error;
}
}
/* Handle the common case of integer comparison here, as an
* optimization, to avoid a call to sqlMemCompare()
*/
- if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt)) != 0) {
- if ((pIn1->flags & pIn3->flags & MEM_Int) != 0) {
+ if (mem_is_int(pIn1) && mem_is_int(pIn3)) {
+ if (!mem_is_uint(pIn1) && !mem_is_uint(pIn3)) {
if (pIn3->u.i > pIn1->u.i)
res = +1;
else if (pIn3->u.i < pIn1->u.i)
@@ -1866,7 +1851,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- if ((pIn1->flags & pIn3->flags & MEM_UInt) != 0) {
+ if (mem_is_uint(pIn1) && mem_is_uint(pIn3)) {
if (pIn3->u.u > pIn1->u.u)
res = +1;
else if (pIn3->u.u < pIn1->u.u)
@@ -1875,8 +1860,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- if ((pIn1->flags & MEM_UInt) != 0 &&
- (pIn3->flags & MEM_Int) != 0) {
+ if (mem_is_uint(pIn1) && !mem_is_uint(pIn3)) {
res = -1;
goto compare_op;
}
@@ -1884,21 +1868,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
goto compare_op;
}
} else if (type == FIELD_TYPE_STRING) {
- if ((flags1 & MEM_Str) == 0 &&
- (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- testcase( pIn1->flags & MEM_Int);
- testcase( pIn1->flags & MEM_Real);
+ if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
sqlVdbeMemStringify(pIn1);
- testcase( (flags1&MEM_Dyn) != (pIn1->flags&MEM_Dyn));
flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
assert(pIn1!=pIn3);
}
- if ((flags3 & MEM_Str) == 0 &&
- (flags3 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- testcase( pIn3->flags & MEM_Int);
- testcase( pIn3->flags & MEM_Real);
+ if (!mem_is_str(pIn3) && mem_is_num(pIn3)) {
sqlVdbeMemStringify(pIn3);
- testcase( (flags3&MEM_Dyn) != (pIn3->flags&MEM_Dyn));
flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
}
}
@@ -2106,9 +2082,9 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
int v2; /* Right operand: 0==FALSE, 1==TRUE, 2==UNKNOWN or NULL */
pIn1 = &aMem[pOp->p1];
- if (pIn1->flags & MEM_Null) {
+ if (mem_is_null(pIn1)) {
v1 = 2;
- } else if ((pIn1->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn1)) {
v1 = pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2116,9 +2092,9 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
goto abort_due_to_error;
}
pIn2 = &aMem[pOp->p2];
- if (pIn2->flags & MEM_Null) {
+ if (mem_is_null(pIn2)) {
v2 = 2;
- } else if ((pIn2->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn2)) {
v2 = pIn2->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2149,8 +2125,8 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
pIn1 = &aMem[pOp->p1];
pOut = vdbe_prepare_null_out(p, pOp->p2);
pOut->field_type = FIELD_TYPE_BOOLEAN;
- if ((pIn1->flags & MEM_Null)==0) {
- if ((pIn1->flags & MEM_Bool) == 0) {
+ if (!mem_is_null(pIn1)) {
+ if (!mem_is_bool(pIn1)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(pIn1), "boolean");
goto abort_due_to_error;
@@ -2172,7 +2148,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
pOut = vdbe_prepare_null_out(p, pOp->p2);
/* Force NULL be of type INTEGER. */
pOut->field_type = FIELD_TYPE_INTEGER;
- if ((pIn1->flags & MEM_Null)==0) {
+ if (!mem_is_null(pIn1)) {
int64_t i;
bool is_neg;
if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
@@ -2217,9 +2193,9 @@ case OP_If: /* jump, in1 */
case OP_IfNot: { /* jump, in1 */
int c;
pIn1 = &aMem[pOp->p1];
- if (pIn1->flags & MEM_Null) {
+ if (mem_is_null(pIn1)) {
c = pOp->p3;
- } else if ((pIn1->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn1)) {
c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2240,8 +2216,8 @@ case OP_IfNot: { /* jump, in1 */
*/
case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */
pIn1 = &aMem[pOp->p1];
- VdbeBranchTaken( (pIn1->flags & MEM_Null)!=0, 2);
- if ((pIn1->flags & MEM_Null)!=0) {
+ VdbeBranchTaken(mem_is_null(pIn1), 2);
+ if (mem_is_null(pIn1)) {
goto jump_to_p2;
}
break;
@@ -2254,8 +2230,8 @@ case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */
*/
case OP_NotNull: { /* same as TK_NOTNULL, jump, in1 */
pIn1 = &aMem[pOp->p1];
- VdbeBranchTaken( (pIn1->flags & MEM_Null)==0, 2);
- if ((pIn1->flags & MEM_Null)==0) {
+ VdbeBranchTaken(!mem_is_null(pIn1), 2);
+ if (!mem_is_null(pIn1)) {
goto jump_to_p2;
}
break;
@@ -2309,7 +2285,7 @@ case OP_Column: {
if (pC->eCurType==CURTYPE_PSEUDO) {
assert(pC->uc.pseudoTableReg>0);
pReg = &aMem[pC->uc.pseudoTableReg];
- assert(pReg->flags & MEM_Blob);
+ assert(mem_is_bin(pReg));
assert(memIsValid(pReg));
vdbe_field_ref_prepare_data(&pC->field_ref,
pReg->z, pReg->n);
@@ -2338,7 +2314,7 @@ case OP_Column: {
if (vdbe_field_ref_fetch(&pC->field_ref, p2, pDest) != 0)
goto abort_due_to_error;
- if ((pDest->flags & MEM_Null) &&
+ if (mem_is_null(pDest) &&
(uint32_t) p2 >= pC->field_ref.field_count &&
default_val_mem != NULL) {
sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
@@ -2393,7 +2369,7 @@ case OP_ApplyType: {
if (!sql_type_is_numeric(type))
goto type_mismatch;
/* Implicit cast is allowed only from numeric type. */
- if (!mp_type_is_numeric(mem_mp_type(pIn1)))
+ if (!mem_is_num(pIn1))
goto type_mismatch;
/* Try to convert numeric-to-numeric. */
if (mem_convert_to_numeric(pIn1, type) != 0)
@@ -3011,18 +2987,18 @@ case OP_SeekGT: { /* jump, in3 */
* the seek, so convert it.
*/
pIn3 = &aMem[int_field];
- if ((pIn3->flags & MEM_Null) != 0)
+ if (mem_is_null(pIn3))
goto skip_truncate;
- if ((pIn3->flags & MEM_Str) != 0)
+ if (mem_is_str(pIn3))
mem_apply_numeric_type(pIn3);
int64_t i;
- if ((pIn3->flags & MEM_Int) == MEM_Int) {
- i = pIn3->u.i;
- is_neg = true;
- } else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
+ if (mem_is_uint(pIn3)) {
i = pIn3->u.u;
is_neg = false;
- } else if ((pIn3->flags & MEM_Real) == MEM_Real) {
+ } else if (mem_is_int(pIn3)) {
+ i = pIn3->u.i;
+ is_neg = true;
+ } else if (mem_is_double(pIn3)) {
if (pIn3->u.r > (double)INT64_MAX)
i = INT64_MAX;
else if (pIn3->u.r < (double)INT64_MIN)
@@ -3040,8 +3016,8 @@ case OP_SeekGT: { /* jump, in3 */
/* If the P3 value could not be converted into an integer without
* loss of information, then special processing is required...
*/
- if ((pIn3->flags & (MEM_Int | MEM_UInt)) == 0) {
- if ((pIn3->flags & MEM_Real)==0) {
+ if (!mem_is_int(pIn3)) {
+ if (!mem_is_double(pIn3)) {
/* If the P3 value cannot be converted into any kind of a number,
* then the seek is not possible, so jump to P2
*/
@@ -3247,7 +3223,8 @@ case OP_Found: { /* jump, in3 */
#ifdef SQL_DEBUG
for(ii=0; ii<r.nField; ii++) {
assert(memIsValid(&r.aMem[ii]));
- assert((r.aMem[ii].flags & MEM_Zero)==0 || r.aMem[ii].n==0);
+ assert(!mem_is_zerobin(&r.aMem[ii]) ||
+ r.aMem[ii].n == 0);
if (ii != 0)
REGISTER_TRACE(p, pOp->p3+ii, &r.aMem[ii]);
}
@@ -3257,7 +3234,7 @@ case OP_Found: { /* jump, in3 */
} else {
pFree = pIdxKey = sqlVdbeAllocUnpackedRecord(db, pC->key_def);
if (pIdxKey==0) goto no_mem;
- assert(pIn3->flags & MEM_Blob );
+ assert(mem_is_bin(pIn3));
(void)ExpandBlob(pIn3);
sqlVdbeRecordUnpackMsgpack(pC->key_def,
pIn3->z, pIdxKey);
@@ -3271,7 +3248,7 @@ case OP_Found: { /* jump, in3 */
* conflict
*/
for(ii=0; ii<pIdxKey->nField; ii++) {
- if (pIdxKey->aMem[ii].flags & MEM_Null) {
+ if (mem_is_null(&pIdxKey->aMem[ii])) {
takeJump = 1;
break;
}
@@ -3381,11 +3358,11 @@ case OP_FCopy: { /* out2 */
pIn1 = &aMem[pOp->p1];
}
- if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && (pIn1->flags & MEM_Null)) {
+ if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) != 0 && mem_is_null(pIn1)) {
pOut = vdbe_prepare_null_out(p, pOp->p2);
} else {
assert(memIsValid(pIn1));
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ assert(mem_is_int(pIn1));
pOut = vdbe_prepare_null_out(p, pOp->p2);
mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
@@ -3520,7 +3497,7 @@ case OP_SorterData: {
assert(isSorter(pC));
if (sqlVdbeSorterRowkey(pC, pOut) != 0)
goto abort_due_to_error;
- assert(pOut->flags & MEM_Blob);
+ assert(mem_is_bin(pOut));
assert(pOp->p1>=0 && pOp->p1<p->nCursor);
p->apCsr[pOp->p3]->cacheStatus = CACHE_STALE;
break;
@@ -3878,7 +3855,7 @@ case OP_SorterInsert: { /* in2 */
assert(cursor != NULL);
assert(isSorter(cursor));
pIn2 = &aMem[pOp->p2];
- assert((pIn2->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(pIn2));
if (ExpandBlob(pIn2) != 0 ||
sqlVdbeSorterWrite(cursor, pIn2) != 0)
goto abort_due_to_error;
@@ -3912,7 +3889,7 @@ case OP_SorterInsert: { /* in2 */
case OP_IdxReplace:
case OP_IdxInsert: {
pIn2 = &aMem[pOp->p1];
- assert((pIn2->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(pIn2));
if (ExpandBlob(pIn2) != 0)
goto abort_due_to_error;
struct space *space;
@@ -3923,7 +3900,7 @@ case OP_IdxInsert: {
assert(space != NULL);
if (space->def->id != 0) {
/* Make sure that memory has been allocated on region. */
- assert(aMem[pOp->p1].flags & MEM_Ephem);
+ assert(mem_is_ephemeral(&aMem[pOp->p1]));
if (pOp->opcode == OP_IdxInsert) {
rc = tarantoolsqlInsert(space, pIn2->z,
pIn2->z + pIn2->n);
@@ -3957,7 +3934,7 @@ case OP_IdxInsert: {
}
if ((pOp->p5 & OPFLAG_NCHANGE) != 0)
p->nChange++;
- if (pOp->p3 > 0 && ((aMem[pOp->p3].flags) & MEM_Null) != 0) {
+ if (pOp->p3 > 0 && mem_is_null(&aMem[pOp->p3])) {
assert(space->sequence != NULL);
int64_t value;
if (sequence_get_value(space->sequence, &value) != 0)
@@ -4003,10 +3980,10 @@ case OP_Update: {
assert(pOp->p4type == P4_SPACEPTR);
struct Mem *key_mem = &aMem[pOp->p2];
- assert((key_mem->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(key_mem));
struct Mem *upd_fields_mem = &aMem[pOp->p3];
- assert((upd_fields_mem->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(upd_fields_mem));
uint32_t *upd_fields = (uint32_t *)upd_fields_mem->z;
uint32_t upd_fields_cnt = upd_fields_mem->n / sizeof(uint32_t);
@@ -4438,7 +4415,7 @@ case OP_Program: { /* jump */
* the trigger program. If this trigger has been fired before, then pRt
* is already allocated. Otherwise, it must be initialized.
*/
- if ((pRt->flags&MEM_Frame)==0) {
+ if (!mem_is_frame(pRt)) {
/* SubProgram.nMem is set to the number of memory cells used by the
* program stored in SubProgram.aOp. As well as these, one memory
* cell is required for each cursor used by the program. Set local
@@ -4578,8 +4555,8 @@ case OP_FkIfZero: { /* jump */
*/
case OP_IfPos: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
- if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
+ assert(mem_is_int(pIn1));
+ if (mem_is_uint(pIn1) && pIn1->u.u != 0) {
assert(pOp->p3 >= 0);
uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
/*
@@ -4613,8 +4590,8 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
pIn3 = &aMem[pOp->p3];
pOut = vdbe_prepare_null_out(p, pOp->p2);
- assert((pIn1->flags & MEM_UInt) != 0);
- assert((pIn3->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
+ assert(mem_is_uint(pIn3));
uint64_t x = pIn1->u.u;
uint64_t rhs = pIn3->u.u;
bool unused;
@@ -4637,7 +4614,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
if (pIn1->u.u > 0) {
pIn1->u.u--;
goto jump_to_p2;
@@ -4653,7 +4630,7 @@ case OP_IfNotZero: { /* jump, in1 */
*/
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
if (pIn1->u.u > 0)
pIn1->u.u--;
if (pIn1->u.u == 0) goto jump_to_p2;
@@ -4750,7 +4727,7 @@ case OP_AggStep: {
mem_destroy(&t);
goto abort_due_to_error;
}
- assert(t.flags==MEM_Null);
+ assert(mem_is_null(&t));
if (pCtx->skipFlag) {
assert(pOp[-1].opcode==OP_CollSeq);
i = pOp[-1].p1;
@@ -4776,7 +4753,7 @@ case OP_AggFinal: {
Mem *pMem;
assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
pMem = &aMem[pOp->p1];
- assert((pMem->flags & ~(MEM_Null|MEM_Agg))==0);
+ assert(mem_is_null(pMem) || mem_is_agg(pMem));
if (sql_vdbemem_finalize(pMem, pOp->p4.func) != 0)
goto abort_due_to_error;
UPDATE_MAX_BLOBSIZE(pMem);
@@ -4903,7 +4880,7 @@ case OP_SetSession: {
struct session_setting *setting = &session_settings[sid];
switch (setting->field_type) {
case FIELD_TYPE_BOOLEAN: {
- if ((pIn1->flags & MEM_Bool) == 0)
+ if (!mem_is_bool(pIn1))
goto invalid_type;
bool value = pIn1->u.b;
size_t size = mp_sizeof_bool(value);
@@ -4914,7 +4891,7 @@ case OP_SetSession: {
break;
}
case FIELD_TYPE_STRING: {
- if ((pIn1->flags & MEM_Str) == 0)
+ if (!mem_is_str(pIn1))
goto invalid_type;
const char *str = pIn1->z;
uint32_t size = mp_sizeof_str(pIn1->n);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index cbf32ccdf..b4ad8b774 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -333,17 +333,6 @@ int sqlVdbeList(Vdbe *);
int sqlVdbeHalt(Vdbe *);
-/**
- * In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
- * messagepacks are stored as binary string (i.e. featuring
- * MEM_Blob internal type).
- */
-#define mp_type_is_bloblike(X) ((X) == MP_BIN || (X) == MP_ARRAY || (X) == MP_MAP)
-
-/** Return TRUE if MP_type of X is numeric, FALSE otherwise. */
-#define mp_type_is_numeric(X) ((X) == MP_INT || (X) == MP_UINT ||\
- (X) == MP_DOUBLE)
-
const char *sqlOpcodeName(int);
int sqlVdbeCloseStatement(Vdbe *, int);
void sqlVdbeFrameDelete(VdbeFrame *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index a195f8dfd..2a3561d42 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -380,7 +380,7 @@ static SQL_NOINLINE void *
createAggContext(sql_context * p, int nByte)
{
Mem *pMem = p->pMem;
- assert((pMem->flags & MEM_Agg) == 0);
+ assert(!mem_is_agg(pMem));
if (nByte <= 0) {
sqlVdbeMemSetNull(pMem);
pMem->z = 0;
@@ -407,7 +407,7 @@ sql_aggregate_context(sql_context * p, int nByte)
assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
testcase(nByte < 0);
- if ((p->pMem->flags & MEM_Agg) == 0) {
+ if (!mem_is_agg(p->pMem)) {
return createAggContext(p, nByte);
} else {
return (void *)p->pMem->z;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 850158572..52e100454 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1253,7 +1253,7 @@ sqlVdbeList(Vdbe * p)
*/
assert(p->nMem > 9);
pSub = &p->aMem[9];
- if (pSub->flags & MEM_Blob) {
+ if (mem_is_bin(pSub)) {
/* On the first call to sql_step(), pSub will hold a NULL. It is
* initialized to a BLOB by the P4_SUBPROGRAM processing logic below
*/
@@ -1722,7 +1722,7 @@ Cleanup(Vdbe * p)
assert(p->apCsr[i] == 0);
if (p->aMem) {
for (i = 0; i < p->nMem; i++)
- assert(p->aMem[i].flags == MEM_Undefined);
+ assert(mem_is_invalid(&p->aMem[i]));
}
#endif
@@ -2330,7 +2330,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
assert(iVar > 0);
if (v) {
Mem *pMem = &v->aVar[iVar - 1];
- if (0 == (pMem->flags & MEM_Null)) {
+ if (!mem_is_null(pMem)) {
sql_value *pRet = sqlValueNew(v->db);
if (pRet) {
sqlVdbeMemCopy((Mem *) pRet, pMem);
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index 927f85559..a9a5f45af 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2218,7 +2218,7 @@ sqlVdbeSorterCompare(const VdbeCursor * pCsr, /* Sorter cursor */
pKey = vdbeSorterRowkey(pSorter, &nKey);
sqlVdbeRecordUnpackMsgpack(pCsr->key_def, pKey, r2);
for (i = 0; i < nKeyCol; i++) {
- if (r2->aMem[i].flags & MEM_Null) {
+ if (mem_is_null(&r2->aMem[i])) {
*pRes = -1;
return 0;
}
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 0c002dbee..93de722cb 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -312,9 +312,8 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
pVal =
sqlVdbeGetBoundValue(pReprepare, iCol,
FIELD_TYPE_SCALAR);
- if (pVal && sql_value_type(pVal) == MP_STR) {
+ if (pVal != NULL && mem_is_str(pVal))
z = (char *)sql_value_text(pVal);
- }
assert(pRight->op == TK_VARIABLE || pRight->op == TK_REGISTER);
} else if (op == TK_STRING) {
z = pRight->u.zToken;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions()
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
@ 2021-04-11 17:59 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 16:09 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 17:59 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for the fixes!
>> For the integers we have several functions because we split
>> unsigned, signed, and always negative integers. So we would
>> need more int-like names. For instance,
>>
>> mem_set_uint(uint64_t) - for MEM_UInt.
>> mem_set_nint(int64_t) - for MEM_Int.
>> mem_set_int(int64_t) - for both, checks the sign inside.
>> mem_set_sint(int64_t, bool) - for both, takes the sign flag
>> in the second argument
>>
>> This can be discussed. The main point - shorter is better IMO.
>>
> I do not hink that splitting is needed. I see it more like field_type -> name of
> function + some functions for internal use.
This does not work already, because MEM_Int != FIELD_TYPE_INTEGER and
mem_is_int() does not check for MEM_Int only.
>>> }
>>> }
>>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>>> index ec6aaab64..abc9291ef 100644
>>> --- a/src/box/sql/mem.c
>>> +++ b/src/box/sql/mem.c
>>> @@ -37,6 +37,142 @@
>>> #include "box/tuple.h"
>>> #include "mpstream/mpstream.h"
>>>
>>> +bool
>>> +mem_is_null(const struct Mem *mem)
>>> +{
>>> + return (mem->flags & MEM_Null) != 0;
>>> +}
>>
>> 4. Maybe better move them all to mem.h. These one-liners easily
>> can be inlined (the ones which are <= 3 lines long could be moved).
>>
> In one of the patches I move MEM types to mem.c so they are not visible from
> outside anymore. I think it is right way, at least for now. We may return
> MEM types back after we convert them to enum, so there won't be a possiblity
> of setting two or more MEM types at the same moment.
But there is now already. AFAIS, MEM_Str might be set along with some
other type. Or was it fixed somewhere in this patchset? See one of
my comments below.
>>> +}
>>> +
>>> +bool
>>> +mem_is_frame(const struct Mem *mem)
>>> +{
>>> + return (mem->flags & MEM_Frame) != 0;
>>> +}
>>> +
>>> +bool
>>> +mem_is_undefined(const struct Mem *mem)
>>> +{
>>> + return (mem->flags & MEM_Undefined) != 0;
>>> +}
>>> +
>>> +bool
>>> +mem_is_static(const struct Mem *mem)
>>> +{
>>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
>>> + (mem->flags & MEM_Static) != 0;
>>> +}
>>> +
>>> +bool
>>> +mem_is_ephemeral(const struct Mem *mem)
>>> +{
>>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
>>> + (mem->flags & MEM_Ephem) != 0;
>>
>> 7. How can it be that MEM_Ephem is set, but Str/Blob are not?
>>
> There is actually a possiblity. After sqlVdbeMemAboutToChange() is called MEM
> may become invalid after which MEM_Undefined is changed to MEM_Ephem and then
Where is undefined changed to ephem?
> MEM become valid again. For now I disable this SCopyFrom mechanism, but did
> not remove it completely. May be we will enable it later.
<...>
See 4 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 805dc7054..25b2e75ee 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -40,6 +40,149 @@
<...>
> +
> +bool
> +mem_is_map(const struct Mem *mem)
> +{
> + return (mem->flags & MEM_Blob) != 0 &&
> + (mem->flags & MEM_Subtype) != 0 &&
1. You could check that in one operation:
(mem->flags & (MEM_Blob | MEM_Subtype)) == (MEM_Blob | MEM_Subtype)
> + mem->subtype == SQL_SUBTYPE_MSGPACK &&
> + mp_typeof(*mem->z) == MP_MAP;
> +}
<...>
> +
> +bool
> +mem_is_cleared(const struct Mem *mem)
> +{
> + return (mem->flags & MEM_Null) != 0 && (mem->flags & MEM_Cleared) != 0;
2. Can be 1 operation:
(mem->flags & (MEM_Null | MEM_Cleared)) == (MEM_Null | MEM_Cleared)
But another question is how is it possible that Cleared is set, but
Null isn't?
> +}
> +
> +bool
> +mem_is_zerobin(const struct Mem *mem)
> +{
> + return (mem->flags & MEM_Blob) != 0 && (mem->flags & MEM_Zero) != 0;
3. The same, can be done in one operation. And the same question - how is it
possible that Zero is set, but Blob isn't?
> +}
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 7cc72dc38..f054a0f43 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c> @@ -1884,21 +1868,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> goto compare_op;
> }
> } else if (type == FIELD_TYPE_STRING) {
> - if ((flags1 & MEM_Str) == 0 &&
> - (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
> - testcase( pIn1->flags & MEM_Int);
> - testcase( pIn1->flags & MEM_Real);
> + if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
4. Are going to do anything with that hack when a string can be stored in
the same mem as the original value? Otherwise you can see yourself how
ugly and confusing the 'mem_is' checks might look.
Besides, all the mem functions doing something with the mem based on its
pure types mask won't work on such mutant mems I suppose. Because there
is more than 1 type.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions()
2021-04-11 17:59 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 16:09 ` Mergen Imeev via Tarantool-patches
2021-04-14 22:48 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 16:09 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Hi! Thank you for the review. After some thought I decided that you were right
and added new function, mem_is_nin(). Also, I moved mem_is_*() functions from
mem.c to mem.c and made them 'static inline'. Due to this I dropped commit
"sql: move MEM flags to mem.c".
Also, here won't be diff since I squashed commits more that once. New patch
below.
On Sun, Apr 11, 2021 at 07:59:05PM +0200, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
>
> >> For the integers we have several functions because we split
> >> unsigned, signed, and always negative integers. So we would
> >> need more int-like names. For instance,
> >>
> >> mem_set_uint(uint64_t) - for MEM_UInt.
> >> mem_set_nint(int64_t) - for MEM_Int.
> >> mem_set_int(int64_t) - for both, checks the sign inside.
> >> mem_set_sint(int64_t, bool) - for both, takes the sign flag
> >> in the second argument
> >>
> >> This can be discussed. The main point - shorter is better IMO.
> >>
> > I do not hink that splitting is needed. I see it more like field_type -> name of
> > function + some functions for internal use.
>
> This does not work already, because MEM_Int != FIELD_TYPE_INTEGER and
> mem_is_int() does not check for MEM_Int only.
>
Thanks, fixed. At the moment I added mem_is_nint(), but I plan to also add
mem_set_nint() and mem_get_nint().
> >>> }
> >>> }
> >>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> >>> index ec6aaab64..abc9291ef 100644
> >>> --- a/src/box/sql/mem.c
> >>> +++ b/src/box/sql/mem.c
> >>> @@ -37,6 +37,142 @@
> >>> #include "box/tuple.h"
> >>> #include "mpstream/mpstream.h"
> >>>
> >>> +bool
> >>> +mem_is_null(const struct Mem *mem)
> >>> +{
> >>> + return (mem->flags & MEM_Null) != 0;
> >>> +}
> >>
> >> 4. Maybe better move them all to mem.h. These one-liners easily
> >> can be inlined (the ones which are <= 3 lines long could be moved).
> >>
> > In one of the patches I move MEM types to mem.c so they are not visible from
> > outside anymore. I think it is right way, at least for now. We may return
> > MEM types back after we convert them to enum, so there won't be a possiblity
> > of setting two or more MEM types at the same moment.
>
> But there is now already. AFAIS, MEM_Str might be set along with some
> other type. Or was it fixed somewhere in this patchset? See one of
> my comments below.
>
I actually believe that this was broken for quite some time. There is still one
place where MEM is checked for str+int, however is legacy. I will remove this
during changing MEM-types from MEM-flags to enum mem_type (issue #4906) which I
plan to do in Q2. In general, after this patch-set there won't be any mutant
mems. Well, except for the one in allocateCursor(). Not sure that it may be
called MEM, actually.
> >>> +}
> >>> +
> >>> +bool
> >>> +mem_is_frame(const struct Mem *mem)
> >>> +{
> >>> + return (mem->flags & MEM_Frame) != 0;
> >>> +}
> >>> +
> >>> +bool
> >>> +mem_is_undefined(const struct Mem *mem)
> >>> +{
> >>> + return (mem->flags & MEM_Undefined) != 0;
> >>> +}
> >>> +
> >>> +bool
> >>> +mem_is_static(const struct Mem *mem)
> >>> +{
> >>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> >>> + (mem->flags & MEM_Static) != 0;
> >>> +}
> >>> +
> >>> +bool
> >>> +mem_is_ephemeral(const struct Mem *mem)
> >>> +{
> >>> + return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> >>> + (mem->flags & MEM_Ephem) != 0;
> >>
> >> 7. How can it be that MEM_Ephem is set, but Str/Blob are not?
> >>
> > There is actually a possiblity. After sqlVdbeMemAboutToChange() is called MEM
> > may become invalid after which MEM_Undefined is changed to MEM_Ephem and then
>
> Where is undefined changed to ephem?
>
> > MEM become valid again. For now I disable this SCopyFrom mechanism, but did
> > not remove it completely. May be we will enable it later.
>
> <...>
>
> See 4 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 805dc7054..25b2e75ee 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -40,6 +40,149 @@
>
> <...>
>
> > +
> > +bool
> > +mem_is_map(const struct Mem *mem)
> > +{
> > + return (mem->flags & MEM_Blob) != 0 &&
> > + (mem->flags & MEM_Subtype) != 0 &&
>
> 1. You could check that in one operation:
>
> (mem->flags & (MEM_Blob | MEM_Subtype)) == (MEM_Blob | MEM_Subtype)
>
Fixed. Moved check for MEM_blob and SQL_SUBTYPE_MSGPACK to asserts since
MEM_Subtype means that both are set. At least should mean this.
> > + mem->subtype == SQL_SUBTYPE_MSGPACK &&
> > + mp_typeof(*mem->z) == MP_MAP;
> > +}
>
> <...>
>
> > +
> > +bool
> > +mem_is_cleared(const struct Mem *mem)
> > +{
> > + return (mem->flags & MEM_Null) != 0 && (mem->flags & MEM_Cleared) != 0;
>
Fixed. Moved check for MEM_Null to assert.
> 2. Can be 1 operation:
>
> (mem->flags & (MEM_Null | MEM_Cleared)) == (MEM_Null | MEM_Cleared)
>
> But another question is how is it possible that Cleared is set, but
> Null isn't?
>
Fixed. Moved check for MEM_Null to assert.
> > +}
> > +
> > +bool
> > +mem_is_zerobin(const struct Mem *mem)
> > +{
> > + return (mem->flags & MEM_Blob) != 0 && (mem->flags & MEM_Zero) != 0;
>
> 3. The same, can be done in one operation. And the same question - how is it
> possible that Zero is set, but Blob isn't?
>
Also imposible. Moved check for MEM_Blob to assert.
> > +}
> > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > index 7cc72dc38..f054a0f43 100644
> > --- a/src/box/sql/vdbe.c
> > +++ b/src/box/sql/vdbe.c> @@ -1884,21 +1868,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> > goto compare_op;
> > }
> > } else if (type == FIELD_TYPE_STRING) {
> > - if ((flags1 & MEM_Str) == 0 &&
> > - (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
> > - testcase( pIn1->flags & MEM_Int);
> > - testcase( pIn1->flags & MEM_Real);
> > + if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
>
> 4. Are going to do anything with that hack when a string can be stored in
> the same mem as the original value? Otherwise you can see yourself how
> ugly and confusing the 'mem_is' checks might look.
>
> Besides, all the mem functions doing something with the mem based on its
> pure types mask won't work on such mutant mems I suppose. Because there
> is more than 1 type.
New patch:
commit 1bcb55b185443cc09ce9b2c843abf25248814306
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 2 11:27:48 2021 +0300
sql: introduce mem_is_*() functions()
This patch introduces mem_is_*() functions that allows to check current
MEM state.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 13d8dd32c..a0108220f 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -92,10 +92,10 @@ minmaxFunc(sql_context * context, int argc, sql_value ** argv)
pColl = sqlGetFuncCollSeq(context);
assert(mask == -1 || mask == 0);
iBest = 0;
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
for (i = 1; i < argc; i++) {
- if (sql_value_is_null(argv[i]))
+ if (mem_is_null(argv[i]))
return;
if ((sqlMemCompare(argv[iBest], argv[i], pColl) ^ mask) >=
0) {
@@ -430,11 +430,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
context->is_aborted = true;
return;
}
- if (sql_value_is_null(argv[1])
- || (argc == 3 && sql_value_is_null(argv[2]))
- ) {
+ if (mem_is_null(argv[1]) || (argc == 3 && mem_is_null(argv[2])))
return;
- }
p0type = sql_value_type(argv[0]);
p1 = sql_value_int(argv[1]);
if (p0type == MP_BIN) {
@@ -532,16 +529,15 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
return;
}
if (argc == 2) {
- if (sql_value_is_null(argv[1]))
+ if (mem_is_null(argv[1]))
return;
n = sql_value_int(argv[1]);
if (n < 0)
n = 0;
}
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
- enum mp_type mp_type = sql_value_type(argv[0]);
- if (mp_type_is_bloblike(mp_type)) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "numeric");
context->is_aborted = true;
@@ -601,8 +597,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv) \
const char *z2; \
int n; \
UNUSED_PARAMETER(argc); \
- int arg_type = sql_value_type(argv[0]); \
- if (mp_type_is_bloblike(arg_type)) { \
+ if (mem_is_bin(argv[0])) { \
diag_set(ClientError, ER_INCONSISTENT_TYPES, "text", \
"varbinary"); \
context->is_aborted = true; \
@@ -683,7 +678,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
unsigned char *p;
assert(argc == 1);
UNUSED_PARAMETER(argc);
- if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "numeric");
context->is_aborted = true;
@@ -1133,7 +1128,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
break;
}
default:{
- assert(sql_value_is_null(argv[0]));
+ assert(mem_is_null(argv[0]));
sql_result_text(context, "NULL", 4, SQL_STATIC);
break;
}
@@ -1272,13 +1267,13 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
assert(zStr == sql_value_text(argv[0])); /* No encoding change */
zPattern = sql_value_text(argv[1]);
if (zPattern == 0) {
- assert(sql_value_is_null(argv[1])
+ assert(mem_is_null(argv[1])
|| sql_context_db_handle(context)->mallocFailed);
return;
}
nPattern = sql_value_bytes(argv[1]);
if (nPattern == 0) {
- assert(! sql_value_is_null(argv[1]));
+ assert(!mem_is_null(argv[1]));
sql_result_value(context, argv[0]);
return;
}
@@ -1442,10 +1437,9 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg)
{
/* In case of VARBINARY type default trim octet is X'00'. */
const unsigned char *default_trim;
- enum mp_type val_type = sql_value_type(arg);
- if (val_type == MP_NIL)
+ if (mem_is_null(arg))
return;
- if (mp_type_is_bloblike(val_type))
+ if (mem_is_bin(arg))
default_trim = (const unsigned char *) "\0";
else
default_trim = (const unsigned char *) " ";
@@ -1574,8 +1568,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
1, 2, 6, 2, 3, 0, 1, 0, 2, 0, 2, 0, 0, 0, 0, 0,
};
assert(argc == 1);
- enum mp_type mp_type = sql_value_type(argv[0]);
- if (mp_type_is_bloblike(mp_type)) {
+ if (mem_is_bin(argv[0])) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(argv[0]), "text");
context->is_aborted = true;
@@ -1733,9 +1726,8 @@ countStep(sql_context * context, int argc, sql_value ** argv)
return;
}
p = sql_aggregate_context(context, sizeof(*p));
- if ((argc == 0 || ! sql_value_is_null(argv[0])) && p) {
+ if ((argc == 0 || !mem_is_null(argv[0])) && p != NULL)
p->n++;
- }
}
static void
@@ -1762,7 +1754,7 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
if (!pBest)
return;
- if (sql_value_is_null(argv[0])) {
+ if (mem_is_null(argv[0])) {
if (pBest->flags)
sqlSkipAccumulatorLoad(context);
} else if (pBest->flags) {
@@ -1816,7 +1808,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
context->is_aborted = true;
return;
}
- if (sql_value_is_null(argv[0]))
+ if (mem_is_null(argv[0]))
return;
pAccum =
(StrAccum *) sql_aggregate_context(context, sizeof(*pAccum));
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index e9062b602..5eef15c62 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1044,8 +1044,7 @@ mem_convert_to_integer(struct Mem *mem)
int
mem_convert_to_numeric(struct Mem *mem, enum field_type type)
{
- assert(mp_type_is_numeric(mem_mp_type(mem)) &&
- sql_type_is_numeric(type));
+ assert(mem_is_num(mem) && sql_type_is_numeric(type));
assert(type != FIELD_TYPE_NUMBER);
if (type == FIELD_TYPE_DOUBLE)
return mem_convert_to_double(mem);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 82b3084fb..041d8a414 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,21 +87,6 @@ struct Mem {
*/
#define MEMCELLSIZE offsetof(Mem,zMalloc)
-/**
- * Return a string that represent content of MEM. String is either allocated
- * using static_alloc() of just a static variable.
- */
-const char *
-mem_str(const struct Mem *mem);
-
-/** Initialize MEM and set NULL. */
-void
-mem_create(struct Mem *mem);
-
-/** Free all allocated memory in MEM and set MEM to NULL. */
-void
-mem_destroy(struct Mem *mem);
-
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -154,6 +139,173 @@ static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
"value of type mask must consist of corresponding to memory "\
"type bits");
+static inline bool
+mem_is_null(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Null) != 0;
+}
+
+static inline bool
+mem_is_uint(const struct Mem *mem)
+{
+ return (mem->flags & MEM_UInt) != 0;
+}
+
+static inline bool
+mem_is_nint(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Int) != 0;
+}
+
+static inline bool
+mem_is_str(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Str) != 0;
+}
+
+static inline bool
+mem_is_num(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0;
+}
+
+static inline bool
+mem_is_double(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Real) != 0;
+}
+
+static inline bool
+mem_is_int(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Int | MEM_UInt)) != 0;
+}
+
+static inline bool
+mem_is_bool(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Bool) != 0;
+}
+
+static inline bool
+mem_is_bin(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Blob) != 0;
+}
+
+static inline bool
+mem_is_map(const struct Mem *mem)
+{
+ assert((mem->flags & MEM_Subtype) == 0 || (mem->flags & MEM_Blob) != 0);
+ assert((mem->flags & MEM_Subtype) == 0 ||
+ mem->subtype == SQL_SUBTYPE_MSGPACK);
+ return (mem->flags & MEM_Subtype) != 0 && mp_typeof(*mem->z) == MP_MAP;
+}
+
+static inline bool
+mem_is_array(const struct Mem *mem)
+{
+ assert((mem->flags & MEM_Subtype) == 0 || (mem->flags & MEM_Blob) != 0);
+ assert((mem->flags & MEM_Subtype) == 0 ||
+ mem->subtype == SQL_SUBTYPE_MSGPACK);
+ return (mem->flags & MEM_Subtype) != 0 &&
+ mp_typeof(*mem->z) == MP_ARRAY;
+}
+
+static inline bool
+mem_is_agg(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Agg) != 0;
+}
+
+static inline bool
+mem_is_bytes(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Blob | MEM_Str)) != 0;
+}
+
+static inline bool
+mem_is_frame(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Frame) != 0;
+}
+
+static inline bool
+mem_is_invalid(const struct Mem *mem)
+{
+ return (mem->flags & MEM_Undefined) != 0;
+}
+
+static inline bool
+mem_is_static(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Static) != 0;
+}
+
+static inline bool
+mem_is_ephemeral(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Ephem) != 0;
+}
+
+static inline bool
+mem_is_dynamic(const struct Mem *mem)
+{
+ assert((mem->flags & (MEM_Str | MEM_Blob)) != 0);
+ return (mem->flags & MEM_Dyn) != 0;
+}
+
+static inline bool
+mem_is_allocated(const struct Mem *mem)
+{
+ return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == mem->zMalloc;
+}
+
+static inline bool
+mem_is_cleared(const struct Mem *mem)
+{
+ assert((mem->flags & MEM_Cleared) == 0 || (mem->flags & MEM_Null) != 0);
+ return (mem->flags & MEM_Cleared) != 0;
+}
+
+static inline bool
+mem_is_zerobin(const struct Mem *mem)
+{
+ assert((mem->flags & MEM_Zero) == 0 || (mem->flags & MEM_Blob) != 0);
+ return (mem->flags & MEM_Zero) != 0;
+}
+
+static inline bool
+mem_is_same_type(const struct Mem *mem1, const struct Mem *mem2)
+{
+ return (mem1->flags & MEM_PURE_TYPE_MASK) ==
+ (mem2->flags & MEM_PURE_TYPE_MASK);
+}
+
+static inline bool
+mem_is_any_null(const struct Mem *mem1, const struct Mem *mem2)
+{
+ return ((mem1->flags | mem2->flags) & MEM_Null) != 0;
+}
+
+/**
+ * Return a string that represent content of MEM. String is either allocated
+ * using static_alloc() of just a static variable.
+ */
+const char *
+mem_str(const struct Mem *mem);
+
+/** Initialize MEM and set NULL. */
+void
+mem_create(struct Mem *mem);
+
+/** Free all allocated memory in MEM and set MEM to NULL. */
+void
+mem_destroy(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -375,12 +527,6 @@ columnNullValue(void);
/** Checkers. */
-static inline bool
-sql_value_is_null(struct Mem *value)
-{
- return sql_value_type(value) == MP_NIL;
-}
-
int sqlVdbeMemTooBig(Mem *);
/* Return TRUE if Mem X contains dynamically allocated content - anything
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 0fa388ae9..b9107fccc 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1092,7 +1092,7 @@ selectInnerLoop(Parse * pParse, /* The parser context */
* re-use second for Null op-code.
*
* Change to an OP_Null sets the
- * MEM_Cleared bit on the first
+ * Cleared flag on the first
* register of the previous value.
* This will cause the OP_Ne below
* to always fail on the first
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7cc72dc38..eb7d77f7e 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -120,8 +120,8 @@ int sql_sort_count = 0;
#endif
/*
- * The next global variable records the size of the largest MEM_Blob
- * or MEM_Str that has been used by a VDBE opcode. The test procedures
+ * The next global variable records the size of the largest varbinary
+ * or string that has been used by a VDBE opcode. The test procedures
* use this information to make sure that the zero-blob functionality
* is working correctly. This variable has no function other than to
* help verify the correct operation of the library.
@@ -131,9 +131,8 @@ int sql_max_blobsize = 0;
static void
updateMaxBlobsize(Mem *p)
{
- if ((p->flags & (MEM_Str|MEM_Blob))!=0 && p->n>sql_max_blobsize) {
+ if (mem_is_bytes(p) && p->n > sql_max_blobsize)
sql_max_blobsize = p->n;
- }
}
#endif
@@ -425,8 +424,7 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
* Add 0 termination (at most for strings)
* Not sure why do we check MEM_Ephem
*/
- if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) ==
- (MEM_Ephem | MEM_Str)) {
+ if (mem_is_str(dest_mem) && mem_is_ephemeral(dest_mem)) {
int len = dest_mem->n;
if (dest_mem->szMalloc < len + 1) {
if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0)
@@ -659,7 +657,7 @@ case OP_Gosub: { /* jump */
*/
case OP_Return: { /* in1 */
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags==MEM_UInt);
+ assert(mem_is_uint(pIn1));
pOp = &aOp[pIn1->u.u];
pIn1->flags = MEM_Undefined;
break;
@@ -698,7 +696,7 @@ case OP_InitCoroutine: { /* jump */
case OP_EndCoroutine: { /* in1 */
VdbeOp *pCaller;
pIn1 = &aMem[pOp->p1];
- assert(pIn1->flags == MEM_UInt);
+ assert(mem_is_uint(pIn1));
assert(pIn1->u.u < (uint64_t) p->nOp);
pCaller = &aOp[pIn1->u.u];
assert(pCaller->opcode==OP_Yield);
@@ -1108,7 +1106,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
pIn2 = &aMem[pOp->p2];
pOut = vdbe_prepare_null_out(p, pOp->p3);
assert(pIn1!=pOut);
- if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn2)) {
/* Force NULL be of type STRING. */
pOut->field_type = FIELD_TYPE_STRING;
break;
@@ -1117,10 +1115,8 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* Concatenation operation can be applied only to
* strings and blobs.
*/
- uint32_t str_type_p1 = pIn1->flags & (MEM_Blob | MEM_Str);
- uint32_t str_type_p2 = pIn2->flags & (MEM_Blob | MEM_Str);
- if (str_type_p1 == 0 || str_type_p2 == 0) {
- char *inconsistent_type = str_type_p1 == 0 ?
+ if (!mem_is_bytes(pIn1) || !mem_is_bytes(pIn2)) {
+ char *inconsistent_type = !mem_is_bytes(pIn1) ?
mem_type_to_str(pIn1) :
mem_type_to_str(pIn2);
diag_set(ClientError, ER_INCONSISTENT_TYPES,
@@ -1129,7 +1125,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
}
/* Moreover, both operands must be of the same type. */
- if (str_type_p1 != str_type_p2) {
+ if (!mem_is_same_type(pIn1, pIn2)) {
diag_set(ClientError, ER_INCONSISTENT_TYPES,
mem_type_to_str(pIn2), mem_type_to_str(pIn1));
goto abort_due_to_error;
@@ -1143,7 +1139,7 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
goto no_mem;
}
- if (pIn1->flags & MEM_Str)
+ if (mem_is_str(pIn1))
MemSetTypeFlag(pOut, MEM_Str);
else
MemSetTypeFlag(pOut, MEM_Blob);
@@ -1202,7 +1198,6 @@ case OP_Subtract: /* same as TK_MINUS, in1, in2, out3 */
case OP_Multiply: /* same as TK_STAR, in1, in2, out3 */
case OP_Divide: /* same as TK_SLASH, in1, in2, out3 */
case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
- u32 flags; /* Combined MEM_* flags from both inputs */
u16 type1; /* Numeric type of left operand */
u16 type2; /* Numeric type of right operand */
i64 iA; /* Integer value of left operand */
@@ -1215,14 +1210,14 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
pIn2 = &aMem[pOp->p2];
type2 = numericType(pIn2);
pOut = vdbe_prepare_null_out(p, pOp->p3);
- flags = pIn1->flags | pIn2->flags;
- if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
+ if (mem_is_any_null(pIn1, pIn2))
+ goto arithmetic_result_is_null;
if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
(type2 & (MEM_Int | MEM_UInt)) != 0) {
iA = pIn1->u.i;
iB = pIn2->u.i;
- bool is_lhs_neg = pIn1->flags & MEM_Int;
- bool is_rhs_neg = pIn2->flags & MEM_Int;
+ bool is_lhs_neg = mem_is_nint(pIn1);
+ bool is_rhs_neg = mem_is_nint(pIn2);
bool is_res_neg;
switch( pOp->opcode) {
case OP_Add: {
@@ -1429,7 +1424,7 @@ case OP_BuiltinFunction: {
goto abort_due_to_error;
/* Copy the result of the function into register P3 */
- if (pOut->flags & (MEM_Str|MEM_Blob)) {
+ if (mem_is_bytes(pOut)) {
if (sqlVdbeMemTooBig(pCtx->pOut)) goto too_big;
}
@@ -1488,7 +1483,7 @@ case OP_FunctionByName: {
* Copy the result of the function invocation into
* register P3.
*/
- if ((pOut->flags & (MEM_Str | MEM_Blob)) != 0)
+ if (mem_is_bytes(pOut))
if (sqlVdbeMemTooBig(pOut)) goto too_big;
REGISTER_TRACE(p, pOp->p3, pOut);
@@ -1538,7 +1533,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
pIn1 = &aMem[pOp->p1];
pIn2 = &aMem[pOp->p2];
pOut = vdbe_prepare_null_out(p, pOp->p3);
- if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn2)) {
/* Force NULL be of type INTEGER. */
pOut->field_type = FIELD_TYPE_INTEGER;
break;
@@ -1597,7 +1592,7 @@ case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
case OP_AddImm: { /* in1 */
pIn1 = &aMem[pOp->p1];
memAboutToChange(p, pIn1);
- assert((pIn1->flags & MEM_UInt) != 0 && pOp->p2 >= 0);
+ assert(mem_is_uint(pIn1) && pOp->p2 >= 0);
pIn1->u.u += pOp->p2;
break;
}
@@ -1611,9 +1606,9 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+ if (!mem_is_int(pIn1)) {
mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+ if (!mem_is_int(pIn1)) {
if (pOp->p2==0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(pIn1), "integer");
@@ -1637,7 +1632,7 @@ case OP_MustBeInt: { /* jump, in1 */
*/
case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
- if ((pIn1->flags & (MEM_Int | MEM_UInt)) != 0) {
+ if (mem_is_int(pIn1)) {
sqlVdbeMemRealify(pIn1);
}
break;
@@ -1778,7 +1773,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
flags3 = pIn3->flags;
enum field_type ft_p1 = pIn1->field_type;
enum field_type ft_p3 = pIn3->field_type;
- if ((flags1 | flags3)&MEM_Null) {
+ if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
/* If SQL_NULLEQ is set (which will only happen if the operator is
@@ -1786,11 +1781,10 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
* or not both operands are null.
*/
assert(pOp->opcode==OP_Eq || pOp->opcode==OP_Ne);
- assert((flags1 & MEM_Cleared)==0);
+ assert(!mem_is_cleared(pIn1));
assert((pOp->p5 & SQL_JUMPIFNULL)==0);
- if ((flags1&flags3&MEM_Null)!=0
- && (flags3&MEM_Cleared)==0
- ) {
+ if (mem_is_same_type(pIn1, pIn3) &&
+ !mem_is_cleared(pIn3)) {
res = 0; /* Operands are equal */
} else {
res = 1; /* Operands are not equal */
@@ -1812,22 +1806,17 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
- } else if (((flags1 | flags3) & MEM_Bool) != 0 ||
- ((flags1 | flags3) & MEM_Blob) != 0) {
- /*
- * If one of values is of type BOOLEAN or VARBINARY,
- * then the second one must be of the same type as
- * well. Otherwise an error is raised.
- */
- int type_arg1 = flags1 & (MEM_Bool | MEM_Blob);
- int type_arg3 = flags3 & (MEM_Bool | MEM_Blob);
- if (type_arg1 != type_arg3) {
- char *inconsistent_type = type_arg1 != 0 ?
+ } else if (mem_is_bool(pIn1) || mem_is_bool(pIn3) ||
+ mem_is_bin(pIn1) || mem_is_bin(pIn3)) {
+ if (!mem_is_same_type(pIn1, pIn3)) {
+ char *inconsistent_type = mem_is_bool(pIn1) ||
+ mem_is_bin(pIn1) ?
mem_type_to_str(pIn3) :
mem_type_to_str(pIn1);
- char *expected_type = type_arg1 != 0 ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn3);
+ char *expected_type = mem_is_bool(pIn1) ||
+ mem_is_bin(pIn1) ?
+ mem_type_to_str(pIn1) :
+ mem_type_to_str(pIn3);
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
inconsistent_type, expected_type);
goto abort_due_to_error;
@@ -1836,28 +1825,24 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
} else {
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (sql_type_is_numeric(type)) {
- if ((flags1 | flags3)&MEM_Str) {
- if ((flags1 & MEM_Str) == MEM_Str) {
- mem_apply_numeric_type(pIn1);
- testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
- flags3 = pIn3->flags;
- }
- if ((flags3 & MEM_Str) == MEM_Str) {
- if (mem_apply_numeric_type(pIn3) != 0) {
- diag_set(ClientError,
- ER_SQL_TYPE_MISMATCH,
- mem_str(pIn3),
- "numeric");
- goto abort_due_to_error;
- }
-
+ if (mem_is_str(pIn1)) {
+ mem_apply_numeric_type(pIn1);
+ flags3 = pIn3->flags;
+ }
+ if (mem_is_str(pIn3)) {
+ if (mem_apply_numeric_type(pIn3) != 0) {
+ diag_set(ClientError,
+ ER_SQL_TYPE_MISMATCH,
+ mem_str(pIn3),
+ "numeric");
+ goto abort_due_to_error;
}
}
/* Handle the common case of integer comparison here, as an
* optimization, to avoid a call to sqlMemCompare()
*/
- if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt)) != 0) {
- if ((pIn1->flags & pIn3->flags & MEM_Int) != 0) {
+ if (mem_is_int(pIn1) && mem_is_int(pIn3)) {
+ if (mem_is_nint(pIn1) && mem_is_nint(pIn3)) {
if (pIn3->u.i > pIn1->u.i)
res = +1;
else if (pIn3->u.i < pIn1->u.i)
@@ -1866,7 +1851,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- if ((pIn1->flags & pIn3->flags & MEM_UInt) != 0) {
+ if (mem_is_uint(pIn1) && mem_is_uint(pIn3)) {
if (pIn3->u.u > pIn1->u.u)
res = +1;
else if (pIn3->u.u < pIn1->u.u)
@@ -1875,8 +1860,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- if ((pIn1->flags & MEM_UInt) != 0 &&
- (pIn3->flags & MEM_Int) != 0) {
+ if (mem_is_uint(pIn1) && mem_is_nint(pIn3)) {
res = -1;
goto compare_op;
}
@@ -1884,21 +1868,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
goto compare_op;
}
} else if (type == FIELD_TYPE_STRING) {
- if ((flags1 & MEM_Str) == 0 &&
- (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- testcase( pIn1->flags & MEM_Int);
- testcase( pIn1->flags & MEM_Real);
+ if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
sqlVdbeMemStringify(pIn1);
- testcase( (flags1&MEM_Dyn) != (pIn1->flags&MEM_Dyn));
flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
assert(pIn1!=pIn3);
}
- if ((flags3 & MEM_Str) == 0 &&
- (flags3 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- testcase( pIn3->flags & MEM_Int);
- testcase( pIn3->flags & MEM_Real);
+ if (!mem_is_str(pIn3) && mem_is_num(pIn3)) {
sqlVdbeMemStringify(pIn3);
- testcase( (flags3&MEM_Dyn) != (pIn3->flags&MEM_Dyn));
flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
}
}
@@ -2106,9 +2082,9 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
int v2; /* Right operand: 0==FALSE, 1==TRUE, 2==UNKNOWN or NULL */
pIn1 = &aMem[pOp->p1];
- if (pIn1->flags & MEM_Null) {
+ if (mem_is_null(pIn1)) {
v1 = 2;
- } else if ((pIn1->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn1)) {
v1 = pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2116,9 +2092,9 @@ case OP_Or: { /* same as TK_OR, in1, in2, out3 */
goto abort_due_to_error;
}
pIn2 = &aMem[pOp->p2];
- if (pIn2->flags & MEM_Null) {
+ if (mem_is_null(pIn2)) {
v2 = 2;
- } else if ((pIn2->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn2)) {
v2 = pIn2->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2149,8 +2125,8 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
pIn1 = &aMem[pOp->p1];
pOut = vdbe_prepare_null_out(p, pOp->p2);
pOut->field_type = FIELD_TYPE_BOOLEAN;
- if ((pIn1->flags & MEM_Null)==0) {
- if ((pIn1->flags & MEM_Bool) == 0) {
+ if (!mem_is_null(pIn1)) {
+ if (!mem_is_bool(pIn1)) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(pIn1), "boolean");
goto abort_due_to_error;
@@ -2172,7 +2148,7 @@ case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
pOut = vdbe_prepare_null_out(p, pOp->p2);
/* Force NULL be of type INTEGER. */
pOut->field_type = FIELD_TYPE_INTEGER;
- if ((pIn1->flags & MEM_Null)==0) {
+ if (!mem_is_null(pIn1)) {
int64_t i;
bool is_neg;
if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
@@ -2217,9 +2193,9 @@ case OP_If: /* jump, in1 */
case OP_IfNot: { /* jump, in1 */
int c;
pIn1 = &aMem[pOp->p1];
- if (pIn1->flags & MEM_Null) {
+ if (mem_is_null(pIn1)) {
c = pOp->p3;
- } else if ((pIn1->flags & MEM_Bool) != 0) {
+ } else if (mem_is_bool(pIn1)) {
c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
} else {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2240,8 +2216,8 @@ case OP_IfNot: { /* jump, in1 */
*/
case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */
pIn1 = &aMem[pOp->p1];
- VdbeBranchTaken( (pIn1->flags & MEM_Null)!=0, 2);
- if ((pIn1->flags & MEM_Null)!=0) {
+ VdbeBranchTaken(mem_is_null(pIn1), 2);
+ if (mem_is_null(pIn1)) {
goto jump_to_p2;
}
break;
@@ -2254,8 +2230,8 @@ case OP_IsNull: { /* same as TK_ISNULL, jump, in1 */
*/
case OP_NotNull: { /* same as TK_NOTNULL, jump, in1 */
pIn1 = &aMem[pOp->p1];
- VdbeBranchTaken( (pIn1->flags & MEM_Null)==0, 2);
- if ((pIn1->flags & MEM_Null)==0) {
+ VdbeBranchTaken(!mem_is_null(pIn1), 2);
+ if (!mem_is_null(pIn1)) {
goto jump_to_p2;
}
break;
@@ -2309,7 +2285,7 @@ case OP_Column: {
if (pC->eCurType==CURTYPE_PSEUDO) {
assert(pC->uc.pseudoTableReg>0);
pReg = &aMem[pC->uc.pseudoTableReg];
- assert(pReg->flags & MEM_Blob);
+ assert(mem_is_bin(pReg));
assert(memIsValid(pReg));
vdbe_field_ref_prepare_data(&pC->field_ref,
pReg->z, pReg->n);
@@ -2338,7 +2314,7 @@ case OP_Column: {
if (vdbe_field_ref_fetch(&pC->field_ref, p2, pDest) != 0)
goto abort_due_to_error;
- if ((pDest->flags & MEM_Null) &&
+ if (mem_is_null(pDest) &&
(uint32_t) p2 >= pC->field_ref.field_count &&
default_val_mem != NULL) {
sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
@@ -2393,7 +2369,7 @@ case OP_ApplyType: {
if (!sql_type_is_numeric(type))
goto type_mismatch;
/* Implicit cast is allowed only from numeric type. */
- if (!mp_type_is_numeric(mem_mp_type(pIn1)))
+ if (!mem_is_num(pIn1))
goto type_mismatch;
/* Try to convert numeric-to-numeric. */
if (mem_convert_to_numeric(pIn1, type) != 0)
@@ -3011,18 +2987,18 @@ case OP_SeekGT: { /* jump, in3 */
* the seek, so convert it.
*/
pIn3 = &aMem[int_field];
- if ((pIn3->flags & MEM_Null) != 0)
+ if (mem_is_null(pIn3))
goto skip_truncate;
- if ((pIn3->flags & MEM_Str) != 0)
+ if (mem_is_str(pIn3))
mem_apply_numeric_type(pIn3);
int64_t i;
- if ((pIn3->flags & MEM_Int) == MEM_Int) {
- i = pIn3->u.i;
- is_neg = true;
- } else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
+ if (mem_is_uint(pIn3)) {
i = pIn3->u.u;
is_neg = false;
- } else if ((pIn3->flags & MEM_Real) == MEM_Real) {
+ } else if (mem_is_nint(pIn3)) {
+ i = pIn3->u.i;
+ is_neg = true;
+ } else if (mem_is_double(pIn3)) {
if (pIn3->u.r > (double)INT64_MAX)
i = INT64_MAX;
else if (pIn3->u.r < (double)INT64_MIN)
@@ -3040,8 +3016,8 @@ case OP_SeekGT: { /* jump, in3 */
/* If the P3 value could not be converted into an integer without
* loss of information, then special processing is required...
*/
- if ((pIn3->flags & (MEM_Int | MEM_UInt)) == 0) {
- if ((pIn3->flags & MEM_Real)==0) {
+ if (!mem_is_int(pIn3)) {
+ if (!mem_is_double(pIn3)) {
/* If the P3 value cannot be converted into any kind of a number,
* then the seek is not possible, so jump to P2
*/
@@ -3247,7 +3223,8 @@ case OP_Found: { /* jump, in3 */
#ifdef SQL_DEBUG
for(ii=0; ii<r.nField; ii++) {
assert(memIsValid(&r.aMem[ii]));
- assert((r.aMem[ii].flags & MEM_Zero)==0 || r.aMem[ii].n==0);
+ assert(!mem_is_zerobin(&r.aMem[ii]) ||
+ r.aMem[ii].n == 0);
if (ii != 0)
REGISTER_TRACE(p, pOp->p3+ii, &r.aMem[ii]);
}
@@ -3257,7 +3234,7 @@ case OP_Found: { /* jump, in3 */
} else {
pFree = pIdxKey = sqlVdbeAllocUnpackedRecord(db, pC->key_def);
if (pIdxKey==0) goto no_mem;
- assert(pIn3->flags & MEM_Blob );
+ assert(mem_is_bin(pIn3));
(void)ExpandBlob(pIn3);
sqlVdbeRecordUnpackMsgpack(pC->key_def,
pIn3->z, pIdxKey);
@@ -3271,7 +3248,7 @@ case OP_Found: { /* jump, in3 */
* conflict
*/
for(ii=0; ii<pIdxKey->nField; ii++) {
- if (pIdxKey->aMem[ii].flags & MEM_Null) {
+ if (mem_is_null(&pIdxKey->aMem[ii])) {
takeJump = 1;
break;
}
@@ -3381,11 +3358,11 @@ case OP_FCopy: { /* out2 */
pIn1 = &aMem[pOp->p1];
}
- if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && (pIn1->flags & MEM_Null)) {
+ if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) != 0 && mem_is_null(pIn1)) {
pOut = vdbe_prepare_null_out(p, pOp->p2);
} else {
assert(memIsValid(pIn1));
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+ assert(mem_is_int(pIn1));
pOut = vdbe_prepare_null_out(p, pOp->p2);
mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
@@ -3520,7 +3497,7 @@ case OP_SorterData: {
assert(isSorter(pC));
if (sqlVdbeSorterRowkey(pC, pOut) != 0)
goto abort_due_to_error;
- assert(pOut->flags & MEM_Blob);
+ assert(mem_is_bin(pOut));
assert(pOp->p1>=0 && pOp->p1<p->nCursor);
p->apCsr[pOp->p3]->cacheStatus = CACHE_STALE;
break;
@@ -3878,7 +3855,7 @@ case OP_SorterInsert: { /* in2 */
assert(cursor != NULL);
assert(isSorter(cursor));
pIn2 = &aMem[pOp->p2];
- assert((pIn2->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(pIn2));
if (ExpandBlob(pIn2) != 0 ||
sqlVdbeSorterWrite(cursor, pIn2) != 0)
goto abort_due_to_error;
@@ -3912,7 +3889,7 @@ case OP_SorterInsert: { /* in2 */
case OP_IdxReplace:
case OP_IdxInsert: {
pIn2 = &aMem[pOp->p1];
- assert((pIn2->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(pIn2));
if (ExpandBlob(pIn2) != 0)
goto abort_due_to_error;
struct space *space;
@@ -3923,7 +3900,7 @@ case OP_IdxInsert: {
assert(space != NULL);
if (space->def->id != 0) {
/* Make sure that memory has been allocated on region. */
- assert(aMem[pOp->p1].flags & MEM_Ephem);
+ assert(mem_is_ephemeral(&aMem[pOp->p1]));
if (pOp->opcode == OP_IdxInsert) {
rc = tarantoolsqlInsert(space, pIn2->z,
pIn2->z + pIn2->n);
@@ -3957,7 +3934,7 @@ case OP_IdxInsert: {
}
if ((pOp->p5 & OPFLAG_NCHANGE) != 0)
p->nChange++;
- if (pOp->p3 > 0 && ((aMem[pOp->p3].flags) & MEM_Null) != 0) {
+ if (pOp->p3 > 0 && mem_is_null(&aMem[pOp->p3])) {
assert(space->sequence != NULL);
int64_t value;
if (sequence_get_value(space->sequence, &value) != 0)
@@ -4003,10 +3980,10 @@ case OP_Update: {
assert(pOp->p4type == P4_SPACEPTR);
struct Mem *key_mem = &aMem[pOp->p2];
- assert((key_mem->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(key_mem));
struct Mem *upd_fields_mem = &aMem[pOp->p3];
- assert((upd_fields_mem->flags & MEM_Blob) != 0);
+ assert(mem_is_bin(upd_fields_mem));
uint32_t *upd_fields = (uint32_t *)upd_fields_mem->z;
uint32_t upd_fields_cnt = upd_fields_mem->n / sizeof(uint32_t);
@@ -4438,7 +4415,7 @@ case OP_Program: { /* jump */
* the trigger program. If this trigger has been fired before, then pRt
* is already allocated. Otherwise, it must be initialized.
*/
- if ((pRt->flags&MEM_Frame)==0) {
+ if (!mem_is_frame(pRt)) {
/* SubProgram.nMem is set to the number of memory cells used by the
* program stored in SubProgram.aOp. As well as these, one memory
* cell is required for each cursor used by the program. Set local
@@ -4578,8 +4555,8 @@ case OP_FkIfZero: { /* jump */
*/
case OP_IfPos: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
- if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
+ assert(mem_is_int(pIn1));
+ if (mem_is_uint(pIn1) && pIn1->u.u != 0) {
assert(pOp->p3 >= 0);
uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
/*
@@ -4613,8 +4590,8 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
pIn3 = &aMem[pOp->p3];
pOut = vdbe_prepare_null_out(p, pOp->p2);
- assert((pIn1->flags & MEM_UInt) != 0);
- assert((pIn3->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
+ assert(mem_is_uint(pIn3));
uint64_t x = pIn1->u.u;
uint64_t rhs = pIn3->u.u;
bool unused;
@@ -4637,7 +4614,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
*/
case OP_IfNotZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
if (pIn1->u.u > 0) {
pIn1->u.u--;
goto jump_to_p2;
@@ -4653,7 +4630,7 @@ case OP_IfNotZero: { /* jump, in1 */
*/
case OP_DecrJumpZero: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- assert((pIn1->flags & MEM_UInt) != 0);
+ assert(mem_is_uint(pIn1));
if (pIn1->u.u > 0)
pIn1->u.u--;
if (pIn1->u.u == 0) goto jump_to_p2;
@@ -4750,7 +4727,7 @@ case OP_AggStep: {
mem_destroy(&t);
goto abort_due_to_error;
}
- assert(t.flags==MEM_Null);
+ assert(mem_is_null(&t));
if (pCtx->skipFlag) {
assert(pOp[-1].opcode==OP_CollSeq);
i = pOp[-1].p1;
@@ -4776,7 +4753,7 @@ case OP_AggFinal: {
Mem *pMem;
assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
pMem = &aMem[pOp->p1];
- assert((pMem->flags & ~(MEM_Null|MEM_Agg))==0);
+ assert(mem_is_null(pMem) || mem_is_agg(pMem));
if (sql_vdbemem_finalize(pMem, pOp->p4.func) != 0)
goto abort_due_to_error;
UPDATE_MAX_BLOBSIZE(pMem);
@@ -4903,7 +4880,7 @@ case OP_SetSession: {
struct session_setting *setting = &session_settings[sid];
switch (setting->field_type) {
case FIELD_TYPE_BOOLEAN: {
- if ((pIn1->flags & MEM_Bool) == 0)
+ if (!mem_is_bool(pIn1))
goto invalid_type;
bool value = pIn1->u.b;
size_t size = mp_sizeof_bool(value);
@@ -4914,7 +4891,7 @@ case OP_SetSession: {
break;
}
case FIELD_TYPE_STRING: {
- if ((pIn1->flags & MEM_Str) == 0)
+ if (!mem_is_str(pIn1))
goto invalid_type;
const char *str = pIn1->z;
uint32_t size = mp_sizeof_str(pIn1->n);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index cbf32ccdf..b4ad8b774 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -333,17 +333,6 @@ int sqlVdbeList(Vdbe *);
int sqlVdbeHalt(Vdbe *);
-/**
- * In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
- * messagepacks are stored as binary string (i.e. featuring
- * MEM_Blob internal type).
- */
-#define mp_type_is_bloblike(X) ((X) == MP_BIN || (X) == MP_ARRAY || (X) == MP_MAP)
-
-/** Return TRUE if MP_type of X is numeric, FALSE otherwise. */
-#define mp_type_is_numeric(X) ((X) == MP_INT || (X) == MP_UINT ||\
- (X) == MP_DOUBLE)
-
const char *sqlOpcodeName(int);
int sqlVdbeCloseStatement(Vdbe *, int);
void sqlVdbeFrameDelete(VdbeFrame *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index a195f8dfd..2a3561d42 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -380,7 +380,7 @@ static SQL_NOINLINE void *
createAggContext(sql_context * p, int nByte)
{
Mem *pMem = p->pMem;
- assert((pMem->flags & MEM_Agg) == 0);
+ assert(!mem_is_agg(pMem));
if (nByte <= 0) {
sqlVdbeMemSetNull(pMem);
pMem->z = 0;
@@ -407,7 +407,7 @@ sql_aggregate_context(sql_context * p, int nByte)
assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
testcase(nByte < 0);
- if ((p->pMem->flags & MEM_Agg) == 0) {
+ if (!mem_is_agg(p->pMem)) {
return createAggContext(p, nByte);
} else {
return (void *)p->pMem->z;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index e49b9664b..7ccc6a957 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1253,7 +1253,7 @@ sqlVdbeList(Vdbe * p)
*/
assert(p->nMem > 9);
pSub = &p->aMem[9];
- if (pSub->flags & MEM_Blob) {
+ if (mem_is_bin(pSub)) {
/* On the first call to sql_step(), pSub will hold a NULL. It is
* initialized to a BLOB by the P4_SUBPROGRAM processing logic below
*/
@@ -1722,7 +1722,7 @@ Cleanup(Vdbe * p)
assert(p->apCsr[i] == 0);
if (p->aMem) {
for (i = 0; i < p->nMem; i++)
- assert(p->aMem[i].flags == MEM_Undefined);
+ assert(mem_is_invalid(&p->aMem[i]));
}
#endif
@@ -2330,7 +2330,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
assert(iVar > 0);
if (v) {
Mem *pMem = &v->aVar[iVar - 1];
- if (0 == (pMem->flags & MEM_Null)) {
+ if (!mem_is_null(pMem)) {
sql_value *pRet = sqlValueNew(v->db);
if (pRet) {
sqlVdbeMemCopy((Mem *) pRet, pMem);
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index 927f85559..a9a5f45af 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2218,7 +2218,7 @@ sqlVdbeSorterCompare(const VdbeCursor * pCsr, /* Sorter cursor */
pKey = vdbeSorterRowkey(pSorter, &nKey);
sqlVdbeRecordUnpackMsgpack(pCsr->key_def, pKey, r2);
for (i = 0; i < nKeyCol; i++) {
- if (r2->aMem[i].flags & MEM_Null) {
+ if (mem_is_null(&r2->aMem[i])) {
*pRes = -1;
return 0;
}
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 0c002dbee..93de722cb 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -312,9 +312,8 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
pVal =
sqlVdbeGetBoundValue(pReprepare, iCol,
FIELD_TYPE_SCALAR);
- if (pVal && sql_value_type(pVal) == MP_STR) {
+ if (pVal != NULL && mem_is_str(pVal))
z = (char *)sql_value_text(pVal);
- }
assert(pRight->op == TK_VARIABLE || pRight->op == TK_REGISTER);
} else if (op == TK_STRING) {
z = pRight->u.zToken;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions()
2021-04-13 16:09 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 22:48 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 23:07 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 22:48 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Thanks for working on this!
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index 82b3084fb..041d8a414 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -87,21 +87,6 @@ struct Mem {
> */
> #define MEMCELLSIZE offsetof(Mem,zMalloc)
>
> -/**
> - * Return a string that represent content of MEM. String is either allocated
> - * using static_alloc() of just a static variable.
> - */
> -const char *
> -mem_str(const struct Mem *mem);
> -
> -/** Initialize MEM and set NULL. */
> -void
> -mem_create(struct Mem *mem);
> -
> -/** Free all allocated memory in MEM and set MEM to NULL. */
> -void
> -mem_destroy(struct Mem *mem);
Why did you move them? They were just introduced in the previous
commits.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions()
2021-04-14 22:48 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 23:07 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 23:07 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer below.
On Thu, Apr 15, 2021 at 12:48:34AM +0200, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> > diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> > index 82b3084fb..041d8a414 100644
> > --- a/src/box/sql/mem.h
> > +++ b/src/box/sql/mem.h
> > @@ -87,21 +87,6 @@ struct Mem {
> > */
> > #define MEMCELLSIZE offsetof(Mem,zMalloc)
> >
> > -/**
> > - * Return a string that represent content of MEM. String is either allocated
> > - * using static_alloc() of just a static variable.
> > - */
> > -const char *
> > -mem_str(const struct Mem *mem);
> > -
> > -/** Initialize MEM and set NULL. */
> > -void
> > -mem_create(struct Mem *mem);
> > -
> > -/** Free all allocated memory in MEM and set MEM to NULL. */
> > -void
> > -mem_destroy(struct Mem *mem);
>
> Why did you move them? They were just introduced in the previous
> commits.
Actually, I moved MEM_Flags since they should be defined before mem_is_*()
functions. Looks like for git moving these functions were more appropriate
solution.
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (11 preceding siblings ...)
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:36 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:06 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
` (28 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:36 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:01, Vladislav Shpilevoy wrote:
> I appreciate the work you did here!
>
> See 2 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_copy(). This function copies value from source
>> MEM to destination MEM. In case value is string or binary and have not
>> static allocation type, it is copied to newly allocated memory.
>>
>> Part of #5818
>> ---
>> src/box/sql/func.c | 4 ++--
>> src/box/sql/mem.c | 54 +++++++++++++++++++++++++------------------
>> src/box/sql/mem.h | 9 +++++++-
>> src/box/sql/vdbeapi.c | 2 +-
>> src/box/sql/vdbeaux.c | 2 +-
>> 5 files changed, 44 insertions(+), 27 deletions(-)
>>
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index 81b537d9b..6b6081150 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -2079,13 +2079,13 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
>> bool is_max = (func->flags & SQL_FUNC_MAX) != 0;
>> cmp = sqlMemCompare(pBest, pArg, pColl);
>> if ((is_max && cmp < 0) || (!is_max && cmp > 0)) {
>> - sqlVdbeMemCopy(pBest, pArg);
>
> 1. Still is "used" in stat4ValueFromExpr().
>
Fixed.
>> + mem_copy(pBest, pArg);
>> } else {
>> sqlSkipAccumulatorLoad(context);
>> }
>> } else {
>> pBest->db = sql_context_db_handle(context);
>> - sqlVdbeMemCopy(pBest, pArg);
>> + mem_copy(pBest, pArg);
>> }
>> }
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index abc9291ef..f12441d7c 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -257,6 +257,38 @@ mem_destroy(struct Mem *mem)
>> mem->zMalloc = NULL;
>> }
>>
>> +int
>> +mem_copy(struct Mem *to, const struct Mem *from)
>> +{
>> + mem_clear(to);
>> + to->u = from->u;
>> + to->flags = from->flags;
>> + to->subtype = from->subtype;
>> + to->field_type = from->field_type;
>> + to->n = from->n;
>> + to->z = from->z;
>> + if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
>> + return 0;
>> + if ((to->flags & MEM_Static) != 0)
>> + return 0;
>> + if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob)) {
>> + if (sqlVdbeMemExpandBlob(to) != 0)
>> + return -1;
>> + return 0;
>
> 2. You can make `returnsqlVdbeMemExpandBlob(to);`, no need to check its result.
>
Fixed.
> Also what was wrong with sqlVdbeMemCopy's way of using sqlVdbeMemMakeWriteable?
>
I see that this as a hack. It changes dynamic or allocated type (only type!) to
ephemeral and then calls sqlVdbeMemMakeWriteable(), which converts ephemeral
value to allocated value. Isn't it better to just directly copy?
>> + }
>> + if (to->szMalloc == 0)
>> + to->zMalloc = sqlDbMallocRaw(to->db, to->n);
>> + else
>> + to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
>> + if (to->zMalloc == NULL)
>> + return -1;
>> + to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
>> + memcpy(to->zMalloc, to->z, to->n);
>> + to->z = to->zMalloc;
>> + to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
>> + return 0;
>> +}
>> +
>> static inline bool
>> mem_has_msgpack_subtype(struct Mem *mem)
>> {
New patch:
commit 14bb43fcd1f34bef5c92279ae6c274d1e437cc86
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 23 00:53:03 2021 +0300
sql: introduce mem_copy()
This patch introduces mem_copy(). This function copies value from source
MEM to destination MEM. In case value is string or binary and have not
static allocation type, it is copied to newly allocated memory.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index a0108220f..0b85bf365 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1769,13 +1769,13 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
bool is_max = (func->flags & SQL_FUNC_MAX) != 0;
cmp = sqlMemCompare(pBest, pArg, pColl);
if ((is_max && cmp < 0) || (!is_max && cmp > 0)) {
- sqlVdbeMemCopy(pBest, pArg);
+ mem_copy(pBest, pArg);
} else {
sqlSkipAccumulatorLoad(context);
}
} else {
pBest->db = sql_context_db_handle(context);
- sqlVdbeMemCopy(pBest, pArg);
+ mem_copy(pBest, pArg);
}
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 25b2e75ee..ea3917fe3 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -267,6 +267,35 @@ mem_destroy(struct Mem *mem)
mem->zMalloc = NULL;
}
+int
+mem_copy(struct Mem *to, const struct Mem *from)
+{
+ mem_clear(to);
+ to->u = from->u;
+ to->flags = from->flags;
+ to->subtype = from->subtype;
+ to->field_type = from->field_type;
+ to->n = from->n;
+ to->z = from->z;
+ if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
+ return 0;
+ if ((to->flags & MEM_Static) != 0)
+ return 0;
+ if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob))
+ return sqlVdbeMemExpandBlob(to);
+ if (to->szMalloc == 0)
+ to->zMalloc = sqlDbMallocRaw(to->db, to->n);
+ else
+ to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
+ if (to->zMalloc == NULL)
+ return -1;
+ to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
+ memcpy(to->zMalloc, to->z, to->n);
+ to->z = to->zMalloc;
+ to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1960,28 +1989,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
return 0;
}
-/*
- * Make a full copy of pFrom into pTo. Prior contents of pTo are
- * freed before the copy is made.
- */
-int
-sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
-{
- int rc = 0;
-
- mem_clear(pTo);
- memcpy(pTo, pFrom, MEMCELLSIZE);
- pTo->flags &= ~MEM_Dyn;
- if (pTo->flags & (MEM_Str | MEM_Blob)) {
- if (0 == (pFrom->flags & MEM_Static)) {
- pTo->flags |= MEM_Ephem;
- rc = sqlVdbeMemMakeWriteable(pTo);
- }
- }
-
- return rc;
-}
-
void
sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index e4e586d4d..af36f31a2 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -168,6 +168,14 @@ mem_create(struct Mem *mem);
void
mem_destroy(struct Mem *mem);
+/**
+ * Copy content of MEM from one MEM to another. In case source MEM contains
+ * string or binary and allocation type is not STATIC, this value is copied to
+ * newly allocated by destination MEM memory.
+ */
+int
+mem_copy(struct Mem *to, const struct Mem *from);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -467,7 +475,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
int
vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-int sqlVdbeMemCopy(Mem *, const Mem *);
void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemMakeWriteable(Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index f054a0f43..abd49b9bb 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1008,11 +1008,7 @@ case OP_Copy: {
pOut = &aMem[pOp->p2];
assert(pOut!=pIn1);
while( 1) {
- sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
- Deephemeralize(pOut);
-#ifdef SQL_DEBUG
- pOut->pScopyFrom = 0;
-#endif
+ mem_copy(pOut, pIn1);
REGISTER_TRACE(p, pOp->p2+pOp->p3-n, pOut);
if ((n--)==0) break;
pOut++;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 2a3561d42..7951996ea 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -229,7 +229,7 @@ sql_result_text64(sql_context * pCtx,
void
sql_result_value(sql_context * pCtx, sql_value * pValue)
{
- sqlVdbeMemCopy(pCtx->pOut, pValue);
+ mem_copy(pCtx->pOut, pValue);
}
void
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 52e100454..bec8a532a 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2333,7 +2333,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
if (!mem_is_null(pMem)) {
sql_value *pRet = sqlValueNew(v->db);
if (pRet) {
- sqlVdbeMemCopy((Mem *) pRet, pMem);
+ mem_copy(pRet, pMem);
sql_value_apply_type(pRet, aff);
}
return pRet;
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index bb87bb902..91cba9962 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -415,8 +415,7 @@ stat4ValueFromExpr(Parse * pParse, /* Parse context */
if ((v = pParse->pReprepare) != 0) {
pVal = valueNew(db, pAlloc);
if (pVal) {
- rc = sqlVdbeMemCopy((Mem *) pVal,
- &v->aVar[iBindVar - 1]);
+ rc = mem_copy(pVal, &v->aVar[iBindVar - 1]);
if (rc == 0)
sql_value_apply_type(pVal, type);
pVal->db = pParse->db;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy()
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:06 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 16:18 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:06 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for the patch!
>> Also what was wrong with sqlVdbeMemCopy's way of using sqlVdbeMemMakeWriteable?
>>
> I see that this as a hack. It changes dynamic or allocated type (only type!) to
> ephemeral and then calls sqlVdbeMemMakeWriteable(), which converts ephemeral
> value to allocated value. Isn't it better to just directly copy?
Yes, your way sounds better.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 25b2e75ee..ea3917fe3 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -267,6 +267,35 @@ mem_destroy(struct Mem *mem)
> mem->zMalloc = NULL;
> }
>
> +int
> +mem_copy(struct Mem *to, const struct Mem *from)
> +{
> + mem_clear(to);
> + to->u = from->u;
> + to->flags = from->flags;
> + to->subtype = from->subtype;
> + to->field_type = from->field_type;
> + to->n = from->n;
> + to->z = from->z;
> + if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
> + return 0;
> + if ((to->flags & MEM_Static) != 0)
> + return 0;
> + if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob))
> + return sqlVdbeMemExpandBlob(to);
> + if (to->szMalloc == 0)
> + to->zMalloc = sqlDbMallocRaw(to->db, to->n);
> + else
> + to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
You can call realloc always. It turns into malloc when
the pointer is NULL, which is the case for szMalloc == 0
I think.
> + if (to->zMalloc == NULL)
> + return -1;
> + to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
> + memcpy(to->zMalloc, to->z, to->n);
> + to->z = to->zMalloc;
> + to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy()
2021-04-11 18:06 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 16:18 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 16:18 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Sun, Apr 11, 2021 at 08:06:24PM +0200, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> >> Also what was wrong with sqlVdbeMemCopy's way of using sqlVdbeMemMakeWriteable?
> >>
> > I see that this as a hack. It changes dynamic or allocated type (only type!) to
> > ephemeral and then calls sqlVdbeMemMakeWriteable(), which converts ephemeral
> > value to allocated value. Isn't it better to just directly copy?
>
> Yes, your way sounds better.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 25b2e75ee..ea3917fe3 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -267,6 +267,35 @@ mem_destroy(struct Mem *mem)
> > mem->zMalloc = NULL;
> > }
> >
> > +int
> > +mem_copy(struct Mem *to, const struct Mem *from)
> > +{
> > + mem_clear(to);
> > + to->u = from->u;
> > + to->flags = from->flags;
> > + to->subtype = from->subtype;
> > + to->field_type = from->field_type;
> > + to->n = from->n;
> > + to->z = from->z;
> > + if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
> > + return 0;
> > + if ((to->flags & MEM_Static) != 0)
> > + return 0;
> > + if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob))
> > + return sqlVdbeMemExpandBlob(to);
> > + if (to->szMalloc == 0)
> > + to->zMalloc = sqlDbMallocRaw(to->db, to->n);
> > + else
> > + to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
>
> You can call realloc always. It turns into malloc when
> the pointer is NULL, which is the case for szMalloc == 0
> I think.
>
Thanks, fixed.
> > + if (to->zMalloc == NULL)
> > + return -1;
> > + to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
> > + memcpy(to->zMalloc, to->z, to->n);
> > + to->z = to->zMalloc;
> > + to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
> > + return 0;
> > +}
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 78a4fe1a5..79b330141 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -141,10 +141,7 @@ mem_copy(struct Mem *to, const struct Mem *from)
return 0;
if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob))
return sqlVdbeMemExpandBlob(to);
- if (to->szMalloc == 0)
- to->zMalloc = sqlDbMallocRaw(to->db, to->n);
- else
- to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
+ to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
if (to->zMalloc == NULL)
return -1;
to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
New patch:
commit 38aceb6474bf967fc464561f5ea6e5e934257924
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 23 00:53:03 2021 +0300
sql: introduce mem_copy()
This patch introduces mem_copy(). This function copies value from source
MEM to destination MEM. In case value is string or binary and have not
static allocation type, it is copied to newly allocated memory.
Part of #5818
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index a0108220f..0b85bf365 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1769,13 +1769,13 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
bool is_max = (func->flags & SQL_FUNC_MAX) != 0;
cmp = sqlMemCompare(pBest, pArg, pColl);
if ((is_max && cmp < 0) || (!is_max && cmp > 0)) {
- sqlVdbeMemCopy(pBest, pArg);
+ mem_copy(pBest, pArg);
} else {
sqlSkipAccumulatorLoad(context);
}
} else {
pBest->db = sql_context_db_handle(context);
- sqlVdbeMemCopy(pBest, pArg);
+ mem_copy(pBest, pArg);
}
}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5eef15c62..79b330141 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -125,6 +125,32 @@ mem_destroy(struct Mem *mem)
mem->z = NULL;
}
+int
+mem_copy(struct Mem *to, const struct Mem *from)
+{
+ mem_clear(to);
+ to->u = from->u;
+ to->flags = from->flags;
+ to->subtype = from->subtype;
+ to->field_type = from->field_type;
+ to->n = from->n;
+ to->z = from->z;
+ if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
+ return 0;
+ if ((to->flags & MEM_Static) != 0)
+ return 0;
+ if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob))
+ return sqlVdbeMemExpandBlob(to);
+ to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
+ if (to->zMalloc == NULL)
+ return -1;
+ to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
+ memcpy(to->zMalloc, to->z, to->n);
+ to->z = to->zMalloc;
+ to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1818,28 +1844,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
return 0;
}
-/*
- * Make a full copy of pFrom into pTo. Prior contents of pTo are
- * freed before the copy is made.
- */
-int
-sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
-{
- int rc = 0;
-
- mem_clear(pTo);
- memcpy(pTo, pFrom, MEMCELLSIZE);
- pTo->flags &= ~MEM_Dyn;
- if (pTo->flags & (MEM_Str | MEM_Blob)) {
- if (0 == (pFrom->flags & MEM_Static)) {
- pTo->flags |= MEM_Ephem;
- rc = sqlVdbeMemMakeWriteable(pTo);
- }
- }
-
- return rc;
-}
-
void
sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 041d8a414..86dcdaec0 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -306,6 +306,14 @@ mem_create(struct Mem *mem);
void
mem_destroy(struct Mem *mem);
+/**
+ * Copy content of MEM from one MEM to another. In case source MEM contains
+ * string or binary and allocation type is not STATIC, this value is copied to
+ * newly allocated by destination MEM memory.
+ */
+int
+mem_copy(struct Mem *to, const struct Mem *from);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -553,7 +561,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
int
vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-int sqlVdbeMemCopy(Mem *, const Mem *);
void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemMakeWriteable(Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index eb7d77f7e..a4718cef7 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1008,11 +1008,7 @@ case OP_Copy: {
pOut = &aMem[pOp->p2];
assert(pOut!=pIn1);
while( 1) {
- sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
- Deephemeralize(pOut);
-#ifdef SQL_DEBUG
- pOut->pScopyFrom = 0;
-#endif
+ mem_copy(pOut, pIn1);
REGISTER_TRACE(p, pOp->p2+pOp->p3-n, pOut);
if ((n--)==0) break;
pOut++;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 2a3561d42..7951996ea 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -229,7 +229,7 @@ sql_result_text64(sql_context * pCtx,
void
sql_result_value(sql_context * pCtx, sql_value * pValue)
{
- sqlVdbeMemCopy(pCtx->pOut, pValue);
+ mem_copy(pCtx->pOut, pValue);
}
void
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 7ccc6a957..cd89fc899 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2333,7 +2333,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
if (!mem_is_null(pMem)) {
sql_value *pRet = sqlValueNew(v->db);
if (pRet) {
- sqlVdbeMemCopy((Mem *) pRet, pMem);
+ mem_copy(pRet, pMem);
sql_value_apply_type(pRet, aff);
}
return pRet;
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index bb87bb902..91cba9962 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -415,8 +415,7 @@ stat4ValueFromExpr(Parse * pParse, /* Parse context */
if ((v = pParse->pReprepare) != 0) {
pVal = valueNew(db, pAlloc);
if (pVal) {
- rc = sqlVdbeMemCopy((Mem *) pVal,
- &v->aVar[iBindVar - 1]);
+ rc = mem_copy(pVal, &v->aVar[iBindVar - 1]);
if (rc == 0)
sql_value_apply_type(pVal, type);
pVal->db = pParse->db;
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (12 preceding siblings ...)
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:36 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:37 ` [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move() Mergen Imeev via Tarantool-patches
` (27 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:36 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:01, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 3 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch intoduces mem_copy_as_ephemeral(). This function copies value
>> from source MEM to destination MEM. In case value is of string or binary
>> type and its allocation type is not static, it copied as value with
>> ephemeral allocation type.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 58 ++++++++++++++++++++--------------------------
>> src/box/sql/mem.h | 15 ++++++------
>> src/box/sql/vdbe.c | 10 ++++----
>> 3 files changed, 38 insertions(+), 45 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index f12441d7c..30c568970 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -289,6 +289,25 @@ mem_copy(struct Mem *to, const struct Mem *from)
>> return 0;
>> }
>>
>> +int
>> +mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
>
> 1. Why didn't you keep its return type void?
>
Fixed, now this function has void return type.
>> +{
>> + mem_clear(to);
>> + to->u = from->u;
>> + to->flags = from->flags;
>> + to->subtype = from->subtype;
>> + to->field_type = from->field_type;
>> + to->n = from->n;
>> + to->z = from->z;
>> + if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
>> + return 0;
>> + if ((to->flags & (MEM_Static | MEM_Ephem)) != 0)
>> + return 0;
>> + to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Zero | MEM_Subtype);
>> + to->flags |= MEM_Ephem;
>> + return 0;
>> +}
>> @@ -593,9 +598,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
>> int i;
>> Mem *pX;
>> for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
>> - if (pX->pScopyFrom == pMem) {
>> - pX->flags |= MEM_Undefined;
>> - pX->pScopyFrom = 0;
>> + if ((pX->flags & (MEM_Blob | MEM_Str)) != 0 &&
>> + (pX->flags & (MEM_Ephem | MEM_Static)) == 0) {
>> + if (pX->pScopyFrom == pMem) {
>> + pX->flags |= MEM_Undefined;
>> + pX->pScopyFrom = 0;
>> + }
>
> 2. Why did you change that?
>
This check is only useful for strings and binaries, since they may be lost due
to change of another MEM. Also, due to this function it was possible that value
of type other than MEM_Blob or MEM_Str will have MEM_Ephem set. This is wrong, I
believe.
>> }
>> }
>> pMem->pScopyFrom = 0;
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 05e0f78c1..55083fb23 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -979,7 +979,7 @@ case OP_Copy: {
>> pOut = &aMem[pOp->p2];
>> assert(pOut!=pIn1);
>> while( 1) {
>> - sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
>> + mem_copy_as_ephemeral(pOut, pIn1);
>> Deephemeralize(pOut);
>
> 3. You could turn mem_copy_as_ephemeral + Deephemeralize into mem_copy
> the previous commit where mem_copy was introduced.
Thanks, fixed in previous patch.
New patch:
commit 91cec4dbcc1d931b70cdba3c70d77e7c58a00675
Author: Mergen Imeev <imeevma@gmail.com>
Date: Thu Mar 4 18:19:06 2021 +0300
sql: introduce mem_copy_as_ephemeral()
This patch intoduces mem_copy_as_ephemeral(). This function copies value
from source MEM to destination MEM. In case value is of string or binary
type and its allocation type is not static, it copied as value with
ephemeral allocation type.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index ea3917fe3..f75661e04 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -296,6 +296,25 @@ mem_copy(struct Mem *to, const struct Mem *from)
return 0;
}
+void
+mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
+{
+ mem_clear(to);
+ to->u = from->u;
+ to->flags = from->flags;
+ to->subtype = from->subtype;
+ to->field_type = from->field_type;
+ to->n = from->n;
+ to->z = from->z;
+ if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
+ return;
+ if ((to->flags & (MEM_Static | MEM_Ephem)) != 0)
+ return;
+ to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Zero | MEM_Subtype);
+ to->flags |= MEM_Ephem;
+ return;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -407,20 +426,6 @@ vdbeMemAddTerminator(Mem * pMem)
return 0;
}
-/*
- * Make an shallow copy of pFrom into pTo. Prior contents of
- * pTo are freed. The pFrom->z field is not duplicated. If
- * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
- * and flags gets srcType (either MEM_Ephem or MEM_Static).
- */
-static SQL_NOINLINE void
-vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
-{
- mem_clear(pTo);
- assert(!VdbeMemDynamic(pTo));
- sqlVdbeMemShallowCopy(pTo, pFrom, eType);
-}
-
/*
* Both *pMem1 and *pMem2 contain string values. Compare the two values
* using the collation sequence pColl. As usual, return a negative , zero
@@ -1989,22 +1994,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
return 0;
}
-void
-sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
-{
- assert(pTo->db == pFrom->db);
- if (VdbeMemDynamic(pTo)) {
- vdbeClrCopy(pTo, pFrom, srcType);
- return;
- }
- memcpy(pTo, pFrom, MEMCELLSIZE);
- if ((pFrom->flags & MEM_Static) == 0) {
- pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
- assert(srcType == MEM_Ephem || srcType == MEM_Static);
- pTo->flags |= srcType;
- }
-}
-
/*
* Transfer the contents of pFrom to pTo. Any existing value in pTo is
* freed. If pFrom contains ephemeral data, a copy is made.
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index af36f31a2..3898888ff 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -81,12 +81,6 @@ struct Mem {
#endif
};
-/*
- * Size of struct Mem not including the Mem.zMalloc member or anything that
- * follows.
- */
-#define MEMCELLSIZE offsetof(Mem,zMalloc)
-
bool
mem_is_null(const struct Mem *mem);
@@ -176,6 +170,14 @@ mem_destroy(struct Mem *mem);
int
mem_copy(struct Mem *to, const struct Mem *from);
+/**
+ * Copy content of MEM from one MEM to another. In case source MEM contains
+ * string or binary and allocation type is not STATIC, this value is copied as
+ * value with ephemeral allocation type.
+ */
+void
+mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -475,7 +477,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
int
vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
void sqlVdbeMemMove(Mem *, Mem *);
int sqlVdbeMemMakeWriteable(Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index abd49b9bb..bbeabf238 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -74,9 +74,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
int i;
Mem *pX;
for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
- if (pX->pScopyFrom == pMem) {
- pX->flags |= MEM_Undefined;
- pX->pScopyFrom = 0;
+ if (mem_is_bytes(pX) && !mem_is_ephemeral(pX) &&
+ !mem_is_static(pX)) {
+ if (pX->pScopyFrom == pMem) {
+ pX->flags |= MEM_Undefined;
+ pX->pScopyFrom = 0;
+ }
}
}
pMem->pScopyFrom = 0;
@@ -946,7 +949,7 @@ case OP_Variable: { /* out2 */
goto too_big;
}
pOut = vdbe_prepare_null_out(p, pOp->p2);
- sqlVdbeMemShallowCopy(pOut, pVar, MEM_Static);
+ mem_copy_as_ephemeral(pOut, pVar);
UPDATE_MAX_BLOBSIZE(pOut);
break;
}
@@ -1034,7 +1037,7 @@ case OP_SCopy: { /* out2 */
pIn1 = &aMem[pOp->p1];
pOut = &aMem[pOp->p2];
assert(pOut!=pIn1);
- sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
+ mem_copy_as_ephemeral(pOut, pIn1);
#ifdef SQL_DEBUG
if (pOut->pScopyFrom==0) pOut->pScopyFrom = pIn1;
#endif
@@ -2313,7 +2316,7 @@ case OP_Column: {
if (mem_is_null(pDest) &&
(uint32_t) p2 >= pC->field_ref.field_count &&
default_val_mem != NULL) {
- sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
+ mem_copy_as_ephemeral(pDest, default_val_mem);
}
pDest->field_type = field_type;
op_column_out:
@@ -4491,7 +4494,7 @@ case OP_Param: { /* out2 */
pOut = vdbe_prepare_null_out(p, pOp->p2);
pFrame = p->pFrame;
pIn = &pFrame->aMem[pOp->p1 + pFrame->aOp[pFrame->pc].p1];
- sqlVdbeMemShallowCopy(pOut, pIn, MEM_Ephem);
+ mem_copy_as_ephemeral(pOut, pIn);
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral()
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 16:31 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:10 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for working on this!
>>> @@ -593,9 +598,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
>>> int i;
>>> Mem *pX;
>>> for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
>>> - if (pX->pScopyFrom == pMem) {
>>> - pX->flags |= MEM_Undefined;
>>> - pX->pScopyFrom = 0;
>>> + if ((pX->flags & (MEM_Blob | MEM_Str)) != 0 &&
>>> + (pX->flags & (MEM_Ephem | MEM_Static)) == 0) {
>>> + if (pX->pScopyFrom == pMem) {
>>> + pX->flags |= MEM_Undefined;
>>> + pX->pScopyFrom = 0;
>>> + }
>>
>> 2. Why did you change that?
>>
> This check is only useful for strings and binaries, since they may be lost due
> to change of another MEM. Also, due to this function it was possible that value
> of type other than MEM_Blob or MEM_Str will have MEM_Ephem set. This is wrong, I
> believe.
Due to which function? sqlVdbeMemAboutToChange() does not set MEM_Ephem.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral()
2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 16:31 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 16:31 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review. My answer below.
On Sun, Apr 11, 2021 at 08:10:19PM +0200, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> >>> @@ -593,9 +598,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
> >>> int i;
> >>> Mem *pX;
> >>> for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
> >>> - if (pX->pScopyFrom == pMem) {
> >>> - pX->flags |= MEM_Undefined;
> >>> - pX->pScopyFrom = 0;
> >>> + if ((pX->flags & (MEM_Blob | MEM_Str)) != 0 &&
> >>> + (pX->flags & (MEM_Ephem | MEM_Static)) == 0) {
> >>> + if (pX->pScopyFrom == pMem) {
> >>> + pX->flags |= MEM_Undefined;
> >>> + pX->pScopyFrom = 0;
> >>> + }
> >>
> >> 2. Why did you change that?
> >>
> > This check is only useful for strings and binaries, since they may be lost due
> > to change of another MEM. Also, due to this function it was possible that value
> > of type other than MEM_Blob or MEM_Str will have MEM_Ephem set. This is wrong, I
> > believe.
>
> Due to which function? sqlVdbeMemAboutToChange() does not set MEM_Ephem.
I'm sorry, I was wrong. Actually it was sqlVdbeMemShallowCopy() fault that it
was possible to MEM to have MEM_Ephem flag even though it was not a string or
varbinary. But, since sqlVdbeMemMakeWriteable() was called in such cases
after sqlVdbeMemShallowCopy(), this flag was removed, I think. About
sqlVdbeMemAboutToChange() and MEM_undefined - I am not sure why I thought so.
Most likely I come to this conclusion during investigation of why memIsValid()
check fails when I removed sqlVdbeMemMakeWriteable() - this was so because MEM
with MEM_Ephem was invalidated in sqlVdbeMemAboutToChange() even though the MEM
wasn't string or varbinary.
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (13 preceding siblings ...)
2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:37 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
` (26 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:37 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch reworks mem_move(). This function moves all content of source
MEM to destination MEM. Source mem is set to NULL.
Part of #5818
---
src/box/sql/mem.c | 57 +++++++++----------------------------------
src/box/sql/mem.h | 8 ++++--
src/box/sql/vdbe.c | 23 +----------------
src/box/sql/vdbeapi.c | 2 +-
4 files changed, 19 insertions(+), 71 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index f75661e04..d56fe56c6 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -315,6 +315,17 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
return;
}
+int
+mem_move(struct Mem *to, struct Mem *from)
+{
+ mem_destroy(to);
+ memcpy(to, from, sizeof(*to));
+ from->flags = MEM_Null;
+ from->szMalloc = 0;
+ from->zMalloc = NULL;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1994,52 +2005,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
return 0;
}
-/*
- * Transfer the contents of pFrom to pTo. Any existing value in pTo is
- * freed. If pFrom contains ephemeral data, a copy is made.
- *
- * pFrom contains an SQL NULL when this routine returns.
- */
-void
-sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
-{
- assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
-
- mem_destroy(pTo);
- memcpy(pTo, pFrom, sizeof(Mem));
- pFrom->flags = MEM_Null;
- pFrom->szMalloc = 0;
-}
-
-/*
- * Change pMem so that its MEM_Str or MEM_Blob value is stored in
- * MEM.zMalloc, where it can be safely written.
- *
- * Return 0 on success or -1 if malloc fails.
- */
-int
-sqlVdbeMemMakeWriteable(Mem * pMem)
-{
- if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
- if (ExpandBlob(pMem))
- return -1;
- if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
- if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
- return -1;
- }
- pMem->z[pMem->n] = 0;
- pMem->z[pMem->n + 1] = 0;
- pMem->flags |= MEM_Term;
- }
- }
- pMem->flags &= ~MEM_Ephem;
-#ifdef SQL_DEBUG
- pMem->pScopyFrom = 0;
-#endif
-
- return 0;
-}
-
int
sql_vdbemem_finalize(struct Mem *mem, struct func *func)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 3898888ff..394055db9 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -178,6 +178,12 @@ mem_copy(struct Mem *to, const struct Mem *from);
void
mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
+/**
+ * Move all content of source MEM to destination MEM. Source MEM is set to NULL.
+ */
+int
+mem_move(struct Mem *to, struct Mem *from);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -477,8 +483,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
int
vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-void sqlVdbeMemMove(Mem *, Mem *);
-int sqlVdbeMemMakeWriteable(Mem *);
/**
* Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index bbeabf238..0c19acff5 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -197,21 +197,6 @@ vdbeTakeBranch(int iSrcLine, u8 I, u8 M)
}
#endif
-/*
- * An ephemeral string value (signified by the MEM_Ephem flag) contains
- * a pointer to a dynamically allocated string where some other entity
- * is responsible for deallocating that string. Because the register
- * does not control the string, it might be deleted without the register
- * knowing it.
- *
- * This routine converts an ephemeral string into a dynamically allocated
- * string that the register itself controls. In other words, it
- * converts an MEM_Ephem string into a string with P.z==P.zMalloc.
- */
-#define Deephemeralize(P) \
- if (((P)->flags&MEM_Ephem)!=0 \
- && sqlVdbeMemMakeWriteable(P)) { goto no_mem;}
-
/* Return true if the cursor was opened using the OP_OpenSorter opcode. */
#define isSorter(x) ((x)->eCurType==CURTYPE_SORTER)
@@ -981,13 +966,7 @@ case OP_Move: {
assert(pIn1<=&aMem[(p->nMem+1 - p->nCursor)]);
assert(memIsValid(pIn1));
memAboutToChange(p, pOut);
- sqlVdbeMemMove(pOut, pIn1);
-#ifdef SQL_DEBUG
- if (pOut->pScopyFrom>=&aMem[p1] && pOut->pScopyFrom<pOut) {
- pOut->pScopyFrom += pOp->p2 - p1;
- }
-#endif
- Deephemeralize(pOut);
+ mem_move(pOut, pIn1);
REGISTER_TRACE(p, p2++, pOut);
pIn1++;
pOut++;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7951996ea..8e69e3c38 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -980,7 +980,7 @@ sqlTransferBindings(sql_stmt * pFromStmt, sql_stmt * pToStmt)
assert(pTo->db == pFrom->db);
assert(pTo->nVar == pFrom->nVar);
for (i = 0; i < pFrom->nVar; i++) {
- sqlVdbeMemMove(&pTo->aVar[i], &pFrom->aVar[i]);
+ mem_move(&pTo->aVar[i], &pFrom->aVar[i]);
}
return 0;
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move()
2021-04-09 17:37 ` [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move() Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 16:38 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:10 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
I appreciate the work you did here!
On 09.04.2021 19:37, Mergen Imeev via Tarantool-patches wrote:
> This patch reworks mem_move(). This function moves all content of source
> MEM to destination MEM. Source mem is set to NULL.
>
> Part of #5818
> ---
> src/box/sql/mem.c | 57 +++++++++----------------------------------
> src/box/sql/mem.h | 8 ++++--
> src/box/sql/vdbe.c | 23 +----------------
> src/box/sql/vdbeapi.c | 2 +-
> 4 files changed, 19 insertions(+), 71 deletions(-)
>
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index f75661e04..d56fe56c6 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -315,6 +315,17 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
> return;
> }
>
> +int
It can be 'void'. The function never fails.
> +mem_move(struct Mem *to, struct Mem *from)
> +{
> + mem_destroy(to);
> + memcpy(to, from, sizeof(*to));
> + from->flags = MEM_Null;
> + from->szMalloc = 0;
> + from->zMalloc = NULL;
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move()
2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 16:38 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 16:38 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Sun, Apr 11, 2021 at 08:10:50PM +0200, Vladislav Shpilevoy wrote:
> I appreciate the work you did here!
>
> On 09.04.2021 19:37, Mergen Imeev via Tarantool-patches wrote:
> > This patch reworks mem_move(). This function moves all content of source
> > MEM to destination MEM. Source mem is set to NULL.
> >
> > Part of #5818
> > ---
> > src/box/sql/mem.c | 57 +++++++++----------------------------------
> > src/box/sql/mem.h | 8 ++++--
> > src/box/sql/vdbe.c | 23 +----------------
> > src/box/sql/vdbeapi.c | 2 +-
> > 4 files changed, 19 insertions(+), 71 deletions(-)
> >
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index f75661e04..d56fe56c6 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -315,6 +315,17 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
> > return;
> > }
> >
> > +int
>
> It can be 'void'. The function never fails.
>
Fixed.
> > +mem_move(struct Mem *to, struct Mem *from)
> > +{
> > + mem_destroy(to);
> > + memcpy(to, from, sizeof(*to));
> > + from->flags = MEM_Null;
> > + from->szMalloc = 0;
> > + from->zMalloc = NULL;
> > + return 0;
> > +}
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index f583dc7c0..2991f902d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -170,7 +170,7 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
return;
}
-int
+void
mem_move(struct Mem *to, struct Mem *from)
{
mem_destroy(to);
@@ -178,7 +178,6 @@ mem_move(struct Mem *to, struct Mem *from)
from->flags = MEM_Null;
from->szMalloc = 0;
from->zMalloc = NULL;
- return 0;
}
static inline bool
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 496fef1e3..beb9826a1 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -319,7 +319,7 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
/**
* Move all content of source MEM to destination MEM. Source MEM is set to NULL.
*/
-int
+void
mem_move(struct Mem *to, struct Mem *from);
/**
New patch:
commit e54a891eebb0ae9af9cde630628a5cdb03e00698
Author: Mergen Imeev <imeevma@gmail.com>
Date: Fri Mar 5 15:30:23 2021 +0300
sql: rework mem_move()
This patch reworks mem_move(). This function moves all content of source
MEM to destination MEM. Source mem is set to NULL.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index befedf04c..2991f902d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -170,6 +170,16 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
return;
}
+void
+mem_move(struct Mem *to, struct Mem *from)
+{
+ mem_destroy(to);
+ memcpy(to, from, sizeof(*to));
+ from->flags = MEM_Null;
+ from->szMalloc = 0;
+ from->zMalloc = NULL;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1849,52 +1859,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
return 0;
}
-/*
- * Transfer the contents of pFrom to pTo. Any existing value in pTo is
- * freed. If pFrom contains ephemeral data, a copy is made.
- *
- * pFrom contains an SQL NULL when this routine returns.
- */
-void
-sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
-{
- assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
-
- mem_destroy(pTo);
- memcpy(pTo, pFrom, sizeof(Mem));
- pFrom->flags = MEM_Null;
- pFrom->szMalloc = 0;
-}
-
-/*
- * Change pMem so that its MEM_Str or MEM_Blob value is stored in
- * MEM.zMalloc, where it can be safely written.
- *
- * Return 0 on success or -1 if malloc fails.
- */
-int
-sqlVdbeMemMakeWriteable(Mem * pMem)
-{
- if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
- if (ExpandBlob(pMem))
- return -1;
- if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
- if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
- return -1;
- }
- pMem->z[pMem->n] = 0;
- pMem->z[pMem->n + 1] = 0;
- pMem->flags |= MEM_Term;
- }
- }
- pMem->flags &= ~MEM_Ephem;
-#ifdef SQL_DEBUG
- pMem->pScopyFrom = 0;
-#endif
-
- return 0;
-}
-
int
sql_vdbemem_finalize(struct Mem *mem, struct func *func)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index e30e5466e..beb9826a1 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -316,6 +316,12 @@ mem_copy(struct Mem *to, const struct Mem *from);
void
mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
+/**
+ * Move all content of source MEM to destination MEM. Source MEM is set to NULL.
+ */
+void
+mem_move(struct Mem *to, struct Mem *from);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -563,8 +569,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
int
vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-void sqlVdbeMemMove(Mem *, Mem *);
-int sqlVdbeMemMakeWriteable(Mem *);
/**
* Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 50d9dd78c..b3b20a374 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -197,21 +197,6 @@ vdbeTakeBranch(int iSrcLine, u8 I, u8 M)
}
#endif
-/*
- * An ephemeral string value (signified by the MEM_Ephem flag) contains
- * a pointer to a dynamically allocated string where some other entity
- * is responsible for deallocating that string. Because the register
- * does not control the string, it might be deleted without the register
- * knowing it.
- *
- * This routine converts an ephemeral string into a dynamically allocated
- * string that the register itself controls. In other words, it
- * converts an MEM_Ephem string into a string with P.z==P.zMalloc.
- */
-#define Deephemeralize(P) \
- if (((P)->flags&MEM_Ephem)!=0 \
- && sqlVdbeMemMakeWriteable(P)) { goto no_mem;}
-
/* Return true if the cursor was opened using the OP_OpenSorter opcode. */
#define isSorter(x) ((x)->eCurType==CURTYPE_SORTER)
@@ -981,13 +966,7 @@ case OP_Move: {
assert(pIn1<=&aMem[(p->nMem+1 - p->nCursor)]);
assert(memIsValid(pIn1));
memAboutToChange(p, pOut);
- sqlVdbeMemMove(pOut, pIn1);
-#ifdef SQL_DEBUG
- if (pOut->pScopyFrom>=&aMem[p1] && pOut->pScopyFrom<pOut) {
- pOut->pScopyFrom += pOp->p2 - p1;
- }
-#endif
- Deephemeralize(pOut);
+ mem_move(pOut, pIn1);
REGISTER_TRACE(p, p2++, pOut);
pIn1++;
pOut++;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7951996ea..8e69e3c38 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -980,7 +980,7 @@ sqlTransferBindings(sql_stmt * pFromStmt, sql_stmt * pToStmt)
assert(pTo->db == pFrom->db);
assert(pTo->nVar == pFrom->nVar);
for (i = 0; i < pFrom->nVar; i++) {
- sqlVdbeMemMove(&pTo->aVar[i], &pFrom->aVar[i]);
+ mem_move(&pTo->aVar[i], &pFrom->aVar[i]);
}
return 0;
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (14 preceding siblings ...)
2021-04-09 17:37 ` [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
` (25 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:57 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:02, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 3 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> The original vdbe_decode_msgpack_into_mem() returns a MEM that contains
>> string and binary values as ephemeral. This patch renames this function
>> to vdbe_decode_msgpack_into_mem_ephemeral() and introduces new
>> vdbe_decode_msgpack_into_mem(), which returns a MEM that contains string
>> and binary values in newly allocated memory.
>>
>> This patch actually changes behavior in this case:
>
> 1. Changes how? I don't see any changes in the tests.
>
I make this change because it doesn't affect test.
Example of changed behaviour:
CREATE TABLE t1(m VARBINARY primary key);
INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379');
SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
Before this patch:
tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
---
- metadata:
- name: COLUMN_1
type: integer
- name: MX
type: string
rows:
- [2, 'y']
- [1, 'y']
...
After this patch.
tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
---
- metadata:
- name: COLUMN_1
type: integer
- name: MX
type: string
rows:
- [2, 'x']
- [1, 'y']
...
A bit more is written in issue #5890.
>> CREATE TABLE t1(m VARBINARY primary key);
>> INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379');
>> SELECT count(*), substr(m,2,1) AS m FROM t1 GROUP BY m;
>> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
>>
>> But it doesn't change behaviour for this:
>>
>> CREATE TABLE t2(m STRING primary key);
>> INSERT INTO t2 VALUES('ax'), ('bx'), ('cy');
>> SELECT count(*), substr(m,2,1) AS m FROM t2 GROUP BY m;
>> SELECT count(*), substr(m,2,1) AS mx FROM t2 GROUP BY mx;
>>
>> Part of #5818
>> Part of #5890
>> ---
>> src/box/sql/mem.c | 16 +++++++++++++++-
>> src/box/sql/mem.h | 17 ++++++++++++++++-
>> src/box/sql/vdbe.c | 18 ------------------
>> src/box/sql/vdbeaux.c | 2 +-
>> 4 files changed, 32 insertions(+), 21 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 3d42ac63c..a2316cc90 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -2253,7 +2253,8 @@ sqlVdbeRecordCompareMsgpack(const void *key1,
>> }
>>
>> int
>> -vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
>> +vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
>> + uint32_t *len)
>
> 2. The function name is getting Java vibes. I propose to rename it to
> mem_from_mp_ephemeral() and mem_from_mp() correspondingly. They also should
> start taking the mem as a first argument.
>
Thank you! Fixed.
>> {
>> const char *start_buf = buf;
>> switch (mp_typeof(*buf)) {
>> @@ -2354,6 +2355,19 @@ install_blob:
>> return 0;
>> }
>>
>> +int
>> +vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
>> +{
>> + if (vdbe_decode_msgpack_into_ephemeral_mem(buf, mem, len) != 0)
>> + return -1;
>> + if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
>> + assert((mem->flags & MEM_Ephem) != 0);
>> + if (sqlVdbeMemGrow(mem, mem->n, 1) != 0)
>> + return -1;
>
> 3. Maybe it is worth adding a function like mem_materialize() or
> mem_make_writable() for that kind of work.
>
Not sure. For now decided to not add a new function.
>> + }
>> + return 0;
>> +}
New patch:
commit 8c81c3e86a628e5777145b30f69392dd5a0fd873
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sat Mar 13 15:43:38 2021 +0300
sql: rework vdbe_decode_msgpack_into_mem()
The original vdbe_decode_msgpack_into_mem() returns a MEM that contains
string and binary values as ephemeral. This patch renames this function
to mem_from_mp_ephemeral() and introduces new function mem_from_mp(),
which returns a MEM that contains string and binary values in newly
allocated memory.
This patch changes behavior for this query:
CREATE TABLE t1(m VARBINARY primary key);
INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379');
SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
Before this patch:
tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
---
- metadata:
- name: COLUMN_1
type: integer
- name: MX
type: string
rows:
- [2, 'y']
- [1, 'y']
...
After this patch.
tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
---
- metadata:
- name: COLUMN_1
type: integer
- name: MX
type: string
rows:
- [2, 'x']
- [1, 'y']
...
Part of #5818
Closes #5890
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index d56fe56c6..7d06e256c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -2214,7 +2214,7 @@ sqlVdbeRecordCompareMsgpack(const void *key1,
}
int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
+mem_from_mp_ephemeral(struct Mem *mem, const char *buf, uint32_t *len)
{
const char *start_buf = buf;
switch (mp_typeof(*buf)) {
@@ -2315,6 +2315,19 @@ install_blob:
return 0;
}
+int
+mem_from_mp(struct Mem *mem, const char *buf, uint32_t *len)
+{
+ if (mem_from_mp_ephemeral(mem, buf, len) != 0)
+ return -1;
+ if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
+ assert((mem->flags & MEM_Ephem) != 0);
+ if (sqlVdbeMemGrow(mem, mem->n, 1) != 0)
+ return -1;
+ }
+ return 0;
+}
+
void
mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 394055db9..55f8f0c9f 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -520,16 +520,30 @@ int sqlVdbeRecordCompareMsgpack(const void *key1,
struct UnpackedRecord *key2);
/**
- * Decode msgpack and save value into VDBE memory cell.
+ * Decode msgpack and save value into VDBE memory cell. String and binary string
+ * values set as ephemeral.
*
+ * @param mem Memory cell to write value into.
* @param buf Buffer to deserialize msgpack from.
+ * @param len[out] Length of decoded part.
+ * @retval Return code: < 0 in case of error.
+ * @retval 0 on success.
+ */
+int
+mem_from_mp_ephemeral(struct Mem *mem, const char *buf, uint32_t *len);
+
+/**
+ * Decode msgpack and save value into VDBE memory cell. String and binary string
+ * values copied to newly allocated memory.
+ *
* @param mem Memory cell to write value into.
+ * @param buf Buffer to deserialize msgpack from.
* @param len[out] Length of decoded part.
* @retval Return code: < 0 in case of error.
* @retval 0 on success.
*/
int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len);
+mem_from_mp(struct Mem *mem, const char *buf, uint32_t *len);
/**
* Perform encoding memory variable to stream.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 0c19acff5..378c7a043 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -405,26 +405,8 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
assert(sqlVdbeCheckMemInvariants(dest_mem) != 0);
const char *data = vdbe_field_ref_fetch_data(field_ref, fieldno);
uint32_t dummy;
- if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
+ if (mem_from_mp(dest_mem, data, &dummy) != 0)
return -1;
-
- /*
- * Add 0 termination (at most for strings)
- * Not sure why do we check MEM_Ephem
- */
- if (mem_is_str(dest_mem) && mem_is_ephemeral(dest_mem)) {
- int len = dest_mem->n;
- if (dest_mem->szMalloc < len + 1) {
- if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0)
- return -1;
- } else {
- dest_mem->z =
- memcpy(dest_mem->zMalloc, dest_mem->z, len);
- dest_mem->flags &= ~MEM_Ephem;
- }
- dest_mem->z[len] = 0;
- dest_mem->flags |= MEM_Term;
- }
UPDATE_MAX_BLOBSIZE(dest_mem);
return 0;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index bec8a532a..dff108412 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2358,7 +2358,7 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec
pMem->szMalloc = 0;
pMem->z = 0;
uint32_t sz = 0;
- vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
+ mem_from_mp_ephemeral(pMem, zParse, &sz);
assert(sz != 0);
zParse += sz;
pMem++;
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 91cba9962..ba5c08a00 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -563,7 +563,7 @@ sql_stat4_column(struct sql *db, const char *record, uint32_t col_num,
}
}
uint32_t unused;
- return vdbe_decode_msgpack_into_mem(a, mem, &unused);
+ return mem_from_mp(mem, a, &unused);
}
/*
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (15 preceding siblings ...)
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches
2021-04-14 22:58 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
` (24 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:57 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Function sql_column_to_messagepack() has almost the same functionality
as sql_vdbe_mem_encode_tuple(). Due to this it is not necessary to
have sql_column_to_messagepack(), so it is removed in this commit.
Part of #5818
---
src/box/execute.c | 130 ++----------------------------------------
src/box/sql/mem.c | 6 --
src/box/sql/mem.h | 3 -
src/box/sql/sqlInt.h | 15 +----
src/box/sql/vdbeapi.c | 15 ++---
5 files changed, 13 insertions(+), 156 deletions(-)
diff --git a/src/box/execute.c b/src/box/execute.c
index e14da20be..f9c161ce4 100644
--- a/src/box/execute.c
+++ b/src/box/execute.c
@@ -131,113 +131,10 @@ port_sql_create(struct port *port, struct sql_stmt *stmt,
port_sql->do_finalize = do_finalize;
}
-/**
- * Serialize a single column of a result set row.
- * @param stmt Prepared and started statement. At least one
- * sql_step must be called.
- * @param i Column number.
- * @param region Allocator for column value.
- *
- * @retval 0 Success.
- * @retval -1 Out of memory when resizing the output buffer.
- */
-static inline int
-sql_column_to_messagepack(struct sql_stmt *stmt, int i,
- struct region *region)
-{
- size_t size;
- enum mp_type type = sql_column_type(stmt, i);
- switch (type) {
- case MP_INT: {
- int64_t n = sql_column_int64(stmt, i);
- size = mp_sizeof_int(n);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_int(pos, n);
- break;
- }
- case MP_UINT: {
- uint64_t n = sql_column_uint64(stmt, i);
- size = mp_sizeof_uint(n);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_uint(pos, n);
- break;
- }
- case MP_DOUBLE: {
- double d = sql_column_double(stmt, i);
- size = mp_sizeof_double(d);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_double(pos, d);
- break;
- }
- case MP_STR: {
- uint32_t len = sql_column_bytes(stmt, i);
- size = mp_sizeof_str(len);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- const char *s;
- s = (const char *)sql_column_text(stmt, i);
- mp_encode_str(pos, s, len);
- break;
- }
- case MP_BIN:
- case MP_MAP:
- case MP_ARRAY: {
- uint32_t len = sql_column_bytes(stmt, i);
- const char *s =
- (const char *)sql_column_blob(stmt, i);
- if (sql_column_subtype(stmt, i) == SQL_SUBTYPE_MSGPACK) {
- size = len;
- char *pos = (char *)region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- memcpy(pos, s, len);
- } else {
- size = mp_sizeof_bin(len);
- char *pos = (char *)region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_bin(pos, s, len);
- }
- break;
- }
- case MP_BOOL: {
- bool b = sql_column_boolean(stmt, i);
- size = mp_sizeof_bool(b);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_bool(pos, b);
- break;
- }
- case MP_NIL: {
- size = mp_sizeof_nil();
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL)
- goto oom;
- mp_encode_nil(pos);
- break;
- }
- default:
- unreachable();
- }
- return 0;
-oom:
- diag_set(OutOfMemory, size, "region_alloc", "SQL value");
- return -1;
-}
-
/**
* Convert sql row into a tuple and append to a port.
* @param stmt Started prepared statement. At least one
* sql_step must be done.
- * @param column_count Statement's column count.
* @param region Runtime allocator for temporary objects.
* @param port Port to store tuples.
*
@@ -245,29 +142,11 @@ oom:
* @retval -1 Memory error.
*/
static inline int
-sql_row_to_port(struct sql_stmt *stmt, int column_count,
- struct region *region, struct port *port)
+sql_row_to_port(struct sql_stmt *stmt, struct region *region, struct port *port)
{
- assert(column_count > 0);
- size_t size = mp_sizeof_array(column_count);
+ uint32_t size;
size_t svp = region_used(region);
- char *pos = (char *) region_alloc(region, size);
- if (pos == NULL) {
- diag_set(OutOfMemory, size, "region_alloc", "SQL row");
- return -1;
- }
- mp_encode_array(pos, column_count);
-
- for (int i = 0; i < column_count; ++i) {
- if (sql_column_to_messagepack(stmt, i, region) != 0)
- goto error;
- }
- size = region_used(region) - svp;
- pos = (char *) region_join(region, size);
- if (pos == NULL) {
- diag_set(OutOfMemory, size, "region_join", "pos");
- goto error;
- }
+ char *pos = sql_result_to_msgpack(stmt, &size, region);
struct tuple *tuple =
tuple_new(box_tuple_format_default(), pos, pos + size);
if (tuple == NULL)
@@ -677,8 +556,7 @@ sql_execute(struct sql_stmt *stmt, struct port *port, struct region *region)
if (column_count > 0) {
/* Either ROW or DONE or ERROR. */
while ((rc = sql_step(stmt)) == SQL_ROW) {
- if (sql_row_to_port(stmt, column_count, region,
- port) != 0)
+ if (sql_row_to_port(stmt, region, port) != 0)
return -1;
}
assert(rc == SQL_DONE || rc != 0);
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 7d06e256c..b417c1007 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1752,12 +1752,6 @@ sqlValueText(sql_value * pVal)
return valueToText(pVal);
}
-enum sql_subtype
-sql_value_subtype(sql_value * pVal)
-{
- return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
-}
-
/*
* Return a pointer to static memory containing an SQL NULL value.
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 55f8f0c9f..0faf223ef 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -449,9 +449,6 @@ const void *sqlValueText(struct Mem *);
#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
-enum sql_subtype
-sql_value_subtype(sql_value * pVal);
-
const Mem *
columnNullValue(void);
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index e075224c6..259ba3833 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -351,16 +351,6 @@ sql_stricmp(const char *, const char *);
int
sql_strnicmp(const char *, const char *, int);
-
-/**
- * Get row column subtype.
- * @param stmt row data to process.
- * @param i column index.
- * @retval SQL subtype if any, 0 else.
- */
-enum sql_subtype
-sql_column_subtype(struct sql_stmt *stmt, int i);
-
sql *
sql_context_db_handle(sql_context *);
@@ -471,8 +461,9 @@ const unsigned char *
sql_column_text(sql_stmt *,
int iCol);
-enum mp_type
-sql_column_type(sql_stmt *stmt, int field_no);
+char *
+sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+ struct region *region);
/*
* Terminate the current execution of an SQL statement and reset
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 8e69e3c38..3e9f1ff6f 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -515,16 +515,13 @@ sql_column_text(sql_stmt * pStmt, int i)
return sql_value_text(columnMem(pStmt, i));
}
-enum mp_type
-sql_column_type(sql_stmt * pStmt, int i)
-{
- return sql_value_type(columnMem(pStmt, i));
-}
-
-enum sql_subtype
-sql_column_subtype(struct sql_stmt *stmt, int i)
+char *
+sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+ struct region *region)
{
- return sql_value_subtype(columnMem(stmt, i));
+ struct Vdbe *vdbe = (struct Vdbe *)stmt;
+ return sql_vdbe_mem_encode_tuple(vdbe->pResultSet, vdbe->nResColumn,
+ tuple_size, region);
}
/*
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack()
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
@ 2021-04-14 22:58 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 23:14 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 22:58 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
I appreciate the work you did here!
> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> index e075224c6..259ba3833 100644
> --- a/src/box/sql/sqlInt.h
> +++ b/src/box/sql/sqlInt.h
> @@ -471,8 +461,9 @@ const unsigned char *
> sql_column_text(sql_stmt *,
> int iCol);
>
> -enum mp_type
> -sql_column_type(sql_stmt *stmt, int field_no);
> +char *
> +sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
> + struct region *region);
I noticed just now - this is a method of sql_stmt, and therefore
must be called 'sql_stmt_result_to_msgpack()'. With 'sql_stmt'
prefix.
^ permalink raw reply [flat|nested] 107+ messages in thread
* Re: [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack()
2021-04-14 22:58 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 23:14 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 23:14 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer below.
On Thu, Apr 15, 2021 at 12:58:57AM +0200, Vladislav Shpilevoy wrote:
> I appreciate the work you did here!
>
> > diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> > index e075224c6..259ba3833 100644
> > --- a/src/box/sql/sqlInt.h
> > +++ b/src/box/sql/sqlInt.h
> > @@ -471,8 +461,9 @@ const unsigned char *
> > sql_column_text(sql_stmt *,
> > int iCol);
> >
> > -enum mp_type
> > -sql_column_type(sql_stmt *stmt, int field_no);
> > +char *
> > +sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
> > + struct region *region);
>
> I noticed just now - this is a method of sql_stmt, and therefore
> must be called 'sql_stmt_result_to_msgpack()'. With 'sql_stmt'
> prefix.
Thanks, fixed.
diff --git a/src/box/execute.c b/src/box/execute.c
index f9c161ce4..70101d036 100644
--- a/src/box/execute.c
+++ b/src/box/execute.c
@@ -146,7 +146,7 @@ sql_row_to_port(struct sql_stmt *stmt, struct region *region, struct port *port)
{
uint32_t size;
size_t svp = region_used(region);
- char *pos = sql_result_to_msgpack(stmt, &size, region);
+ char *pos = sql_stmt_result_to_msgpack(stmt, &size, region);
struct tuple *tuple =
tuple_new(box_tuple_format_default(), pos, pos + size);
if (tuple == NULL)
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 259ba3833..663f2841d 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -462,8 +462,8 @@ sql_column_text(sql_stmt *,
int iCol);
char *
-sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
- struct region *region);
+sql_stmt_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+ struct region *region);
/*
* Terminate the current execution of an SQL statement and reset
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 3e9f1ff6f..7216ac12b 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -516,8 +516,8 @@ sql_column_text(sql_stmt * pStmt, int i)
}
char *
-sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
- struct region *region)
+sql_stmt_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+ struct region *region)
{
struct Vdbe *vdbe = (struct Vdbe *)stmt;
return sql_vdbe_mem_encode_tuple(vdbe->pResultSet, vdbe->nResColumn,
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (16 preceding siblings ...)
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:11 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM Mergen Imeev via Tarantool-patches
` (23 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:57 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_concat(). Function mem_concat() concatenates
values from two MEMs in case these values are strings or binaries and
writes the result to the third MEM.
Part of #5818
---
src/box/sql/mem.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++
src/box/sql/mem.h | 8 ++++++
src/box/sql/vdbe.c | 50 ++----------------------------------
3 files changed, 74 insertions(+), 48 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index b417c1007..2d76ef88d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from)
return 0;
}
+static bool
+is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
+ enum field_type type)
+{
+ mem_clear(result);
+ result->field_type = type;
+ return (((a->flags | b->flags) & MEM_Null) != 0);
+}
+
+int
+mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
+{
+ assert(result != b);
+ if (a != result) {
+ if (is_result_null(a, b, result, FIELD_TYPE_STRING))
+ return 0;
+ } else {
+ if (((a->flags | b->flags) & MEM_Null) != 0) {
+ mem_clear(a);
+ result->field_type = FIELD_TYPE_STRING;
+ return 0;
+ }
+ }
+
+ /* Concatenation operation can be applied only to strings and blobs. */
+ if ((b->flags & (MEM_Str | MEM_Blob)) == 0) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ "text or varbinary", mem_type_to_str(b));
+ return -1;
+ }
+ if ((a->flags & (MEM_Str | MEM_Blob)) == 0) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ "text or varbinary", mem_type_to_str(a));
+ return -1;
+ }
+
+ /* Moreover, both operands must be of the same type. */
+ if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ mem_type_to_str(a), mem_type_to_str(b));
+ return -1;
+ }
+
+ if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0)
+ return -1;
+
+ uint32_t size = a->n + b->n;
+ if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big");
+ return -1;
+ }
+ if (sqlVdbeMemGrow(result, size, result == a) != 0)
+ return -1;
+
+ result->flags = a->flags & (MEM_Str | MEM_Blob);
+ if ((result->flags & MEM_Blob) != 0)
+ result->field_type = FIELD_TYPE_VARBINARY;
+ if (result != a)
+ memcpy(result->z, a->z, a->n);
+ memcpy(&result->z[a->n], b->z, b->n);
+ result->n = size;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 0faf223ef..df273026b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -184,6 +184,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
int
mem_move(struct Mem *to, struct Mem *from);
+/**
+ * Concatenate strings or binaries from the first and the second MEMs and write
+ * to the result MEM. In case the first MEM or the second MEM is NULL, the
+ * result MEM is set to NULL even if the result MEM is actually the first MEM.
+ */
+int
+mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 378c7a043..6f3475147 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1060,57 +1060,11 @@ case OP_ResultRow: {
* types (i.e. TEXT and BLOB).
*/
case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
- i64 nByte;
-
pIn1 = &aMem[pOp->p1];
pIn2 = &aMem[pOp->p2];
- pOut = vdbe_prepare_null_out(p, pOp->p3);
- assert(pIn1!=pOut);
- if (mem_is_any_null(pIn1, pIn2)) {
- /* Force NULL be of type STRING. */
- pOut->field_type = FIELD_TYPE_STRING;
- break;
- }
- /*
- * Concatenation operation can be applied only to
- * strings and blobs.
- */
- if (!mem_is_bytes(pIn1) || !mem_is_bytes(pIn2)) {
- char *inconsistent_type = !mem_is_bytes(pIn1) ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn2);
- diag_set(ClientError, ER_INCONSISTENT_TYPES,
- "text or varbinary", inconsistent_type);
- goto abort_due_to_error;
- }
-
- /* Moreover, both operands must be of the same type. */
- if (!mem_is_same_type(pIn1, pIn2)) {
- diag_set(ClientError, ER_INCONSISTENT_TYPES,
- mem_type_to_str(pIn2), mem_type_to_str(pIn1));
- goto abort_due_to_error;
- }
- if (ExpandBlob(pIn1) != 0 || ExpandBlob(pIn2) != 0)
+ pOut = &aMem[pOp->p3];
+ if (mem_concat(pIn2, pIn1, pOut) != 0)
goto abort_due_to_error;
- nByte = pIn1->n + pIn2->n;
- if (nByte>db->aLimit[SQL_LIMIT_LENGTH]) {
- goto too_big;
- }
- if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
- goto no_mem;
- }
- if (mem_is_str(pIn1))
- MemSetTypeFlag(pOut, MEM_Str);
- else
- MemSetTypeFlag(pOut, MEM_Blob);
- if (pOut!=pIn2) {
- memcpy(pOut->z, pIn2->z, pIn2->n);
- }
- memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);
- pOut->z[nByte]=0;
- pOut->z[nByte+1] = 0;
- pOut->flags |= MEM_Term;
- pOut->n = (int)nByte;
UPDATE_MAX_BLOBSIZE(pOut);
break;
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat()
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:11 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 16:57 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:11 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Good job on the patch!
See 2 comments below.
On 09.04.2021 19:57, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_concat(). Function mem_concat() concatenates
> values from two MEMs in case these values are strings or binaries and
> writes the result to the third MEM.
>
> Part of #5818
> ---
> src/box/sql/mem.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++
> src/box/sql/mem.h | 8 ++++++
> src/box/sql/vdbe.c | 50 ++----------------------------------
> 3 files changed, 74 insertions(+), 48 deletions(-)
>
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index b417c1007..2d76ef88d 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from)
> return 0;
> }
>
> +static bool
> +is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
> + enum field_type type)
1. Functions called 'is_*' never should change anything.
Another question is why do you even need it? It is used in a single place,
where it could be just inlined. And is not used in a place, where it could
be applied.
> +{
> + mem_clear(result);
> + result->field_type = type;
> + return (((a->flags | b->flags) & MEM_Null) != 0);
> +}
> +
> +int
> +mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
> +{
> + assert(result != b);
> + if (a != result) {
> + if (is_result_null(a, b, result, FIELD_TYPE_STRING))
> + return 0;
> + } else {
> + if (((a->flags | b->flags) & MEM_Null) != 0) {
> + mem_clear(a);
> + result->field_type = FIELD_TYPE_STRING;
> + return 0;
> + }
> + }
> +
> + /* Concatenation operation can be applied only to strings and blobs. */
> + if ((b->flags & (MEM_Str | MEM_Blob)) == 0) {
> + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> + "text or varbinary", mem_type_to_str(b));
> + return -1;
> + }
> + if ((a->flags & (MEM_Str | MEM_Blob)) == 0) {
> + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> + "text or varbinary", mem_type_to_str(a));
> + return -1;
> + }
> +
> + /* Moreover, both operands must be of the same type. */
> + if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) {
> + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> + mem_type_to_str(a), mem_type_to_str(b));
> + return -1;
> + }
> +
> + if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0)
> + return -1;
> +
> + uint32_t size = a->n + b->n;
> + if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) {
> + diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big");
> + return -1;
> + }
> + if (sqlVdbeMemGrow(result, size, result == a) != 0)
> + return -1;
> +
> + result->flags = a->flags & (MEM_Str | MEM_Blob);
2. Why isn't result cleared? What if it was an Agg, or Frame?
I see before your patch they called vdbe_prepare_null_out(), which
cleared the mem.
> + if ((result->flags & MEM_Blob) != 0)
> + result->field_type = FIELD_TYPE_VARBINARY;
> + if (result != a)
> + memcpy(result->z, a->z, a->n);
> + memcpy(&result->z[a->n], b->z, b->n);
> + result->n = size;
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat()
2021-04-11 18:11 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 16:57 ` Mergen Imeev via Tarantool-patches
2021-04-14 23:04 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 16:57 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers, diff and new patch below.
On Sun, Apr 11, 2021 at 08:11:53PM +0200, Vladislav Shpilevoy wrote:
> Good job on the patch!
>
> See 2 comments below.
>
> On 09.04.2021 19:57, Mergen Imeev via Tarantool-patches wrote:
> > This patch introduces mem_concat(). Function mem_concat() concatenates
> > values from two MEMs in case these values are strings or binaries and
> > writes the result to the third MEM.
> >
> > Part of #5818
> > ---
> > src/box/sql/mem.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++
> > src/box/sql/mem.h | 8 ++++++
> > src/box/sql/vdbe.c | 50 ++----------------------------------
> > 3 files changed, 74 insertions(+), 48 deletions(-)
> >
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index b417c1007..2d76ef88d 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from)
> > return 0;
> > }
> >
> > +static bool
> > +is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
> > + enum field_type type)
>
> 1. Functions called 'is_*' never should change anything.
>
Fixed. Renamed to check_result_null().
> Another question is why do you even need it? It is used in a single place,
> where it could be just inlined. And is not used in a place, where it could
> be applied.
>
I added it here since it was the first commit, which used it. This functions
will be used in all arithmetic and bitwise operations with two operands.
> > +{
> > + mem_clear(result);
> > + result->field_type = type;
> > + return (((a->flags | b->flags) & MEM_Null) != 0);
> > +}
> > +
> > +int
> > +mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
> > +{
> > + assert(result != b);
> > + if (a != result) {
> > + if (is_result_null(a, b, result, FIELD_TYPE_STRING))
> > + return 0;
> > + } else {
> > + if (((a->flags | b->flags) & MEM_Null) != 0) {
> > + mem_clear(a);
> > + result->field_type = FIELD_TYPE_STRING;
> > + return 0;
> > + }
> > + }
> > +
> > + /* Concatenation operation can be applied only to strings and blobs. */
> > + if ((b->flags & (MEM_Str | MEM_Blob)) == 0) {
> > + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> > + "text or varbinary", mem_type_to_str(b));
> > + return -1;
> > + }
> > + if ((a->flags & (MEM_Str | MEM_Blob)) == 0) {
> > + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> > + "text or varbinary", mem_type_to_str(a));
> > + return -1;
> > + }
> > +
> > + /* Moreover, both operands must be of the same type. */
> > + if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) {
> > + diag_set(ClientError, ER_INCONSISTENT_TYPES,
> > + mem_type_to_str(a), mem_type_to_str(b));
> > + return -1;
> > + }
> > +
> > + if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0)
> > + return -1;
> > +
> > + uint32_t size = a->n + b->n;
> > + if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) {
> > + diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big");
> > + return -1;
> > + }
> > + if (sqlVdbeMemGrow(result, size, result == a) != 0)
> > + return -1;
> > +
> > + result->flags = a->flags & (MEM_Str | MEM_Blob);
>
> 2. Why isn't result cleared? What if it was an Agg, or Frame?
> I see before your patch they called vdbe_prepare_null_out(), which
> cleared the mem.
>
In case result != left result is cleared in check_result_null(). In the other
case it is cleared if one of operands is NULL. If it is not cleared than
result == left, which means that if it is not varbinary of string, the error
will be returned. There shouldn't be any problem during destruction, I think.
Also, if we clear result when result == left we will get NULL as result of
concatenation no matter what left and right were. I do not know why result
was cleared previously, since it may lead to result described above.
> > + if ((result->flags & MEM_Blob) != 0)
> > + result->field_type = FIELD_TYPE_VARBINARY;
> > + if (result != a)
> > + memcpy(result->z, a->z, a->n);
> > + memcpy(&result->z[a->n], b->z, b->n);
> > + result->n = size;
> > + return 0;
> > +}
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 67c8d90ee..f3bb62369 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -181,8 +181,8 @@ mem_move(struct Mem *to, struct Mem *from)
}
static bool
-is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
- enum field_type type)
+check_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
+ enum field_type type)
{
mem_clear(result);
result->field_type = type;
@@ -194,7 +194,7 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
{
assert(result != b);
if (a != result) {
- if (is_result_null(a, b, result, FIELD_TYPE_STRING))
+ if (check_result_null(a, b, result, FIELD_TYPE_STRING))
return 0;
} else {
if (((a->flags | b->flags) & MEM_Null) != 0) {
New patch:
commit e95f6aea6698865e5c864fa8ae1b5393c4faf553
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sat Mar 13 21:30:54 2021 +0300
sql: introduce mem_concat()
This patch introduces mem_concat(). Function mem_concat() concatenates
values from two MEMs in case these values are strings or binaries and
writes the result to the third MEM.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index a7caac014..f3bb62369 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -180,6 +180,70 @@ mem_move(struct Mem *to, struct Mem *from)
from->zMalloc = NULL;
}
+static bool
+check_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
+ enum field_type type)
+{
+ mem_clear(result);
+ result->field_type = type;
+ return (((a->flags | b->flags) & MEM_Null) != 0);
+}
+
+int
+mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
+{
+ assert(result != b);
+ if (a != result) {
+ if (check_result_null(a, b, result, FIELD_TYPE_STRING))
+ return 0;
+ } else {
+ if (((a->flags | b->flags) & MEM_Null) != 0) {
+ mem_clear(a);
+ result->field_type = FIELD_TYPE_STRING;
+ return 0;
+ }
+ }
+
+ /* Concatenation operation can be applied only to strings and blobs. */
+ if ((b->flags & (MEM_Str | MEM_Blob)) == 0) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ "text or varbinary", mem_type_to_str(b));
+ return -1;
+ }
+ if ((a->flags & (MEM_Str | MEM_Blob)) == 0) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ "text or varbinary", mem_type_to_str(a));
+ return -1;
+ }
+
+ /* Moreover, both operands must be of the same type. */
+ if ((b->flags & MEM_Str) != (a->flags & MEM_Str)) {
+ diag_set(ClientError, ER_INCONSISTENT_TYPES,
+ mem_type_to_str(a), mem_type_to_str(b));
+ return -1;
+ }
+
+ if (ExpandBlob(a) != 0 || ExpandBlob(b) != 0)
+ return -1;
+
+ uint32_t size = a->n + b->n;
+ if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big");
+ return -1;
+ }
+ if (sqlVdbeMemGrow(result, size, result == a) != 0)
+ return -1;
+
+ result->flags = a->flags & (MEM_Str | MEM_Blob);
+ if ((result->flags & MEM_Blob) != 0)
+ result->field_type = FIELD_TYPE_VARBINARY;
+ if (result != a)
+ memcpy(result->z, a->z, a->n);
+ memcpy(&result->z[a->n], b->z, b->n);
+ result->n = size;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 37f6ac5df..d17ed0593 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -322,6 +322,14 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
void
mem_move(struct Mem *to, struct Mem *from);
+/**
+ * Concatenate strings or binaries from the first and the second MEMs and write
+ * to the result MEM. In case the first MEM or the second MEM is NULL, the
+ * result MEM is set to NULL even if the result MEM is actually the first MEM.
+ */
+int
+mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e73ed173d..c017c5091 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1060,57 +1060,11 @@ case OP_ResultRow: {
* types (i.e. TEXT and BLOB).
*/
case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
- i64 nByte;
-
pIn1 = &aMem[pOp->p1];
pIn2 = &aMem[pOp->p2];
- pOut = vdbe_prepare_null_out(p, pOp->p3);
- assert(pIn1!=pOut);
- if (mem_is_any_null(pIn1, pIn2)) {
- /* Force NULL be of type STRING. */
- pOut->field_type = FIELD_TYPE_STRING;
- break;
- }
- /*
- * Concatenation operation can be applied only to
- * strings and blobs.
- */
- if (!mem_is_bytes(pIn1) || !mem_is_bytes(pIn2)) {
- char *inconsistent_type = !mem_is_bytes(pIn1) ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn2);
- diag_set(ClientError, ER_INCONSISTENT_TYPES,
- "text or varbinary", inconsistent_type);
- goto abort_due_to_error;
- }
-
- /* Moreover, both operands must be of the same type. */
- if (!mem_is_same_type(pIn1, pIn2)) {
- diag_set(ClientError, ER_INCONSISTENT_TYPES,
- mem_type_to_str(pIn2), mem_type_to_str(pIn1));
- goto abort_due_to_error;
- }
- if (ExpandBlob(pIn1) != 0 || ExpandBlob(pIn2) != 0)
+ pOut = &aMem[pOp->p3];
+ if (mem_concat(pIn2, pIn1, pOut) != 0)
goto abort_due_to_error;
- nByte = pIn1->n + pIn2->n;
- if (nByte>db->aLimit[SQL_LIMIT_LENGTH]) {
- goto too_big;
- }
- if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
- goto no_mem;
- }
- if (mem_is_str(pIn1))
- MemSetTypeFlag(pOut, MEM_Str);
- else
- MemSetTypeFlag(pOut, MEM_Blob);
- if (pOut!=pIn2) {
- memcpy(pOut->z, pIn2->z, pIn2->n);
- }
- memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);
- pOut->z[nByte]=0;
- pOut->z[nByte+1] = 0;
- pOut->flags |= MEM_Term;
- pOut->n = (int)nByte;
UPDATE_MAX_BLOBSIZE(pOut);
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat()
2021-04-13 16:57 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 23:04 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 23:22 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 23:04 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Thanks for the discussion!
>>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>>> index b417c1007..2d76ef88d 100644
>>> --- a/src/box/sql/mem.c
>>> +++ b/src/box/sql/mem.c
>>> @@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from)
>>> return 0;
>>> }
>>>
>>> +static bool
>>> +is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
>>> + enum field_type type)
>>
>> 1. Functions called 'is_*' never should change anything.
>>
> Fixed. Renamed to check_result_null().
>
>> Another question is why do you even need it? It is used in a single place,
>> where it could be just inlined. And is not used in a place, where it could
>> be applied.
>>
> I added it here since it was the first commit, which used it. This functions
> will be used in all arithmetic and bitwise operations with two operands.
I hope it is not too late to rename it to try_return_null()? Because
sorry, but 'check' also has a meaning - it is just like 'is' (does not
change anything), but sets a diag. At least this is what I see in the
code usually, and try to force in the new patches.
^ permalink raw reply [flat|nested] 107+ messages in thread
* Re: [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat()
2021-04-14 23:04 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 23:22 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 23:22 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer and diff below.
On Thu, Apr 15, 2021 at 01:04:27AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the discussion!
>
> >>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> >>> index b417c1007..2d76ef88d 100644
> >>> --- a/src/box/sql/mem.c
> >>> +++ b/src/box/sql/mem.c
> >>> @@ -326,6 +326,70 @@ mem_move(struct Mem *to, struct Mem *from)
> >>> return 0;
> >>> }
> >>>
> >>> +static bool
> >>> +is_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
> >>> + enum field_type type)
> >>
> >> 1. Functions called 'is_*' never should change anything.
> >>
> > Fixed. Renamed to check_result_null().
> >
> >> Another question is why do you even need it? It is used in a single place,
> >> where it could be just inlined. And is not used in a place, where it could
> >> be applied.
> >>
> > I added it here since it was the first commit, which used it. This functions
> > will be used in all arithmetic and bitwise operations with two operands.
>
> I hope it is not too late to rename it to try_return_null()? Because
> sorry, but 'check' also has a meaning - it is just like 'is' (does not
> change anything), but sets a diag. At least this is what I see in the
> code usually, and try to force in the new patches.
Fixed, thank you! Also fixed in next patches.
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index e06f964bd..2f2f859e3 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -182,8 +182,8 @@ mem_move(struct Mem *to, struct Mem *from)
}
static bool
-check_result_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
- enum field_type type)
+try_return_null(const struct Mem *a, const struct Mem *b, struct Mem *result,
+ enum field_type type)
{
mem_clear(result);
result->field_type = type;
@@ -195,7 +195,7 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
{
assert(result != b);
if (a != result) {
- if (check_result_null(a, b, result, FIELD_TYPE_STRING))
+ if (try_return_null(a, b, result, FIELD_TYPE_STRING))
return 0;
} else {
if (((a->flags | b->flags) & MEM_Null) != 0) {
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (17 preceding siblings ...)
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:13 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
` (22 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:57 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:02, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 3 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_arithmetic(). Function mem_arithmetic()
>> executes arithmetic operations on the first and the second MEMs and
>> writes the result to the third MEM.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 224 +++++++++++++++++++++++++++++++++++++--------
>> src/box/sql/mem.h | 6 +-
>> src/box/sql/vdbe.c | 111 +---------------------
>> 3 files changed, 193 insertions(+), 148 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index f160439c9..6120939d8 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -370,6 +370,192 @@ mem_concat(struct Mem *left, struct Mem *right, struct Mem *result)
>> return 0;
>> }
>>
>> +int
>> +mem_arithmetic(const struct Mem *left, const struct Mem *right,
>> + struct Mem *result, int op)
>> +{
>> + sqlVdbeMemSetNull(result);
>
> 1. Lets use mem_clear() directly. You delete sqlVdbeMemSetNull() later
> anyway.
>
Fixed.
>> + result->field_type = FIELD_TYPE_NUMBER;
>> + if (((left->flags | right->flags) & MEM_Null) != 0)
>> + return 0;
>> +
>> + int64_t il;
>> + bool is_l_neg;
>> + double dl;
>> + uint16_t type_left = 0;
>
> 2. Looks surprising when you use 'l' in 3 variables before
> and suddenly 'left' here.
>
> Tbh, the old names with A and B looked easier to read. Up to you,
> I can live with both.
>
Fixed, I think.
>> + if ((left->flags & MEM_Real) != 0) {
>> + dl = left->u.r;
>> + type_left = MEM_Real;
>> + } else if ((left->flags & MEM_Int) != 0) {
>> + il = left->u.i;
>> + type_left = MEM_Int;
>> + is_l_neg = true;
>> + } else if ((left->flags & MEM_UInt) != 0) {
>> + il = left->u.i;
>> + type_left = MEM_UInt;
>> + is_l_neg = false;
>> + } else if ((left->flags & (MEM_Str | MEM_Blob)) != 0) {
>> + if (sql_atoi64(left->z, &il, &is_l_neg, left->n) == 0)
>> + type_left = is_l_neg ? MEM_Int : MEM_UInt;
>> + else if (sqlAtoF(left->z, &dl, left->n) != 0)
>> + type_left = MEM_Real;
>> + }
>> +
>> + int64_t ir;
>> + bool is_r_neg;
>> + double dr;
>> + uint16_t type_right = 0;
>> + if ((right->flags & MEM_Real) != 0) {
>> + dr = right->u.r;
>> + type_right = MEM_Real;
>> + } else if ((right->flags & MEM_Int) != 0) {
>> + ir = right->u.i;
>> + type_right = MEM_Int;
>> + is_r_neg = true;
>> + } else if ((right->flags & MEM_UInt) != 0) {
>> + ir = right->u.i;
>> + type_right = MEM_UInt;
>> + is_r_neg = false;
>> + } else if ((right->flags & (MEM_Str | MEM_Blob)) != 0) {
>> + if (sql_atoi64(right->z, &ir, &is_r_neg, right->n) == 0)
>> + type_right = is_r_neg ? MEM_Int : MEM_UInt;
>> + else if (sqlAtoF(right->z, &dr, right->n) != 0)
>> + type_right = MEM_Real;
>> + }
>> +
>> + if ((type_right & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
>> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
>> + mem_str(right), "numeric");
>> + return -1;
>> + }
>> + if ((type_left & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
>> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
>> + mem_str(left), "numeric");
>> + return -1;
>> + }
>> + if (((type_left | type_right) & MEM_Real) != 0) {
>> + if (type_left == MEM_Int)
>> + dl = (double)il;
>> + else if (type_left == MEM_UInt)
>> + dl = (double)(uint64_t)il;
>> +
>> + if (type_right == MEM_Int)
>> + dr = (double)ir;
>> + else if (type_right == MEM_UInt)
>> + dr = (double)(uint64_t)ir;
>> +
>> + double dres;
>> + switch(op) {
>> + case OP_Add:
>> + dres = dl + dr;
>> + break;
>> + case OP_Subtract:
>> + dres = dl - dr;
>> + break;
>> + case OP_Multiply:
>> + dres = dl * dr;
>> + break;
>> + case OP_Divide:
>> + if (dr == 0.) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "division by zero");
>> + return -1;
>> + }
>> + dres = dl / dr;
>> + break;
>> + case OP_Remainder: {
>> + int64_t il = (int64_t)dl;
>> + int64_t ir = (int64_t)dr;
>> + if (ir == 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "division by zero");
>> + return -1;
>> + }
>> + if (ir == -1)
>> + ir = 1;
>> + dres = (double)(il % ir);
>> + break;
>> + }
>> + default:
>> + unreachable();
>> + }
>> + if (sqlIsNaN(dres))
>> + return 0;
>> + result->u.r = dres;
>> + result->flags = MEM_Real;
>> + return 0;
>> + }
>> + int64_t ires;
>> + /*
>> + * TODO: This is wrong. Both these flags should already be set. This
>> + * assignment done to not change behaviour of the function, which
>> + * is currently bugged.
>> + */
>> + is_l_neg = (left->flags & MEM_Int) != 0;
>> + is_r_neg = (right->flags & MEM_Int) != 0;
>> + bool is_res_neg;
>> + switch(op) {
>> + case OP_Add:
>> + if (sql_add_int(il, is_l_neg, ir, is_r_neg, &ires,
>> + &is_res_neg) != 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "integer is overflowed");
>> + return -1;
>> + }
>> + break;
>> + case OP_Subtract:
>> + if (sql_sub_int(il, is_l_neg, ir, is_r_neg, &ires,
>> + &is_res_neg) != 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "integer is overflowed");
>> + return -1;
>> + }
>> + break;
>> + case OP_Multiply:
>> + if (sql_mul_int(il, is_l_neg, ir, is_r_neg, &ires,
>> + &is_res_neg) != 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "integer is overflowed");
>> + return -1;
>> + }
>> + break;
>> + case OP_Divide:
>> + if (ir == 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "division by zero");
>> + return -1;
>> + }
>> + if (sql_div_int(il, is_l_neg, ir, is_r_neg, &ires,
>> + &is_res_neg) != 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "integer is overflowed");
>> + return -1;
>> + }
>> + break;
>> + case OP_Remainder: {
>> + if (ir == 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "division by zero");
>> + return -1;
>> + }
>> + if (ir == -1)
>> + ir = 1;
>> + if (sql_rem_int(il, is_l_neg, ir, is_r_neg, &ires,
>> + &is_res_neg) != 0) {
>> + diag_set(ClientError, ER_SQL_EXECUTE,
>> + "integer is overflowed");
>> + return -1;
>> + }
>> + break;
>> + }
>> + default:
>> + unreachable();
>> + }
>> + result->u.i = ires;
>> + result->flags = is_res_neg ? MEM_Int : MEM_UInt;
>> + return 0;
>
> 3. The original code looked shorter and easier to read. Why did you
> pad it out so much?
>
> Additionally, since you now have a function for doing arith, you
> could make one function for each opcode with common parts as a
> static functions in mem.c, and call them right from 'case's in
> vdbe.c. So instead of one big case,case,case,case: arith which
> also has a switch inside, we would have proper case: plus,
> case: multiply, etc. Would be faster and would make each
> individual function simpler I think.
>
> The names could be mem_arith_plus(), mem_arith_mul(), mem_arith_minus(),
> etc.
Fixed. I named new functions mem_add(), mem_sub(), mem_mul(), mem_div() and
mem_rem(). Each of them simpler than this function.
New patch:
commit 25bd849d8ebd45d3865770f1c4c6fc0c02d53d6a
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 11:51:52 2021 +0300
sql: introduce arithmetic operations for MEM
This patch introduces mem_add(), mem_sub(), mem_mul(), mem_div() and
mem_rem(), which perform arithmetic operations on two MEMs. Operands
must contain values of numeric types or values that can be converted
to a number according to implicit casting rules.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 2d76ef88d..859e337aa 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -390,6 +390,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
return 0;
}
+struct sql_num {
+ union {
+ int64_t i;
+ uint64_t u;
+ double d;
+ };
+ int type;
+ bool is_neg;
+};
+
+static int
+get_number(const struct Mem *mem, struct sql_num *number)
+{
+ if ((mem->flags & MEM_Real) != 0) {
+ number->d = mem->u.r;
+ number->type = MEM_Real;
+ return 0;
+ }
+ if ((mem->flags & MEM_Int) != 0) {
+ number->i = mem->u.i;
+ number->type = MEM_Int;
+ number->is_neg = true;
+ return 0;
+ }
+ if ((mem->flags & MEM_UInt) != 0) {
+ number->u = mem->u.u;
+ number->type = MEM_UInt;
+ number->is_neg = false;
+ return 0;
+ }
+ if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
+ return -1;
+ if ((mem->flags & MEM_Subtype) != 0)
+ return -1;
+ if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
+ number->type = number->is_neg ? MEM_Int : MEM_UInt;
+ /*
+ * The next line should be removed along with the is_neg field
+ * of struct sql_num. The integer type tells us about the sign.
+ * However, if it is removed, the behavior of arithmetic
+ * operations will change.
+ */
+ number->is_neg = (mem->flags & MEM_Int) != 0;
+ return 0;
+ }
+ if (sqlAtoF(mem->z, &number->d, mem->n) != 0) {
+ number->type = MEM_Real;
+ return 0;
+ }
+ return -1;
+}
+
+static int
+arithmetic_prepare(const struct Mem *left, const struct Mem *right,
+ struct sql_num *a, struct sql_num *b)
+{
+ if (get_number(right, b) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(right),
+ "numeric");
+ return -1;
+ }
+ if (get_number(left, a) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
+ "numeric");
+ return -1;
+ }
+ assert(a->type != 0 && b->type != 0);
+ if (a->type == MEM_Real && b->type != MEM_Real) {
+ b->d = b->type == MEM_Int ? (double)b->i : (double)b->u;
+ b->type = MEM_Real;
+ return 0;
+ }
+ if (a->type != MEM_Real && b->type == MEM_Real) {
+ a->d = a->type == MEM_Int ? (double)a->i : (double)a->u;
+ a->type = MEM_Real;
+ return 0;
+ }
+ return 0;
+}
+
+int
+mem_add(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d + b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_add_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_sub(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d - b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_sub_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_mul(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d * b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_mul_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ if (b.d == 0.) {
+ diag_set(ClientError, ER_SQL_EXECUTE,
+ "division by zero");
+ return -1;
+ }
+ result->u.r = a.d / b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ if (b.i == 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
+ return -1;
+ }
+ int64_t res;
+ bool is_neg;
+ if (sql_div_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ /*
+ * TODO: This operation works wrong when double d > INT64_MAX and
+ * d < UINT64_MAX. Also, there may be precision losses due to
+ * conversion integer to double and back.
+ */
+ a.i = a.type == MEM_Real ? (int64_t)a.d : a.i;
+ b.i = b.type == MEM_Real ? (int64_t)b.d : b.i;
+ if (b.i == 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
+ return -1;
+ }
+ int64_t res;
+ bool is_neg;
+ if (sql_rem_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -594,44 +828,6 @@ sql_value_type(sql_value *pVal)
return mem_mp_type(mem);
}
-
-/*
- * pMem currently only holds a string type (or maybe a BLOB that we can
- * interpret as a string if we want to). Compute its corresponding
- * numeric type, if has one. Set the pMem->u.r and pMem->u.i fields
- * accordingly.
- */
-static u16 SQL_NOINLINE
-computeNumericType(Mem *pMem)
-{
- assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
- assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
- if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
- return 0;
- bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
- return is_neg ? MEM_Int : MEM_UInt;
- return MEM_Real;
-}
-
-/*
- * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
- * none.
- *
- * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
- * But it does set pMem->u.r and pMem->u.i appropriately.
- */
-u16
-numericType(Mem *pMem)
-{
- if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
- return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
- if (pMem->flags & (MEM_Str|MEM_Blob)) {
- return computeNumericType(pMem);
- }
- return 0;
-}
-
/*
* The sqlValueBytes() routine returns the number of bytes in the
* sql_value object assuming that it uses the encoding "enc".
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index df273026b..69a7d9f7a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -192,6 +192,40 @@ mem_move(struct Mem *to, struct Mem *from);
int
mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
+/**
+ * Add the first MEM to the second MEM and write the result to the third MEM.
+ */
+int
+mem_add(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Subtract the second MEM from the first MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_sub(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Multiply the first MEM by the second MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_mul(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Divide the first MEM by the second MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Divide the first MEM by the second MEM and write integer part of the result
+ * to the third MEM.
+ */
+int
+mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
@@ -261,8 +295,6 @@ mem_mp_type(struct Mem *mem);
enum mp_type
sql_value_type(struct Mem *);
-u16
-numericType(Mem *pMem);
int sqlValueBytes(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6f3475147..67e1cc85a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1076,6 +1076,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Add: { /* same as TK_PLUS, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_add(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Multiply P1 P2 P3 * *
* Synopsis: r[P3]=r[P1]*r[P2]
*
@@ -1084,6 +1093,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Multiply: { /* same as TK_STAR, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_mul(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Subtract P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]-r[P1]
*
@@ -1091,6 +1109,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Subtract: { /* same as TK_MINUS, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_sub(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Divide P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]/r[P1]
*
@@ -1099,6 +1126,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* register P1 is zero, then the result is NULL. If either input is
* NULL, the result is NULL.
*/
+case OP_Divide: { /* same as TK_SLASH, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_div(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Remainder P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]%r[P1]
*
@@ -1107,120 +1143,13 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* If the value in register P1 is zero the result is NULL.
* If either operand is NULL, the result is NULL.
*/
-case OP_Add: /* same as TK_PLUS, in1, in2, out3 */
-case OP_Subtract: /* same as TK_MINUS, in1, in2, out3 */
-case OP_Multiply: /* same as TK_STAR, in1, in2, out3 */
-case OP_Divide: /* same as TK_SLASH, in1, in2, out3 */
case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
- u16 type1; /* Numeric type of left operand */
- u16 type2; /* Numeric type of right operand */
- i64 iA; /* Integer value of left operand */
- i64 iB; /* Integer value of right operand */
- double rA; /* Real value of left operand */
- double rB; /* Real value of right operand */
-
pIn1 = &aMem[pOp->p1];
- type1 = numericType(pIn1);
pIn2 = &aMem[pOp->p2];
- type2 = numericType(pIn2);
- pOut = vdbe_prepare_null_out(p, pOp->p3);
- if (mem_is_any_null(pIn1, pIn2))
- goto arithmetic_result_is_null;
- if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
- (type2 & (MEM_Int | MEM_UInt)) != 0) {
- iA = pIn1->u.i;
- iB = pIn2->u.i;
- bool is_lhs_neg = mem_is_int(pIn1) && !mem_is_uint(pIn1);
- bool is_rhs_neg = mem_is_int(pIn2) && !mem_is_uint(pIn2);
- bool is_res_neg;
- switch( pOp->opcode) {
- case OP_Add: {
- if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Subtract: {
- if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Multiply: {
- if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Divide: {
- if (iA == 0)
- goto division_by_zero;
- if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- default: {
- if (iA == 0)
- goto division_by_zero;
- if (iA==-1) iA = 1;
- if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- }
- mem_set_int(pOut, iB, is_res_neg);
- } else {
- if (sqlVdbeRealValue(pIn1, &rA) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "numeric");
- goto abort_due_to_error;
- }
- if (sqlVdbeRealValue(pIn2, &rB) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn2), "numeric");
- goto abort_due_to_error;
- }
- assert(((type1 | type2) & MEM_Real) != 0);
- switch( pOp->opcode) {
- case OP_Add: rB += rA; break;
- case OP_Subtract: rB -= rA; break;
- case OP_Multiply: rB *= rA; break;
- case OP_Divide: {
- if (rA == (double)0)
- goto division_by_zero;
- rB /= rA;
- break;
- }
- default: {
- iA = (i64)rA;
- iB = (i64)rB;
- if (iA == 0)
- goto division_by_zero;
- if (iA==-1) iA = 1;
- rB = (double)(iB % iA);
- break;
- }
- }
- if (sqlIsNaN(rB)) {
- goto arithmetic_result_is_null;
- }
- mem_set_double(pOut, rB);
- }
- break;
-
-arithmetic_result_is_null:
- /* Force NULL be of type NUMBER. */
- pOut->field_type = FIELD_TYPE_NUMBER;
+ pOut = &aMem[pOp->p3];
+ if (mem_rem(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
break;
-
-division_by_zero:
- diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
- goto abort_due_to_error;
-integer_overflow:
- diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
- goto abort_due_to_error;
}
/* Opcode: CollSeq P1 * * P4
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:13 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 17:06 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:13 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Good job on the fixes!
>> The names could be mem_arith_plus(), mem_arith_mul(), mem_arith_minus(),
>> etc.
> Fixed. I named new functions mem_add(), mem_sub(), mem_mul(), mem_div() and
> mem_rem(). Each of them simpler than this function.
The last operation is called modulo. Usually shortened to mod. But I see
we already use 'rem' in some other place, and in the token name. Up to you.
See 1 comment below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 2d76ef88d..859e337aa 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -390,6 +390,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
> +
> +static int
> +get_number(const struct Mem *mem, struct sql_num *number)
> +{
> + if ((mem->flags & MEM_Real) != 0) {
> + number->d = mem->u.r;
> + number->type = MEM_Real;
> + return 0;
> + }
> + if ((mem->flags & MEM_Int) != 0) {
> + number->i = mem->u.i;
> + number->type = MEM_Int;
> + number->is_neg = true;
> + return 0;
> + }
> + if ((mem->flags & MEM_UInt) != 0) {
> + number->u = mem->u.u;
> + number->type = MEM_UInt;
> + number->is_neg = false;
> + return 0;
> + }
> + if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
> + return -1;
> + if ((mem->flags & MEM_Subtype) != 0)
> + return -1;
> + if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
> + number->type = number->is_neg ? MEM_Int : MEM_UInt;
> + /*
> + * The next line should be removed along with the is_neg field
> + * of struct sql_num. The integer type tells us about the sign.
> + * However, if it is removed, the behavior of arithmetic
> + * operations will change.
> + */
> + number->is_neg = (mem->flags & MEM_Int) != 0;
I don't understand that. How is it possible it mismatches the
value returned from sql_atoi64()? And why isn't it just 'false' then?
Because a few lines above you already checked (mem->flags & MEM_Int) != 0
and it was false.
> + return 0;
> + }
> + if (sqlAtoF(mem->z, &number->d, mem->n) != 0) {
> + number->type = MEM_Real;
> + return 0;
> + }
> + return -1;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM
2021-04-11 18:13 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 17:06 ` Mergen Imeev via Tarantool-patches
2021-04-14 23:10 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 17:06 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers below.
On Sun, Apr 11, 2021 at 08:13:27PM +0200, Vladislav Shpilevoy wrote:
> Good job on the fixes!
>
> >> The names could be mem_arith_plus(), mem_arith_mul(), mem_arith_minus(),
> >> etc.
> > Fixed. I named new functions mem_add(), mem_sub(), mem_mul(), mem_div() and
> > mem_rem(). Each of them simpler than this function.
>
> The last operation is called modulo. Usually shortened to mod. But I see
> we already use 'rem' in some other place, and in the token name. Up to you.
>
I will leave it as it is for now. This function should be fixed at some point
(see comments in its body).
> See 1 comment below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 2d76ef88d..859e337aa 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -390,6 +390,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
> > +
> > +static int
> > +get_number(const struct Mem *mem, struct sql_num *number)
> > +{
> > + if ((mem->flags & MEM_Real) != 0) {
> > + number->d = mem->u.r;
> > + number->type = MEM_Real;
> > + return 0;
> > + }
> > + if ((mem->flags & MEM_Int) != 0) {
> > + number->i = mem->u.i;
> > + number->type = MEM_Int;
> > + number->is_neg = true;
> > + return 0;
> > + }
> > + if ((mem->flags & MEM_UInt) != 0) {
> > + number->u = mem->u.u;
> > + number->type = MEM_UInt;
> > + number->is_neg = false;
> > + return 0;
> > + }
> > + if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
> > + return -1;
> > + if ((mem->flags & MEM_Subtype) != 0)
> > + return -1;
> > + if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
> > + number->type = number->is_neg ? MEM_Int : MEM_UInt;
> > + /*
> > + * The next line should be removed along with the is_neg field
> > + * of struct sql_num. The integer type tells us about the sign.
> > + * However, if it is removed, the behavior of arithmetic
> > + * operations will change.
> > + */
> > + number->is_neg = (mem->flags & MEM_Int) != 0;
>
> I don't understand that. How is it possible it mismatches the
> value returned from sql_atoi64()? And why isn't it just 'false' then?
> Because a few lines above you already checked (mem->flags & MEM_Int) != 0
> and it was false.
>
Not exactly right. For example:
tarantool> box.execute([[SELECT '-5' + 2;]])
---
- metadata:
- name: COLUMN_1
type: integer
rows:
- [18446744073709551613]
...
As you see, this is wrong. This is due to the fact, that MEM of type string do
not have MEM_Int set. Even though this is wrong, it is expected behaviour. I
created an issue for this: #5756. Since I didn't want to change this behaviour,
I added is_neg field to struct sql_num. This is clearly a hack and should be
fixed.
> > + return 0;
> > + }
> > + if (sqlAtoF(mem->z, &number->d, mem->n) != 0) {
> > + number->type = MEM_Real;
> > + return 0;
> > + }
> > + return -1;
> > +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM
2021-04-13 17:06 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 23:10 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 23:33 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 23:10 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Thanks for the fixes!
>>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>>> index 2d76ef88d..859e337aa 100644
>>> --- a/src/box/sql/mem.c
>>> +++ b/src/box/sql/mem.c
>>> @@ -390,6 +390,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
>>> +
>>> +static int
>>> +get_number(const struct Mem *mem, struct sql_num *number)
>>> +{
>>> + if ((mem->flags & MEM_Real) != 0) {
>>> + number->d = mem->u.r;
>>> + number->type = MEM_Real;
>>> + return 0;
>>> + }
>>> + if ((mem->flags & MEM_Int) != 0) {
>>> + number->i = mem->u.i;
>>> + number->type = MEM_Int;
>>> + number->is_neg = true;
>>> + return 0;
>>> + }
>>> + if ((mem->flags & MEM_UInt) != 0) {
>>> + number->u = mem->u.u;
>>> + number->type = MEM_UInt;
>>> + number->is_neg = false;
>>> + return 0;
>>> + }
>>> + if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
>>> + return -1;
>>> + if ((mem->flags & MEM_Subtype) != 0)
>>> + return -1;
>>> + if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
>>> + number->type = number->is_neg ? MEM_Int : MEM_UInt;
>>> + /*
>>> + * The next line should be removed along with the is_neg field
>>> + * of struct sql_num. The integer type tells us about the sign.
>>> + * However, if it is removed, the behavior of arithmetic
>>> + * operations will change.
>>> + */
>>> + number->is_neg = (mem->flags & MEM_Int) != 0;
>>
>> I don't understand that. How is it possible it mismatches the
>> value returned from sql_atoi64()? And why isn't it just 'false' then?
>> Because a few lines above you already checked (mem->flags & MEM_Int) != 0
>> and it was false.
>>
> Not exactly right. For example:
>
> tarantool> box.execute([[SELECT '-5' + 2;]])
> ---
> - metadata:
> - name: COLUMN_1
> type: integer
> rows:
> - [18446744073709551613]
> ...
>
> As you see, this is wrong. This is due to the fact, that MEM of type string do
> not have MEM_Int set. Even though this is wrong, it is expected behaviour. I
> created an issue for this: #5756. Since I didn't want to change this behaviour,
> I added is_neg field to struct sql_num. This is clearly a hack and should be
> fixed.
But that does not answer the second part of my question - why can't
I set it to false here always?
====================
@@ -286,7 +286,7 @@ get_number(const struct Mem *mem, struct sql_num *number)
* However, if it is removed, the behavior of arithmetic
* operations will change.
*/
- number->is_neg = (mem->flags & MEM_Int) != 0;
+ number->is_neg = false;
return 0;
}
====================
Because (mem->flags & MEM_Int) == 0, otherwise it would return earlier above.
Also 'is_neg' is not used at all now in all places where get_number() is called.
At least in this commit. I would propose to add it in the commit which needs it
or remove it then now.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM
2021-04-14 23:10 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 23:33 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 23:33 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer below. I also included whole patch here
since there was merge conflicts due to changes function name in the last patch.
On Thu, Apr 15, 2021 at 01:10:42AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
>
> >>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> >>> index 2d76ef88d..859e337aa 100644
> >>> --- a/src/box/sql/mem.c
> >>> +++ b/src/box/sql/mem.c
> >>> @@ -390,6 +390,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
> >>> +
> >>> +static int
> >>> +get_number(const struct Mem *mem, struct sql_num *number)
> >>> +{
> >>> + if ((mem->flags & MEM_Real) != 0) {
> >>> + number->d = mem->u.r;
> >>> + number->type = MEM_Real;
> >>> + return 0;
> >>> + }
> >>> + if ((mem->flags & MEM_Int) != 0) {
> >>> + number->i = mem->u.i;
> >>> + number->type = MEM_Int;
> >>> + number->is_neg = true;
> >>> + return 0;
> >>> + }
> >>> + if ((mem->flags & MEM_UInt) != 0) {
> >>> + number->u = mem->u.u;
> >>> + number->type = MEM_UInt;
> >>> + number->is_neg = false;
> >>> + return 0;
> >>> + }
> >>> + if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
> >>> + return -1;
> >>> + if ((mem->flags & MEM_Subtype) != 0)
> >>> + return -1;
> >>> + if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
> >>> + number->type = number->is_neg ? MEM_Int : MEM_UInt;
> >>> + /*
> >>> + * The next line should be removed along with the is_neg field
> >>> + * of struct sql_num. The integer type tells us about the sign.
> >>> + * However, if it is removed, the behavior of arithmetic
> >>> + * operations will change.
> >>> + */
> >>> + number->is_neg = (mem->flags & MEM_Int) != 0;
> >>
> >> I don't understand that. How is it possible it mismatches the
> >> value returned from sql_atoi64()? And why isn't it just 'false' then?
> >> Because a few lines above you already checked (mem->flags & MEM_Int) != 0
> >> and it was false.
> >>
> > Not exactly right. For example:
> >
> > tarantool> box.execute([[SELECT '-5' + 2;]])
> > ---
> > - metadata:
> > - name: COLUMN_1
> > type: integer
> > rows:
> > - [18446744073709551613]
> > ...
> >
> > As you see, this is wrong. This is due to the fact, that MEM of type string do
> > not have MEM_Int set. Even though this is wrong, it is expected behaviour. I
> > created an issue for this: #5756. Since I didn't want to change this behaviour,
> > I added is_neg field to struct sql_num. This is clearly a hack and should be
> > fixed.
>
> But that does not answer the second part of my question - why can't
> I set it to false here always?
>
You are right, I forgot about this.
> ====================
> @@ -286,7 +286,7 @@ get_number(const struct Mem *mem, struct sql_num *number)
> * However, if it is removed, the behavior of arithmetic
> * operations will change.
> */
> - number->is_neg = (mem->flags & MEM_Int) != 0;
> + number->is_neg = false;
> return 0;
> }
> ====================
>
Thank you. I applied this diff and tested.
> Because (mem->flags & MEM_Int) == 0, otherwise it would return earlier above.
>
> Also 'is_neg' is not used at all now in all places where get_number() is called.
> At least in this commit. I would propose to add it in the commit which needs it
> or remove it then now.
No, it is used in functions sql_add_int(), sql_sub_int(), etc. Actually, this is
the only patch that use this field.
New patch:
commit 184a2407e92466657dc44d4b7f9ff80599141010
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 11:51:52 2021 +0300
sql: introduce arithmetic operations for MEM
This patch introduces mem_add(), mem_sub(), mem_mul(), mem_div() and
mem_rem(), which perform arithmetic operations on two MEMs. Operands
must contain values of numeric types or values that can be converted
to a number according to implicit casting rules.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 2f2f859e3..a8bbfd3ea 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -245,6 +245,240 @@ mem_concat(struct Mem *a, struct Mem *b, struct Mem *result)
return 0;
}
+struct sql_num {
+ union {
+ int64_t i;
+ uint64_t u;
+ double d;
+ };
+ int type;
+ bool is_neg;
+};
+
+static int
+get_number(const struct Mem *mem, struct sql_num *number)
+{
+ if ((mem->flags & MEM_Real) != 0) {
+ number->d = mem->u.r;
+ number->type = MEM_Real;
+ return 0;
+ }
+ if ((mem->flags & MEM_Int) != 0) {
+ number->i = mem->u.i;
+ number->type = MEM_Int;
+ number->is_neg = true;
+ return 0;
+ }
+ if ((mem->flags & MEM_UInt) != 0) {
+ number->u = mem->u.u;
+ number->type = MEM_UInt;
+ number->is_neg = false;
+ return 0;
+ }
+ if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
+ return -1;
+ if ((mem->flags & MEM_Subtype) != 0)
+ return -1;
+ if (sql_atoi64(mem->z, &number->i, &number->is_neg, mem->n) == 0) {
+ number->type = number->is_neg ? MEM_Int : MEM_UInt;
+ /*
+ * The next line should be removed along with the is_neg field
+ * of struct sql_num. The integer type tells us about the sign.
+ * However, if it is removed, the behavior of arithmetic
+ * operations will change.
+ */
+ number->is_neg = false;
+ return 0;
+ }
+ if (sqlAtoF(mem->z, &number->d, mem->n) != 0) {
+ number->type = MEM_Real;
+ return 0;
+ }
+ return -1;
+}
+
+static int
+arithmetic_prepare(const struct Mem *left, const struct Mem *right,
+ struct sql_num *a, struct sql_num *b)
+{
+ if (get_number(right, b) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(right),
+ "numeric");
+ return -1;
+ }
+ if (get_number(left, a) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
+ "numeric");
+ return -1;
+ }
+ assert(a->type != 0 && b->type != 0);
+ if (a->type == MEM_Real && b->type != MEM_Real) {
+ b->d = b->type == MEM_Int ? (double)b->i : (double)b->u;
+ b->type = MEM_Real;
+ return 0;
+ }
+ if (a->type != MEM_Real && b->type == MEM_Real) {
+ a->d = a->type == MEM_Int ? (double)a->i : (double)a->u;
+ a->type = MEM_Real;
+ return 0;
+ }
+ return 0;
+}
+
+int
+mem_add(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (try_return_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d + b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_add_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_sub(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (try_return_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d - b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_sub_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_mul(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (try_return_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ result->u.r = a.d * b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ int64_t res;
+ bool is_neg;
+ if (sql_mul_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (try_return_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ if (a.type == MEM_Real) {
+ if (b.d == 0.) {
+ diag_set(ClientError, ER_SQL_EXECUTE,
+ "division by zero");
+ return -1;
+ }
+ result->u.r = a.d / b.d;
+ result->flags = MEM_Real;
+ return 0;
+ }
+
+ if (b.i == 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
+ return -1;
+ }
+ int64_t res;
+ bool is_neg;
+ if (sql_div_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (try_return_null(left, right, result, FIELD_TYPE_NUMBER))
+ return 0;
+
+ struct sql_num a, b;
+ if (arithmetic_prepare(left, right, &a, &b) != 0)
+ return -1;
+
+ assert(a.type != MEM_Real || a.type == b.type);
+ /*
+ * TODO: This operation works wrong when double d > INT64_MAX and
+ * d < UINT64_MAX. Also, there may be precision losses due to
+ * conversion integer to double and back.
+ */
+ a.i = a.type == MEM_Real ? (int64_t)a.d : a.i;
+ b.i = b.type == MEM_Real ? (int64_t)b.d : b.i;
+ if (b.i == 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
+ return -1;
+ }
+ int64_t res;
+ bool is_neg;
+ if (sql_rem_int(a.i, a.is_neg, b.i, b.is_neg, &res, &is_neg) != 0) {
+ diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
+ return -1;
+ }
+ result->u.i = res;
+ result->flags = is_neg ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -449,44 +683,6 @@ sql_value_type(sql_value *pVal)
return mem_mp_type(mem);
}
-
-/*
- * pMem currently only holds a string type (or maybe a BLOB that we can
- * interpret as a string if we want to). Compute its corresponding
- * numeric type, if has one. Set the pMem->u.r and pMem->u.i fields
- * accordingly.
- */
-static u16 SQL_NOINLINE
-computeNumericType(Mem *pMem)
-{
- assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
- assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
- if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
- return 0;
- bool is_neg;
- if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
- return is_neg ? MEM_Int : MEM_UInt;
- return MEM_Real;
-}
-
-/*
- * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
- * none.
- *
- * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
- * But it does set pMem->u.r and pMem->u.i appropriately.
- */
-u16
-numericType(Mem *pMem)
-{
- if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
- return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
- if (pMem->flags & (MEM_Str|MEM_Blob)) {
- return computeNumericType(pMem);
- }
- return 0;
-}
-
/*
* The sqlValueBytes() routine returns the number of bytes in the
* sql_value object assuming that it uses the encoding "enc".
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d17ed0593..9539fbbd1 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -330,6 +330,40 @@ mem_move(struct Mem *to, struct Mem *from);
int
mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
+/**
+ * Add the first MEM to the second MEM and write the result to the third MEM.
+ */
+int
+mem_add(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Subtract the second MEM from the first MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_sub(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Multiply the first MEM by the second MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_mul(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Divide the first MEM by the second MEM and write the result to the third
+ * MEM.
+ */
+int
+mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Divide the first MEM by the second MEM and write integer part of the result
+ * to the third MEM.
+ */
+int
+mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -347,8 +381,6 @@ mem_mp_type(struct Mem *mem);
enum mp_type
sql_value_type(struct Mem *);
-u16
-numericType(Mem *pMem);
int sqlValueBytes(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e78229581..fb2a5ccc1 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1076,6 +1076,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Add: { /* same as TK_PLUS, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_add(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Multiply P1 P2 P3 * *
* Synopsis: r[P3]=r[P1]*r[P2]
*
@@ -1084,6 +1093,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Multiply: { /* same as TK_STAR, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_mul(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Subtract P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]-r[P1]
*
@@ -1091,6 +1109,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* and store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_Subtract: { /* same as TK_MINUS, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_sub(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Divide P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]/r[P1]
*
@@ -1099,6 +1126,15 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* register P1 is zero, then the result is NULL. If either input is
* NULL, the result is NULL.
*/
+case OP_Divide: { /* same as TK_SLASH, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_div(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ break;
+}
+
/* Opcode: Remainder P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]%r[P1]
*
@@ -1107,120 +1143,13 @@ case OP_Concat: { /* same as TK_CONCAT, in1, in2, out3 */
* If the value in register P1 is zero the result is NULL.
* If either operand is NULL, the result is NULL.
*/
-case OP_Add: /* same as TK_PLUS, in1, in2, out3 */
-case OP_Subtract: /* same as TK_MINUS, in1, in2, out3 */
-case OP_Multiply: /* same as TK_STAR, in1, in2, out3 */
-case OP_Divide: /* same as TK_SLASH, in1, in2, out3 */
case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
- u16 type1; /* Numeric type of left operand */
- u16 type2; /* Numeric type of right operand */
- i64 iA; /* Integer value of left operand */
- i64 iB; /* Integer value of right operand */
- double rA; /* Real value of left operand */
- double rB; /* Real value of right operand */
-
pIn1 = &aMem[pOp->p1];
- type1 = numericType(pIn1);
pIn2 = &aMem[pOp->p2];
- type2 = numericType(pIn2);
- pOut = vdbe_prepare_null_out(p, pOp->p3);
- if (mem_is_any_null(pIn1, pIn2))
- goto arithmetic_result_is_null;
- if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
- (type2 & (MEM_Int | MEM_UInt)) != 0) {
- iA = pIn1->u.i;
- iB = pIn2->u.i;
- bool is_lhs_neg = mem_is_nint(pIn1);
- bool is_rhs_neg = mem_is_nint(pIn2);
- bool is_res_neg;
- switch( pOp->opcode) {
- case OP_Add: {
- if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Subtract: {
- if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Multiply: {
- if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- case OP_Divide: {
- if (iA == 0)
- goto division_by_zero;
- if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- default: {
- if (iA == 0)
- goto division_by_zero;
- if (iA==-1) iA = 1;
- if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
- (int64_t *) &iB, &is_res_neg) != 0)
- goto integer_overflow;
- break;
- }
- }
- mem_set_int(pOut, iB, is_res_neg);
- } else {
- if (sqlVdbeRealValue(pIn1, &rA) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "numeric");
- goto abort_due_to_error;
- }
- if (sqlVdbeRealValue(pIn2, &rB) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn2), "numeric");
- goto abort_due_to_error;
- }
- assert(((type1 | type2) & MEM_Real) != 0);
- switch( pOp->opcode) {
- case OP_Add: rB += rA; break;
- case OP_Subtract: rB -= rA; break;
- case OP_Multiply: rB *= rA; break;
- case OP_Divide: {
- if (rA == (double)0)
- goto division_by_zero;
- rB /= rA;
- break;
- }
- default: {
- iA = (i64)rA;
- iB = (i64)rB;
- if (iA == 0)
- goto division_by_zero;
- if (iA==-1) iA = 1;
- rB = (double)(iB % iA);
- break;
- }
- }
- if (sqlIsNaN(rB)) {
- goto arithmetic_result_is_null;
- }
- mem_set_double(pOut, rB);
- }
- break;
-
-arithmetic_result_is_null:
- /* Force NULL be of type NUMBER. */
- pOut->field_type = FIELD_TYPE_NUMBER;
+ pOut = &aMem[pOp->p3];
+ if (mem_rem(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
break;
-
-division_by_zero:
- diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
- goto abort_due_to_error;
-integer_overflow:
- diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
- goto abort_due_to_error;
}
/* Opcode: CollSeq P1 * * P4
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (18 preceding siblings ...)
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM Mergen Imeev via Tarantool-patches
@ 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches
2021-04-11 18:16 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM Mergen Imeev via Tarantool-patches
` (21 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 17:57 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:03, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> I don't understand. Why are there still sqlMemCompare(), sqlBlobCompare()
> if you also have mem_compare(), compare_blobs(), compare_numbers()?
>
Functions compare_blobs(), compare_numbers() and compare_strings() are used to
simplify mem_compare(). Also, now I use them in sqlMemCompare(). Function
mem_compare() is used to perform comparison of two values in opcodes OP_Ge,
OP_Le, etc. Function sqlMemCompare() is used for comparison in other parts.
There two functions Have a bit different way to compare MEMs, which is whong, I
believe. Still, removing each of them will change behaviour, so I decided to
left them both be for now. Function sqlBlobCompare() is the same as function
compare_blobs() (actually it is vice versa), so I removed this function.
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 6120939d8..8119644ed 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -556,6 +556,247 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>> return 0;
>> }
>>
>> +static int
>> +compare_blobs(const struct Mem *left, const struct Mem *right, int *result)> +{
>> + int nl = left->n;
>> + int nr = right->n;
>> + int minlen = MIN(nl, nr);
>> +
>> + /*
>> + * It is possible to have a Blob value that has some non-zero content
>> + * followed by zero content. But that only comes up for Blobs formed
>> + * by the OP_MakeRecord opcode, and such Blobs never get passed into
>> + * mem_compare().
>> + */
>> + assert((left->flags & MEM_Zero) == 0 || nl == 0);
>> + assert((right->flags & MEM_Zero) == 0 || nr == 0);
>> +
>> + if (left->flags & right->flags & MEM_Zero) {
>
> Please, use explicit != 0, in the other places below too.
Fixed.
New patch:
commit 5a71515faac5f305fc3bd1ebbc20d1dd65bf027c
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 13:13:34 2021 +0300
sql: introduce mem_compare()
This patch introduces mem_compare(). Function mem_compare() compares the
first and the second MEMs and returns result of comparison.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 859e337aa..eee72a7fe 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -624,6 +624,211 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
return 0;
}
+static int
+compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
+{
+ int an = a->n;
+ int bn = b->n;
+ int minlen = MIN(an, bn);
+
+ /*
+ * It is possible to have a Blob value that has some non-zero content
+ * followed by zero content. But that only comes up for Blobs formed
+ * by the OP_MakeRecord opcode, and such Blobs never get passed into
+ * mem_compare().
+ */
+ assert((a->flags & MEM_Zero) == 0 || an == 0);
+ assert((b->flags & MEM_Zero) == 0 || bn == 0);
+
+ if ((a->flags & b->flags & MEM_Zero) != 0) {
+ *result = a->u.nZero - b->u.nZero;
+ return 0;
+ }
+ if ((a->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (b->z[i] != 0) {
+ *result = -1;
+ return 0;
+ }
+ }
+ *result = a->u.nZero - bn;
+ return 0;
+ }
+ if ((b->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (a->z[i] != 0){
+ *result = 1;
+ return 0;
+ }
+ }
+ *result = b->u.nZero - an;
+ return 0;
+ }
+ *result = memcmp(a->z, b->z, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
+static int
+compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
+{
+ struct sql_num a, b;
+ /* TODO: Here should be check for right value type. */
+ if (get_number(right, &b) != 0) {
+ *result = -1;
+ return 0;
+ }
+ if (get_number(left, &a) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
+ "numeric");
+ return -1;
+ }
+ if (a.type == MEM_Real) {
+ if (b.type == MEM_Real) {
+ if (a.d > b.d)
+ *result = 1;
+ else if (a.d < b.d)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = double_compare_nint64(a.d, b.i, 1);
+ else
+ *result = double_compare_uint64(a.d, b.u, 1);
+ return 0;
+ }
+ if (a.type == MEM_Int) {
+ if (b.type == MEM_Int) {
+ if (a.i > b.i)
+ *result = 1;
+ else if (a.i < b.i)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_UInt)
+ *result = -1;
+ else
+ *result = double_compare_nint64(b.d, a.i, -1);
+ return 0;
+ }
+ assert(a.type == MEM_UInt);
+ if (b.type == MEM_UInt) {
+ if (a.u > b.u)
+ *result = 1;
+ else if (a.u < b.u)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = 1;
+ else
+ *result = double_compare_uint64(b.d, a.u, -1);
+ return 0;
+}
+
+static int
+compare_strings(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll)
+{
+ char *a;
+ uint32_t an;
+ char bufl[BUF_SIZE];
+ if ((left->flags & MEM_Str) != 0) {
+ a = left->z;
+ an = left->n;
+ } else {
+ assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ a = &bufl[0];
+ if ((left->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, a, "%lld", left->u.i);
+ else if ((left->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, a, "%llu", left->u.u);
+ else
+ sql_snprintf(BUF_SIZE, a, "%!.15g", left->u.r);
+ an = strlen(a);
+ }
+
+ char *b;
+ uint32_t bn;
+ char bufr[BUF_SIZE];
+ if ((right->flags & MEM_Str) != 0) {
+ b = right->z;
+ bn = right->n;
+ } else {
+ assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ b = &bufr[0];
+ if ((right->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, b, "%lld", right->u.i);
+ else if ((right->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, b, "%llu", right->u.u);
+ else
+ sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
+ bn = strlen(b);
+ }
+ if (coll) {
+ *result = coll->cmp(a, an, b, bn, coll);
+ return 0;
+ }
+ uint32_t minlen = MIN(an, bn);
+ *result = memcmp(a, b, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
+int
+mem_compare(const struct Mem *left, const struct Mem *right, int *result,
+ enum field_type type, struct coll *coll)
+{
+ assert(((left->flags | right->flags) & MEM_Null) == 0);
+ int flags_any = left->flags | right->flags;
+ int flags_all = left->flags & right->flags;
+
+ if ((flags_all & MEM_Bool) != 0) {
+ if (left->u.b == right->u.b)
+ *result = 0;
+ else if (left->u.b)
+ *result = 1;
+ else
+ *result = -1;
+ return 0;
+ }
+ if ((flags_any & MEM_Bool) != 0) {
+ char *str = (left->flags & MEM_Bool) == 0 ?
+ mem_type_to_str(left) : mem_type_to_str(right);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "boolean");
+ return -1;
+ }
+
+ if ((flags_all & MEM_Blob) != 0)
+ return compare_blobs(left, right, result);
+ if ((flags_any & MEM_Blob) != 0) {
+ char *str = (left->flags & MEM_Blob) == 0 ?
+ mem_type_to_str(left) : mem_type_to_str(right);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "varbinary");
+ return -1;
+ }
+
+ if (type == FIELD_TYPE_STRING)
+ return compare_strings(left, right, result, coll);
+
+ if (sql_type_is_numeric(type) ||
+ (flags_any & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+ return compare_numbers(left, right, result);
+
+ assert((left->flags & MEM_Str) != 0 && (right->flags & MEM_Str) != 0);
+ return compare_strings(left, right, result, coll);
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -2071,45 +2276,6 @@ sqlVdbeMemTooBig(Mem * p)
return 0;
}
-/*
- * Compare two blobs. Return negative, zero, or positive if the first
- * is less than, equal to, or greater than the second, respectively.
- * If one blob is a prefix of the other, then the shorter is the lessor.
- */
-static SQL_NOINLINE int
-sqlBlobCompare(const Mem * pB1, const Mem * pB2)
-{
- int c;
- int n1 = pB1->n;
- int n2 = pB2->n;
-
- /* It is possible to have a Blob value that has some non-zero content
- * followed by zero content. But that only comes up for Blobs formed
- * by the OP_MakeRecord opcode, and such Blobs never get passed into
- * sqlMemCompare().
- */
- assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
- assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
-
- if ((pB1->flags | pB2->flags) & MEM_Zero) {
- if (pB1->flags & pB2->flags & MEM_Zero) {
- return pB1->u.nZero - pB2->u.nZero;
- } else if (pB1->flags & MEM_Zero) {
- if (!isAllZero(pB2->z, pB2->n))
- return -1;
- return pB1->u.nZero - n2;
- } else {
- if (!isAllZero(pB1->z, pB1->n))
- return +1;
- return n1 - pB2->u.nZero;
- }
- }
- c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
- if (c)
- return c;
- return n1 - n2;
-}
-
/*
* Compare the values contained by the two memory cells, returning
* negative, zero or positive if pMem1 is less than, equal to, or greater
@@ -2123,6 +2289,7 @@ int
sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
{
int f1, f2;
+ int res;
int combined_flags;
f1 = pMem1->flags;
@@ -2152,57 +2319,12 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
/* At least one of the two values is a number
*/
if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- if ((f1 & f2 & MEM_Int) != 0) {
- if (pMem1->u.i < pMem2->u.i)
- return -1;
- if (pMem1->u.i > pMem2->u.i)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_UInt) != 0) {
- if (pMem1->u.u < pMem2->u.u)
- return -1;
- if (pMem1->u.u > pMem2->u.u)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_Real) != 0) {
- if (pMem1->u.r < pMem2->u.r)
- return -1;
- if (pMem1->u.r > pMem2->u.r)
- return +1;
- return 0;
- }
- if ((f1 & MEM_Int) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_nint64(pMem2->u.r,
- pMem1->u.i, -1);
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_UInt) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_uint64(pMem2->u.r,
- pMem1->u.u, -1);
- } else if ((f2 & MEM_Int) != 0) {
- return +1;
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_Real) != 0) {
- if ((f2 & MEM_Int) != 0) {
- return double_compare_nint64(pMem1->u.r,
- pMem2->u.i, 1);
- } else if ((f2 & MEM_UInt) != 0) {
- return double_compare_uint64(pMem1->u.r,
- pMem2->u.u, 1);
- } else {
- return -1;
- }
- }
- return +1;
+ if ((f1 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return +1;
+ if ((f2 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return -1;
+ compare_numbers(pMem1, pMem2, &res);
+ return res;
}
/* If one value is a string and the other is a blob, the string is less.
@@ -2215,27 +2337,13 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
if ((f2 & MEM_Str) == 0) {
return -1;
}
- /* The collation sequence must be defined at this point, even if
- * the user deletes the collation sequence after the vdbe program is
- * compiled (this was not always the case).
- */
- if (pColl) {
- return vdbeCompareMemString(pMem1, pMem2, pColl);
- } else {
- size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
- int res;
- res = memcmp(pMem1->z, pMem2->z, n);
- if (res == 0)
- res = (int)pMem1->n - (int)pMem2->n;
- return res;
- }
- /* If a NULL pointer was passed as the collate function, fall through
- * to the blob case and use memcmp().
- */
+ compare_strings(pMem1, pMem2, &res, pColl);
+ return res;
}
/* Both values must be blobs. Compare using memcmp(). */
- return sqlBlobCompare(pMem1, pMem2);
+ compare_blobs(pMem1, pMem2, &res);
+ return res;
}
bool
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 69a7d9f7a..6c022d8d8 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -226,6 +226,11 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
int
mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+/** Compare two non-NULL MEMs and return the result of comparison. */
+int
+mem_compare(const struct Mem *left, const struct Mem *right, int *result,
+ enum field_type type, struct coll *coll);
+
/* One or more of the following flags are set to indicate the validOK
* representations of the value stored in the Mem struct.
*
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 67e1cc85a..6a923a8b6 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1607,15 +1607,9 @@ case OP_Le: /* same as TK_LE, jump, in1, in3 */
case OP_Gt: /* same as TK_GT, jump, in1, in3 */
case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
int res, res2; /* Result of the comparison of pIn1 against pIn3 */
- u32 flags1; /* Copy of initial value of pIn1->flags */
- u32 flags3; /* Copy of initial value of pIn3->flags */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
- flags1 = pIn1->flags;
- flags3 = pIn3->flags;
- enum field_type ft_p1 = pIn1->field_type;
- enum field_type ft_p3 = pIn3->field_type;
if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
@@ -1649,82 +1643,11 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
- } else if (mem_is_bool(pIn1) || mem_is_bool(pIn3) ||
- mem_is_bin(pIn1) || mem_is_bin(pIn3)) {
- if (!mem_is_same_type(pIn1, pIn3)) {
- char *inconsistent_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn3) :
- mem_type_to_str(pIn1);
- char *expected_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn3);
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- inconsistent_type, expected_type);
- goto abort_due_to_error;
- }
- res = sqlMemCompare(pIn3, pIn1, NULL);
} else {
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
- if (sql_type_is_numeric(type)) {
- if (mem_is_str(pIn1)) {
- mem_apply_numeric_type(pIn1);
- flags3 = pIn3->flags;
- }
- if (mem_is_str(pIn3)) {
- if (mem_apply_numeric_type(pIn3) != 0) {
- diag_set(ClientError,
- ER_SQL_TYPE_MISMATCH,
- mem_str(pIn3),
- "numeric");
- goto abort_due_to_error;
- }
- }
- /* Handle the common case of integer comparison here, as an
- * optimization, to avoid a call to sqlMemCompare()
- */
- if (mem_is_int(pIn1) && mem_is_int(pIn3)) {
- if (!mem_is_uint(pIn1) && !mem_is_uint(pIn3)) {
- if (pIn3->u.i > pIn1->u.i)
- res = +1;
- else if (pIn3->u.i < pIn1->u.i)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && mem_is_uint(pIn3)) {
- if (pIn3->u.u > pIn1->u.u)
- res = +1;
- else if (pIn3->u.u < pIn1->u.u)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && !mem_is_uint(pIn3)) {
- res = -1;
- goto compare_op;
- }
- res = 1;
- goto compare_op;
- }
- } else if (type == FIELD_TYPE_STRING) {
- if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
- sqlVdbeMemStringify(pIn1);
- flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
- assert(pIn1!=pIn3);
- }
- if (!mem_is_str(pIn3) && mem_is_num(pIn3)) {
- sqlVdbeMemStringify(pIn3);
- flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
- }
- }
- assert(pOp->p4type==P4_COLLSEQ || pOp->p4.pColl==0);
- res = sqlMemCompare(pIn3, pIn1, pOp->p4.pColl);
+ if (mem_compare(pIn3, pIn1, &res, type, pOp->p4.pColl))
+ goto abort_due_to_error;
}
- compare_op:
switch( pOp->opcode) {
case OP_Eq: res2 = res==0; break;
case OP_Ne: res2 = res; break;
@@ -1734,14 +1657,6 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
default: res2 = res>=0; break;
}
- /* Undo any changes made by mem_apply_type() to the input registers. */
- assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
- pIn1->flags = flags1;
- pIn1->field_type = ft_p1;
- assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
- pIn3->flags = flags3;
- pIn3->field_type = ft_p3;
-
if (pOp->p5 & SQL_STOREP2) {
iCompare = res;
res2 = res2!=0; /* For this path res2 must be exactly 0 or 1 */
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare()
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
@ 2021-04-11 18:16 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 18:33 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-11 18:16 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Nice fixes!
This is the last email for today, I will continue the review of
the patchset tomorrow.
See 6 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 859e337aa..eee72a7fe 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -624,6 +624,211 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
> return 0;
> }
>
> +static int
> +compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
> +{
1. Would be good to have an assertion here that both types are MEM_Blob.
> + int an = a->n;
> + int bn = b->n;
> + int minlen = MIN(an, bn);
> +
> + /*
> + * It is possible to have a Blob value that has some non-zero content
> + * followed by zero content. But that only comes up for Blobs formed
> + * by the OP_MakeRecord opcode, and such Blobs never get passed into
> + * mem_compare().
> + */
> + assert((a->flags & MEM_Zero) == 0 || an == 0);
> + assert((b->flags & MEM_Zero) == 0 || bn == 0);
> +
> + if ((a->flags & b->flags & MEM_Zero) != 0) {
> + *result = a->u.nZero - b->u.nZero;
> + return 0;
> + }
> + if ((a->flags & MEM_Zero) != 0) {
> + for (int i = 0; i < minlen; ++i) {
> + if (b->z[i] != 0) {
> + *result = -1;
> + return 0;
> + }
> + }
> + *result = a->u.nZero - bn;
> + return 0;
> + }
> + if ((b->flags & MEM_Zero) != 0) {
> + for (int i = 0; i < minlen; ++i) {
> + if (a->z[i] != 0){
> + *result = 1;
> + return 0;
> + }
> + }
> + *result = b->u.nZero - an;
> + return 0;
> + }
> + *result = memcmp(a->z, b->z, minlen);
> + if (*result != 0)
> + return 0;
> + *result = an - bn;
> + return 0;
2. compare_blobs never fails. So you can drop result out argument
and return the comparison result as 'return'.
> +}
> +
> +static int
> +compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
> +{
> + struct sql_num a, b;
> + /* TODO: Here should be check for right value type. */
3. What if 'b' is a string, which can't be converted to a number?
> + if (get_number(right, &b) != 0) {
> + *result = -1;
> + return 0;
> + }
> + if (get_number(left, &a) != 0) {> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
> + "numeric");
> + return -1;
> + }
> + if (a.type == MEM_Real) {
> + if (b.type == MEM_Real) {
> + if (a.d > b.d)
> + *result = 1;
> + else if (a.d < b.d)
> + *result = -1;
> + else
> + *result = 0;
> + return 0;
> + }
> + if (b.type == MEM_Int)
> + *result = double_compare_nint64(a.d, b.i, 1);
> + else
> + *result = double_compare_uint64(a.d, b.u, 1);
> + return 0;
> + }
> + if (a.type == MEM_Int) {
> + if (b.type == MEM_Int) {
> + if (a.i > b.i)
> + *result = 1;
> + else if (a.i < b.i)
> + *result = -1;
> + else
> + *result = 0;
> + return 0;
> + }
> + if (b.type == MEM_UInt)
> + *result = -1;
> + else
> + *result = double_compare_nint64(b.d, a.i, -1);
> + return 0;
> + }
> + assert(a.type == MEM_UInt);
> + if (b.type == MEM_UInt) {
> + if (a.u > b.u)
> + *result = 1;
> + else if (a.u < b.u)
> + *result = -1;
> + else
> + *result = 0;
> + return 0;
> + }
> + if (b.type == MEM_Int)
> + *result = 1;
> + else
> + *result = double_compare_uint64(b.d, a.u, -1);
> + return 0;
> +}
> +
> +static int
> +compare_strings(const struct Mem *left, const struct Mem *right, int *result,
> + const struct coll *coll)
> +{
> + char *a;
> + uint32_t an;
> + char bufl[BUF_SIZE];
> + if ((left->flags & MEM_Str) != 0) {
> + a = left->z;
> + an = left->n;
> + } else {
> + assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
> + a = &bufl[0];
> + if ((left->flags & MEM_Int) != 0)
> + sql_snprintf(BUF_SIZE, a, "%lld", left->u.i);
> + else if ((left->flags & MEM_UInt) != 0)
> + sql_snprintf(BUF_SIZE, a, "%llu", left->u.u);
> + else
> + sql_snprintf(BUF_SIZE, a, "%!.15g", left->u.r);
> + an = strlen(a);
> + }
> +
> + char *b;
> + uint32_t bn;
> + char bufr[BUF_SIZE];
> + if ((right->flags & MEM_Str) != 0) {
> + b = right->z;
> + bn = right->n;
> + } else {
> + assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
> + b = &bufr[0];
> + if ((right->flags & MEM_Int) != 0)
> + sql_snprintf(BUF_SIZE, b, "%lld", right->u.i);
> + else if ((right->flags & MEM_UInt) != 0)
> + sql_snprintf(BUF_SIZE, b, "%llu", right->u.u);
> + else
> + sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
> + bn = strlen(b);
> + }
> + if (coll) {
> + *result = coll->cmp(a, an, b, bn, coll);
> + return 0;
> + }
> + uint32_t minlen = MIN(an, bn);
> + *result = memcmp(a, b, minlen);
> + if (*result != 0)
> + return 0;
> + *result = an - bn;
> + return 0;
4. It can't fail either. You can return result as 'return'.
> +}
> +
> +int
> +mem_compare(const struct Mem *left, const struct Mem *right, int *result,
> + enum field_type type, struct coll *coll)
> +{
> + assert(((left->flags | right->flags) & MEM_Null) == 0);
> + int flags_any = left->flags | right->flags;
5. 'any' isn't needed until the next branch. You can move it right above
if ((flags_any & MEM_Bool) != 0) {
> + int flags_all = left->flags & right->flags;
> +
> + if ((flags_all & MEM_Bool) != 0) {
> + if (left->u.b == right->u.b)
> + *result = 0;
> + else if (left->u.b)
> + *result = 1;
> + else
> + *result = -1;
> + return 0;
> + }
> + if ((flags_any & MEM_Bool) != 0) {
> + char *str = (left->flags & MEM_Bool) == 0 ?
> + mem_type_to_str(left) : mem_type_to_str(right);
> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "boolean");
> + return -1;
> + }
> +
> + if ((flags_all & MEM_Blob) != 0)
> + return compare_blobs(left, right, result);
> + if ((flags_any & MEM_Blob) != 0) {
> + char *str = (left->flags & MEM_Blob) == 0 ?
> + mem_type_to_str(left) : mem_type_to_str(right);
> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "varbinary");
> + return -1;
> + }
> +
> + if (type == FIELD_TYPE_STRING)
> + return compare_strings(left, right, result, coll);
> +
> + if (sql_type_is_numeric(type) ||
> + (flags_any & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
> + return compare_numbers(left, right, result);
> +
> + assert((left->flags & MEM_Str) != 0 && (right->flags & MEM_Str) != 0);
> + return compare_strings(left, right, result, coll);
> +}
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index 69a7d9f7a..6c022d8d8 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -226,6 +226,11 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
> int
> mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
>
> +/** Compare two non-NULL MEMs and return the result of comparison. */
> +int
> +mem_compare(const struct Mem *left, const struct Mem *right, int *result,
> + enum field_type type, struct coll *coll);
6. What is 'type'?
Can you keep it out of the comparator somehow? For example, make it
3 functions: mem_cmp_as_str, mem_cmp_as_num, and just a generic mem_cmp
without any types calling the first 2 comparators. Otherwise the type
thing is super ugly. It leaks the details of opcodes into mem.c.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare()
2021-04-11 18:16 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 18:33 ` Mergen Imeev via Tarantool-patches
2021-04-14 23:20 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 18:33 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers, diff and new patch below. The patch was
almost completely rewritten.
On Sun, Apr 11, 2021 at 08:16:30PM +0200, Vladislav Shpilevoy wrote:
> Nice fixes!
>
> This is the last email for today, I will continue the review of
> the patchset tomorrow.
>
> See 6 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 859e337aa..eee72a7fe 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -624,6 +624,211 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
> > return 0;
> > }
> >
> > +static int
> > +compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
> > +{
>
> 1. Would be good to have an assertion here that both types are MEM_Blob.
>
Instead of assert I added 'if' here. Also, now this function is not static. See
the last comment.
> > + int an = a->n;
> > + int bn = b->n;
> > + int minlen = MIN(an, bn);
> > +
> > + /*
> > + * It is possible to have a Blob value that has some non-zero content
> > + * followed by zero content. But that only comes up for Blobs formed
> > + * by the OP_MakeRecord opcode, and such Blobs never get passed into
> > + * mem_compare().
> > + */
> > + assert((a->flags & MEM_Zero) == 0 || an == 0);
> > + assert((b->flags & MEM_Zero) == 0 || bn == 0);
> > +
> > + if ((a->flags & b->flags & MEM_Zero) != 0) {
> > + *result = a->u.nZero - b->u.nZero;
> > + return 0;
> > + }
> > + if ((a->flags & MEM_Zero) != 0) {
> > + for (int i = 0; i < minlen; ++i) {
> > + if (b->z[i] != 0) {
> > + *result = -1;
> > + return 0;
> > + }
> > + }
> > + *result = a->u.nZero - bn;
> > + return 0;
> > + }
> > + if ((b->flags & MEM_Zero) != 0) {
> > + for (int i = 0; i < minlen; ++i) {
> > + if (a->z[i] != 0){
> > + *result = 1;
> > + return 0;
> > + }
> > + }
> > + *result = b->u.nZero - an;
> > + return 0;
> > + }
> > + *result = memcmp(a->z, b->z, minlen);
> > + if (*result != 0)
> > + return 0;
> > + *result = an - bn;
> > + return 0;
>
> 2. compare_blobs never fails. So you can drop result out argument
> and return the comparison result as 'return'.
>
See the last comment.
> > +}
> > +
> > +static int
> > +compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
> > +{
> > + struct sql_num a, b;
> > + /* TODO: Here should be check for right value type. */
>
> 3. What if 'b' is a string, which can't be converted to a number?
>
Then string will be greater than number. See issue #5891.
> > + if (get_number(right, &b) != 0) {
> > + *result = -1;
> > + return 0;
> > + }
> > + if (get_number(left, &a) != 0) {> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
> > + "numeric");
> > + return -1;
> > + }
> > + if (a.type == MEM_Real) {
> > + if (b.type == MEM_Real) {
> > + if (a.d > b.d)
> > + *result = 1;
> > + else if (a.d < b.d)
> > + *result = -1;
> > + else
> > + *result = 0;
> > + return 0;
> > + }
> > + if (b.type == MEM_Int)
> > + *result = double_compare_nint64(a.d, b.i, 1);
> > + else
> > + *result = double_compare_uint64(a.d, b.u, 1);
> > + return 0;
> > + }
> > + if (a.type == MEM_Int) {
> > + if (b.type == MEM_Int) {
> > + if (a.i > b.i)
> > + *result = 1;
> > + else if (a.i < b.i)
> > + *result = -1;
> > + else
> > + *result = 0;
> > + return 0;
> > + }
> > + if (b.type == MEM_UInt)
> > + *result = -1;
> > + else
> > + *result = double_compare_nint64(b.d, a.i, -1);
> > + return 0;
> > + }
> > + assert(a.type == MEM_UInt);
> > + if (b.type == MEM_UInt) {
> > + if (a.u > b.u)
> > + *result = 1;
> > + else if (a.u < b.u)
> > + *result = -1;
> > + else
> > + *result = 0;
> > + return 0;
> > + }
> > + if (b.type == MEM_Int)
> > + *result = 1;
> > + else
> > + *result = double_compare_uint64(b.d, a.u, -1);
> > + return 0;
> > +}
> > +
> > +static int
> > +compare_strings(const struct Mem *left, const struct Mem *right, int *result,
> > + const struct coll *coll)
> > +{
> > + char *a;
> > + uint32_t an;
> > + char bufl[BUF_SIZE];
> > + if ((left->flags & MEM_Str) != 0) {
> > + a = left->z;
> > + an = left->n;
> > + } else {
> > + assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
> > + a = &bufl[0];
> > + if ((left->flags & MEM_Int) != 0)
> > + sql_snprintf(BUF_SIZE, a, "%lld", left->u.i);
> > + else if ((left->flags & MEM_UInt) != 0)
> > + sql_snprintf(BUF_SIZE, a, "%llu", left->u.u);
> > + else
> > + sql_snprintf(BUF_SIZE, a, "%!.15g", left->u.r);
> > + an = strlen(a);
> > + }
> > +
> > + char *b;
> > + uint32_t bn;
> > + char bufr[BUF_SIZE];
> > + if ((right->flags & MEM_Str) != 0) {
> > + b = right->z;
> > + bn = right->n;
> > + } else {
> > + assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
> > + b = &bufr[0];
> > + if ((right->flags & MEM_Int) != 0)
> > + sql_snprintf(BUF_SIZE, b, "%lld", right->u.i);
> > + else if ((right->flags & MEM_UInt) != 0)
> > + sql_snprintf(BUF_SIZE, b, "%llu", right->u.u);
> > + else
> > + sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
> > + bn = strlen(b);
> > + }
> > + if (coll) {
> > + *result = coll->cmp(a, an, b, bn, coll);
> > + return 0;
> > + }
> > + uint32_t minlen = MIN(an, bn);
> > + *result = memcmp(a, b, minlen);
> > + if (*result != 0)
> > + return 0;
> > + *result = an - bn;
> > + return 0;
>
> 4. It can't fail either. You can return result as 'return'.
>
True, but I left it as it is since now it is now static. See the last comment.
> > +}
> > +
> > +int
> > +mem_compare(const struct Mem *left, const struct Mem *right, int *result,
> > + enum field_type type, struct coll *coll)
> > +{
> > + assert(((left->flags | right->flags) & MEM_Null) == 0);
> > + int flags_any = left->flags | right->flags;
>
> 5. 'any' isn't needed until the next branch. You can move it right above
>
Dropped this function.
> if ((flags_any & MEM_Bool) != 0) {
>
> > + int flags_all = left->flags & right->flags;
> > +
> > + if ((flags_all & MEM_Bool) != 0) {
> > + if (left->u.b == right->u.b)
> > + *result = 0;
> > + else if (left->u.b)
> > + *result = 1;
> > + else
> > + *result = -1;
> > + return 0;
> > + }
> > + if ((flags_any & MEM_Bool) != 0) {
> > + char *str = (left->flags & MEM_Bool) == 0 ?
> > + mem_type_to_str(left) : mem_type_to_str(right);
> > + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "boolean");
> > + return -1;
> > + }
> > +
> > + if ((flags_all & MEM_Blob) != 0)
> > + return compare_blobs(left, right, result);
> > + if ((flags_any & MEM_Blob) != 0) {
> > + char *str = (left->flags & MEM_Blob) == 0 ?
> > + mem_type_to_str(left) : mem_type_to_str(right);
> > + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "varbinary");
> > + return -1;
> > + }
> > +
> > + if (type == FIELD_TYPE_STRING)
> > + return compare_strings(left, right, result, coll);
> > +
> > + if (sql_type_is_numeric(type) ||
> > + (flags_any & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
> > + return compare_numbers(left, right, result);
> > +
> > + assert((left->flags & MEM_Str) != 0 && (right->flags & MEM_Str) != 0);
> > + return compare_strings(left, right, result, coll);
> > +}
> > diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> > index 69a7d9f7a..6c022d8d8 100644
> > --- a/src/box/sql/mem.h
> > +++ b/src/box/sql/mem.h
> > @@ -226,6 +226,11 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
> > int
> > mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
> >
> > +/** Compare two non-NULL MEMs and return the result of comparison. */
> > +int
> > +mem_compare(const struct Mem *left, const struct Mem *right, int *result,
> > + enum field_type type, struct coll *coll);
>
> 6. What is 'type'?
>
I believe that is field type used during '... a IN (...);' Not sure that it is
the only place where type is given to the opcodes.
> Can you keep it out of the comparator somehow? For example, make it
> 3 functions: mem_cmp_as_str, mem_cmp_as_num, and just a generic mem_cmp
> without any types calling the first 2 comparators. Otherwise the type
> thing is super ugly. It leaks the details of opcodes into mem.c.
Thank you for suggestion. I was bothered by this too, but couldn't think of a
solution. I dropped mem_compare() and instead I now have set of mem_cmp_*()
functions. In their descriptions I wrote that they convert their values
according to implicit cast rules but this is not exactly true since now
our implicit cast rules are in mess. Though, in the end we should come to
some conclusion, I believe. And then we should fix all comparison opcodes. I
hope.
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5116af519..9de57bcb4 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -478,9 +478,25 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
return 0;
}
-static int
-compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
{
+ if ((a->flags & MEM_Bool) == 0 || (b->flags & MEM_Bool) == 0)
+ return -1;
+ if (a->u.b == b->u.b)
+ *result = 0;
+ else if (a->u.b)
+ *result = 1;
+ else
+ *result = -1;
+ return 0;
+}
+
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result)
+{
+ if ((a->flags & MEM_Blob) == 0 || (b->flags & MEM_Blob) == 0)
+ return -1;
int an = a->n;
int bn = b->n;
int minlen = MIN(an, bn);
@@ -525,8 +541,8 @@ compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
return 0;
}
-static int
-compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
+int
+mem_cmp_num(const struct Mem *left, const struct Mem *right, int *result)
{
struct sql_num a, b;
/* TODO: Here should be check for right value type. */
@@ -534,11 +550,8 @@ compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
*result = -1;
return 0;
}
- if (get_number(left, &a) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
- "numeric");
+ if (get_number(left, &a) != 0)
return -1;
- }
if (a.type == MEM_Real) {
if (b.type == MEM_Real) {
if (a.d > b.d)
@@ -588,9 +601,9 @@ compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
return 0;
}
-static int
-compare_strings(const struct Mem *left, const struct Mem *right, int *result,
- const struct coll *coll)
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll)
{
char *a;
uint32_t an;
@@ -639,50 +652,6 @@ compare_strings(const struct Mem *left, const struct Mem *right, int *result,
return 0;
}
-int
-mem_compare(const struct Mem *left, const struct Mem *right, int *result,
- enum field_type type, struct coll *coll)
-{
- assert(((left->flags | right->flags) & MEM_Null) == 0);
- int flags_any = left->flags | right->flags;
- int flags_all = left->flags & right->flags;
-
- if ((flags_all & MEM_Bool) != 0) {
- if (left->u.b == right->u.b)
- *result = 0;
- else if (left->u.b)
- *result = 1;
- else
- *result = -1;
- return 0;
- }
- if ((flags_any & MEM_Bool) != 0) {
- char *str = (left->flags & MEM_Bool) == 0 ?
- mem_type_to_str(left) : mem_type_to_str(right);
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "boolean");
- return -1;
- }
-
- if ((flags_all & MEM_Blob) != 0)
- return compare_blobs(left, right, result);
- if ((flags_any & MEM_Blob) != 0) {
- char *str = (left->flags & MEM_Blob) == 0 ?
- mem_type_to_str(left) : mem_type_to_str(right);
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str, "varbinary");
- return -1;
- }
-
- if (type == FIELD_TYPE_STRING)
- return compare_strings(left, right, result, coll);
-
- if (sql_type_is_numeric(type) ||
- (flags_any & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
- return compare_numbers(left, right, result);
-
- assert((left->flags & MEM_Str) != 0 && (right->flags & MEM_Str) != 0);
- return compare_strings(left, right, result, coll);
-}
-
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -2177,7 +2146,7 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
return +1;
if ((f2 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
return -1;
- compare_numbers(pMem1, pMem2, &res);
+ mem_cmp_num(pMem1, pMem2, &res);
return res;
}
@@ -2191,12 +2160,12 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
if ((f2 & MEM_Str) == 0) {
return -1;
}
- compare_strings(pMem1, pMem2, &res, pColl);
+ mem_cmp_str(pMem1, pMem2, &res, pColl);
return res;
}
/* Both values must be blobs. Compare using memcmp(). */
- compare_blobs(pMem1, pMem2, &res);
+ mem_cmp_bin(pMem1, pMem2, &res);
return res;
}
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 80793ef29..7e498356b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -364,10 +364,38 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
int
mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
-/** Compare two non-NULL MEMs and return the result of comparison. */
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * BOOLEAN type or their values are converted to VARBINARY according to implicit
+ * cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * VARBINARY type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * STRING type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * NUMBER type or their values are converted to NUMBER according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
int
-mem_compare(const struct Mem *left, const struct Mem *right, int *result,
- enum field_type type, struct coll *coll);
+mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
/**
* Simple type to str convertor. It is used to simplify
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 65fd63b2b..f917c7de9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1610,6 +1610,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
+ enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
@@ -1643,11 +1644,54 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
+ } else if (mem_is_bool(pIn3) || mem_is_bool(pIn1)) {
+ if (mem_cmp_bool(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bool(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "boolean");
+ goto abort_due_to_error;
+ }
+ } else if (mem_is_bin(pIn3) || mem_is_bin(pIn1)) {
+ if (mem_cmp_bin(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bin(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "varbinary");
+ goto abort_due_to_error;
+ }
+ } else if (type == FIELD_TYPE_STRING) {
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
+ goto abort_due_to_error;
+ }
+ } else if (sql_type_is_numeric(type) || mem_is_num(pIn3) ||
+ mem_is_num(pIn1)) {
+ type = FIELD_TYPE_NUMBER;
+ if (mem_cmp_num(pIn3, pIn1, &res) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "numeric");
+ goto abort_due_to_error;
+ }
} else {
- enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
- if (mem_compare(pIn3, pIn1, &res, type, pOp->p4.pColl))
+ type = FIELD_TYPE_STRING;
+ assert(mem_is_str(pIn3) && mem_is_same_type(pIn3, pIn1));
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
goto abort_due_to_error;
+ }
}
+
switch( pOp->opcode) {
case OP_Eq: res2 = res==0; break;
case OP_Ne: res2 = res; break;
New patch:
commit ed2c851550f8d658dfea7f14f758222acef1665e
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 13:13:34 2021 +0300
sql: introduce mem_cmp_*() functions
This patch introduces set of mem_cmp_*() functions. These functions are
used to compare MEMs.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 9a4a9b92e..9de57bcb4 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -478,6 +478,180 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
return 0;
}
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
+{
+ if ((a->flags & MEM_Bool) == 0 || (b->flags & MEM_Bool) == 0)
+ return -1;
+ if (a->u.b == b->u.b)
+ *result = 0;
+ else if (a->u.b)
+ *result = 1;
+ else
+ *result = -1;
+ return 0;
+}
+
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result)
+{
+ if ((a->flags & MEM_Blob) == 0 || (b->flags & MEM_Blob) == 0)
+ return -1;
+ int an = a->n;
+ int bn = b->n;
+ int minlen = MIN(an, bn);
+
+ /*
+ * It is possible to have a Blob value that has some non-zero content
+ * followed by zero content. But that only comes up for Blobs formed
+ * by the OP_MakeRecord opcode, and such Blobs never get passed into
+ * mem_compare().
+ */
+ assert((a->flags & MEM_Zero) == 0 || an == 0);
+ assert((b->flags & MEM_Zero) == 0 || bn == 0);
+
+ if ((a->flags & b->flags & MEM_Zero) != 0) {
+ *result = a->u.nZero - b->u.nZero;
+ return 0;
+ }
+ if ((a->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (b->z[i] != 0) {
+ *result = -1;
+ return 0;
+ }
+ }
+ *result = a->u.nZero - bn;
+ return 0;
+ }
+ if ((b->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (a->z[i] != 0){
+ *result = 1;
+ return 0;
+ }
+ }
+ *result = b->u.nZero - an;
+ return 0;
+ }
+ *result = memcmp(a->z, b->z, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
+int
+mem_cmp_num(const struct Mem *left, const struct Mem *right, int *result)
+{
+ struct sql_num a, b;
+ /* TODO: Here should be check for right value type. */
+ if (get_number(right, &b) != 0) {
+ *result = -1;
+ return 0;
+ }
+ if (get_number(left, &a) != 0)
+ return -1;
+ if (a.type == MEM_Real) {
+ if (b.type == MEM_Real) {
+ if (a.d > b.d)
+ *result = 1;
+ else if (a.d < b.d)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = double_compare_nint64(a.d, b.i, 1);
+ else
+ *result = double_compare_uint64(a.d, b.u, 1);
+ return 0;
+ }
+ if (a.type == MEM_Int) {
+ if (b.type == MEM_Int) {
+ if (a.i > b.i)
+ *result = 1;
+ else if (a.i < b.i)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_UInt)
+ *result = -1;
+ else
+ *result = double_compare_nint64(b.d, a.i, -1);
+ return 0;
+ }
+ assert(a.type == MEM_UInt);
+ if (b.type == MEM_UInt) {
+ if (a.u > b.u)
+ *result = 1;
+ else if (a.u < b.u)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = 1;
+ else
+ *result = double_compare_uint64(b.d, a.u, -1);
+ return 0;
+}
+
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll)
+{
+ char *a;
+ uint32_t an;
+ char bufl[BUF_SIZE];
+ if ((left->flags & MEM_Str) != 0) {
+ a = left->z;
+ an = left->n;
+ } else {
+ assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ a = &bufl[0];
+ if ((left->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, a, "%lld", left->u.i);
+ else if ((left->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, a, "%llu", left->u.u);
+ else
+ sql_snprintf(BUF_SIZE, a, "%!.15g", left->u.r);
+ an = strlen(a);
+ }
+
+ char *b;
+ uint32_t bn;
+ char bufr[BUF_SIZE];
+ if ((right->flags & MEM_Str) != 0) {
+ b = right->z;
+ bn = right->n;
+ } else {
+ assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ b = &bufr[0];
+ if ((right->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, b, "%lld", right->u.i);
+ else if ((right->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, b, "%llu", right->u.u);
+ else
+ sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
+ bn = strlen(b);
+ }
+ if (coll) {
+ *result = coll->cmp(a, an, b, bn, coll);
+ return 0;
+ }
+ uint32_t minlen = MIN(an, bn);
+ *result = memcmp(a, b, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1925,45 +2099,6 @@ sqlVdbeMemTooBig(Mem * p)
return 0;
}
-/*
- * Compare two blobs. Return negative, zero, or positive if the first
- * is less than, equal to, or greater than the second, respectively.
- * If one blob is a prefix of the other, then the shorter is the lessor.
- */
-static SQL_NOINLINE int
-sqlBlobCompare(const Mem * pB1, const Mem * pB2)
-{
- int c;
- int n1 = pB1->n;
- int n2 = pB2->n;
-
- /* It is possible to have a Blob value that has some non-zero content
- * followed by zero content. But that only comes up for Blobs formed
- * by the OP_MakeRecord opcode, and such Blobs never get passed into
- * sqlMemCompare().
- */
- assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
- assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
-
- if ((pB1->flags | pB2->flags) & MEM_Zero) {
- if (pB1->flags & pB2->flags & MEM_Zero) {
- return pB1->u.nZero - pB2->u.nZero;
- } else if (pB1->flags & MEM_Zero) {
- if (!isAllZero(pB2->z, pB2->n))
- return -1;
- return pB1->u.nZero - n2;
- } else {
- if (!isAllZero(pB1->z, pB1->n))
- return +1;
- return n1 - pB2->u.nZero;
- }
- }
- c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
- if (c)
- return c;
- return n1 - n2;
-}
-
/*
* Compare the values contained by the two memory cells, returning
* negative, zero or positive if pMem1 is less than, equal to, or greater
@@ -1977,6 +2112,7 @@ int
sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
{
int f1, f2;
+ int res;
int combined_flags;
f1 = pMem1->flags;
@@ -2006,57 +2142,12 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
/* At least one of the two values is a number
*/
if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- if ((f1 & f2 & MEM_Int) != 0) {
- if (pMem1->u.i < pMem2->u.i)
- return -1;
- if (pMem1->u.i > pMem2->u.i)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_UInt) != 0) {
- if (pMem1->u.u < pMem2->u.u)
- return -1;
- if (pMem1->u.u > pMem2->u.u)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_Real) != 0) {
- if (pMem1->u.r < pMem2->u.r)
- return -1;
- if (pMem1->u.r > pMem2->u.r)
- return +1;
- return 0;
- }
- if ((f1 & MEM_Int) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_nint64(pMem2->u.r,
- pMem1->u.i, -1);
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_UInt) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_uint64(pMem2->u.r,
- pMem1->u.u, -1);
- } else if ((f2 & MEM_Int) != 0) {
- return +1;
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_Real) != 0) {
- if ((f2 & MEM_Int) != 0) {
- return double_compare_nint64(pMem1->u.r,
- pMem2->u.i, 1);
- } else if ((f2 & MEM_UInt) != 0) {
- return double_compare_uint64(pMem1->u.r,
- pMem2->u.u, 1);
- } else {
- return -1;
- }
- }
- return +1;
+ if ((f1 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return +1;
+ if ((f2 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return -1;
+ mem_cmp_num(pMem1, pMem2, &res);
+ return res;
}
/* If one value is a string and the other is a blob, the string is less.
@@ -2069,27 +2160,13 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
if ((f2 & MEM_Str) == 0) {
return -1;
}
- /* The collation sequence must be defined at this point, even if
- * the user deletes the collation sequence after the vdbe program is
- * compiled (this was not always the case).
- */
- if (pColl) {
- return vdbeCompareMemString(pMem1, pMem2, pColl);
- } else {
- size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
- int res;
- res = memcmp(pMem1->z, pMem2->z, n);
- if (res == 0)
- res = (int)pMem1->n - (int)pMem2->n;
- return res;
- }
- /* If a NULL pointer was passed as the collate function, fall through
- * to the blob case and use memcmp().
- */
+ mem_cmp_str(pMem1, pMem2, &res, pColl);
+ return res;
}
/* Both values must be blobs. Compare using memcmp(). */
- return sqlBlobCompare(pMem1, pMem2);
+ mem_cmp_bin(pMem1, pMem2, &res);
+ return res;
}
bool
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9539fbbd1..7e498356b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -364,6 +364,39 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
int
mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * BOOLEAN type or their values are converted to VARBINARY according to implicit
+ * cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * VARBINARY type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * STRING type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * NUMBER type or their values are converted to NUMBER according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 0849875e2..f917c7de9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1607,15 +1607,10 @@ case OP_Le: /* same as TK_LE, jump, in1, in3 */
case OP_Gt: /* same as TK_GT, jump, in1, in3 */
case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
int res, res2; /* Result of the comparison of pIn1 against pIn3 */
- u32 flags1; /* Copy of initial value of pIn1->flags */
- u32 flags3; /* Copy of initial value of pIn3->flags */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
- flags1 = pIn1->flags;
- flags3 = pIn3->flags;
- enum field_type ft_p1 = pIn1->field_type;
- enum field_type ft_p3 = pIn3->field_type;
+ enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
@@ -1649,82 +1644,54 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
- } else if (mem_is_bool(pIn1) || mem_is_bool(pIn3) ||
- mem_is_bin(pIn1) || mem_is_bin(pIn3)) {
- if (!mem_is_same_type(pIn1, pIn3)) {
- char *inconsistent_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn3) :
- mem_type_to_str(pIn1);
- char *expected_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn3);
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- inconsistent_type, expected_type);
+ } else if (mem_is_bool(pIn3) || mem_is_bool(pIn1)) {
+ if (mem_cmp_bool(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bool(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "boolean");
+ goto abort_due_to_error;
+ }
+ } else if (mem_is_bin(pIn3) || mem_is_bin(pIn1)) {
+ if (mem_cmp_bin(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bin(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "varbinary");
+ goto abort_due_to_error;
+ }
+ } else if (type == FIELD_TYPE_STRING) {
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
+ goto abort_due_to_error;
+ }
+ } else if (sql_type_is_numeric(type) || mem_is_num(pIn3) ||
+ mem_is_num(pIn1)) {
+ type = FIELD_TYPE_NUMBER;
+ if (mem_cmp_num(pIn3, pIn1, &res) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "numeric");
goto abort_due_to_error;
}
- res = sqlMemCompare(pIn3, pIn1, NULL);
} else {
- enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
- if (sql_type_is_numeric(type)) {
- if (mem_is_str(pIn1)) {
- mem_apply_numeric_type(pIn1);
- flags3 = pIn3->flags;
- }
- if (mem_is_str(pIn3)) {
- if (mem_apply_numeric_type(pIn3) != 0) {
- diag_set(ClientError,
- ER_SQL_TYPE_MISMATCH,
- mem_str(pIn3),
- "numeric");
- goto abort_due_to_error;
- }
- }
- /* Handle the common case of integer comparison here, as an
- * optimization, to avoid a call to sqlMemCompare()
- */
- if (mem_is_int(pIn1) && mem_is_int(pIn3)) {
- if (mem_is_nint(pIn1) && mem_is_nint(pIn3)) {
- if (pIn3->u.i > pIn1->u.i)
- res = +1;
- else if (pIn3->u.i < pIn1->u.i)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && mem_is_uint(pIn3)) {
- if (pIn3->u.u > pIn1->u.u)
- res = +1;
- else if (pIn3->u.u < pIn1->u.u)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && mem_is_nint(pIn3)) {
- res = -1;
- goto compare_op;
- }
- res = 1;
- goto compare_op;
- }
- } else if (type == FIELD_TYPE_STRING) {
- if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
- sqlVdbeMemStringify(pIn1);
- flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
- assert(pIn1!=pIn3);
- }
- if (!mem_is_str(pIn3) && mem_is_num(pIn3)) {
- sqlVdbeMemStringify(pIn3);
- flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
- }
+ type = FIELD_TYPE_STRING;
+ assert(mem_is_str(pIn3) && mem_is_same_type(pIn3, pIn1));
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
+ goto abort_due_to_error;
}
- assert(pOp->p4type==P4_COLLSEQ || pOp->p4.pColl==0);
- res = sqlMemCompare(pIn3, pIn1, pOp->p4.pColl);
}
- compare_op:
+
switch( pOp->opcode) {
case OP_Eq: res2 = res==0; break;
case OP_Ne: res2 = res; break;
@@ -1734,14 +1701,6 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
default: res2 = res>=0; break;
}
- /* Undo any changes made by mem_apply_type() to the input registers. */
- assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
- pIn1->flags = flags1;
- pIn1->field_type = ft_p1;
- assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
- pIn3->flags = flags3;
- pIn3->field_type = ft_p3;
-
if (pOp->p5 & SQL_STOREP2) {
iCompare = res;
res2 = res2!=0; /* For this path res2 must be exactly 0 or 1 */
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare()
2021-04-13 18:33 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 23:20 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-14 23:40 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 23:20 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Nice fixes!
Consider mine below:
====================
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 9de57bcb4..a7c2c16cb 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -481,7 +481,7 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
int
mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
{
- if ((a->flags & MEM_Bool) == 0 || (b->flags & MEM_Bool) == 0)
+ if ((a->flags & b->flags & MEM_Bool) == 0)
return -1;
if (a->u.b == b->u.b)
*result = 0;
@@ -495,7 +495,7 @@ mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
int
mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result)
{
- if ((a->flags & MEM_Blob) == 0 || (b->flags & MEM_Blob) == 0)
+ if ((a->flags & b->flags & MEM_Blob) == 0)
return -1;
int an = a->n;
int bn = b->n;
@@ -640,7 +640,7 @@ mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
bn = strlen(b);
}
- if (coll) {
+ if (coll != NULL) {
*result = coll->cmp(a, an, b, bn, coll);
return 0;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare()
2021-04-14 23:20 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-14 23:40 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-14 23:40 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for review and suggestion! I applied your diff and tested. New patch
below.
On Thu, Apr 15, 2021 at 01:20:21AM +0200, Vladislav Shpilevoy wrote:
> Nice fixes!
>
> Consider mine below:
>
> ====================
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 9de57bcb4..a7c2c16cb 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -481,7 +481,7 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
> int
> mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
> {
> - if ((a->flags & MEM_Bool) == 0 || (b->flags & MEM_Bool) == 0)
> + if ((a->flags & b->flags & MEM_Bool) == 0)
> return -1;
> if (a->u.b == b->u.b)
> *result = 0;
> @@ -495,7 +495,7 @@ mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
> int
> mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result)
> {
> - if ((a->flags & MEM_Blob) == 0 || (b->flags & MEM_Blob) == 0)
> + if ((a->flags & b->flags & MEM_Blob) == 0)
> return -1;
> int an = a->n;
> int bn = b->n;
> @@ -640,7 +640,7 @@ mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
> sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
> bn = strlen(b);
> }
> - if (coll) {
> + if (coll != NULL) {
> *result = coll->cmp(a, an, b, bn, coll);
> return 0;
> }
New patch:
commit 37f32ea303fe8c01683a4f77ac87350b15f4a82e
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 13:13:34 2021 +0300
sql: introduce mem_cmp_*() functions
This patch introduces set of mem_cmp_*() functions. These functions are
used to compare MEMs.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index a8bbfd3ea..529982624 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -479,6 +479,180 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
return 0;
}
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result)
+{
+ if ((a->flags & b->flags & MEM_Bool) == 0)
+ return -1;
+ if (a->u.b == b->u.b)
+ *result = 0;
+ else if (a->u.b)
+ *result = 1;
+ else
+ *result = -1;
+ return 0;
+}
+
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result)
+{
+ if ((a->flags & b->flags & MEM_Blob) == 0)
+ return -1;
+ int an = a->n;
+ int bn = b->n;
+ int minlen = MIN(an, bn);
+
+ /*
+ * It is possible to have a Blob value that has some non-zero content
+ * followed by zero content. But that only comes up for Blobs formed
+ * by the OP_MakeRecord opcode, and such Blobs never get passed into
+ * mem_compare().
+ */
+ assert((a->flags & MEM_Zero) == 0 || an == 0);
+ assert((b->flags & MEM_Zero) == 0 || bn == 0);
+
+ if ((a->flags & b->flags & MEM_Zero) != 0) {
+ *result = a->u.nZero - b->u.nZero;
+ return 0;
+ }
+ if ((a->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (b->z[i] != 0) {
+ *result = -1;
+ return 0;
+ }
+ }
+ *result = a->u.nZero - bn;
+ return 0;
+ }
+ if ((b->flags & MEM_Zero) != 0) {
+ for (int i = 0; i < minlen; ++i) {
+ if (a->z[i] != 0){
+ *result = 1;
+ return 0;
+ }
+ }
+ *result = b->u.nZero - an;
+ return 0;
+ }
+ *result = memcmp(a->z, b->z, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
+int
+mem_cmp_num(const struct Mem *left, const struct Mem *right, int *result)
+{
+ struct sql_num a, b;
+ /* TODO: Here should be check for right value type. */
+ if (get_number(right, &b) != 0) {
+ *result = -1;
+ return 0;
+ }
+ if (get_number(left, &a) != 0)
+ return -1;
+ if (a.type == MEM_Real) {
+ if (b.type == MEM_Real) {
+ if (a.d > b.d)
+ *result = 1;
+ else if (a.d < b.d)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = double_compare_nint64(a.d, b.i, 1);
+ else
+ *result = double_compare_uint64(a.d, b.u, 1);
+ return 0;
+ }
+ if (a.type == MEM_Int) {
+ if (b.type == MEM_Int) {
+ if (a.i > b.i)
+ *result = 1;
+ else if (a.i < b.i)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_UInt)
+ *result = -1;
+ else
+ *result = double_compare_nint64(b.d, a.i, -1);
+ return 0;
+ }
+ assert(a.type == MEM_UInt);
+ if (b.type == MEM_UInt) {
+ if (a.u > b.u)
+ *result = 1;
+ else if (a.u < b.u)
+ *result = -1;
+ else
+ *result = 0;
+ return 0;
+ }
+ if (b.type == MEM_Int)
+ *result = 1;
+ else
+ *result = double_compare_uint64(b.d, a.u, -1);
+ return 0;
+}
+
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll)
+{
+ char *a;
+ uint32_t an;
+ char bufl[BUF_SIZE];
+ if ((left->flags & MEM_Str) != 0) {
+ a = left->z;
+ an = left->n;
+ } else {
+ assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ a = &bufl[0];
+ if ((left->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, a, "%lld", left->u.i);
+ else if ((left->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, a, "%llu", left->u.u);
+ else
+ sql_snprintf(BUF_SIZE, a, "%!.15g", left->u.r);
+ an = strlen(a);
+ }
+
+ char *b;
+ uint32_t bn;
+ char bufr[BUF_SIZE];
+ if ((right->flags & MEM_Str) != 0) {
+ b = right->z;
+ bn = right->n;
+ } else {
+ assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+ b = &bufr[0];
+ if ((right->flags & MEM_Int) != 0)
+ sql_snprintf(BUF_SIZE, b, "%lld", right->u.i);
+ else if ((right->flags & MEM_UInt) != 0)
+ sql_snprintf(BUF_SIZE, b, "%llu", right->u.u);
+ else
+ sql_snprintf(BUF_SIZE, b, "%!.15g", right->u.r);
+ bn = strlen(b);
+ }
+ if (coll != NULL) {
+ *result = coll->cmp(a, an, b, bn, coll);
+ return 0;
+ }
+ uint32_t minlen = MIN(an, bn);
+ *result = memcmp(a, b, minlen);
+ if (*result != 0)
+ return 0;
+ *result = an - bn;
+ return 0;
+}
+
static inline bool
mem_has_msgpack_subtype(struct Mem *mem)
{
@@ -1926,45 +2100,6 @@ sqlVdbeMemTooBig(Mem * p)
return 0;
}
-/*
- * Compare two blobs. Return negative, zero, or positive if the first
- * is less than, equal to, or greater than the second, respectively.
- * If one blob is a prefix of the other, then the shorter is the lessor.
- */
-static SQL_NOINLINE int
-sqlBlobCompare(const Mem * pB1, const Mem * pB2)
-{
- int c;
- int n1 = pB1->n;
- int n2 = pB2->n;
-
- /* It is possible to have a Blob value that has some non-zero content
- * followed by zero content. But that only comes up for Blobs formed
- * by the OP_MakeRecord opcode, and such Blobs never get passed into
- * sqlMemCompare().
- */
- assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
- assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
-
- if ((pB1->flags | pB2->flags) & MEM_Zero) {
- if (pB1->flags & pB2->flags & MEM_Zero) {
- return pB1->u.nZero - pB2->u.nZero;
- } else if (pB1->flags & MEM_Zero) {
- if (!isAllZero(pB2->z, pB2->n))
- return -1;
- return pB1->u.nZero - n2;
- } else {
- if (!isAllZero(pB1->z, pB1->n))
- return +1;
- return n1 - pB2->u.nZero;
- }
- }
- c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
- if (c)
- return c;
- return n1 - n2;
-}
-
/*
* Compare the values contained by the two memory cells, returning
* negative, zero or positive if pMem1 is less than, equal to, or greater
@@ -1978,6 +2113,7 @@ int
sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
{
int f1, f2;
+ int res;
int combined_flags;
f1 = pMem1->flags;
@@ -2007,57 +2143,12 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
/* At least one of the two values is a number
*/
if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
- if ((f1 & f2 & MEM_Int) != 0) {
- if (pMem1->u.i < pMem2->u.i)
- return -1;
- if (pMem1->u.i > pMem2->u.i)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_UInt) != 0) {
- if (pMem1->u.u < pMem2->u.u)
- return -1;
- if (pMem1->u.u > pMem2->u.u)
- return +1;
- return 0;
- }
- if ((f1 & f2 & MEM_Real) != 0) {
- if (pMem1->u.r < pMem2->u.r)
- return -1;
- if (pMem1->u.r > pMem2->u.r)
- return +1;
- return 0;
- }
- if ((f1 & MEM_Int) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_nint64(pMem2->u.r,
- pMem1->u.i, -1);
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_UInt) != 0) {
- if ((f2 & MEM_Real) != 0) {
- return double_compare_uint64(pMem2->u.r,
- pMem1->u.u, -1);
- } else if ((f2 & MEM_Int) != 0) {
- return +1;
- } else {
- return -1;
- }
- }
- if ((f1 & MEM_Real) != 0) {
- if ((f2 & MEM_Int) != 0) {
- return double_compare_nint64(pMem1->u.r,
- pMem2->u.i, 1);
- } else if ((f2 & MEM_UInt) != 0) {
- return double_compare_uint64(pMem1->u.r,
- pMem2->u.u, 1);
- } else {
- return -1;
- }
- }
- return +1;
+ if ((f1 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return +1;
+ if ((f2 & (MEM_Real | MEM_Int | MEM_UInt)) == 0)
+ return -1;
+ mem_cmp_num(pMem1, pMem2, &res);
+ return res;
}
/* If one value is a string and the other is a blob, the string is less.
@@ -2070,27 +2161,13 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
if ((f2 & MEM_Str) == 0) {
return -1;
}
- /* The collation sequence must be defined at this point, even if
- * the user deletes the collation sequence after the vdbe program is
- * compiled (this was not always the case).
- */
- if (pColl) {
- return vdbeCompareMemString(pMem1, pMem2, pColl);
- } else {
- size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
- int res;
- res = memcmp(pMem1->z, pMem2->z, n);
- if (res == 0)
- res = (int)pMem1->n - (int)pMem2->n;
- return res;
- }
- /* If a NULL pointer was passed as the collate function, fall through
- * to the blob case and use memcmp().
- */
+ mem_cmp_str(pMem1, pMem2, &res, pColl);
+ return res;
}
/* Both values must be blobs. Compare using memcmp(). */
- return sqlBlobCompare(pMem1, pMem2);
+ mem_cmp_bin(pMem1, pMem2, &res);
+ return res;
}
bool
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9539fbbd1..7e498356b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -364,6 +364,39 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
int
mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * BOOLEAN type or their values are converted to VARBINARY according to implicit
+ * cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bool(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * VARBINARY type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_bin(const struct Mem *a, const struct Mem *b, int *result);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * STRING type or their values are converted to VARBINARY according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
+ const struct coll *coll);
+
+/**
+ * Compare two MEMs and return the result of comparison. MEMs should be of
+ * NUMBER type or their values are converted to NUMBER according to
+ * implicit cast rules. Original MEMs are not changed.
+ */
+int
+mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index fb2a5ccc1..2fbbf32a0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1607,15 +1607,10 @@ case OP_Le: /* same as TK_LE, jump, in1, in3 */
case OP_Gt: /* same as TK_GT, jump, in1, in3 */
case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
int res, res2; /* Result of the comparison of pIn1 against pIn3 */
- u32 flags1; /* Copy of initial value of pIn1->flags */
- u32 flags3; /* Copy of initial value of pIn3->flags */
pIn1 = &aMem[pOp->p1];
pIn3 = &aMem[pOp->p3];
- flags1 = pIn1->flags;
- flags3 = pIn3->flags;
- enum field_type ft_p1 = pIn1->field_type;
- enum field_type ft_p3 = pIn3->field_type;
+ enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (mem_is_any_null(pIn1, pIn3)) {
/* One or both operands are NULL */
if (pOp->p5 & SQL_NULLEQ) {
@@ -1649,82 +1644,54 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
}
break;
}
- } else if (mem_is_bool(pIn1) || mem_is_bool(pIn3) ||
- mem_is_bin(pIn1) || mem_is_bin(pIn3)) {
- if (!mem_is_same_type(pIn1, pIn3)) {
- char *inconsistent_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn3) :
- mem_type_to_str(pIn1);
- char *expected_type = mem_is_bool(pIn1) ||
- mem_is_bin(pIn1) ?
- mem_type_to_str(pIn1) :
- mem_type_to_str(pIn3);
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- inconsistent_type, expected_type);
+ } else if (mem_is_bool(pIn3) || mem_is_bool(pIn1)) {
+ if (mem_cmp_bool(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bool(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "boolean");
+ goto abort_due_to_error;
+ }
+ } else if (mem_is_bin(pIn3) || mem_is_bin(pIn1)) {
+ if (mem_cmp_bin(pIn3, pIn1, &res) != 0) {
+ char *str = !mem_is_bin(pIn3) ?
+ mem_type_to_str(pIn3) :
+ mem_type_to_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "varbinary");
+ goto abort_due_to_error;
+ }
+ } else if (type == FIELD_TYPE_STRING) {
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
+ goto abort_due_to_error;
+ }
+ } else if (sql_type_is_numeric(type) || mem_is_num(pIn3) ||
+ mem_is_num(pIn1)) {
+ type = FIELD_TYPE_NUMBER;
+ if (mem_cmp_num(pIn3, pIn1, &res) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "numeric");
goto abort_due_to_error;
}
- res = sqlMemCompare(pIn3, pIn1, NULL);
} else {
- enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
- if (sql_type_is_numeric(type)) {
- if (mem_is_str(pIn1)) {
- mem_apply_numeric_type(pIn1);
- flags3 = pIn3->flags;
- }
- if (mem_is_str(pIn3)) {
- if (mem_apply_numeric_type(pIn3) != 0) {
- diag_set(ClientError,
- ER_SQL_TYPE_MISMATCH,
- mem_str(pIn3),
- "numeric");
- goto abort_due_to_error;
- }
- }
- /* Handle the common case of integer comparison here, as an
- * optimization, to avoid a call to sqlMemCompare()
- */
- if (mem_is_int(pIn1) && mem_is_int(pIn3)) {
- if (mem_is_nint(pIn1) && mem_is_nint(pIn3)) {
- if (pIn3->u.i > pIn1->u.i)
- res = +1;
- else if (pIn3->u.i < pIn1->u.i)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && mem_is_uint(pIn3)) {
- if (pIn3->u.u > pIn1->u.u)
- res = +1;
- else if (pIn3->u.u < pIn1->u.u)
- res = -1;
- else
- res = 0;
- goto compare_op;
- }
- if (mem_is_uint(pIn1) && mem_is_nint(pIn3)) {
- res = -1;
- goto compare_op;
- }
- res = 1;
- goto compare_op;
- }
- } else if (type == FIELD_TYPE_STRING) {
- if (!mem_is_str(pIn1) && mem_is_num(pIn1)) {
- sqlVdbeMemStringify(pIn1);
- flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
- assert(pIn1!=pIn3);
- }
- if (!mem_is_str(pIn3) && mem_is_num(pIn3)) {
- sqlVdbeMemStringify(pIn3);
- flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
- }
+ type = FIELD_TYPE_STRING;
+ assert(mem_is_str(pIn3) && mem_is_same_type(pIn3, pIn1));
+ if (mem_cmp_str(pIn3, pIn1, &res, pOp->p4.pColl) != 0) {
+ const char *str = mem_apply_type(pIn3, type) != 0 ?
+ mem_str(pIn3) : mem_str(pIn1);
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, str,
+ "string");
+ goto abort_due_to_error;
}
- assert(pOp->p4type==P4_COLLSEQ || pOp->p4.pColl==0);
- res = sqlMemCompare(pIn3, pIn1, pOp->p4.pColl);
}
- compare_op:
+
switch( pOp->opcode) {
case OP_Eq: res2 = res==0; break;
case OP_Ne: res2 = res; break;
@@ -1734,14 +1701,6 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
default: res2 = res>=0; break;
}
- /* Undo any changes made by mem_apply_type() to the input registers. */
- assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
- pIn1->flags = flags1;
- pIn1->field_type = ft_p1;
- assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
- pIn3->flags = flags3;
- pIn3->field_type = ft_p3;
-
if (pOp->p5 & SQL_STOREP2) {
iCompare = res;
res2 = res2!=0; /* For this path res2 must be exactly 0 or 1 */
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (19 preceding siblings ...)
2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
@ 2021-04-09 18:11 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:31 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 22/52] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches
` (20 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 18:11 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:03, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_bitwise(). Function mem_bitwise() executes
>> bitwise operations with two operands and writes the result to the third
>> MEM.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 51 +++++++++++++++++++++++++++++++++++++++++++
>> src/box/sql/mem.h | 3 +++
>> src/box/sql/vdbe.c | 54 +++-------------------------------------------
>> 3 files changed, 57 insertions(+), 51 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 8119644ed..2b455e39f 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -556,6 +556,57 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>> return 0;
>> }
>>
>> +int
>> +mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
>
> Would be better to split it into separate functions. Also why is OP_BitNot
> separated? How is it much different from, say, OP_BitAnd?
>
Done. I splitted this function. Also, I moved function mem_bit_not() to this
patch.
> Besides, having OP_BitNot in mem.c would allow to make sqlVdbeIntValue static
> inside mem.c.
I didn't make this function static since I will remove it in a few patches.
New patch:
commit cc2e656974210a7f4f8e3c5402adf1bd90c5576f
Author: Mergen Imeev <imeevma@gmail.com>
Date: Sun Mar 14 22:20:11 2021 +0300
sql: introduce bitwise operations for MEM
This patch introduces mem_bit_and(), mem_bit_or(), mem_shift_left(),
mem_shift_right(), and mem_bit_not(), which perform bitwise operations
on MEM.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index eee72a7fe..aeb801c7c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -624,6 +624,115 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
return 0;
}
+static int
+bitwise_prepare(const struct Mem *left, const struct Mem *right,
+ int64_t *a, int64_t *b)
+{
+ bool unused;
+ if (sqlVdbeIntValue(left, a, &unused) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(left),
+ "integer");
+ return -1;
+ }
+ if (sqlVdbeIntValue(right, b, &unused) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(right),
+ "integer");
+ return -1;
+ }
+ return 0;
+}
+
+int
+mem_bit_and(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
+ return 0;
+ int64_t a;
+ int64_t b;
+ if (bitwise_prepare(left, right, &a, &b) != 0)
+ return -1;
+ result->u.i = a & b;
+ result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_bit_or(const struct Mem *left, const struct Mem *right, struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
+ return 0;
+ int64_t a;
+ int64_t b;
+ if (bitwise_prepare(left, right, &a, &b) != 0)
+ return -1;
+ result->u.i = a | b;
+ result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_shift_left(const struct Mem *left, const struct Mem *right,
+ struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
+ return 0;
+ int64_t a;
+ int64_t b;
+ if (bitwise_prepare(left, right, &a, &b) != 0)
+ return -1;
+ if (b <= -64)
+ result->u.i = a >= 0 ? 0 : -1;
+ else if (b < 0)
+ result->u.i = a >> -b;
+ else if (b > 64)
+ result->u.i = 0;
+ else
+ result->u.i = a << b;
+ result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_shift_right(const struct Mem *left, const struct Mem *right,
+ struct Mem *result)
+{
+ if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
+ return 0;
+ int64_t a;
+ int64_t b;
+ if (bitwise_prepare(left, right, &a, &b) != 0)
+ return -1;
+ if (b <= -64)
+ result->u.i = 0;
+ else if (b < 0)
+ result->u.i = a << -b;
+ else if (b > 64)
+ result->u.i = a >= 0 ? 0 : -1;
+ else
+ result->u.i = a >> b;
+ result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
+int
+mem_bit_not(const struct Mem *mem, struct Mem *result)
+{
+ mem_clear(result);
+ result->field_type = FIELD_TYPE_INTEGER;
+ if ((mem->flags & MEM_Null) != 0)
+ return 0;
+ int64_t i;
+ bool unused;
+ if (sqlVdbeIntValue(mem, &i, &unused) != 0) {
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(mem),
+ "integer");
+ return -1;
+ }
+ result->u.i = ~i;
+ result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+ return 0;
+}
+
static int
compare_blobs(const struct Mem *a, const struct Mem *b, int *result)
{
@@ -2065,7 +2174,7 @@ mem_value_bool(const struct Mem *mem, bool *b)
* If pMem represents a string value, its encoding might be changed.
*/
int
-sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
+sqlVdbeIntValue(const struct Mem *pMem, int64_t *i, bool *is_neg)
{
int flags;
assert(EIGHT_BYTE_ALIGNMENT(pMem));
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 6c022d8d8..e3b55644e 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -226,6 +226,35 @@ mem_div(const struct Mem *left, const struct Mem *right, struct Mem *result);
int
mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result);
+/** Perform a bitwise AND for two MEMs and write the result to the third MEM. */
+int
+mem_bit_and(const struct Mem *left, const struct Mem *right,
+ struct Mem *result);
+
+/** Perform a bitwise OR for two MEMs and write the result to the third MEM. */
+int
+mem_bit_or(const struct Mem *left, const struct Mem *right, struct Mem *result);
+
+/**
+ * Perform a bitwise left shift for the first MEM by value from the second MEM
+ * and write the result to the third MEM.
+ */
+int
+mem_shift_left(const struct Mem *left, const struct Mem *right,
+ struct Mem *result);
+
+/**
+ * Perform a bitwise right shift for the first MEM by value from the second MEM
+ * and write the result to the third MEM.
+ */
+int
+mem_shift_right(const struct Mem *left, const struct Mem *right,
+ struct Mem *result);
+
+/** Perform a bitwise NOT to the MEM and write the result to the second MEM. */
+int
+mem_bit_not(const struct Mem *mem, struct Mem *result);
+
/** Compare two non-NULL MEMs and return the result of comparison. */
int
mem_compare(const struct Mem *left, const struct Mem *right, int *result,
@@ -464,7 +493,7 @@ releaseMemArray(Mem * p, int N);
int
mem_value_bool(const struct Mem *mem, bool *b);
-int sqlVdbeIntValue(struct Mem *, int64_t *, bool *is_neg);
+int sqlVdbeIntValue(const struct Mem *, int64_t *, bool *is_neg);
int sqlVdbeRealValue(struct Mem *, double *);
const void *
sql_value_blob(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6a923a8b6..2ad681fa4 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1341,6 +1341,16 @@ case OP_FunctionByName: {
* store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_BitAnd: { /* same as TK_BITAND, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_bit_and(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ assert(pOut->field_type == FIELD_TYPE_INTEGER);
+ break;
+}
+
/* Opcode: BitOr P1 P2 P3 * *
* Synopsis: r[P3]=r[P1]|r[P2]
*
@@ -1348,6 +1358,16 @@ case OP_FunctionByName: {
* store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_BitOr: { /* same as TK_BITOR, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_bit_or(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ assert(pOut->field_type == FIELD_TYPE_INTEGER);
+ break;
+}
+
/* Opcode: ShiftLeft P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]<<r[P1]
*
@@ -1356,6 +1376,16 @@ case OP_FunctionByName: {
* Store the result in register P3.
* If either input is NULL, the result is NULL.
*/
+case OP_ShiftLeft: { /* same as TK_LSHIFT, in1, in2, out3 */
+ pIn1 = &aMem[pOp->p1];
+ pIn2 = &aMem[pOp->p2];
+ pOut = &aMem[pOp->p3];
+ if (mem_shift_left(pIn2, pIn1, pOut) != 0)
+ goto abort_due_to_error;
+ assert(pOut->field_type == FIELD_TYPE_INTEGER);
+ break;
+}
+
/* Opcode: ShiftRight P1 P2 P3 * *
* Synopsis: r[P3]=r[P2]>>r[P1]
*
@@ -1364,64 +1394,13 @@ case OP_FunctionByName: {
* Store the result in register P3.
* If either input is NULL, the result is NULL.
*/
-case OP_BitAnd: /* same as TK_BITAND, in1, in2, out3 */
-case OP_BitOr: /* same as TK_BITOR, in1, in2, out3 */
-case OP_ShiftLeft: /* same as TK_LSHIFT, in1, in2, out3 */
case OP_ShiftRight: { /* same as TK_RSHIFT, in1, in2, out3 */
- i64 iA;
- u64 uA;
- i64 iB;
- u8 op;
-
pIn1 = &aMem[pOp->p1];
pIn2 = &aMem[pOp->p2];
- pOut = vdbe_prepare_null_out(p, pOp->p3);
- if (mem_is_any_null(pIn1, pIn2)) {
- /* Force NULL be of type INTEGER. */
- pOut->field_type = FIELD_TYPE_INTEGER;
- break;
- }
- bool unused;
- if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn2), "integer");
- goto abort_due_to_error;
- }
- if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "integer");
+ pOut = &aMem[pOp->p3];
+ if (mem_shift_right(pIn2, pIn1, pOut) != 0)
goto abort_due_to_error;
- }
- op = pOp->opcode;
- if (op==OP_BitAnd) {
- iA &= iB;
- } else if (op==OP_BitOr) {
- iA |= iB;
- } else if (iB!=0) {
- assert(op==OP_ShiftRight || op==OP_ShiftLeft);
-
- /* If shifting by a negative amount, shift in the other direction */
- if (iB<0) {
- assert(OP_ShiftRight==OP_ShiftLeft+1);
- op = 2*OP_ShiftLeft + 1 - op;
- iB = iB>(-64) ? -iB : 64;
- }
-
- if (iB>=64) {
- iA = (iA>=0 || op==OP_ShiftLeft) ? 0 : -1;
- } else {
- memcpy(&uA, &iA, sizeof(uA));
- if (op==OP_ShiftLeft) {
- uA <<= iB;
- } else {
- uA >>= iB;
- /* Sign-extend on a right shift of a negative number */
- if (iA<0) uA |= ((((u64)0xffffffff)<<32)|0xffffffff) << (64-iB);
- }
- memcpy(&iA, &uA, sizeof(iA));
- }
- }
- mem_set_i64(pOut, iA);
+ assert(pOut->field_type == FIELD_TYPE_INTEGER);
break;
}
@@ -1903,19 +1882,9 @@ case OP_Not: { /* same as TK_NOT, in1, out2 */
*/
case OP_BitNot: { /* same as TK_BITNOT, in1, out2 */
pIn1 = &aMem[pOp->p1];
- pOut = vdbe_prepare_null_out(p, pOp->p2);
- /* Force NULL be of type INTEGER. */
- pOut->field_type = FIELD_TYPE_INTEGER;
- if (!mem_is_null(pIn1)) {
- int64_t i;
- bool is_neg;
- if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "integer");
- goto abort_due_to_error;
- }
- mem_set_i64(pOut, ~i);
- }
+ pOut = &aMem[pOp->p2];
+ if (mem_bit_not(pIn1, pOut) != 0)
+ goto abort_due_to_error;
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:31 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 20:49 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:31 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Hi! Thanks for the fixes!
See 2 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index eee72a7fe..aeb801c7c 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -624,6 +624,115 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
<...>
> +
> +int
> +mem_shift_right(const struct Mem *left, const struct Mem *right,
> + struct Mem *result)
> +{
> + if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
> + return 0;
> + int64_t a;
> + int64_t b;
> + if (bitwise_prepare(left, right, &a, &b) != 0)
> + return -1;
> + if (b <= -64)
> + result->u.i = 0;
> + else if (b < 0)
> + result->u.i = a << -b;
> + else if (b > 64)
> + result->u.i = a >= 0 ? 0 : -1;
> + else
> + result->u.i = a >> b;
1. Right shit has different meaning for negative and positive
numbers. This code produces invalid output:
tarantool> box.execute('SELECT 9223372036854775808 >> 3')
---
- metadata:
- name: COLUMN_1
type: integer
rows:
- [-1152921504606846976]
...
The number should have decreased, but it should not have changed
its sign. It should be 1152921504606846976, positive. But I see
the same bug exists on the master branch, even though it had
special handling for negative numbers, which is also broken.
Is there a ticket for that? Can you fix it right away?
> + result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
> + return 0;
> +}
> +
> +int
> +mem_bit_not(const struct Mem *mem, struct Mem *result)
> +{
> + mem_clear(result);
> + result->field_type = FIELD_TYPE_INTEGER;
> + if ((mem->flags & MEM_Null) != 0)
> + return 0;
> + int64_t i;
> + bool unused;
> + if (sqlVdbeIntValue(mem, &i, &unused) != 0) {
> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(mem),
> + "integer");
> + return -1;
> + }
> + result->u.i = ~i;
> + result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
2. What if the original value was positive, and the user also
expected to get a positive result? For example, what if he did
~CAST(value AS UNSIGNED)? Or is it useless and I am expected to
do CAST(~value as UNSIGNED)?
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM
2021-04-12 23:31 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 20:49 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 20:49 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers below.
On Tue, Apr 13, 2021 at 01:31:39AM +0200, Vladislav Shpilevoy wrote:
> Hi! Thanks for the fixes!
>
> See 2 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index eee72a7fe..aeb801c7c 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -624,6 +624,115 @@ mem_rem(const struct Mem *left, const struct Mem *right, struct Mem *result)
>
> <...>
>
> > +
> > +int
> > +mem_shift_right(const struct Mem *left, const struct Mem *right,
> > + struct Mem *result)
> > +{
> > + if (is_result_null(left, right, result, FIELD_TYPE_INTEGER))
> > + return 0;
> > + int64_t a;
> > + int64_t b;
> > + if (bitwise_prepare(left, right, &a, &b) != 0)
> > + return -1;
> > + if (b <= -64)
> > + result->u.i = 0;
> > + else if (b < 0)
> > + result->u.i = a << -b;
> > + else if (b > 64)
> > + result->u.i = a >= 0 ? 0 : -1;
> > + else
> > + result->u.i = a >> b;
>
> 1. Right shit has different meaning for negative and positive
> numbers. This code produces invalid output:
>
> tarantool> box.execute('SELECT 9223372036854775808 >> 3')
> ---
> - metadata:
> - name: COLUMN_1
> type: integer
> rows:
> - [-1152921504606846976]
> ...
>
> The number should have decreased, but it should not have changed
> its sign. It should be 1152921504606846976, positive. But I see
> the same bug exists on the master branch, even though it had
> special handling for negative numbers, which is also broken.
>
> Is there a ticket for that? Can you fix it right away?
>
There is one issue about these operations: #5364. I think these bugs shuld be
fixed in this issue.
> > + result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
> > + return 0;
> > +}
> > +
> > +int
> > +mem_bit_not(const struct Mem *mem, struct Mem *result)
> > +{
> > + mem_clear(result);
> > + result->field_type = FIELD_TYPE_INTEGER;
> > + if ((mem->flags & MEM_Null) != 0)
> > + return 0;
> > + int64_t i;
> > + bool unused;
> > + if (sqlVdbeIntValue(mem, &i, &unused) != 0) {
> > + diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(mem),
> > + "integer");
> > + return -1;
> > + }
> > + result->u.i = ~i;
> > + result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
>
> 2. What if the original value was positive, and the user also
> expected to get a positive result? For example, what if he did
> ~CAST(value AS UNSIGNED)? Or is it useless and I am expected to
> do CAST(~value as UNSIGNED)?
I believe there is no way to get unsigned value more that INT64_MAX. Casts also
won't work:
tarantool> box.execute('select ~1;')
---
- metadata:
- name: COLUMN_1
type: integer
rows:
- [-2]
...
tarantool> box.execute('select CAST(~1 AS UNSIGNED);')
---
- null
- 'Type mismatch: can not convert -2 to unsigned'
...
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 22/52] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (20 preceding siblings ...)
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM Mergen Imeev via Tarantool-patches
@ 2021-04-09 18:11 ` Mergen Imeev via Tarantool-patches
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 23/52] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches
` (19 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 18:11 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch adds initialization for newly created MEM objects in
sqlVdbeAllocUnpackedRecord(). Changing a MEM without being
initialized may give us unexpected result.
Part of #5818
---
src/box/sql/vdbeaux.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index dff108412..e0d815958 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2264,6 +2264,8 @@ sqlVdbeAllocUnpackedRecord(struct sql *db, struct key_def *key_def)
if (!p)
return 0;
p->aMem = (Mem *) & ((char *)p)[ROUND8(sizeof(UnpackedRecord))];
+ for (uint32_t i = 0; i < key_def->part_count + 1; ++i)
+ mem_create(&p->aMem[i]);
p->key_def = key_def;
p->nField = key_def->part_count + 1;
return p;
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 23/52] sql: introduce mem_set_null()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (21 preceding siblings ...)
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 22/52] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches
@ 2021-04-09 18:11 ` Mergen Imeev via Tarantool-patches
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int() Mergen Imeev via Tarantool-patches
` (18 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 18:11 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_set_null(). This function clears MEM and sets
it to NULL.
Part of #5818
---
src/box/sql/mem.c | 64 +++++++++++++++++++------------------------
src/box/sql/mem.h | 7 +++--
src/box/sql/vdbe.c | 14 ++++------
src/box/sql/vdbeapi.c | 4 +--
4 files changed, 40 insertions(+), 49 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index aeb801c7c..937fa3272 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -267,6 +267,12 @@ mem_destroy(struct Mem *mem)
mem->zMalloc = NULL;
}
+void
+mem_set_null(struct Mem *mem)
+{
+ mem_clear(mem);
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2058,24 +2064,6 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
return 0;
}
-/*
- * Delete any previous value and set the value stored in *pMem to NULL.
- *
- * This routine calls the Mem.xDel destructor to dispose of values that
- * require the destructor. But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use mem_destroy(), which both calls this
- * routine to invoke the destructor and deallocates Mem.zMalloc.
- *
- * Use this routine to reset the Mem prior to insert a new value.
- *
- * Use mem_destroy() to complete erase the Mem prior to abandoning it.
- */
-void
-sqlVdbeMemSetNull(Mem * pMem)
-{
- mem_clear(pMem);
-}
-
/*
* Delete any previous value and set the value to be a BLOB of length
* n containing all zeros.
@@ -2106,12 +2094,6 @@ sqlValueSetStr(sql_value * v, /* Value to be set */
sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
}
-void
-sqlValueSetNull(sql_value * p)
-{
- sqlVdbeMemSetNull((Mem *) p);
-}
-
/*
* Free an sql_value object
*/
@@ -3001,21 +2983,27 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
NULL, -1 - i, &field) < 0) {
goto error;
}
+ mem_clear(&val[i]);
switch (field.type) {
case MP_BOOL:
- mem_set_bool(&val[i], field.bval);
+ val[i].flags = MEM_Bool;
+ val[i].u.b = field.bval;
break;
case MP_FLOAT:
- mem_set_double(&val[i], field.fval);
+ val[i].flags = MEM_Real;
+ val[i].u.r = field.fval;
break;
case MP_DOUBLE:
- mem_set_double(&val[i], field.dval);
+ val[i].flags = MEM_Real;
+ val[i].u.r = field.dval;
break;
case MP_INT:
- mem_set_i64(&val[i], field.ival);
+ val[i].flags = MEM_Int;
+ val[i].u.i = field.ival;
break;
case MP_UINT:
- mem_set_u64(&val[i], field.ival);
+ val[i].flags = MEM_UInt;
+ val[i].u.i = field.ival;
break;
case MP_STR:
if (sqlVdbeMemSetStr(&val[i], field.sval.data,
@@ -3024,7 +3012,6 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
goto error;
break;
case MP_NIL:
- sqlVdbeMemSetNull(&val[i]);
break;
default:
diag_set(ClientError, ER_SQL_EXECUTE,
@@ -3071,22 +3058,28 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
data = pe->mp;
}
uint32_t len;
+ mem_clear(&val[i]);
const char *str;
switch (mp_typeof(*data)) {
case MP_BOOL:
- mem_set_bool(&val[i], mp_decode_bool(&data));
+ val[i].flags = MEM_Bool;
+ val[i].u.b = mp_decode_bool(&data);
break;
case MP_FLOAT:
- mem_set_double(&val[i], mp_decode_float(&data));
+ val[i].flags = MEM_Real;
+ val[i].u.r = mp_decode_float(&data);
break;
case MP_DOUBLE:
- mem_set_double(&val[i], mp_decode_double(&data));
+ val[i].flags = MEM_Real;
+ val[i].u.r = mp_decode_double(&data);
break;
case MP_INT:
- mem_set_i64(&val[i], mp_decode_int(&data));
+ val[i].flags = MEM_Int;
+ val[i].u.i = mp_decode_int(&data);
break;
case MP_UINT:
- mem_set_u64(&val[i], mp_decode_uint(&data));
+ val[i].flags = MEM_UInt;
+ val[i].u.u = mp_decode_uint(&data);
break;
case MP_STR:
str = mp_decode_str(&data, &len);
@@ -3095,7 +3088,6 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
goto error;
break;
case MP_NIL:
- sqlVdbeMemSetNull(&val[i]);
break;
default:
diag_set(ClientError, ER_SQL_EXECUTE,
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index e3b55644e..581ef007a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -162,6 +162,10 @@ mem_create(struct Mem *mem);
void
mem_destroy(struct Mem *mem);
+/** Clear MEM and set it to NULL. */
+void
+mem_set_null(struct Mem *mem);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -468,12 +472,9 @@ mem_set_double(struct Mem *mem, double value);
int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void
-sqlVdbeMemSetNull(struct Mem *);
-void
sqlVdbeMemSetZeroBlob(struct Mem *, int);
void sqlValueSetStr(struct Mem *, int, const void *,
void (*)(void *));
-void sqlValueSetNull(struct Mem *);
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 2ad681fa4..c7d21ddd5 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -276,7 +276,7 @@ vdbe_prepare_null_out(struct Vdbe *v, int n)
assert(n <= (v->nMem + 1 - v->nCursor));
struct Mem *out = &v->aMem[n];
memAboutToChange(v, out);
- sqlVdbeMemSetNull(out);
+ mem_set_null(out);
out->field_type = field_type_MAX;
return out;
}
@@ -862,19 +862,17 @@ case OP_String: { /* out2 */
*/
case OP_Null: { /* out2 */
int cnt;
- u16 nullFlag;
pOut = vdbe_prepare_null_out(p, pOp->p2);
cnt = pOp->p3-pOp->p2;
assert(pOp->p3<=(p->nMem+1 - p->nCursor));
- pOut->flags = nullFlag = pOp->p1 ? (MEM_Null|MEM_Cleared) : MEM_Null;
- pOut->n = 0;
+ if (pOp->p1 != 0)
+ pOut->flags = MEM_Null | MEM_Cleared;
while( cnt>0) {
pOut++;
memAboutToChange(p, pOut);
- sqlVdbeMemSetNull(pOut);
- pOut->flags = nullFlag;
- pOut->field_type = field_type_MAX;
- pOut->n = 0;
+ mem_set_null(pOut);
+ if (pOp->p1 != 0)
+ pOut->flags = MEM_Null | MEM_Cleared;
cnt--;
}
break;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 3e9f1ff6f..4398b41b1 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -201,7 +201,7 @@ sql_result_bool(struct sql_context *ctx, bool value)
void
sql_result_null(sql_context * pCtx)
{
- sqlVdbeMemSetNull(pCtx->pOut);
+ mem_set_null(pCtx->pOut);
}
void
@@ -382,7 +382,7 @@ createAggContext(sql_context * p, int nByte)
Mem *pMem = p->pMem;
assert(!mem_is_agg(pMem));
if (nByte <= 0) {
- sqlVdbeMemSetNull(pMem);
+ mem_set_null(pMem);
pMem->z = 0;
} else {
sqlVdbeMemClearAndResize(pMem, nByte);
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (22 preceding siblings ...)
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 23/52] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches
@ 2021-04-09 18:11 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:32 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 25/52] sql: introduce mem_set_uint() Mergen Imeev via Tarantool-patches
` (17 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 18:11 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:04, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 5 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_set_integer(). Function mem_set_integer()
>> clears MEM and sets it to given integer value.
>>
>> Part of #5818
>> ---
>> src/box/sql/func.c | 6 +--
>> src/box/sql/mem.c | 88 ++++++++++++++++++-------------------------
>> src/box/sql/mem.h | 21 ++++-------
>> src/box/sql/vdbe.c | 10 ++---
>> src/box/sql/vdbeapi.c | 4 +-
>> src/box/sql/vdbeaux.c | 6 +--
>> 6 files changed, 56 insertions(+), 79 deletions(-)
>>
>> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
>> index b61de18d8..8f7550f30 100644
>> --- a/src/box/sql/func.c
>> +++ b/src/box/sql/func.c
>> @@ -278,7 +278,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
>> mem_set_double(&val[i], field.dval);
>> break;
>> case MP_INT:
>> - mem_set_i64(&val[i], field.ival);
>> + mem_set_integer(&val[i], field.ival, true);
>
> 1. It is worth adding a function for setting a negative integer,
> like I mentioned in one of the previous emails. Might make such
> places easier to read.
>
I moved this block of code to mem.c and inlined mem setting.
>> break;
>> case MP_UINT:
>> mem_set_u64(&val[i], field.ival);
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index da2aa5c94..13a587aba 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -557,8 +566,7 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>> default:
>> unreachable();
>> }
>> - result->u.i = ires;
>> - result->flags = is_res_neg ? MEM_Int : MEM_UInt;
>> + mem_set_integer(result, ires, is_res_neg);
>
> 2. mem_set_integer() calls mem_clear(), but you already called
> clear for the result in the beginning of this function. Better
> keep the old version here. Inside of mem functions you can do
> things more efficiently sometimes, without using the public API.
>
Thank, fixed. I inlined mem_set_*() where we do not need clear().
> Also there was no field_type set to FIELD_TYPE_INTEGER before. Why
> did you change that? It was NUMBER.
>
Fixed.
>> @@ -583,13 +592,13 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
>> return -1;
>> }
>> if (op == OP_BitAnd) {
>> - result->u.i = l & r;
>> - result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
>> + res = l & r;
>> + mem_set_integer(result, res, res < 0);
>
> 3. The same. Clear() is called second time and field_type is changed,
> but it wasn't before. Why? The same in some other similar places in the
> patch.
>
Fixed. Inlined.
>> @@ -1359,15 +1368,14 @@ vdbe_mem_numerify(struct Mem *mem)
>> if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
>> return 0;
>> if ((mem->flags & MEM_Bool) != 0) {
>> - mem->u.u = mem->u.b;
>> - MemSetTypeFlag(mem, MEM_UInt);
>> + mem_set_integer(mem, (int64_t)mem->u.b, false);
>
> 4. Why can't you replace it with mem_set_u64()? If this is because you
> need FIELD_TYPE_INTEGER, then see the question above why the field
> type is set now.
>
Fixed. Inlined.
>> return 0;
>> }> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index f0b56033a..92845d66d 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -2999,8 +2999,8 @@ case OP_FCopy: { /* out2 */
>> assert(mem_is_integer(pIn1));
>>
>> pOut = vdbe_prepare_null_out(p, pOp->p2);
>> - mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
>> - pOut->field_type = pIn1->field_type;
>> + if (mem_copy(pOut, pIn1) != 0)
>> + goto abort_due_to_error;
>
> 5. Why? It couldn't fail before, now it can. It copied just
> int before, now it calls the full copy function which looks
> like an overkill.
>
Replaced mem_copy() to mem_copy_as_ephemeral(). In current case mem_copy() and
mem_copy_as_ephemeral() are not different since they both should just copy a few
fields. However, mem_copy_as_ephemeral() cannot return an error.
>> }
>> break;
>> }
New patch:
commit 69aa47b1575b768da04cc579e70038f5be272027
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 11:29:40 2021 +0300
sql: introduce mem_set_int()
This patch introduces mem_set_int(). This function clears MEM and sets
it to given integer value.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 937fa3272..075000218 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -273,6 +273,15 @@ mem_set_null(struct Mem *mem)
mem_clear(mem);
}
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
+{
+ mem_clear(mem);
+ mem->u.i = value;
+ mem->flags = is_neg ? MEM_Int : MEM_UInt;
+ mem->field_type = FIELD_TYPE_INTEGER;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1384,8 +1393,9 @@ vdbe_mem_numerify(struct Mem *mem)
if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
return 0;
if ((mem->flags & MEM_Bool) != 0) {
- mem->u.u = mem->u.b;
- MemSetTypeFlag(mem, MEM_UInt);
+ mem->u.u = (uint64_t)mem->u.b;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
return 0;
}
assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
@@ -1454,20 +1464,23 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return 0;
}
if ((pMem->flags & MEM_Bool) != 0) {
- pMem->u.u = pMem->u.b;
- MemSetTypeFlag(pMem, MEM_UInt);
+ pMem->u.u = (uint64_t)pMem->u.b;
+ pMem->flags = MEM_UInt;
+ pMem->field_type = FIELD_TYPE_UNSIGNED;
return 0;
}
if ((pMem->flags & MEM_Real) != 0) {
- double d;
- if (sqlVdbeRealValue(pMem, &d) != 0)
- return -1;
- if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
- mem_set_int(pMem, d, d <= -1);
+ double d = pMem->u.r;
+ if (d < 0. && d >= (double)INT64_MIN) {
+ pMem->u.i = (int64_t)d;
+ pMem->flags = MEM_Int;
+ pMem->field_type = FIELD_TYPE_INTEGER;
return 0;
}
- if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
- mem_set_u64(pMem, d);
+ if (d >= 0. && d < (double)UINT64_MAX) {
+ pMem->u.u = (uint64_t)d;
+ pMem->flags = MEM_UInt;
+ pMem->field_type = FIELD_TYPE_UNSIGNED;
return 0;
}
return -1;
@@ -1518,8 +1531,11 @@ mem_apply_integer_type(Mem *pMem)
assert(pMem->flags & MEM_Real);
assert(EIGHT_BYTE_ALIGNMENT(pMem));
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
- mem_set_int(pMem, ix, pMem->u.r <= -1);
+ if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
+ pMem->u.i = ix;
+ pMem->flags = pMem->u.r <= -1 ? MEM_Int : MEM_UInt;
+ pMem->field_type = FIELD_TYPE_INTEGER;
+ }
return rc;
}
@@ -1663,13 +1679,20 @@ mem_apply_type(struct Mem *record, enum field_type type)
if (double_compare_uint64(d, UINT64_MAX,
1) > 0)
return 0;
- if ((double)(uint64_t)d == d)
- mem_set_u64(record, (uint64_t)d);
+ if ((double)(uint64_t)d == d) {
+ record->u.u = (uint64_t)d;
+ record->flags = MEM_UInt;
+ record->field_type =
+ FIELD_TYPE_UNSIGNED;
+ }
} else {
if (double_compare_nint64(d, INT64_MIN, 1) < 0)
return 0;
- if ((double)(int64_t)d == d)
- mem_set_int(record, (int64_t)d, true);
+ if ((double)(int64_t)d == d) {
+ record->u.i = (int64_t)d;
+ record->flags = MEM_Int;
+ record->field_type = FIELD_TYPE_INTEGER;
+ }
}
return 0;
}
@@ -1781,7 +1804,9 @@ mem_convert_to_unsigned(struct Mem *mem)
double d = mem->u.r;
if (d < 0.0 || d >= (double)UINT64_MAX)
return -1;
- mem_set_u64(mem, (uint64_t) d);
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
return 0;
}
@@ -1801,10 +1826,15 @@ mem_convert_to_integer(struct Mem *mem)
double d = mem->u.r;
if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
return -1;
- if (d < (double)INT64_MAX)
- mem_set_int(mem, (int64_t) d, d < 0);
- else
- mem_set_int(mem, (uint64_t) d, false);
+ if (d < 0.) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ } else {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ }
return 0;
}
@@ -1923,16 +1953,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
-void
-mem_set_i64(struct Mem *mem, int64_t value)
-{
- mem_clear(mem);
- mem->u.i = value;
- int flag = value < 0 ? MEM_Int : MEM_UInt;
- MemSetTypeFlag(mem, flag);
- mem->field_type = FIELD_TYPE_INTEGER;
-}
-
void
mem_set_u64(struct Mem *mem, uint64_t value)
{
@@ -1942,21 +1962,6 @@ mem_set_u64(struct Mem *mem, uint64_t value)
mem->field_type = FIELD_TYPE_UNSIGNED;
}
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
-{
- mem_clear(mem);
- if (is_neg) {
- assert(value < 0);
- mem->u.i = value;
- MemSetTypeFlag(mem, MEM_Int);
- } else {
- mem->u.u = value;
- MemSetTypeFlag(mem, MEM_UInt);
- }
- mem->field_type = FIELD_TYPE_INTEGER;
-}
-
void
mem_set_double(struct Mem *mem, double value)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 581ef007a..788f3c89f 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -166,6 +166,10 @@ mem_destroy(struct Mem *mem);
void
mem_set_null(struct Mem *mem);
+/** Clear MEM and set it to INTEGER. */
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -447,24 +451,10 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_ptr(struct Mem *mem, void *ptr);
-/**
- * Set integer value. Depending on its sign MEM_Int (in case
- * of negative value) or MEM_UInt flag is set.
- */
-void
-mem_set_i64(struct Mem *mem, int64_t value);
-
/** Set unsigned value and MEM_UInt flag. */
void
mem_set_u64(struct Mem *mem, uint64_t value);
-/**
- * Set integer value. According to is_neg flag value is considered
- * to be signed or unsigned.
- */
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
-
/** Set double value and MEM_Real flag. */
void
mem_set_double(struct Mem *mem, double value);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index c7d21ddd5..0a3de8ac9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -763,7 +763,7 @@ case OP_Halt: {
*/
case OP_Integer: { /* out2 */
pOut = vdbe_prepare_null_out(p, pOp->p2);
- mem_set_i64(pOut, pOp->p1);
+ mem_set_int(pOut, pOp->p1, pOp->p1 < 0);
break;
}
@@ -3090,8 +3090,7 @@ case OP_FCopy: { /* out2 */
assert(mem_is_int(pIn1));
pOut = vdbe_prepare_null_out(p, pOp->p2);
- mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
- pOut->field_type = pIn1->field_type;
+ mem_copy_as_ephemeral(pOut, pIn1);
}
break;
}
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 4398b41b1..737477285 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -189,7 +189,7 @@ sql_result_uint(sql_context *ctx, uint64_t u_val)
void
sql_result_int(sql_context *ctx, int64_t val)
{
- mem_set_i64(ctx->pOut, val);
+ mem_set_int(ctx->pOut, val, val < 0);
}
void
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index e0d815958..8e50d47bf 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1324,13 +1324,13 @@ sqlVdbeList(Vdbe * p)
}
}
- mem_set_i64(pMem, pOp->p1);
+ mem_set_int(pMem, pOp->p1, pOp->p1 < 0);
pMem++;
- mem_set_i64(pMem, pOp->p2);
+ mem_set_int(pMem, pOp->p2, pOp->p2 < 0);
pMem++;
- mem_set_i64(pMem, pOp->p3);
+ mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
pMem++;
if (sqlVdbeMemClearAndResize(pMem, 256)) {
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int()
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:32 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 20:56 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:32 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Good job on the fixes!
See 3 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 937fa3272..075000218 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -1384,8 +1393,9 @@ vdbe_mem_numerify(struct Mem *mem)
> if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
> return 0;
> if ((mem->flags & MEM_Bool) != 0) {
> - mem->u.u = mem->u.b;
> - MemSetTypeFlag(mem, MEM_UInt);
> + mem->u.u = (uint64_t)mem->u.b;
> + mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_UNSIGNED;
1. Field type wasn't set before. Why did you change that?
> return 0;
> }
> assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
> @@ -1454,20 +1464,23 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
> return 0;
> }
> if ((pMem->flags & MEM_Bool) != 0) {
> - pMem->u.u = pMem->u.b;
> - MemSetTypeFlag(pMem, MEM_UInt);
> + pMem->u.u = (uint64_t)pMem->u.b;
> + pMem->flags = MEM_UInt;
> + pMem->field_type = FIELD_TYPE_UNSIGNED;
2. Ditto.
> return 0;
> }
> @@ -1801,10 +1826,15 @@ mem_convert_to_integer(struct Mem *mem)
> double d = mem->u.r;
> if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
> return -1;
> - if (d < (double)INT64_MAX)
> - mem_set_int(mem, (int64_t) d, d < 0);
> - else
> - mem_set_int(mem, (uint64_t) d, false);
> + if (d < 0.) {
> + mem->u.i = (int64_t)d;
> + mem->flags = MEM_Int;
> + mem->field_type = FIELD_TYPE_INTEGER;
> + } else {
> + mem->u.u = (uint64_t)d;
> + mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_UNSIGNED;
3. Previously it was FIELD_TYPE_INTEGER in both cases. Why
did you change that?
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int()
2021-04-12 23:32 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 20:56 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 20:56 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers below.
On Tue, Apr 13, 2021 at 01:32:47AM +0200, Vladislav Shpilevoy wrote:
> Good job on the fixes!
>
> See 3 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 937fa3272..075000218 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -1384,8 +1393,9 @@ vdbe_mem_numerify(struct Mem *mem)
> > if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
> > return 0;
> > if ((mem->flags & MEM_Bool) != 0) {
> > - mem->u.u = mem->u.b;
> > - MemSetTypeFlag(mem, MEM_UInt);
> > + mem->u.u = (uint64_t)mem->u.b;
> > + mem->flags = MEM_UInt;
> > + mem->field_type = FIELD_TYPE_UNSIGNED;
>
> 1. Field type wasn't set before. Why did you change that?
>
Since now I cannot say where field-type and MEM-type are compatible and where
they are not, I just set field-type everywhere. Also, since there is no proper
rule to determine field-type of the MEM I set the smallest field-type possible
for the value of the MEM. This looks like something useless for me, but it is
better than checking if field-type is field_type_MAX everywhere end set
field-type in case it isn't.
> > return 0;
> > }
> > assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
> > @@ -1454,20 +1464,23 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
> > return 0;
> > }
> > if ((pMem->flags & MEM_Bool) != 0) {
> > - pMem->u.u = pMem->u.b;
> > - MemSetTypeFlag(pMem, MEM_UInt);
> > + pMem->u.u = (uint64_t)pMem->u.b;
> > + pMem->flags = MEM_UInt;
> > + pMem->field_type = FIELD_TYPE_UNSIGNED;
>
> 2. Ditto.
>
Same.
> > return 0;
> > }
> > @@ -1801,10 +1826,15 @@ mem_convert_to_integer(struct Mem *mem)
> > double d = mem->u.r;
> > if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
> > return -1;
> > - if (d < (double)INT64_MAX)
> > - mem_set_int(mem, (int64_t) d, d < 0);
> > - else
> > - mem_set_int(mem, (uint64_t) d, false);
> > + if (d < 0.) {
> > + mem->u.i = (int64_t)d;
> > + mem->flags = MEM_Int;
> > + mem->field_type = FIELD_TYPE_INTEGER;
> > + } else {
> > + mem->u.u = (uint64_t)d;
> > + mem->flags = MEM_UInt;
> > + mem->field_type = FIELD_TYPE_UNSIGNED;
>
> 3. Previously it was FIELD_TYPE_INTEGER in both cases. Why
> did you change that?
Same.
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 25/52] sql: introduce mem_set_uint()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (23 preceding siblings ...)
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int() Mergen Imeev via Tarantool-patches
@ 2021-04-09 18:11 ` Mergen Imeev via Tarantool-patches
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 26/52] sql: move mem_set_bool() and mem_set_double() Mergen Imeev via Tarantool-patches
` (16 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 18:11 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:04, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 13a587aba..f7a1a9b8a 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -1765,7 +1774,7 @@ mem_convert_to_unsigned(struct Mem *mem)
>> double d = mem->u.r;
>> if (d < 0.0 || d >= (double)UINT64_MAX)
>> return -1;
>> - mem_set_u64(mem, (uint64_t) d);
>> + mem_set_unsigned(mem, (uint64_t) d);
>
> Whitespace after unary operator.
Fixed.
New patch:
commit 93900f797e7e000af0efe5a1df1f7162f2f56db1
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 12:38:26 2021 +0300
sql: introduce mem_set_uint()
This patch introduces mem_set_uint(). This function clears MEM and sets
it to given unsigned value.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 075000218..d79d471bc 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -282,6 +282,15 @@ mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
mem->field_type = FIELD_TYPE_INTEGER;
}
+void
+mem_set_uint(struct Mem *mem, uint64_t value)
+{
+ mem_clear(mem);
+ mem->u.u = value;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1953,15 +1962,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
-void
-mem_set_u64(struct Mem *mem, uint64_t value)
-{
- mem_clear(mem);
- mem->u.u = value;
- MemSetTypeFlag(mem, MEM_UInt);
- mem->field_type = FIELD_TYPE_UNSIGNED;
-}
-
void
mem_set_double(struct Mem *mem, double value)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 788f3c89f..8bc058d3b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -170,6 +170,10 @@ mem_set_null(struct Mem *mem);
void
mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
+/** Clear MEM and set it to UNSIGNED. */
+void
+mem_set_uint(struct Mem *mem, uint64_t value);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -451,10 +455,6 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_ptr(struct Mem *mem, void *ptr);
-/** Set unsigned value and MEM_UInt flag. */
-void
-mem_set_u64(struct Mem *mem, uint64_t value);
-
/** Set double value and MEM_Real flag. */
void
mem_set_double(struct Mem *mem, double value);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 0a3de8ac9..eb997fa3d 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -609,7 +609,7 @@ case OP_Gosub: { /* jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
memAboutToChange(p, pIn1);
- mem_set_u64(pIn1, pOp - aOp);
+ mem_set_uint(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
/* Most jump operations do a goto to this spot in order to update
@@ -650,7 +650,7 @@ case OP_InitCoroutine: { /* jump */
assert(pOp->p3>0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
assert(!VdbeMemDynamic(pOut));
- mem_set_u64(pOut, pOp->p3 - 1);
+ mem_set_uint(pOut, pOp->p3 - 1);
if (pOp->p2) goto jump_to_p2;
break;
}
@@ -693,7 +693,7 @@ case OP_Yield: { /* in1, jump */
pIn1 = &aMem[pOp->p1];
assert(VdbeMemDynamic(pIn1)==0);
int pcDest = (int)pIn1->u.u;
- mem_set_u64(pIn1, pOp - aOp);
+ mem_set_uint(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
pOp = &aOp[pcDest];
break;
@@ -2233,7 +2233,7 @@ case OP_Count: { /* out2 */
nEntry = tarantoolsqlEphemeralCount(pCrsr);
}
pOut = vdbe_prepare_null_out(p, pOp->p2);
- mem_set_u64(pOut, nEntry);
+ mem_set_uint(pOut, nEntry);
break;
}
@@ -3011,7 +3011,7 @@ case OP_Sequence: { /* out2 */
assert(p->apCsr[pOp->p1]!=0);
pOut = vdbe_prepare_null_out(p, pOp->p2);
int64_t seq_val = p->apCsr[pOp->p1]->seqCount++;
- mem_set_u64(pOut, seq_val);
+ mem_set_uint(pOut, seq_val);
break;
}
@@ -3027,7 +3027,7 @@ case OP_NextSequenceId: {
uint64_t id = 0;
tarantoolSqlNextSeqId(&id);
id++;
- mem_set_u64(pOut, id);
+ mem_set_uint(pOut, id);
break;
}
@@ -3057,7 +3057,7 @@ case OP_NextIdEphemeral: {
goto abort_due_to_error;
}
pOut = vdbe_prepare_null_out(p, pOp->p2);
- mem_set_u64(pOut, rowid);
+ mem_set_uint(pOut, rowid);
break;
}
@@ -4324,7 +4324,7 @@ case OP_OffsetLimit: { /* in1, out2, in3 */
"values should not result in integer overflow");
goto abort_due_to_error;
}
- mem_set_u64(pOut, x);
+ mem_set_uint(pOut, x);
break;
}
@@ -4580,10 +4580,10 @@ case OP_Init: { /* jump */
case OP_IncMaxid: {
assert(pOp->p1 > 0);
pOut = vdbe_prepare_null_out(p, pOp->p1);
-
- if (tarantoolsqlIncrementMaxid(&pOut->u.u) != 0)
+ uint64_t u;
+ if (tarantoolsqlIncrementMaxid(&u) != 0)
goto abort_due_to_error;
- pOut->flags = MEM_UInt;
+ mem_set_uint(pOut, u);
break;
}
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 737477285..6c08e772d 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -183,7 +183,7 @@ sql_result_double(sql_context * pCtx, double rVal)
void
sql_result_uint(sql_context *ctx, uint64_t u_val)
{
- mem_set_u64(ctx->pOut, u_val);
+ mem_set_uint(ctx->pOut, u_val);
}
void
@@ -851,7 +851,7 @@ sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value)
if (vdbeUnbind(p, i) != 0)
return -1;
int rc = sql_bind_type(p, i, "integer");
- mem_set_u64(&p->aVar[i - 1], value);
+ mem_set_uint(&p->aVar[i - 1], value);
return rc;
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 8e50d47bf..ae55e4c29 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1291,7 +1291,7 @@ sqlVdbeList(Vdbe * p)
}
if (p->explain == 1) {
assert(i >= 0);
- mem_set_u64(pMem, i);
+ mem_set_uint(pMem, i);
pMem++;
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 26/52] sql: move mem_set_bool() and mem_set_double()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (24 preceding siblings ...)
2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 25/52] sql: introduce mem_set_uint() Mergen Imeev via Tarantool-patches
@ 2021-04-09 19:45 ` Mergen Imeev via Tarantool-patches
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions Mergen Imeev via Tarantool-patches
` (15 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:45 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
I squashed two patches here since they do the same thing - they move functions
from one place to another.
On 30.03.2021 02:04, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_set_double(). Function mem_set_double()
>> clears MEM and sets it to given unsigned value.
>
> 1. It is not about unsigned.
>
Fixed.
>> Part of #5818
>> ---
>> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
>> index bf3690b7c..49b4e4b1a 100644
>> --- a/src/box/sql/mem.h
>> +++ b/src/box/sql/mem.h
>> @@ -419,10 +422,6 @@ int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
>> void
>> mem_set_ptr(struct Mem *mem, void *ptr);
>>
>> -/** Set double value and MEM_Real flag. */
>> -void
>> -mem_set_double(struct Mem *mem, double value);
>
> 2. Why do you move these functions and their definitions? What was
> wrong with keeping them in place?
>
I think it is better when all these mem_set_*() functions in one block. This is
the only reason, I believe.
>> -
>> int
>> sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
>> void
>>
New patch:
commit fa3fc18275f01e824c6ca8f0bcbf27cb857fb3a8
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 12:42:59 2021 +0300
sql: move mem_set_bool() and mem_set_double()
This patch performs a small refactoring of mem_set_double() and moves
this function and mem_set_bool() to another place so that they are part
of the code block containing all mem_set_*() functions.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index d79d471bc..c16f3a28a 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -291,6 +291,26 @@ mem_set_uint(struct Mem *mem, uint64_t value)
mem->field_type = FIELD_TYPE_UNSIGNED;
}
+void
+mem_set_bool(struct Mem *mem, bool value)
+{
+ mem_clear(mem);
+ mem->u.b = value;
+ mem->flags = MEM_Bool;
+ mem->field_type = FIELD_TYPE_BOOLEAN;
+}
+
+void
+mem_set_double(struct Mem *mem, double value)
+{
+ mem_clear(mem);
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ if (sqlIsNaN(value))
+ return;
+ mem->u.r = value;
+ mem->flags = MEM_Real;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1945,15 +1965,6 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
return 0;
}
-void
-mem_set_bool(struct Mem *mem, bool value)
-{
- mem_clear(mem);
- mem->u.b = value;
- mem->flags = MEM_Bool;
- mem->field_type = FIELD_TYPE_BOOLEAN;
-}
-
void
mem_set_ptr(struct Mem *mem, void *ptr)
{
@@ -1962,17 +1973,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
-void
-mem_set_double(struct Mem *mem, double value)
-{
- mem_clear(mem);
- if (sqlIsNaN(value))
- return;
- mem->u.r = value;
- MemSetTypeFlag(mem, MEM_Real);
- mem->field_type = FIELD_TYPE_DOUBLE;
-}
-
/*
* Change the value of a Mem to be a string or a BLOB.
*
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8bc058d3b..8022f53d8 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -174,6 +174,14 @@ mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
void
mem_set_uint(struct Mem *mem, uint64_t value);
+/** Clear MEM and set it to BOOLEAN. */
+void
+mem_set_bool(struct Mem *mem, bool value);
+
+/** Clear MEM and set it to DOUBLE. */
+void
+mem_set_double(struct Mem *mem, double value);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -444,9 +452,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
-void
-mem_set_bool(struct Mem *mem, bool value);
-
/**
* Set VDBE memory register with given pointer as a data.
* @param mem VDBE memory register to update.
@@ -455,10 +460,6 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_ptr(struct Mem *mem, void *ptr);
-/** Set double value and MEM_Real flag. */
-void
-mem_set_double(struct Mem *mem, double value);
-
int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (25 preceding siblings ...)
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 26/52] sql: move mem_set_bool() and mem_set_double() Mergen Imeev via Tarantool-patches
@ 2021-04-09 19:45 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:34 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0() Mergen Imeev via Tarantool-patches
` (14 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:45 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:05, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
> 1. I think it would be better to have mem_set_str_* as a prefix
> for these functions. 'string' is too long, and by placing allocation
> type in the beginning you complicate grep by mem_set_str_* and
> autocompletion when I type 'mem_set_str_' and want to see all the
> options.
>
Fixed. Now all functions named as mem_set_str_*() or mem_set_str0_*().
> On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_set_*_string() functions. Functions
>> mem_set_*_string() clears MEM and sets it to given string value.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 111 +++++++++++++++++++++++++++++++++---------
>> src/box/sql/mem.h | 26 +++++++++-
>> src/box/sql/sqlInt.h | 4 --
>> src/box/sql/vdbe.c | 5 +-
>> src/box/sql/vdbeapi.c | 47 ++++++++++++++----
>> src/box/sql/vdbeaux.c | 45 +++++++----------
>> 6 files changed, 171 insertions(+), 67 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 47a71fb30..91ef7f3c8 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -2780,13 +2849,9 @@ vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
>> break;
>> }
>> case MP_STR: {
>> - /* XXX u32->int */
>> - mem->n = (int) mp_decode_strl(&buf);
>> - mem->flags = MEM_Str | MEM_Ephem;
>> - mem->field_type = FIELD_TYPE_STRING;
>> -install_blob:
>> - mem->z = (char *)buf;
>> - buf += mem->n;
>> + uint32_t len = mp_decode_strl(&buf);
>> + mem_set_ephemeral_string(mem, (char *)buf, len);
>
> 2. It adds clear() call which is not necessary here. I would
> propose to keep the old code for the sake of speed and
> consistency with the other mp types.
Thanks, fixed.
New patch:
commit f82e5a93ce35b81b682195e92284fa37a707b14e
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 14:28:29 2021 +0300
sql: introduce mem_set_str_*() functions
This patch introduces set of mem_set_str_*() functions. These functions
clears MEM and sets it to given string value. Degree of clearing and
type of allocation of the string is determined by the function used.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index c16f3a28a..87e1bcfd1 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -311,6 +311,89 @@ mem_set_double(struct Mem *mem, double value)
mem->flags = MEM_Real;
}
+static inline void
+set_str_const(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+ assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
+ mem_clear(mem);
+ mem->z = value;
+ mem->n = len;
+ mem->flags = MEM_Str | alloc_type;
+ mem->field_type = FIELD_TYPE_STRING;
+}
+
+static inline void
+set_str_dynamic(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+ assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
+ assert(mem->szMalloc == 0 || value != mem->zMalloc);
+ assert(alloc_type == MEM_Dyn || alloc_type == 0);
+ mem_destroy(mem);
+ mem->z = value;
+ mem->n = len;
+ mem->flags = MEM_Str | alloc_type;
+ mem->field_type = FIELD_TYPE_STRING;
+ if (alloc_type == MEM_Dyn) {
+ mem->xDel = sql_free;
+ } else {
+ mem->xDel = NULL;
+ mem->zMalloc = mem->z;
+ mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
+ }
+}
+
+void
+mem_set_str_ephemeral(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_const(mem, value, len, MEM_Ephem);
+}
+
+void
+mem_set_str_static(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_const(mem, value, len, MEM_Static);
+}
+
+void
+mem_set_str_dynamic(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_dynamic(mem, value, len, MEM_Dyn);
+}
+
+void
+mem_set_str_allocated(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_dynamic(mem, value, len, 0);
+}
+
+void
+mem_set_str0_ephemeral(struct Mem *mem, char *value)
+{
+ set_str_const(mem, value, strlen(value), MEM_Ephem);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_static(struct Mem *mem, char *value)
+{
+ set_str_const(mem, value, strlen(value), MEM_Static);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_dynamic(struct Mem *mem, char *value)
+{
+ set_str_dynamic(mem, value, strlen(value), MEM_Dyn);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_allocated(struct Mem *mem, char *value)
+{
+ set_str_dynamic(mem, value, strlen(value), 0);
+ mem->flags |= MEM_Term;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2085,20 +2168,6 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
pMem->z = 0;
}
-/*
- * Change the string value of an sql_value object
- */
-void
-sqlValueSetStr(sql_value * v, /* Value to be set */
- int n, /* Length of string z */
- const void *z, /* Text of the new string */
- void (*xDel) (void *) /* Destructor for the string */
- )
-{
- if (v)
- sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
-}
-
/*
* Free an sql_value object
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8022f53d8..ecaa1edb8 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -182,6 +182,59 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_double(struct Mem *mem, double value);
+/** Clear MEM and set it to STRING. The string belongs to another object. */
+void
+mem_set_str_ephemeral(struct Mem *mem, char *value, uint32_t len);
+
+/** Clear MEM and set it to STRING. The string is static. */
+void
+mem_set_str_static(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to STRING. The string was allocated by another object
+ * and passed to MEM. MEMs with this allocation type must free given memory
+ * whenever the MEM changes.
+ */
+void
+mem_set_str_dynamic(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to STRING. The string was allocated by another object
+ * and passed to MEM. MEMs with this allocation type only deallocate the string
+ * on destruction. Also, the memory may be reallocated if MEM is set to a
+ * different value of this allocation type.
+ */
+void
+mem_set_str_allocated(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string belongs to
+ * another object.
+ */
+void
+mem_set_str0_ephemeral(struct Mem *mem, char *value);
+
+/** Clear MEM and set it to NULL-terminated STRING. The string is static. */
+void
+mem_set_str0_static(struct Mem *mem, char *value);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string was allocated by
+ * another object and passed to MEM. MEMs with this allocation type must free
+ * given memory whenever the MEM changes.
+ */
+void
+mem_set_str0_dynamic(struct Mem *mem, char *value);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string was allocated by
+ * another object and passed to MEM. MEMs with this allocation type only
+ * deallocate the string on destruction. Also, the memory may be reallocated if
+ * MEM is set to a different value of this allocation type.
+ */
+void
+mem_set_str0_allocated(struct Mem *mem, char *value);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -464,8 +517,6 @@ int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void
sqlVdbeMemSetZeroBlob(struct Mem *, int);
-void sqlValueSetStr(struct Mem *, int, const void *,
- void (*)(void *));
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 259ba3833..7a026d21b 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -641,10 +641,6 @@ sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value);
int
sql_bind_null(sql_stmt *, int);
-int
-sql_bind_text(sql_stmt *, int, const char *, int,
- void (*)(void *));
-
int
sql_bind_text64(sql_stmt *, int, const char *,
sql_uint64, void (*)(void *));
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index eb997fa3d..943179c55 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -841,9 +841,8 @@ case OP_String8: { /* same as TK_STRING, out2 */
case OP_String: { /* out2 */
assert(pOp->p4.z!=0);
pOut = vdbe_prepare_null_out(p, pOp->p2);
- pOut->flags = MEM_Str|MEM_Static|MEM_Term;
- pOut->z = pOp->p4.z;
- pOut->n = pOp->p1;
+ assert(strlen(pOp->p4.z) == (size_t)pOp->p1);
+ mem_set_str0_static(pOut, pOp->p4.z);
UPDATE_MAX_BLOBSIZE(pOut);
break;
}
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 6c08e772d..e0903e3b0 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -125,6 +125,27 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
void (*xDel) (void *) /* Destructor function */
)
{
+ if (xDel == SQL_STATIC) {
+ if (n < 0)
+ mem_set_str0_static(pCtx->pOut, (char *)z);
+ else
+ mem_set_str_static(pCtx->pOut, (char *)z, n);
+ return;
+ }
+ if (xDel == SQL_DYNAMIC) {
+ if (n < 0)
+ mem_set_str0_allocated(pCtx->pOut, (char *)z);
+ else
+ mem_set_str_allocated(pCtx->pOut, (char *)z, n);
+ return;
+ }
+ if (xDel != SQL_TRANSIENT) {
+ if (n < 0)
+ mem_set_str0_dynamic(pCtx->pOut, (char *)z);
+ else
+ mem_set_str_dynamic(pCtx->pOut, (char *)z, n);
+ return;
+ }
if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -762,8 +783,24 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
if (zData == NULL)
return 0;
pVar = &p->aVar[i - 1];
- if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
+ if (xDel == SQL_STATIC) {
+ if (nData < 0)
+ mem_set_str0_static(pVar, (char *)zData);
+ else
+ mem_set_str_static(pVar, (char *)zData, nData);
+ } else if (xDel == SQL_DYNAMIC) {
+ if (nData < 0)
+ mem_set_str0_allocated(pVar, (char *)zData);
+ else
+ mem_set_str_allocated(pVar, (char *)zData, nData);
+ } else if (xDel != SQL_TRANSIENT) {
+ if (nData < 0)
+ mem_set_str0_dynamic(pVar, (char *)zData);
+ else
+ mem_set_str_dynamic(pVar, (char *)zData, nData);
+ } else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
return -1;
+ }
return sql_bind_type(p, i, "text");
}
@@ -876,14 +913,6 @@ sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr)
return rc;
}
-int
-sql_bind_text(sql_stmt * pStmt,
- int i, const char *zData, int nData, void (*xDel) (void *)
- )
-{
- return bindText(pStmt, i, zData, nData, xDel);
-}
-
int
sql_bind_text64(sql_stmt * pStmt,
int i,
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index ae55e4c29..171cb8946 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1295,10 +1295,8 @@ sqlVdbeList(Vdbe * p)
pMem++;
- pMem->flags = MEM_Static | MEM_Str | MEM_Term;
- pMem->z = (char *)sqlOpcodeName(pOp->opcode); /* Opcode */
- assert(pMem->z != 0);
- pMem->n = sqlStrlen30(pMem->z);
+ char *value = (char *)sqlOpcodeName(pOp->opcode);
+ mem_set_str0_static(pMem, value);
pMem++;
/* When an OP_Program opcode is encounter (the only opcode that has
@@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
pMem++;
- if (sqlVdbeMemClearAndResize(pMem, 256)) {
- assert(p->db->mallocFailed);
+ char *buf = sqlDbMallocRaw(sql_get(), 256);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- zP4 = displayP4(pOp, pMem->z, pMem->szMalloc);
-
- if (zP4 != pMem->z) {
- pMem->n = 0;
- sqlVdbeMemSetStr(pMem, zP4, -1, 1, 0);
+ zP4 = displayP4(pOp, buf, sqlDbMallocSize(sql_get(), buf));
+ if (zP4 != buf) {
+ sqlDbFree(sql_get(), buf);
+ mem_set_str0_ephemeral(pMem, zP4);
} else {
- assert(pMem->z != 0);
- pMem->n = sqlStrlen30(pMem->z);
+ mem_set_str0_allocated(pMem, zP4);
}
pMem++;
if (p->explain == 1) {
- if (sqlVdbeMemClearAndResize(pMem, 4)) {
- assert(p->db->mallocFailed);
+ buf = sqlDbMallocRaw(sql_get(), 4);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- pMem->n = 2;
- sql_snprintf(3, pMem->z, "%.2x", pOp->p5); /* P5 */
+ sql_snprintf(3, buf, "%.2x", pOp->p5);
+ mem_set_str0_allocated(pMem, buf);
pMem++;
#ifdef SQL_ENABLE_EXPLAIN_COMMENTS
- if (sqlVdbeMemClearAndResize(pMem, 500)) {
- assert(p->db->mallocFailed);
+ buf = sqlDbMallocRaw(sql_get(), 500);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- pMem->n = displayComment(pOp, zP4, pMem->z, 500);
+ displayComment(pOp, zP4, buf, 500);
+ mem_set_str0_allocated(pMem, buf);
#else
- pMem->flags = MEM_Null; /* Comment */
+ mem_set_null(pMem);
#endif
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:34 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 21:36 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:34 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Nice fixes!
See 2 comments below.
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index 6c08e772d..e0903e3b0 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -125,6 +125,27 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
> void (*xDel) (void *) /* Destructor function */
> )
> {
> + if (xDel == SQL_STATIC) {
> + if (n < 0)
> + mem_set_str0_static(pCtx->pOut, (char *)z);
> + else
> + mem_set_str_static(pCtx->pOut, (char *)z, n);
> + return;
> + }
> + if (xDel == SQL_DYNAMIC) {
> + if (n < 0)
> + mem_set_str0_allocated(pCtx->pOut, (char *)z);
> + else
> + mem_set_str_allocated(pCtx->pOut, (char *)z, n);
1. I don't understand. You check for xDel == SQL_DYNAMIC and yet
you use 'allocated' suffix. Below you check for != TRANSIENT
instead of == DYNAMIC, and use 'dynamic'. Why? All looks messed
up. The same in the next hunk.
> + return;
> + }> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index ae55e4c29..171cb8946 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
> mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
> pMem++;
>
> - if (sqlVdbeMemClearAndResize(pMem, 256)) {
> - assert(p->db->mallocFailed);
> + char *buf = sqlDbMallocRaw(sql_get(), 256);
2. I think you need some kind of mem_set_strlen(), or mem_grow()/mem_reserve(),
or something else to reserve the memory. To extend zMalloc. Otherwise you
can't reuse the memory which might already be in the mem object.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions
2021-04-12 23:34 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 21:36 ` Mergen Imeev via Tarantool-patches
2021-04-14 23:49 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 21:36 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers, diff and new patch below.
On Tue, Apr 13, 2021 at 01:34:22AM +0200, Vladislav Shpilevoy wrote:
> Nice fixes!
>
> See 2 comments below.
>
> > diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> > index 6c08e772d..e0903e3b0 100644
> > --- a/src/box/sql/vdbeapi.c
> > +++ b/src/box/sql/vdbeapi.c
> > @@ -125,6 +125,27 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
> > void (*xDel) (void *) /* Destructor function */
> > )
> > {
> > + if (xDel == SQL_STATIC) {
> > + if (n < 0)
> > + mem_set_str0_static(pCtx->pOut, (char *)z);
> > + else
> > + mem_set_str_static(pCtx->pOut, (char *)z, n);
> > + return;
> > + }
> > + if (xDel == SQL_DYNAMIC) {
> > + if (n < 0)
> > + mem_set_str0_allocated(pCtx->pOut, (char *)z);
> > + else
> > + mem_set_str_allocated(pCtx->pOut, (char *)z, n);
>
> 1. I don't understand. You check for xDel == SQL_DYNAMIC and yet
> you use 'allocated' suffix. Below you check for != TRANSIENT
> instead of == DYNAMIC, and use 'dynamic'. Why? All looks messed
> up. The same in the next hunk.
>
This is how it worked in sqlValueSetStr(). xDel == SQL_DYNAMIC (which is
actually == sqlMallocSize) was used to show that this value should be set as
allocated memory; xDel == SQL_TRANSIENT (== -1) was used to show that value
should be copied; xDel == SQL_STATIC (== 0) for MEM_Static and in case it some
other functions it was treated as free() for given memory, i.e. MEM_Dyn.
> > + return;
> > + }> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> > index ae55e4c29..171cb8946 100644
> > --- a/src/box/sql/vdbeaux.c
> > +++ b/src/box/sql/vdbeaux.c
> > @@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
> > mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
> > pMem++;
> >
> > - if (sqlVdbeMemClearAndResize(pMem, 256)) {
> > - assert(p->db->mallocFailed);
> > + char *buf = sqlDbMallocRaw(sql_get(), 256);
>
> 2. I think you need some kind of mem_set_strlen(), or mem_grow()/mem_reserve(),
> or something else to reserve the memory. To extend zMalloc. Otherwise you
> can't reuse the memory which might already be in the mem object.
Wouldn't mem_copy_*() be enough? In general, allocated by MEM memory should not
be accessed for changing (except for MEM_Agg).
Diff:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d8e640810..f029ee0e5 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -373,6 +373,42 @@ mem_set_str0_dynamic(struct Mem *mem, char *value);
void
mem_set_str0_allocated(struct Mem *mem, char *value);
+static inline void
+mem_set_strl_ephemeral(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_ephemeral(mem, value);
+ else
+ mem_set_str_ephemeral(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_static(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_static(mem, value);
+ else
+ mem_set_str_static(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_dynamic(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_dynamic(mem, value);
+ else
+ mem_set_str_dynamic(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_allocated(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_allocated(mem, value);
+ else
+ mem_set_str_allocated(mem, value, len_hint);
+}
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index e0903e3b0..484c66b29 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -125,27 +125,12 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
void (*xDel) (void *) /* Destructor function */
)
{
- if (xDel == SQL_STATIC) {
- if (n < 0)
- mem_set_str0_static(pCtx->pOut, (char *)z);
- else
- mem_set_str_static(pCtx->pOut, (char *)z, n);
- return;
- }
- if (xDel == SQL_DYNAMIC) {
- if (n < 0)
- mem_set_str0_allocated(pCtx->pOut, (char *)z);
- else
- mem_set_str_allocated(pCtx->pOut, (char *)z, n);
- return;
- }
- if (xDel != SQL_TRANSIENT) {
- if (n < 0)
- mem_set_str0_dynamic(pCtx->pOut, (char *)z);
- else
- mem_set_str_dynamic(pCtx->pOut, (char *)z, n);
- return;
- }
+ if (xDel == SQL_STATIC)
+ return mem_set_strl_static(pCtx->pOut, (char *)z, n);
+ if (xDel == SQL_DYNAMIC)
+ return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
+ if (xDel != SQL_TRANSIENT)
+ return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -783,24 +768,14 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
if (zData == NULL)
return 0;
pVar = &p->aVar[i - 1];
- if (xDel == SQL_STATIC) {
- if (nData < 0)
- mem_set_str0_static(pVar, (char *)zData);
- else
- mem_set_str_static(pVar, (char *)zData, nData);
- } else if (xDel == SQL_DYNAMIC) {
- if (nData < 0)
- mem_set_str0_allocated(pVar, (char *)zData);
- else
- mem_set_str_allocated(pVar, (char *)zData, nData);
- } else if (xDel != SQL_TRANSIENT) {
- if (nData < 0)
- mem_set_str0_dynamic(pVar, (char *)zData);
- else
- mem_set_str_dynamic(pVar, (char *)zData, nData);
- } else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
+ if (xDel == SQL_STATIC)
+ mem_set_strl_static(pVar, (char *)zData, nData);
+ else if (xDel == SQL_DYNAMIC)
+ mem_set_strl_allocated(pVar, (char *)zData, nData);
+ else if (xDel != SQL_TRANSIENT)
+ mem_set_strl_dynamic(pVar, (char *)zData, nData);
+ else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
return -1;
- }
return sql_bind_type(p, i, "text");
}
New patch:
commit 33b35f0b1d8578f99af39d509ac1bdf9abc458cc
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 14:28:29 2021 +0300
sql: introduce mem_set_str_*() functions
This patch introduces set of mem_set_str_*() functions. These functions
clears MEM and sets it to given string value. Degree of clearing and
type of allocation of the string is determined by the function used.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 683094599..5cf067453 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -169,6 +169,89 @@ mem_set_double(struct Mem *mem, double value)
mem->flags = MEM_Real;
}
+static inline void
+set_str_const(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+ assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
+ mem_clear(mem);
+ mem->z = value;
+ mem->n = len;
+ mem->flags = MEM_Str | alloc_type;
+ mem->field_type = FIELD_TYPE_STRING;
+}
+
+static inline void
+set_str_dynamic(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+ assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
+ assert(mem->szMalloc == 0 || value != mem->zMalloc);
+ assert(alloc_type == MEM_Dyn || alloc_type == 0);
+ mem_destroy(mem);
+ mem->z = value;
+ mem->n = len;
+ mem->flags = MEM_Str | alloc_type;
+ mem->field_type = FIELD_TYPE_STRING;
+ if (alloc_type == MEM_Dyn) {
+ mem->xDel = sql_free;
+ } else {
+ mem->xDel = NULL;
+ mem->zMalloc = mem->z;
+ mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
+ }
+}
+
+void
+mem_set_str_ephemeral(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_const(mem, value, len, MEM_Ephem);
+}
+
+void
+mem_set_str_static(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_const(mem, value, len, MEM_Static);
+}
+
+void
+mem_set_str_dynamic(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_dynamic(mem, value, len, MEM_Dyn);
+}
+
+void
+mem_set_str_allocated(struct Mem *mem, char *value, uint32_t len)
+{
+ set_str_dynamic(mem, value, len, 0);
+}
+
+void
+mem_set_str0_ephemeral(struct Mem *mem, char *value)
+{
+ set_str_const(mem, value, strlen(value), MEM_Ephem);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_static(struct Mem *mem, char *value)
+{
+ set_str_const(mem, value, strlen(value), MEM_Static);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_dynamic(struct Mem *mem, char *value)
+{
+ set_str_dynamic(mem, value, strlen(value), MEM_Dyn);
+ mem->flags |= MEM_Term;
+}
+
+void
+mem_set_str0_allocated(struct Mem *mem, char *value)
+{
+ set_str_dynamic(mem, value, strlen(value), 0);
+ mem->flags |= MEM_Term;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1908,20 +1991,6 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
pMem->z = 0;
}
-/*
- * Change the string value of an sql_value object
- */
-void
-sqlValueSetStr(sql_value * v, /* Value to be set */
- int n, /* Length of string z */
- const void *z, /* Text of the new string */
- void (*xDel) (void *) /* Destructor for the string */
- )
-{
- if (v)
- sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
-}
-
/*
* Free an sql_value object
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d573487b0..f029ee0e5 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -320,6 +320,95 @@ mem_set_bool(struct Mem *mem, bool value);
void
mem_set_double(struct Mem *mem, double value);
+/** Clear MEM and set it to STRING. The string belongs to another object. */
+void
+mem_set_str_ephemeral(struct Mem *mem, char *value, uint32_t len);
+
+/** Clear MEM and set it to STRING. The string is static. */
+void
+mem_set_str_static(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to STRING. The string was allocated by another object
+ * and passed to MEM. MEMs with this allocation type must free given memory
+ * whenever the MEM changes.
+ */
+void
+mem_set_str_dynamic(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to STRING. The string was allocated by another object
+ * and passed to MEM. MEMs with this allocation type only deallocate the string
+ * on destruction. Also, the memory may be reallocated if MEM is set to a
+ * different value of this allocation type.
+ */
+void
+mem_set_str_allocated(struct Mem *mem, char *value, uint32_t len);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string belongs to
+ * another object.
+ */
+void
+mem_set_str0_ephemeral(struct Mem *mem, char *value);
+
+/** Clear MEM and set it to NULL-terminated STRING. The string is static. */
+void
+mem_set_str0_static(struct Mem *mem, char *value);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string was allocated by
+ * another object and passed to MEM. MEMs with this allocation type must free
+ * given memory whenever the MEM changes.
+ */
+void
+mem_set_str0_dynamic(struct Mem *mem, char *value);
+
+/**
+ * Clear MEM and set it to NULL-terminated STRING. The string was allocated by
+ * another object and passed to MEM. MEMs with this allocation type only
+ * deallocate the string on destruction. Also, the memory may be reallocated if
+ * MEM is set to a different value of this allocation type.
+ */
+void
+mem_set_str0_allocated(struct Mem *mem, char *value);
+
+static inline void
+mem_set_strl_ephemeral(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_ephemeral(mem, value);
+ else
+ mem_set_str_ephemeral(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_static(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_static(mem, value);
+ else
+ mem_set_str_static(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_dynamic(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_dynamic(mem, value);
+ else
+ mem_set_str_dynamic(mem, value, len_hint);
+}
+
+static inline void
+mem_set_strl_allocated(struct Mem *mem, char *value, int len_hint)
+{
+ if (len_hint < 0)
+ mem_set_str0_allocated(mem, value);
+ else
+ mem_set_str_allocated(mem, value, len_hint);
+}
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -578,8 +667,6 @@ int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void
sqlVdbeMemSetZeroBlob(struct Mem *, int);
-void sqlValueSetStr(struct Mem *, int, const void *,
- void (*)(void *));
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 259ba3833..7a026d21b 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -641,10 +641,6 @@ sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value);
int
sql_bind_null(sql_stmt *, int);
-int
-sql_bind_text(sql_stmt *, int, const char *, int,
- void (*)(void *));
-
int
sql_bind_text64(sql_stmt *, int, const char *,
sql_uint64, void (*)(void *));
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 84704b9d0..d7f01000a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -841,9 +841,8 @@ case OP_String8: { /* same as TK_STRING, out2 */
case OP_String: { /* out2 */
assert(pOp->p4.z!=0);
pOut = vdbe_prepare_null_out(p, pOp->p2);
- pOut->flags = MEM_Str|MEM_Static|MEM_Term;
- pOut->z = pOp->p4.z;
- pOut->n = pOp->p1;
+ assert(strlen(pOp->p4.z) == (size_t)pOp->p1);
+ mem_set_str0_static(pOut, pOp->p4.z);
UPDATE_MAX_BLOBSIZE(pOut);
break;
}
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 6c08e772d..484c66b29 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -125,6 +125,12 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
void (*xDel) (void *) /* Destructor function */
)
{
+ if (xDel == SQL_STATIC)
+ return mem_set_strl_static(pCtx->pOut, (char *)z, n);
+ if (xDel == SQL_DYNAMIC)
+ return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
+ if (xDel != SQL_TRANSIENT)
+ return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -762,7 +768,13 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
if (zData == NULL)
return 0;
pVar = &p->aVar[i - 1];
- if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
+ if (xDel == SQL_STATIC)
+ mem_set_strl_static(pVar, (char *)zData, nData);
+ else if (xDel == SQL_DYNAMIC)
+ mem_set_strl_allocated(pVar, (char *)zData, nData);
+ else if (xDel != SQL_TRANSIENT)
+ mem_set_strl_dynamic(pVar, (char *)zData, nData);
+ else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
return -1;
return sql_bind_type(p, i, "text");
}
@@ -876,14 +888,6 @@ sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr)
return rc;
}
-int
-sql_bind_text(sql_stmt * pStmt,
- int i, const char *zData, int nData, void (*xDel) (void *)
- )
-{
- return bindText(pStmt, i, zData, nData, xDel);
-}
-
int
sql_bind_text64(sql_stmt * pStmt,
int i,
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index c8ec7f4fa..dad9cab03 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1295,10 +1295,8 @@ sqlVdbeList(Vdbe * p)
pMem++;
- pMem->flags = MEM_Static | MEM_Str | MEM_Term;
- pMem->z = (char *)sqlOpcodeName(pOp->opcode); /* Opcode */
- assert(pMem->z != 0);
- pMem->n = sqlStrlen30(pMem->z);
+ char *value = (char *)sqlOpcodeName(pOp->opcode);
+ mem_set_str0_static(pMem, value);
pMem++;
/* When an OP_Program opcode is encounter (the only opcode that has
@@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
pMem++;
- if (sqlVdbeMemClearAndResize(pMem, 256)) {
- assert(p->db->mallocFailed);
+ char *buf = sqlDbMallocRaw(sql_get(), 256);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- zP4 = displayP4(pOp, pMem->z, pMem->szMalloc);
-
- if (zP4 != pMem->z) {
- pMem->n = 0;
- sqlVdbeMemSetStr(pMem, zP4, -1, 1, 0);
+ zP4 = displayP4(pOp, buf, sqlDbMallocSize(sql_get(), buf));
+ if (zP4 != buf) {
+ sqlDbFree(sql_get(), buf);
+ mem_set_str0_ephemeral(pMem, zP4);
} else {
- assert(pMem->z != 0);
- pMem->n = sqlStrlen30(pMem->z);
+ mem_set_str0_allocated(pMem, zP4);
}
pMem++;
if (p->explain == 1) {
- if (sqlVdbeMemClearAndResize(pMem, 4)) {
- assert(p->db->mallocFailed);
+ buf = sqlDbMallocRaw(sql_get(), 4);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- pMem->n = 2;
- sql_snprintf(3, pMem->z, "%.2x", pOp->p5); /* P5 */
+ sql_snprintf(3, buf, "%.2x", pOp->p5);
+ mem_set_str0_allocated(pMem, buf);
pMem++;
#ifdef SQL_ENABLE_EXPLAIN_COMMENTS
- if (sqlVdbeMemClearAndResize(pMem, 500)) {
- assert(p->db->mallocFailed);
+ buf = sqlDbMallocRaw(sql_get(), 500);
+ if (buf == NULL)
return -1;
- }
- pMem->flags = MEM_Str | MEM_Term;
- pMem->n = displayComment(pOp, zP4, pMem->z, 500);
+ displayComment(pOp, zP4, buf, 500);
+ mem_set_str0_allocated(pMem, buf);
#else
- pMem->flags = MEM_Null; /* Comment */
+ mem_set_null(pMem);
#endif
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions
2021-04-13 21:36 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 23:49 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-15 1:25 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 23:49 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Thanks for working on this!
>>> index ae55e4c29..171cb8946 100644
>>> --- a/src/box/sql/vdbeaux.c
>>> +++ b/src/box/sql/vdbeaux.c
>>> @@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
>>> mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
>>> pMem++;
>>>
>>> - if (sqlVdbeMemClearAndResize(pMem, 256)) {
>>> - assert(p->db->mallocFailed);
>>> + char *buf = sqlDbMallocRaw(sql_get(), 256);
>>
>> 2. I think you need some kind of mem_set_strlen(), or mem_grow()/mem_reserve(),
>> or something else to reserve the memory. To extend zMalloc. Otherwise you
>> can't reuse the memory which might already be in the mem object.
> Wouldn't mem_copy_*() be enough? In general, allocated by MEM memory should not
> be accessed for changing (except for MEM_Agg).
Both your current solution and mem_copy suffer from unnecessary copying.
In the old code the string was created in-place. This is what I am talking
about.
But nevermind, it is not a perf-critical place as I see. Can be tweaked
later.
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index 6c08e772d..484c66b29 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -125,6 +125,12 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
> void (*xDel) (void *) /* Destructor function */
> )
> {
> + if (xDel == SQL_STATIC)
> + return mem_set_strl_static(pCtx->pOut, (char *)z, n);
> + if (xDel == SQL_DYNAMIC)
> + return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
> + if (xDel != SQL_TRANSIENT)
> + return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
I just can't help myself. It makes my eyes bleed - for dynamic we use
allocated, and for non-transient we use dynamic! I see why, but I would
want us to hide it as deep inside of mem.c as possible, so as we at
least could change the names in a not very intrusive manner in the
future. Or delete it in future entirely.
Please, move this hunk and the one below to a separate functions
like mem_set_strl(Mem, str, len_hint, destructor). Then it would be in
one terrible place instead of 2 terrible places.
The same for the bins.
> if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
> pCtx->is_aborted = true;
> }
> @@ -762,7 +768,13 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
> if (zData == NULL)
> return 0;
> pVar = &p->aVar[i - 1];
> - if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
> + if (xDel == SQL_STATIC)
> + mem_set_strl_static(pVar, (char *)zData, nData);
> + else if (xDel == SQL_DYNAMIC)
> + mem_set_strl_allocated(pVar, (char *)zData, nData);
> + else if (xDel != SQL_TRANSIENT)
> + mem_set_strl_dynamic(pVar, (char *)zData, nData);
> + else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
> return -1;
> return sql_bind_type(p, i, "text");
> }
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions
2021-04-14 23:49 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-15 1:25 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-15 1:25 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review. My answer and two diffs below. First diff for this
patch, second - for similar patch with binary.
On Thu, Apr 15, 2021 at 01:49:46AM +0200, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> >>> index ae55e4c29..171cb8946 100644
> >>> --- a/src/box/sql/vdbeaux.c
> >>> +++ b/src/box/sql/vdbeaux.c
> >>> @@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
> >>> mem_set_int(pMem, pOp->p3, pOp->p3 < 0);
> >>> pMem++;
> >>>
> >>> - if (sqlVdbeMemClearAndResize(pMem, 256)) {
> >>> - assert(p->db->mallocFailed);
> >>> + char *buf = sqlDbMallocRaw(sql_get(), 256);
> >>
> >> 2. I think you need some kind of mem_set_strlen(), or mem_grow()/mem_reserve(),
> >> or something else to reserve the memory. To extend zMalloc. Otherwise you
> >> can't reuse the memory which might already be in the mem object.
> > Wouldn't mem_copy_*() be enough? In general, allocated by MEM memory should not
> > be accessed for changing (except for MEM_Agg).
>
> Both your current solution and mem_copy suffer from unnecessary copying.
> In the old code the string was created in-place. This is what I am talking
> about.
>
> But nevermind, it is not a perf-critical place as I see. Can be tweaked
> later.
>
> > diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> > index 6c08e772d..484c66b29 100644
> > --- a/src/box/sql/vdbeapi.c
> > +++ b/src/box/sql/vdbeapi.c
> > @@ -125,6 +125,12 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
> > void (*xDel) (void *) /* Destructor function */
> > )
> > {
> > + if (xDel == SQL_STATIC)
> > + return mem_set_strl_static(pCtx->pOut, (char *)z, n);
> > + if (xDel == SQL_DYNAMIC)
> > + return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
> > + if (xDel != SQL_TRANSIENT)
> > + return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
>
> I just can't help myself. It makes my eyes bleed - for dynamic we use
> allocated, and for non-transient we use dynamic! I see why, but I would
> want us to hide it as deep inside of mem.c as possible, so as we at
> least could change the names in a not very intrusive manner in the
> future. Or delete it in future entirely.
>
> Please, move this hunk and the one below to a separate functions
> like mem_set_strl(Mem, str, len_hint, destructor). Then it would be in
> one terrible place instead of 2 terrible places.
>
> The same for the bins.
>
Done.
> > if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
> > pCtx->is_aborted = true;
> > }
> > @@ -762,7 +768,13 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
> > if (zData == NULL)
> > return 0;
> > pVar = &p->aVar[i - 1];
> > - if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
> > + if (xDel == SQL_STATIC)
> > + mem_set_strl_static(pVar, (char *)zData, nData);
> > + else if (xDel == SQL_DYNAMIC)
> > + mem_set_strl_allocated(pVar, (char *)zData, nData);
> > + else if (xDel != SQL_TRANSIENT)
> > + mem_set_strl_dynamic(pVar, (char *)zData, nData);
> > + else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
> > return -1;
> > return sql_bind_type(p, i, "text");
> > }
Diff for str:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f029ee0e5..258bb11e6 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -409,6 +409,18 @@ mem_set_strl_allocated(struct Mem *mem, char *value, int len_hint)
mem_set_str_allocated(mem, value, len_hint);
}
+static inline void
+mem_set_strl(struct Mem *mem, char *value, int len_hint,
+ void (*custom_free)(void *))
+{
+ if (custom_free == SQL_STATIC)
+ return mem_set_strl_static(mem, value, len_hint);
+ if (custom_free == SQL_DYNAMIC)
+ return mem_set_strl_allocated(mem, value, len_hint);
+ if (custom_free != SQL_TRANSIENT)
+ return mem_set_strl_dynamic(mem, value, len_hint);
+}
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 3cd60864f..ae65a53d8 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -125,12 +125,8 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
void (*xDel) (void *) /* Destructor function */
)
{
- if (xDel == SQL_STATIC)
- return mem_set_strl_static(pCtx->pOut, (char *)z, n);
- if (xDel == SQL_DYNAMIC)
- return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
if (xDel != SQL_TRANSIENT)
- return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
+ return mem_set_strl(pCtx->pOut, (char *)z, n, xDel);
if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -768,12 +764,8 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
if (zData == NULL)
return 0;
pVar = &p->aVar[i - 1];
- if (xDel == SQL_STATIC)
- mem_set_strl_static(pVar, (char *)zData, nData);
- else if (xDel == SQL_DYNAMIC)
- mem_set_strl_allocated(pVar, (char *)zData, nData);
- else if (xDel != SQL_TRANSIENT)
- mem_set_strl_dynamic(pVar, (char *)zData, nData);
+ if (xDel != SQL_TRANSIENT)
+ mem_set_strl(pVar, (char *)zData, nData, xDel);
else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
return -1;
return sql_bind_type(p, i, "text");
Diff for bin:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f728ac58a..5b9ac6a96 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -468,6 +468,18 @@ mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size);
void
mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size);
+static inline void
+mem_set_binl(struct Mem *mem, char *value, uint32_t size,
+ void (*custom_free)(void *))
+{
+ if (custom_free == SQL_STATIC)
+ return mem_set_bin_static(mem, value, size);
+ if (custom_free == SQL_DYNAMIC)
+ return mem_set_bin_allocated(mem, value, size);
+ if (custom_free != SQL_TRANSIENT)
+ return mem_set_bin_dynamic(mem, value, size);
+}
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 1ba2f4260..66908e2ec 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -158,12 +158,8 @@ sql_result_blob(sql_context * pCtx,
)
{
assert(n >= 0);
- if (xDel == SQL_STATIC)
- mem_set_bin_static(pCtx->pOut, (char *)z, n);
- else if (xDel == SQL_DYNAMIC)
- mem_set_bin_allocated(pCtx->pOut, (char *)z, n);
- else if (xDel != SQL_TRANSIENT)
- mem_set_bin_dynamic(pCtx->pOut, (char *)z, n);
+ if (xDel != SQL_TRANSIENT)
+ mem_set_binl(pCtx->pOut, (char *)z, n, xDel);
else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -793,12 +789,8 @@ sql_bind_blob(sql_stmt * pStmt,
if (zData == NULL)
return 0;
struct Mem *var = &p->aVar[i - 1];
- if (xDel == SQL_STATIC)
- mem_set_bin_static(var, (char *)zData, nData);
- else if (xDel == SQL_DYNAMIC)
- mem_set_bin_allocated(var, (char *)zData, nData);
- else if (xDel != SQL_TRANSIENT)
- mem_set_bin_dynamic(var, (char *)zData, nData);
+ if (xDel != SQL_TRANSIENT)
+ mem_set_binl(var, (char *)zData, nData, xDel);
else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
return -1;
return sql_bind_type(p, i, "varbinary");
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (26 preceding siblings ...)
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-09 19:45 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:35 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 29/52] sql: introduce mem_set_bin_*() functions Mergen Imeev via Tarantool-patches
` (13 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:45 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_copy_str() and mem_copy_str0() functions.
These functions copy given string to newly allocated by MEM memory.
String given to mem_copy_str0() should be NULL-terminated. This is not
necessary for mem_copy_str().
Part of #5818
---
src/box/sql/mem.c | 40 +++++++++++++++++++++++++++++++++-------
src/box/sql/mem.h | 11 +++++++++++
src/box/sql/vdbeapi.c | 17 ++++++++++++++---
3 files changed, 58 insertions(+), 10 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 87e1bcfd1..045c44e8f 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -394,6 +394,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
mem->flags |= MEM_Term;
}
+int
+mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
+ mem_clear(mem);
+ bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == value;
+ if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
+ return -1;
+ if (!is_own_value)
+ memcpy(mem->z, value, len);
+ mem->n = len;
+ mem->flags = MEM_Str;
+ mem->field_type = FIELD_TYPE_STRING;
+ return 0;
+}
+
+int
+mem_copy_str0(struct Mem *mem, const char *value)
+{
+ uint32_t len = strlen(value);
+ if (mem_copy_str(mem, value, len + 1) != 0)
+ return -1;
+ mem->n = len;
+ mem->flags |= MEM_Term;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1618,8 +1646,8 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
assert(MEM_Str == (MEM_Blob >> 3));
if ((pMem->flags & MEM_Bool) != 0) {
const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
- sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
- SQL_TRANSIENT);
+ if (mem_copy_str0(pMem, str_bool) != 0)
+ return -1;
return 0;
}
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
@@ -3080,9 +3108,8 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
val[i].u.i = field.ival;
break;
case MP_STR:
- if (sqlVdbeMemSetStr(&val[i], field.sval.data,
- field.sval.len, 1,
- SQL_TRANSIENT) != 0)
+ if (mem_copy_str(&val[i], field.sval.data,
+ field.sval.len) != 0)
goto error;
break;
case MP_NIL:
@@ -3157,8 +3184,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
break;
case MP_STR:
str = mp_decode_str(&data, &len);
- if (sqlVdbeMemSetStr(&val[i], str, len,
- 1, SQL_TRANSIENT) != 0)
+ if (mem_copy_str(&val[i], str, len) != 0)
goto error;
break;
case MP_NIL:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index ecaa1edb8..d03651192 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -235,6 +235,17 @@ mem_set_str0_dynamic(struct Mem *mem, char *value);
void
mem_set_str0_allocated(struct Mem *mem, char *value);
+/** Copy string to a newly allocated memory. The MEM type becomes STRING. */
+int
+mem_copy_str(struct Mem *mem, const char *value, uint32_t len);
+
+/**
+ * Copy NULL-terminated string to a newly allocated memory. The MEM type becomes
+ * STRING.
+ */
+int
+mem_copy_str0(struct Mem *mem, const char *value);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index e0903e3b0..ee095f36e 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -146,7 +146,12 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
mem_set_str_dynamic(pCtx->pOut, (char *)z, n);
return;
}
- if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
+ if (n < 0) {
+ if (mem_copy_str0(pCtx->pOut, z) != 0)
+ pCtx->is_aborted = true;
+ return;
+ }
+ if (mem_copy_str(pCtx->pOut, z, n) != 0)
pCtx->is_aborted = true;
}
@@ -798,8 +803,14 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
mem_set_str0_dynamic(pVar, (char *)zData);
else
mem_set_str_dynamic(pVar, (char *)zData, nData);
- } else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
- return -1;
+ } else {
+ if (nData < 0) {
+ if (mem_copy_str0(pVar, zData) != 0)
+ return -1;
+ } else {
+ if (mem_copy_str(pVar, zData, nData) != 0)
+ return -1;
+ }
}
return sql_bind_type(p, i, "text");
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0()
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:35 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:00 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:35 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for the fixes!
See 2 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 87e1bcfd1..045c44e8f 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -394,6 +394,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
> mem->flags |= MEM_Term;
> }
>
> +int
> +mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
> +{
> + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> + mem_clear(mem);
1. Why don't you call clear always? Anyway 'dynamic' memory can't
be reused. It is freed in sqlVdbeMemGrow() AFAIS.
> + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> + mem->z == value;
> + if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
> + return -1;
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index e0903e3b0..ee095f36e 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -798,8 +803,14 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
> mem_set_str0_dynamic(pVar, (char *)zData);
> else
> mem_set_str_dynamic(pVar, (char *)zData, nData);
> - } else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
> - return -1;
> + } else {
> + if (nData < 0) {
> + if (mem_copy_str0(pVar, zData) != 0)
> + return -1;
> + } else {
> + if (mem_copy_str(pVar, zData, nData) != 0)
> + return -1;
> + }
2. For mem_set_str* and mem_copy_str* you could probably have 3
versions: mem_set_strn(const char *, uint32 len),
mem_set_str0(const char *), and mem_set_str(const char *, uint32 len_hint).
In the last version len_hint might be -1 and then strlen() is called.
It would keep the places like that as simple as they were.
> }
> return sql_bind_type(p, i, "text");
> }
>
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0()
2021-04-12 23:35 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:00 ` Mergen Imeev via Tarantool-patches
2021-04-14 23:54 ` Vladislav Shpilevoy via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:00 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below. There won't be diff
since I added mem_copy_strl() during merge caused by previous patch.
On Tue, Apr 13, 2021 at 01:35:19AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
>
> See 2 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 87e1bcfd1..045c44e8f 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -394,6 +394,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
> > mem->flags |= MEM_Term;
> > }
> >
> > +int
> > +mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
> > +{
> > + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> > + mem_clear(mem);
>
> 1. Why don't you call clear always? Anyway 'dynamic' memory can't
> be reused. It is freed in sqlVdbeMemGrow() AFAIS.
>
It is possible that value given to mem_copy_*() is own value of the MEM. If it
is of MEM_Dyn type it will be lost in case of clear() being called. Function
sqlVdbeMemGrow() will free this memory after value will be copied.
> > + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> > + mem->z == value;
> > + if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
> > + return -1;
> > diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> > index e0903e3b0..ee095f36e 100644
> > --- a/src/box/sql/vdbeapi.c
> > +++ b/src/box/sql/vdbeapi.c
> > @@ -798,8 +803,14 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
> > mem_set_str0_dynamic(pVar, (char *)zData);
> > else
> > mem_set_str_dynamic(pVar, (char *)zData, nData);
> > - } else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
> > - return -1;
> > + } else {
> > + if (nData < 0) {
> > + if (mem_copy_str0(pVar, zData) != 0)
> > + return -1;
> > + } else {
> > + if (mem_copy_str(pVar, zData, nData) != 0)
> > + return -1;
> > + }
>
> 2. For mem_set_str* and mem_copy_str* you could probably have 3
> versions: mem_set_strn(const char *, uint32 len),
> mem_set_str0(const char *), and mem_set_str(const char *, uint32 len_hint).
> In the last version len_hint might be -1 and then strlen() is called.
> It would keep the places like that as simple as they were.
>
Done, in previous patch and here. I named functions with len_hint as
mem_set_strl_*() and mem_copy_strl().
> > }
> > return sql_bind_type(p, i, "text");
> > }
> >
New patch:
commit 3e3f08f8c3bc2fb45a62c992cdf4f9ce20890641
Author: Mergen Imeev <imeevma@gmail.com>
Date: Mon Mar 15 16:59:27 2021 +0300
sql: introduce mem_copy_str() and mem_copy_str0()
This patch introduces mem_copy_str() and mem_copy_str0() functions.
These functions copy given string to newly allocated by MEM memory.
String given to mem_copy_str0() should be NULL-terminated. This is not
necessary for mem_copy_str().
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5cf067453..ded20315b 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -252,6 +252,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
mem->flags |= MEM_Term;
}
+int
+mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
+ mem_clear(mem);
+ bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == value;
+ if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
+ return -1;
+ if (!is_own_value)
+ memcpy(mem->z, value, len);
+ mem->n = len;
+ mem->flags = MEM_Str;
+ mem->field_type = FIELD_TYPE_STRING;
+ return 0;
+}
+
+int
+mem_copy_str0(struct Mem *mem, const char *value)
+{
+ uint32_t len = strlen(value);
+ if (mem_copy_str(mem, value, len + 1) != 0)
+ return -1;
+ mem->n = len;
+ mem->flags |= MEM_Term;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1441,8 +1469,8 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
assert(MEM_Str == (MEM_Blob >> 3));
if ((pMem->flags & MEM_Bool) != 0) {
const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
- sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
- SQL_TRANSIENT);
+ if (mem_copy_str0(pMem, str_bool) != 0)
+ return -1;
return 0;
}
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
@@ -2903,9 +2931,8 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
val[i].u.i = field.ival;
break;
case MP_STR:
- if (sqlVdbeMemSetStr(&val[i], field.sval.data,
- field.sval.len, 1,
- SQL_TRANSIENT) != 0)
+ if (mem_copy_str(&val[i], field.sval.data,
+ field.sval.len) != 0)
goto error;
break;
case MP_NIL:
@@ -2980,8 +3007,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
break;
case MP_STR:
str = mp_decode_str(&data, &len);
- if (sqlVdbeMemSetStr(&val[i], str, len,
- 1, SQL_TRANSIENT) != 0)
+ if (mem_copy_str(&val[i], str, len) != 0)
goto error;
break;
case MP_NIL:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f029ee0e5..aaecd064a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -409,6 +409,25 @@ mem_set_strl_allocated(struct Mem *mem, char *value, int len_hint)
mem_set_str_allocated(mem, value, len_hint);
}
+/** Copy string to a newly allocated memory. The MEM type becomes STRING. */
+int
+mem_copy_str(struct Mem *mem, const char *value, uint32_t len);
+
+/**
+ * Copy NULL-terminated string to a newly allocated memory. The MEM type becomes
+ * STRING.
+ */
+int
+mem_copy_str0(struct Mem *mem, const char *value);
+
+static inline int
+mem_copy_strl(struct Mem *mem, const char *value, int len_hint)
+{
+ if (len_hint < 0)
+ return mem_copy_str0(mem, value);
+ return mem_copy_str(mem, value, len_hint);
+}
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 484c66b29..69dd1fb0d 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -131,7 +131,7 @@ setResultStrOrError(sql_context * pCtx, /* Function context */
return mem_set_strl_allocated(pCtx->pOut, (char *)z, n);
if (xDel != SQL_TRANSIENT)
return mem_set_strl_dynamic(pCtx->pOut, (char *)z, n);
- if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
+ if (mem_copy_strl(pCtx->pOut, z, n) != 0)
pCtx->is_aborted = true;
}
@@ -774,7 +774,7 @@ bindText(sql_stmt * pStmt, /* The statement to bind against */
mem_set_strl_allocated(pVar, (char *)zData, nData);
else if (xDel != SQL_TRANSIENT)
mem_set_strl_dynamic(pVar, (char *)zData, nData);
- else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
+ else if (mem_copy_strl(pVar, zData, nData) != 0)
return -1;
return sql_bind_type(p, i, "text");
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0()
2021-04-13 22:00 ` Mergen Imeev via Tarantool-patches
@ 2021-04-14 23:54 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-15 0:30 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-14 23:54 UTC (permalink / raw)
To: Mergen Imeev; +Cc: tarantool-patches
Good job on the patch!
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 5cf067453..ded20315b 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -252,6 +252,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
> mem->flags |= MEM_Term;
> }
>
> +int
> +mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
> +{
> + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> + mem_clear(mem);
> + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> + mem->z == value;
> + if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
> + return -1;
I would propose this change:
====================
@@ -256,14 +256,14 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
int
mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
{
- if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
- mem_clear(mem);
- bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
- mem->z == value;
- if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
+ if ((mem->flags & (MEM_Str | MEM_Blob)) != 0 && mem->z == value) {
+ /* Own value, but might be ephemeral. Make it own if so. */
+ return sqlVdbeMemGrow(mem, len, 1);
+ }
+ mem_clear(mem);
+ if (sqlVdbeMemGrow(mem, len, 0) != 0)
return -1;
- if (!is_own_value)
- memcpy(mem->z, value, len);
+ memcpy(mem->z, value, len);
mem->n = len;
mem->flags = MEM_Str;
mem->field_type = FIELD_TYPE_STRING;
====================
The same for the bins.
> + if (!is_own_value)
> + memcpy(mem->z, value, len);
> + mem->n = len;
> + mem->flags = MEM_Str;
> + mem->field_type = FIELD_TYPE_STRING;
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0()
2021-04-14 23:54 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-15 0:30 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-15 0:30 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you! I applied your patch. However, I added a bit of changes since it was
possible that after mem_copy_str() type of copied value will be MEM_Blob.
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index d98b077ad..7676a29e3 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -258,7 +258,11 @@ mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
{
if ((mem->flags & (MEM_Str | MEM_Blob)) != 0 && mem->z == value) {
/* Own value, but might be ephemeral. Make it own if so. */
- return sqlVdbeMemGrow(mem, len, 1);
+ if (sqlVdbeMemGrow(mem, len, 1) != 0)
+ return -1;
+ mem->flags = MEM_Str;
+ mem->field_type = FIELD_TYPE_STRING;
+ return 0;
}
mem_clear(mem);
if (sqlVdbeMemGrow(mem, len, 0) != 0)
On Thu, Apr 15, 2021 at 01:54:31AM +0200, Vladislav Shpilevoy wrote:
> Good job on the patch!
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 5cf067453..ded20315b 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -252,6 +252,34 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
> > mem->flags |= MEM_Term;
> > }
> >
> > +int
> > +mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
> > +{
> > + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> > + mem_clear(mem);
> > + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> > + mem->z == value;
> > + if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
> > + return -1;
>
> I would propose this change:
>
> ====================
> @@ -256,14 +256,14 @@ mem_set_str0_allocated(struct Mem *mem, char *value)
> int
> mem_copy_str(struct Mem *mem, const char *value, uint32_t len)
> {
> - if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> - mem_clear(mem);
> - bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> - mem->z == value;
> - if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
> + if ((mem->flags & (MEM_Str | MEM_Blob)) != 0 && mem->z == value) {
> + /* Own value, but might be ephemeral. Make it own if so. */
> + return sqlVdbeMemGrow(mem, len, 1);
> + }
> + mem_clear(mem);
> + if (sqlVdbeMemGrow(mem, len, 0) != 0)
> return -1;
> - if (!is_own_value)
> - memcpy(mem->z, value, len);
> + memcpy(mem->z, value, len);
> mem->n = len;
> mem->flags = MEM_Str;
> mem->field_type = FIELD_TYPE_STRING;
> ====================
>
> The same for the bins.
>
> > + if (!is_own_value)
> > + memcpy(mem->z, value, len);
> > + mem->n = len;
> > + mem->flags = MEM_Str;
> > + mem->field_type = FIELD_TYPE_STRING;
> > + return 0;
> > +}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 29/52] sql: introduce mem_set_bin_*() functions
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (27 preceding siblings ...)
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0() Mergen Imeev via Tarantool-patches
@ 2021-04-09 19:45 ` Mergen Imeev via Tarantool-patches
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin() Mergen Imeev via Tarantool-patches
` (12 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:45 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:05, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 59a378e1b..5ee49cdca 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -410,6 +410,61 @@ mem_copy_string0(struct Mem *mem, const char *value)
>> return 0;
>> }
>>
>> +static inline void
>> +mem_set_const_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
>> +{
>> + assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
>> + mem_clear(mem);
>> + mem->z = value;
>> + mem->n = size;
>> + mem->flags = MEM_Blob | alloc_type;
>> + mem->field_type = FIELD_TYPE_VARBINARY;
>> +}
>> +
>> +static inline void
>> +mem_set_dyn_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
>> +{
>> + assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
>> + assert(mem->szMalloc == 0 || value != mem->zMalloc);
>> + assert(alloc_type == MEM_Dyn || alloc_type == 0);
>> + mem_destroy(mem);
>
> 1. Why is it destroy here and clear above?
>
There is a rule which I cannot understand - it says that MEM should contain
either given allocated memory + sql_free() (MEM_Dyn) or "self-allocated" memory
(which may be not self-allocated actually, but still). I see no reason for this,
but I decided to leave it as it is for now. So, in case we set MEM_Dyn or
allocated type (when MEM_Static, MEM_Ephem and MEM_Dyn unset) MEM have to free
all memory. This is why we use destroy here. In case of MEM_Ephem or MEM_Static
we only should free memory if it is of MEM_Dyn type. So that is why we use
clear() there.
>> + mem->z = value;
>> + mem->n = size;
>> + mem->flags = MEM_Blob | alloc_type;
>> + mem->field_type = FIELD_TYPE_VARBINARY;
>> + if (alloc_type == MEM_Dyn) {
>> + mem->xDel = sql_free;
>> + } else {
>> + mem->xDel = NULL;
>> + mem->zMalloc = mem->z;
>> + mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
>> + }
>> +}
>> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
>> index 5b5e5b0c8..0e51e4809 100644
>> --- a/src/box/sql/vdbeapi.c
>> +++ b/src/box/sql/vdbeapi.c
>> @@ -183,7 +182,13 @@ sql_result_blob(sql_context * pCtx,
>> )
>> {
>> assert(n >= 0);
>> - if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
>> + if (xDel == SQL_STATIC)
>> + mem_set_static_binary(pCtx->pOut, (char *)z, n);
>> + else if (xDel == SQL_DYNAMIC)
>> + mem_set_allocated_binary(pCtx->pOut, (char *)z, n);
>> + else if (xDel != SQL_TRANSIENT)
>> + mem_set_dynamic_binary(pCtx->pOut, (char *)z, n);
>> + else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
>> pCtx->is_aborted = true;
>
> 2. It seems to me you need to add a generic mem_set_binary which would
> take the xdel argument. Repeating this tree of ifs in each usage place
> is not any better. The same for the string API.
I think the problem here is mostly with the way built-in functions returns their
result. Don't you think that it is better to change sql_result_*() instead of
adding of a generic mem_set_binary() function?
About bindText() - I think it is better to move sql_bind_column() to
sql/vdbeapi.c and inline such functions as bindText(), sql_bind_double(), etc.
This is the only place they are used, after all.
Still, I didn't do any of these changes. I plan to rework built-in functions
later, along with removing them from _func. As for sql_bind_column() - I just
not sure that this should be done in this patch-set. No problem to add one more
patch to the set, though. What do you think?
New patch:
commit 67af1300324e2548ffa9234e762cdc473ba88e19
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 10:17:33 2021 +0300
sql: introduce mem_set_bin_*() functions
This patch introduces set of mem_set_bin_*() functions. These functions
clears MEM and sets it to given binary value. Degree of clearing and
type of allocation of the binary value is determined by the function
used.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 045c44e8f..2622cdd82 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -422,6 +422,61 @@ mem_copy_str0(struct Mem *mem, const char *value)
return 0;
}
+static inline void
+set_bin_const(struct Mem *mem, char *value, uint32_t size, int alloc_type)
+{
+ assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
+ mem_clear(mem);
+ mem->z = value;
+ mem->n = size;
+ mem->flags = MEM_Blob | alloc_type;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+}
+
+static inline void
+set_bin_dynamic(struct Mem *mem, char *value, uint32_t size, int alloc_type)
+{
+ assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
+ assert(mem->szMalloc == 0 || value != mem->zMalloc);
+ assert(alloc_type == MEM_Dyn || alloc_type == 0);
+ mem_destroy(mem);
+ mem->z = value;
+ mem->n = size;
+ mem->flags = MEM_Blob | alloc_type;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+ if (alloc_type == MEM_Dyn) {
+ mem->xDel = sql_free;
+ } else {
+ mem->xDel = NULL;
+ mem->zMalloc = mem->z;
+ mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
+ }
+}
+
+void
+mem_set_bin_ephemeral(struct Mem *mem, char *value, uint32_t size)
+{
+ set_bin_const(mem, value, size, MEM_Ephem);
+}
+
+void
+mem_set_bin_static(struct Mem *mem, char *value, uint32_t size)
+{
+ set_bin_const(mem, value, size, MEM_Static);
+}
+
+void
+mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size)
+{
+ set_bin_dynamic(mem, value, size, MEM_Dyn);
+}
+
+void
+mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size)
+{
+ set_bin_dynamic(mem, value, size, 0);
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2547,19 +2602,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type)
return field_mp_plain_type_is_compatible(type, mp_type, true);
}
-/* Allocate memory for internal VDBE structure on region. */
-int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
-{
- vdbe_mem->n = size;
- vdbe_mem->z = region_alloc(&fiber()->gc, size);
- if (vdbe_mem->z == NULL)
- return -1;
- vdbe_mem->flags = MEM_Ephem | MEM_Blob;
- assert(sqlVdbeCheckMemInvariants(vdbe_mem));
- return 0;
-}
-
int
sql_vdbemem_finalize(struct Mem *mem, struct func *func)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d03651192..532860ff7 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -246,6 +246,34 @@ mem_copy_str(struct Mem *mem, const char *value, uint32_t len);
int
mem_copy_str0(struct Mem *mem, const char *value);
+/**
+ * Clear MEM and set it to VARBINARY. The binary value belongs to another
+ * object.
+ */
+void
+mem_set_bin_ephemeral(struct Mem *mem, char *value, uint32_t size);
+
+/** Clear MEM and set it to VARBINARY. The binary value is static. */
+void
+mem_set_bin_static(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to VARBINARY. The binary value was allocated by another
+ * object and passed to MEM. MEMs with this allocation type must free given
+ * memory whenever the MEM changes.
+ */
+void
+mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to VARBINARY. The binary value was allocated by another
+ * object and passed to MEM. MEMs with this allocation type only deallocate the
+ * string on destruction. Also, the memory may be reallocated if MEM is set to a
+ * different value of this allocation type.
+ */
+void
+mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -606,9 +634,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
/** MEM manipulate functions. */
-int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-
/**
* Memory cell mem contains the context of an aggregate function.
* This routine calls the finalize method for that function. The
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 943179c55..85fb7a26d 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -886,8 +886,15 @@ case OP_Null: { /* out2 */
case OP_Blob: { /* out2 */
assert(pOp->p1 <= SQL_MAX_LENGTH);
pOut = vdbe_prepare_null_out(p, pOp->p2);
- sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
- if (pOp->p3!=0) {
+ if (pOp->p3 == 0) {
+ /*
+ * TODO: It is possible that vabinary should be stored as
+ * ephemeral or static depending on value. There is no way to
+ * determine right now, so it is stored as static.
+ */
+ mem_set_bin_static(pOut, pOp->p4.z, pOp->p1);
+ } else {
+ sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
pOut->flags |= MEM_Subtype;
pOut->subtype = pOp->p3;
}
@@ -2201,9 +2208,7 @@ case OP_MakeRecord: {
* sure previously allocated memory has gone.
*/
mem_destroy(pOut);
- pOut->flags = MEM_Blob | MEM_Ephem;
- pOut->n = tuple_size;
- pOut->z = tuple;
+ mem_set_bin_ephemeral(pOut, tuple, tuple_size);
}
assert(sqlVdbeCheckMemInvariants(pOut));
assert(pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor));
@@ -3286,9 +3291,14 @@ case OP_RowData: {
}
testcase( n==0);
- if (vdbe_mem_alloc_blob_region(pOut, n) != 0)
+ char *buf = region_alloc(&fiber()->gc, n);
+ if (buf == NULL) {
+ diag_set(OutOfMemory, n, "region_alloc", "buf");
goto abort_due_to_error;
- sqlCursorPayload(pCrsr, 0, n, pOut->z);
+ }
+ sqlCursorPayload(pCrsr, 0, n, buf);
+ mem_set_bin_ephemeral(pOut, buf, n);
+ assert(sqlVdbeCheckMemInvariants(pOut));
UPDATE_MAX_BLOBSIZE(pOut);
REGISTER_TRACE(p, pOp->p2, pOut);
break;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index ee095f36e..8c7a0b6cd 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -111,9 +111,8 @@ sql_metadata_is_full()
* The following routines are used by user-defined functions to specify
* the function result.
*
- * The setStrOrError() function calls sqlVdbeMemSetStr() to store the
- * result as a string or blob but if the string or blob is too large, it
- * then sets the error code.
+ * The setStrOrError() function sets the result as a string or blob but
+ * if the string or blob is too large, it then sets the error code.
*
* The invokeValueDestructor(P,X) routine invokes destructor function X()
* on value P is not going to be used and need to be destroyed.
@@ -183,7 +182,13 @@ sql_result_blob(sql_context * pCtx,
)
{
assert(n >= 0);
- if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
+ if (xDel == SQL_STATIC)
+ mem_set_bin_static(pCtx->pOut, (char *)z, n);
+ else if (xDel == SQL_DYNAMIC)
+ mem_set_bin_allocated(pCtx->pOut, (char *)z, n);
+ else if (xDel != SQL_TRANSIENT)
+ mem_set_bin_dynamic(pCtx->pOut, (char *)z, n);
+ else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
pCtx->is_aborted = true;
}
@@ -832,7 +837,13 @@ sql_bind_blob(sql_stmt * pStmt,
if (zData == NULL)
return 0;
struct Mem *var = &p->aVar[i - 1];
- if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
+ if (xDel == SQL_STATIC)
+ mem_set_bin_static(var, (char *)zData, nData);
+ else if (xDel == SQL_DYNAMIC)
+ mem_set_bin_allocated(var, (char *)zData, nData);
+ else if (xDel != SQL_TRANSIENT)
+ mem_set_bin_dynamic(var, (char *)zData, nData);
+ else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
return -1;
return sql_bind_type(p, i, "varbinary");
}
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (28 preceding siblings ...)
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 29/52] sql: introduce mem_set_bin_*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-09 19:45 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 31/52] sql: introduce mem_set_zerobin() Mergen Imeev via Tarantool-patches
` (11 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:45 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:05, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> On 23.03.2021 10:36, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_copy_binary() function. Function
>> mem_copy_binary() clears MEM, allocates enough memory and copies given
>> binary to allocated memory.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 14 ++++++++++++++
>> src/box/sql/mem.h | 3 +++
>> src/box/sql/vdbe.c | 7 ++-----
>> src/box/sql/vdbeapi.c | 4 ++--
>> src/box/sql/vdbesort.c | 6 +-----
>> 5 files changed, 22 insertions(+), 12 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 5ee49cdca..99beec9ad 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -465,6 +465,20 @@ mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size)
>> mem_set_dyn_bin(mem, value, size, 0);
>> }
>>
>> +int
>> +mem_copy_binary(struct Mem *mem, const char *value, uint32_t size)
>> +{
>
> What if mem is not a binary now? What if it is a frame? Why don't you clear it?
>
Thanks! Fixed here and in mem_copy_str(), mem_copy_str0() functions.
>> + bool is_own_value = (mem->flags & MEM_Blob) != 0 && mem->z == value;
>> + if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
>> + return -1;
>> + if (!is_own_value)
>> + memcpy(mem->z, value, size);
>> + mem->n = size;
>> + mem->flags = MEM_Blob;
>> + mem->field_type = FIELD_TYPE_VARBINARY;
>> + return 0;
>> +}
New patch:
commit ab44b535847f591f638826e19277b85bab9f6189
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 11:55:05 2021 +0300
sql: introduce mem_copy_bin()
This patch introduces mem_copy_bin() function. This function copies
given binary value to a newly allocated by MEM memory.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 2622cdd82..e30795de5 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,19 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+/*
+ * Make sure pMem->z points to a writable allocation of at least
+ * min(n,32) bytes.
+ *
+ * If the bPreserve argument is true, then copy of the content of
+ * pMem->z into the new allocation. pMem must be either a string or
+ * blob if bPreserve is true. If bPreserve is false, any prior content
+ * in pMem->z is discarded.
+ */
+static int
+sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
+
+
bool
mem_is_null(const struct Mem *mem)
{
@@ -477,6 +490,23 @@ mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size)
set_bin_dynamic(mem, value, size, 0);
}
+int
+mem_copy_bin(struct Mem *mem, const char *value, uint32_t size)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
+ mem_clear(mem);
+ bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == value;
+ if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
+ return -1;
+ if (!is_own_value)
+ memcpy(mem->z, value, size);
+ mem->n = size;
+ mem->flags = MEM_Blob;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2046,17 +2076,8 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
return mem_convert_to_integer(mem);
}
-/*
- * Make sure pMem->z points to a writable allocation of at least
- * min(n,32) bytes.
- *
- * If the bPreserve argument is true, then copy of the content of
- * pMem->z into the new allocation. pMem must be either a string or
- * blob if bPreserve is true. If bPreserve is false, any prior content
- * in pMem->z is discarded.
- */
-SQL_NOINLINE int
-sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
+static int
+sqlVdbeMemGrow(struct Mem *pMem, int n, int bPreserve)
{
assert(sqlVdbeCheckMemInvariants(pMem));
testcase(pMem->db == 0);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 532860ff7..e17a722f4 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -274,6 +274,13 @@ mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size);
void
mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size);
+/**
+ * Copy binary value to a newly allocated memory. The MEM type becomes
+ * VARBINARY.
+ */
+int
+mem_copy_bin(struct Mem *mem, const char *value, uint32_t size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -541,7 +548,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
/** Setters = Change MEM value. */
-int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
/**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 85fb7a26d..19a0b041c 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2196,11 +2196,8 @@ case OP_MakeRecord: {
* routine.
*/
if (bIsEphemeral) {
- if (sqlVdbeMemClearAndResize(pOut, tuple_size) != 0)
+ if (mem_copy_bin(pOut, tuple, tuple_size) != 0)
goto abort_due_to_error;
- pOut->flags = MEM_Blob;
- pOut->n = tuple_size;
- memcpy(pOut->z, tuple, tuple_size);
region_truncate(region, used);
} else {
/* Allocate memory on the region for the tuple
@@ -2540,7 +2537,7 @@ case OP_SequenceTest: {
* Open a new cursor that points to a fake table that contains a single
* row of data. The content of that one row is the content of memory
* register P2. In other words, cursor P1 becomes an alias for the
- * MEM_Blob content contained in register P2.
+ * MEM with binary content contained in register P2.
*
* A pseudo-table created by this opcode is used to hold a single
* row output from the sorter so that the row can be decomposed into
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 8c7a0b6cd..d16b92cf6 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -188,7 +188,7 @@ sql_result_blob(sql_context * pCtx,
mem_set_bin_allocated(pCtx->pOut, (char *)z, n);
else if (xDel != SQL_TRANSIENT)
mem_set_bin_dynamic(pCtx->pOut, (char *)z, n);
- else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
+ else if (mem_copy_bin(pCtx->pOut, z, n) != 0)
pCtx->is_aborted = true;
}
@@ -843,7 +843,7 @@ sql_bind_blob(sql_stmt * pStmt,
mem_set_bin_allocated(var, (char *)zData, nData);
else if (xDel != SQL_TRANSIENT)
mem_set_bin_dynamic(var, (char *)zData, nData);
- else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
+ else if (mem_copy_bin(var, zData, nData) != 0)
return -1;
return sql_bind_type(p, i, "varbinary");
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 171cb8946..c294a0286 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1305,19 +1305,26 @@ sqlVdbeList(Vdbe * p)
* has not already been seen.
*/
if (pOp->p4type == P4_SUBPROGRAM) {
- int nByte = (nSub + 1) * sizeof(SubProgram *);
int j;
for (j = 0; j < nSub; j++) {
if (apSub[j] == pOp->p4.pProgram)
break;
}
- if (j == nSub &&
- sqlVdbeMemGrow(pSub, nByte,
- nSub != 0) == 0) {
- apSub = (SubProgram **) pSub->z;
- apSub[nSub++] = pOp->p4.pProgram;
- pSub->flags |= MEM_Blob;
- pSub->n = nSub * sizeof(SubProgram *);
+ if (nSub == 0) {
+ uint32_t size = sizeof(SubProgram *);
+ char *bin = (char *)&pOp->p4.pProgram;
+ if (mem_copy_bin(pSub, bin, size) != 0)
+ return -1;
+ } else if (j == nSub) {
+ struct Mem tmp;
+ mem_create(&tmp);
+ uint32_t size = sizeof(SubProgram *);
+ char *bin = (char *)&pOp->p4.pProgram;
+ mem_set_bin_ephemeral(&tmp, bin, size);
+ int rc = mem_concat(pSub, &tmp, pSub);
+ mem_destroy(&tmp);
+ if (rc != 0)
+ return -1;
}
}
}
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index a9a5f45af..3da425be4 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2164,12 +2164,8 @@ sqlVdbeSorterRowkey(const VdbeCursor * pCsr, Mem * pOut)
assert(pCsr->eCurType == CURTYPE_SORTER);
pSorter = pCsr->uc.pSorter;
pKey = vdbeSorterRowkey(pSorter, &nKey);
- if (sqlVdbeMemClearAndResize(pOut, nKey)) {
+ if (mem_copy_bin(pOut, pKey, nKey) != 0)
return -1;
- }
- pOut->n = nKey;
- MemSetTypeFlag(pOut, MEM_Blob);
- memcpy(pOut->z, pKey, nKey);
return 0;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin()
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:06 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:36 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
I appreciate the work you did here!
See 2 comments below.
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 2622cdd82..e30795de5 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -40,6 +40,19 @@
> #include "lua/utils.h"
> #include "lua/msgpack.h"
>
> +/*
> + * Make sure pMem->z points to a writable allocation of at least
> + * min(n,32) bytes.
> + *
> + * If the bPreserve argument is true, then copy of the content of
> + * pMem->z into the new allocation. pMem must be either a string or
> + * blob if bPreserve is true. If bPreserve is false, any prior content
> + * in pMem->z is discarded.
> + */
> +static int
> +sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
> +
> +
1. Double empty line.
> bool
> mem_is_null(const struct Mem *mem)
> {
> @@ -477,6 +490,23 @@ mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size)
> set_bin_dynamic(mem, value, size, 0);
> }
>
> +int
> +mem_copy_bin(struct Mem *mem, const char *value, uint32_t size)
> +{
> + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> + mem_clear(mem);
2. The same comment as for copy_str.
> + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> + mem->z == value;
> + if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
> + return -1;
> + if (!is_own_value)
> + memcpy(mem->z, value, size);
> + mem->n = size;
> + mem->flags = MEM_Blob;
> + mem->field_type = FIELD_TYPE_VARBINARY;
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin()
2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:06 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:06 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below. Patch was a bit
changed due to merge conflicts with new version of
"sql: introduce mem_is_*() functions".
On Tue, Apr 13, 2021 at 01:36:15AM +0200, Vladislav Shpilevoy wrote:
> I appreciate the work you did here!
>
> See 2 comments below.
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 2622cdd82..e30795de5 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -40,6 +40,19 @@
> > #include "lua/utils.h"
> > #include "lua/msgpack.h"
> >
> > +/*
> > + * Make sure pMem->z points to a writable allocation of at least
> > + * min(n,32) bytes.
> > + *
> > + * If the bPreserve argument is true, then copy of the content of
> > + * pMem->z into the new allocation. pMem must be either a string or
> > + * blob if bPreserve is true. If bPreserve is false, any prior content
> > + * in pMem->z is discarded.
> > + */
> > +static int
> > +sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
> > +
> > +
>
> 1. Double empty line.
>
Fixed during resolving merge conflicts with new version of
"sql: introduce mem_is_*() functions".
> > bool
> > mem_is_null(const struct Mem *mem)
> > {
> > @@ -477,6 +490,23 @@ mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size)
> > set_bin_dynamic(mem, value, size, 0);
> > }
> >
> > +int
> > +mem_copy_bin(struct Mem *mem, const char *value, uint32_t size)
> > +{
> > + if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
> > + mem_clear(mem);
>
> 2. The same comment as for copy_str.
>
Same as there, in case copied value is value of the MEM it copied to and its
allocation type being MEM_Dyn, it will be lost during clear().
> > + bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> > + mem->z == value;
> > + if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
> > + return -1;
> > + if (!is_own_value)
> > + memcpy(mem->z, value, size);
> > + mem->n = size;
> > + mem->flags = MEM_Blob;
> > + mem->field_type = FIELD_TYPE_VARBINARY;
> > + return 0;
> > +}
New patch:
commit 8788956575ecf095d86d63de5eeea2e932fb0bac
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 11:55:05 2021 +0300
sql: introduce mem_copy_bin()
This patch introduces mem_copy_bin() function. This function copies
given binary value to a newly allocated by MEM memory.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 1e7adfcb0..190311d80 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -40,6 +40,18 @@
#include "lua/utils.h"
#include "lua/msgpack.h"
+/*
+ * Make sure pMem->z points to a writable allocation of at least
+ * min(n,32) bytes.
+ *
+ * If the bPreserve argument is true, then copy of the content of
+ * pMem->z into the new allocation. pMem must be either a string or
+ * blob if bPreserve is true. If bPreserve is false, any prior content
+ * in pMem->z is discarded.
+ */
+static int
+sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
+
enum {
BUF_SIZE = 32,
};
@@ -335,6 +347,23 @@ mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size)
set_bin_dynamic(mem, value, size, 0);
}
+int
+mem_copy_bin(struct Mem *mem, const char *value, uint32_t size)
+{
+ if ((mem->flags & (MEM_Agg | MEM_Frame)) != 0)
+ mem_clear(mem);
+ bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+ mem->z == value;
+ if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
+ return -1;
+ if (!is_own_value)
+ memcpy(mem->z, value, size);
+ mem->n = size;
+ mem->flags = MEM_Blob;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1869,17 +1898,8 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
return mem_convert_to_integer(mem);
}
-/*
- * Make sure pMem->z points to a writable allocation of at least
- * min(n,32) bytes.
- *
- * If the bPreserve argument is true, then copy of the content of
- * pMem->z into the new allocation. pMem must be either a string or
- * blob if bPreserve is true. If bPreserve is false, any prior content
- * in pMem->z is discarded.
- */
-SQL_NOINLINE int
-sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
+static int
+sqlVdbeMemGrow(struct Mem *pMem, int n, int bPreserve)
{
assert(sqlVdbeCheckMemInvariants(pMem));
testcase(pMem->db == 0);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 617966cf7..197c16461 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -456,6 +456,13 @@ mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size);
void
mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size);
+/**
+ * Copy binary value to a newly allocated memory. The MEM type becomes
+ * VARBINARY.
+ */
+int
+mem_copy_bin(struct Mem *mem, const char *value, uint32_t size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -699,7 +706,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
/** Setters = Change MEM value. */
-int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
/**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index cd7811846..d573e1e8f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2240,11 +2240,8 @@ case OP_MakeRecord: {
* routine.
*/
if (bIsEphemeral) {
- if (sqlVdbeMemClearAndResize(pOut, tuple_size) != 0)
+ if (mem_copy_bin(pOut, tuple, tuple_size) != 0)
goto abort_due_to_error;
- pOut->flags = MEM_Blob;
- pOut->n = tuple_size;
- memcpy(pOut->z, tuple, tuple_size);
region_truncate(region, used);
} else {
/* Allocate memory on the region for the tuple
@@ -2584,7 +2581,7 @@ case OP_SequenceTest: {
* Open a new cursor that points to a fake table that contains a single
* row of data. The content of that one row is the content of memory
* register P2. In other words, cursor P1 becomes an alias for the
- * MEM_Blob content contained in register P2.
+ * MEM with binary content contained in register P2.
*
* A pseudo-table created by this opcode is used to hold a single
* row output from the sorter so that the row can be decomposed into
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7114f4351..7656d8d67 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -168,7 +168,7 @@ sql_result_blob(sql_context * pCtx,
mem_set_bin_allocated(pCtx->pOut, (char *)z, n);
else if (xDel != SQL_TRANSIENT)
mem_set_bin_dynamic(pCtx->pOut, (char *)z, n);
- else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
+ else if (mem_copy_bin(pCtx->pOut, z, n) != 0)
pCtx->is_aborted = true;
}
@@ -807,7 +807,7 @@ sql_bind_blob(sql_stmt * pStmt,
mem_set_bin_allocated(var, (char *)zData, nData);
else if (xDel != SQL_TRANSIENT)
mem_set_bin_dynamic(var, (char *)zData, nData);
- else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
+ else if (mem_copy_bin(var, zData, nData) != 0)
return -1;
return sql_bind_type(p, i, "varbinary");
}
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index dad9cab03..ffc85b91d 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1305,19 +1305,26 @@ sqlVdbeList(Vdbe * p)
* has not already been seen.
*/
if (pOp->p4type == P4_SUBPROGRAM) {
- int nByte = (nSub + 1) * sizeof(SubProgram *);
int j;
for (j = 0; j < nSub; j++) {
if (apSub[j] == pOp->p4.pProgram)
break;
}
- if (j == nSub &&
- sqlVdbeMemGrow(pSub, nByte,
- nSub != 0) == 0) {
- apSub = (SubProgram **) pSub->z;
- apSub[nSub++] = pOp->p4.pProgram;
- pSub->flags |= MEM_Blob;
- pSub->n = nSub * sizeof(SubProgram *);
+ if (nSub == 0) {
+ uint32_t size = sizeof(SubProgram *);
+ char *bin = (char *)&pOp->p4.pProgram;
+ if (mem_copy_bin(pSub, bin, size) != 0)
+ return -1;
+ } else if (j == nSub) {
+ struct Mem tmp;
+ mem_create(&tmp);
+ uint32_t size = sizeof(SubProgram *);
+ char *bin = (char *)&pOp->p4.pProgram;
+ mem_set_bin_ephemeral(&tmp, bin, size);
+ int rc = mem_concat(pSub, &tmp, pSub);
+ mem_destroy(&tmp);
+ if (rc != 0)
+ return -1;
}
}
}
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index a9a5f45af..3da425be4 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2164,12 +2164,8 @@ sqlVdbeSorterRowkey(const VdbeCursor * pCsr, Mem * pOut)
assert(pCsr->eCurType == CURTYPE_SORTER);
pSorter = pCsr->uc.pSorter;
pKey = vdbeSorterRowkey(pSorter, &nKey);
- if (sqlVdbeMemClearAndResize(pOut, nKey)) {
+ if (mem_copy_bin(pOut, pKey, nKey) != 0)
return -1;
- }
- pOut->n = nKey;
- MemSetTypeFlag(pOut, MEM_Blob);
- memcpy(pOut->z, pKey, nKey);
return 0;
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 31/52] sql: introduce mem_set_zerobin()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (29 preceding siblings ...)
2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:05 ` Mergen Imeev via Tarantool-patches
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array Mergen Imeev via Tarantool-patches
` (10 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:05 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch introduces mem_set_zerobin() function. This function clears
MEM and sets it to binary value that consists of given number of zero
bytes.
Part of #5818
---
src/box/sql/mem.c | 29 +++++++++++++----------------
src/box/sql/mem.h | 8 ++++++--
src/box/sql/vdbeapi.c | 6 +++---
3 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index e30795de5..508b1dee3 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -507,6 +507,19 @@ mem_copy_bin(struct Mem *mem, const char *value, uint32_t size)
return 0;
}
+void
+mem_set_zerobin(struct Mem *mem, int n)
+{
+ mem_destroy(mem);
+ if (n < 0)
+ n = 0;
+ mem->u.nZero = n;
+ mem->z = NULL;
+ mem->n = 0;
+ mem->flags = MEM_Blob | MEM_Zero;
+ mem->field_type = FIELD_TYPE_VARBINARY;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2256,22 +2269,6 @@ sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
return 0;
}
-/*
- * Delete any previous value and set the value to be a BLOB of length
- * n containing all zeros.
- */
-void
-sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
-{
- mem_destroy(pMem);
- pMem->flags = MEM_Blob | MEM_Zero;
- pMem->n = 0;
- if (n < 0)
- n = 0;
- pMem->u.nZero = n;
- pMem->z = 0;
-}
-
/*
* Free an sql_value object
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index e17a722f4..0aeb23496 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -274,6 +274,12 @@ mem_set_bin_dynamic(struct Mem *mem, char *value, uint32_t size);
void
mem_set_bin_allocated(struct Mem *mem, char *value, uint32_t size);
+/**
+ * Clear MEM and set it to VARBINARY. The binary value consist of n zero bytes.
+ */
+void
+mem_set_zerobin(struct Mem *mem, int n);
+
/**
* Copy binary value to a newly allocated memory. The MEM type becomes
* VARBINARY.
@@ -560,8 +566,6 @@ mem_set_ptr(struct Mem *mem, void *ptr);
int
sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
-void
-sqlVdbeMemSetZeroBlob(struct Mem *, int);
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index d16b92cf6..6d9103ff2 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -266,7 +266,7 @@ sql_result_value(sql_context * pCtx, sql_value * pValue)
void
sql_result_zeroblob(sql_context * pCtx, int n)
{
- sqlVdbeMemSetZeroBlob(pCtx->pOut, n);
+ mem_set_zerobin(pCtx->pOut, n);
}
int
@@ -278,7 +278,7 @@ sql_result_zeroblob64(sql_context * pCtx, u64 n)
"is too big");
return -1;
}
- sqlVdbeMemSetZeroBlob(pCtx->pOut, (int)n);
+ mem_set_zerobin(pCtx->pOut, (int)n);
return 0;
}
@@ -956,7 +956,7 @@ sql_bind_zeroblob(sql_stmt * pStmt, int i, int n)
Vdbe *p = (Vdbe *) pStmt;
if (vdbeUnbind(p, i) != 0)
return -1;
- sqlVdbeMemSetZeroBlob(&p->aVar[i - 1], n);
+ mem_set_zerobin(&p->aVar[i - 1], n);
return 0;
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (30 preceding siblings ...)
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 31/52] sql: introduce mem_set_zerobin() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:05 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 33/52] sql: introduce mem_set_invalid() Mergen Imeev via Tarantool-patches
` (9 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:05 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:05, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 7885caaf5..583de00a2 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -508,6 +508,86 @@ mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size)
>> return 0;
>> }
>>
>> +void
>> +mem_set_ephemeral_map(struct Mem *mem, char *value, uint32_t size)
>> +{
>> + assert(mp_typeof(*value) == MP_MAP);
>> + mem_set_const_bin(mem, value, size, MEM_Ephem);
>> + mem->flags |= MEM_Subtype;
>> + mem->subtype = SQL_SUBTYPE_MSGPACK;
>> + mem->field_type = FIELD_TYPE_MAP;
>> +}
>> +
>> +void
>> +mem_set_static_map(struct Mem *mem, char *value, uint32_t size)
>> +{
>> + assert(mp_typeof(*value) == MP_MAP);
>> + mem_set_const_bin(mem, value, size, MEM_Static);
>> + mem->flags |= MEM_Subtype;
>> + mem->subtype = SQL_SUBTYPE_MSGPACK;
>> + mem->field_type = FIELD_TYPE_MAP;
>> +}
>> +
>> +void
>> +mem_set_dynamic_map(struct Mem *mem, char *value, uint32_t size)
>
> I think I lost the clue of what is the difference between dynamic
> and allocated. Maybe worth adding a comment? Or find a better name?
>
Added a comment. Didn't thought about new names for now.
> For instance, if one of them is supposed to copy the map, and the
> other one to steal its ownership, then you could call them
> mem_set_map_copy() and mem_set_map_move(), where move is the same as
> C++ move - steal the resource. The same for the others dynamic/allocated
> terminology in the function names.
>
In some sense dynamic and allocated allocation types steals the ownership of
allocated memory, however there is difference in way they free it. I described
it in comments to the functions.
> I think I also lost the understanding of static vs ephem by now.
Static - available always, ephemeral - may become unavailable at some point.
Thought it is true that the border between these two allocation type not as
clear as should be.
New patch:
commit 26b61648509b08b4cde3307015d398372cdd20f3
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 13:50:54 2021 +0300
sql: introduce mem_set_*() for map and array
This patch introduces set of mem_set_*() functions for MAP and ARRAY.
These functions clears MEM and sets it to given binary value. Binary
value must be MAP or ARRAY. Degree of clearing and type of allocation of
the binary value is determined by the function used.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 508b1dee3..61849cde7 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -520,6 +520,75 @@ mem_set_zerobin(struct Mem *mem, int n)
mem->field_type = FIELD_TYPE_VARBINARY;
}
+static inline void
+set_msgpack_value(struct Mem *mem, char *value, uint32_t size, int alloc_type,
+ enum field_type type)
+{
+ if (alloc_type == MEM_Ephem || alloc_type == MEM_Static)
+ set_bin_const(mem, value, size, alloc_type);
+ else
+ set_bin_dynamic(mem, value, size, alloc_type);
+ mem->flags |= MEM_Subtype;
+ mem->subtype = SQL_SUBTYPE_MSGPACK;
+ mem->field_type = type;
+}
+
+void
+mem_set_map_ephemeral(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_MAP);
+ set_msgpack_value(mem, value, size, MEM_Ephem, FIELD_TYPE_MAP);
+}
+
+void
+mem_set_map_static(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_MAP);
+ set_msgpack_value(mem, value, size, MEM_Static, FIELD_TYPE_MAP);
+}
+
+void
+mem_set_map_dynamic(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_MAP);
+ set_msgpack_value(mem, value, size, MEM_Dyn, FIELD_TYPE_MAP);
+}
+
+void
+mem_set_map_allocated(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_MAP);
+ set_msgpack_value(mem, value, size, 0, FIELD_TYPE_MAP);
+}
+
+void
+mem_set_array_ephemeral(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_ARRAY);
+ set_msgpack_value(mem, value, size, MEM_Ephem, FIELD_TYPE_ARRAY);
+}
+
+void
+mem_set_array_static(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_ARRAY);
+ set_msgpack_value(mem, value, size, MEM_Static, FIELD_TYPE_ARRAY);
+}
+
+void
+mem_set_array_dynamic(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_ARRAY);
+ set_msgpack_value(mem, value, size, MEM_Dyn, FIELD_TYPE_ARRAY);
+}
+
+void
+mem_set_array_allocated(struct Mem *mem, char *value, uint32_t size)
+{
+ assert(mp_typeof(*value) == MP_ARRAY);
+ set_msgpack_value(mem, value, size, 0, FIELD_TYPE_ARRAY);
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2173,102 +2242,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
-/*
- * Change the value of a Mem to be a string or a BLOB.
- *
- * The memory management strategy depends on the value of the xDel
- * parameter. If the value passed is SQL_TRANSIENT, then the
- * string is copied into a (possibly existing) buffer managed by the
- * Mem structure. Otherwise, any existing buffer is freed and the
- * pointer copied.
- *
- * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
- * size limit) then no memory allocation occurs. If the string can be
- * stored without allocating memory, then it is. If a memory allocation
- * is required to store the string, then value of pMem is unchanged. In
- * either case, error is returned.
- */
-int
-sqlVdbeMemSetStr(Mem * pMem, /* Memory cell to set to string value */
- const char *z, /* String pointer */
- int n, /* Bytes in string, or negative */
- u8 not_blob, /* Encoding of z. 0 for BLOBs */
- void (*xDel) (void *) /* Destructor function */
- )
-{
- int nByte = n; /* New value for pMem->n */
- int iLimit; /* Maximum allowed string or blob size */
- u16 flags = 0; /* New value for pMem->flags */
-
- /* If z is a NULL pointer, set pMem to contain an SQL NULL. */
- if (!z) {
- mem_clear(pMem);
- return 0;
- }
-
- if (pMem->db) {
- iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
- } else {
- iLimit = SQL_MAX_LENGTH;
- }
- flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
- if (nByte < 0) {
- assert(not_blob != 0);
- nByte = sqlStrlen30(z);
- if (nByte > iLimit)
- nByte = iLimit + 1;
- flags |= MEM_Term;
- }
-
- /* The following block sets the new values of Mem.z and Mem.xDel. It
- * also sets a flag in local variable "flags" to indicate the memory
- * management (one of MEM_Dyn or MEM_Static).
- */
- if (xDel == SQL_TRANSIENT) {
- int nAlloc = nByte;
- if (flags & MEM_Term) {
- nAlloc += 1; //SQL_UTF8
- }
- if (nByte > iLimit) {
- diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
- "string is too big");
- return -1;
- }
- testcase(nAlloc == 0);
- testcase(nAlloc == 31);
- testcase(nAlloc == 32);
- if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
- return -1;
- }
- memcpy(pMem->z, z, nAlloc);
- } else if (xDel == SQL_DYNAMIC) {
- mem_destroy(pMem);
- pMem->zMalloc = pMem->z = (char *)z;
- pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
- } else {
- mem_destroy(pMem);
- pMem->z = (char *)z;
- pMem->xDel = xDel;
- flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
- }
-
- pMem->n = nByte;
- pMem->flags = flags;
- assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
- if ((pMem->flags & MEM_Str) != 0)
- pMem->field_type = FIELD_TYPE_STRING;
- else
- pMem->field_type = FIELD_TYPE_VARBINARY;
-
- if (nByte > iLimit) {
- diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
- "is too big");
- return -1;
- }
-
- return 0;
-}
-
/*
* Free an sql_value object
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 0aeb23496..b3a602f6e 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -287,6 +287,71 @@ mem_set_zerobin(struct Mem *mem, int n);
int
mem_copy_bin(struct Mem *mem, const char *value, uint32_t size);
+/**
+ * Clear MEM and set it to MAP. The binary value belongs to another object. The
+ * binary value must be msgpack of MAP type.
+ */
+void
+mem_set_map_ephemeral(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to MAP. The binary value is static. The binary value
+ * must be msgpack of MAP type.
+ */
+void
+mem_set_map_static(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to MAP. The binary value was allocated by another object
+ * and passed to MEM. The binary value must be msgpack of MAP type. MEMs with
+ * this allocation type must free given memory whenever the MEM changes.
+ */
+void
+mem_set_map_dynamic(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to MAP. The binary value was allocated by another object
+ * and passed to MEM. The binary value must be msgpack of MAP type. MEMs with
+ * this allocation type only deallocate the string on destruction. Also, the
+ * memory may be reallocated if MEM is set to a different value of this
+ * allocation type.
+ */
+void
+mem_set_map_allocated(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to ARRAY. The binary value belongs to another object.
+ * The binary value must be msgpack of ARRAY type.
+ */
+void
+mem_set_array_ephemeral(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to ARRAY. The binary value is static. The binary value
+ * must be msgpack of ARRAY type.
+ */
+void
+mem_set_array_static(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to ARRAY. The binary value was allocated by another
+ * object and passed to MEM. The binary value must be msgpack of ARRAY type.
+ * MEMs with this allocation type must free given memory whenever the MEM
+ * changes.
+ */
+void
+mem_set_array_dynamic(struct Mem *mem, char *value, uint32_t size);
+
+/**
+ * Clear MEM and set it to ARRAY. The binary value was allocated by another
+ * object and passed to MEM. The binary value must be msgpack of ARRAY type.
+ * MEMs with this allocation type only deallocate the string on destruction.
+ * Also, the memory may be reallocated if MEM is set to a different value of
+ * this allocation type.
+ */
+void
+mem_set_array_allocated(struct Mem *mem, char *value, uint32_t size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -564,8 +629,6 @@ int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
void
mem_set_ptr(struct Mem *mem, void *ptr);
-int
-sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 19a0b041c..9434a4d06 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -894,9 +894,11 @@ case OP_Blob: { /* out2 */
*/
mem_set_bin_static(pOut, pOp->p4.z, pOp->p1);
} else {
- sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
- pOut->flags |= MEM_Subtype;
- pOut->subtype = pOp->p3;
+ assert(pOp->p3 == SQL_SUBTYPE_MSGPACK);
+ if (mp_typeof(*pOp->p4.z) == MP_MAP)
+ mem_set_map_static(pOut, pOp->p4.z, pOp->p1);
+ else
+ mem_set_array_static(pOut, pOp->p4.z, pOp->p1);
}
UPDATE_MAX_BLOBSIZE(pOut);
break;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:08 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:36 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for working on this!
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 508b1dee3..61849cde7 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -520,6 +520,75 @@ mem_set_zerobin(struct Mem *mem, int n)
> mem->field_type = FIELD_TYPE_VARBINARY;
> }
>
> +static inline void
> +set_msgpack_value(struct Mem *mem, char *value, uint32_t size, int alloc_type,
> + enum field_type type)
> +{
> + if (alloc_type == MEM_Ephem || alloc_type == MEM_Static)
> + set_bin_const(mem, value, size, alloc_type);
> + else
> + set_bin_dynamic(mem, value, size, alloc_type);
> + mem->flags |= MEM_Subtype;
> + mem->subtype = SQL_SUBTYPE_MSGPACK;
> + mem->field_type = type;
Previously field_type wasn't set. Why did you change that?
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array
2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:08 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:08 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer below.
On Tue, Apr 13, 2021 at 01:36:45AM +0200, Vladislav Shpilevoy wrote:
> Thanks for working on this!
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index 508b1dee3..61849cde7 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -520,6 +520,75 @@ mem_set_zerobin(struct Mem *mem, int n)
> > mem->field_type = FIELD_TYPE_VARBINARY;
> > }
> >
> > +static inline void
> > +set_msgpack_value(struct Mem *mem, char *value, uint32_t size, int alloc_type,
> > + enum field_type type)
> > +{
> > + if (alloc_type == MEM_Ephem || alloc_type == MEM_Static)
> > + set_bin_const(mem, value, size, alloc_type);
> > + else
> > + set_bin_dynamic(mem, value, size, alloc_type);
> > + mem->flags |= MEM_Subtype;
> > + mem->subtype = SQL_SUBTYPE_MSGPACK;
> > + mem->field_type = type;
>
> Previously field_type wasn't set. Why did you change that?
This is the simplest way I know to make sure that MEM-type and field-type are
compatible.
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 33/52] sql: introduce mem_set_invalid()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (31 preceding siblings ...)
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:05 ` Mergen Imeev via Tarantool-patches
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 34/52] sql: refactor mem_set_ptr() Mergen Imeev via Tarantool-patches
` (8 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:05 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answer and new patch below.
On 30.03.2021 02:06, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> On 23.03.2021 10:36, Mergen Imeev via Tarantool-patches wrote:
>> This patch introduces mem_set_undefined() function. Function
>> mem_set_undefined() invalidates MEM. It does not clears MEM prior to
>> invalidating it.
>
> Why does not it clear it?
Usually MEM_Undefined is just set to MEM without any other changes. I cannot say
for sure that all MEMs that have this flag set did not have MEM_Dyn, MEM_Frame
or MEM_Agg set previously. I added clear() to the function.
New patch:
commit 07b94caea28f23d61ba6f8a31d0ce70c7829dde6
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 14:26:55 2021 +0300
sql: introduce mem_set_invalid()
This patch introduces mem_set_invalid() function. This function clears
MEM and invalidates it.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 61849cde7..b97904f22 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -589,6 +589,13 @@ mem_set_array_allocated(struct Mem *mem, char *value, uint32_t size)
set_msgpack_value(mem, value, size, 0, FIELD_TYPE_ARRAY);
}
+void
+mem_set_invalid(struct Mem *mem)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Undefined;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index b3a602f6e..fe3d8d98f 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -352,6 +352,10 @@ mem_set_array_dynamic(struct Mem *mem, char *value, uint32_t size);
void
mem_set_array_allocated(struct Mem *mem, char *value, uint32_t size);
+/** Clear MEM and set it to invalid state. */
+void
+mem_set_invalid(struct Mem *mem);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9434a4d06..0031d2248 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -77,7 +77,7 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
if (mem_is_bytes(pX) && !mem_is_ephemeral(pX) &&
!mem_is_static(pX)) {
if (pX->pScopyFrom == pMem) {
- pX->flags |= MEM_Undefined;
+ mem_set_invalid(pX);
pX->pScopyFrom = 0;
}
}
@@ -629,7 +629,7 @@ case OP_Return: { /* in1 */
pIn1 = &aMem[pOp->p1];
assert(mem_is_uint(pIn1));
pOp = &aOp[pIn1->u.u];
- pIn1->flags = MEM_Undefined;
+ mem_set_invalid(pIn1);
break;
}
@@ -672,7 +672,7 @@ case OP_EndCoroutine: { /* in1 */
assert(pCaller->opcode==OP_Yield);
assert(pCaller->p2>=0 && pCaller->p2<p->nOp);
pOp = &aOp[pCaller->p2 - 1];
- pIn1->flags = MEM_Undefined;
+ mem_set_invalid(pIn1);
break;
}
@@ -4181,8 +4181,8 @@ case OP_Program: { /* jump */
pEnd = &VdbeFrameMem(pFrame)[pFrame->nChildMem];
for(pMem=VdbeFrameMem(pFrame); pMem!=pEnd; pMem++) {
- pMem->flags = MEM_Undefined;
- pMem->db = db;
+ mem_create(pMem);
+ mem_set_invalid(pMem);
}
} else {
pFrame = pRt->u.pFrame;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index c294a0286..a5e78be06 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1592,7 +1592,7 @@ sqlVdbeMakeReady(Vdbe * p, /* The VDBE */
p->nMem = nMem;
for (int i = 0; i < nMem; ++i) {
mem_create(&p->aMem[i]);
- p->aMem[i].flags = MEM_Undefined;
+ mem_set_invalid(&p->aMem[i]);
}
memset(p->apCsr, 0, nCursor * sizeof(VdbeCursor *));
}
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 34/52] sql: refactor mem_set_ptr()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (32 preceding siblings ...)
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 33/52] sql: introduce mem_set_invalid() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:05 ` Mergen Imeev via Tarantool-patches
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
` (7 subsequent siblings)
41 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:05 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answer and new patch below.
On 30.03.2021 02:06, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> I think mem_set_ptr() name was better. Was shorter and still easy to
> understand.
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index fd2710478..3701741ef 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -595,6 +595,15 @@ mem_set_undefined(struct Mem *mem)
>> mem->field_type = field_type_MAX;
>> }
>>
>> +void
>> +mem_set_pointer(struct Mem *mem, void *ptr)
>> +{
>> + mem_clear(mem);
>> + mem->flags = MEM_Ptr;
>> + mem->u.p = ptr;
>> + mem->field_type = field_type_MAX;
>
> mem_clear() already sets the type.
Fixed.
New patch:
commit 6d76eee6ea147188aced9dfa18603bd7d2c2e744
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 14:36:24 2021 +0300
sql: refactor mem_set_ptr()
This patch refactors mem_set_ptr() function. Also, it moves the function
to all others mem_set_*() functions.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index b97904f22..4091161ce 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -596,6 +596,14 @@ mem_set_invalid(struct Mem *mem)
mem->flags = MEM_Undefined;
}
+void
+mem_set_ptr(struct Mem *mem, void *ptr)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Ptr;
+ mem->u.p = ptr;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -2241,14 +2249,6 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
return 0;
}
-void
-mem_set_ptr(struct Mem *mem, void *ptr)
-{
- mem_destroy(mem);
- mem->flags = MEM_Ptr;
- mem->u.p = ptr;
-}
-
/*
* Free an sql_value object
*/
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index fe3d8d98f..ae5d78ecb 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -356,6 +356,10 @@ mem_set_array_allocated(struct Mem *mem, char *value, uint32_t size);
void
mem_set_invalid(struct Mem *mem);
+/** Clear MEM and set pointer to be its value. */
+void
+mem_set_ptr(struct Mem *mem, void *ptr);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
@@ -625,14 +629,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
-/**
- * Set VDBE memory register with given pointer as a data.
- * @param mem VDBE memory register to update.
- * @param ptr Pointer to use.
- */
-void
-mem_set_ptr(struct Mem *mem, void *ptr);
-
void sqlValueFree(struct Mem *);
struct Mem *sqlValueNew(struct sql *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 0031d2248..c21a6576f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2485,8 +2485,7 @@ case OP_OpenTEphemeral: {
if (space == NULL)
goto abort_due_to_error;
- aMem[pOp->p1].u.p = space;
- aMem[pOp->p1].flags = MEM_Ptr;
+ mem_set_ptr(&aMem[pOp->p1], space);
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (33 preceding siblings ...)
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 34/52] sql: refactor mem_set_ptr() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:05 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg() Mergen Imeev via Tarantool-patches
` (6 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:05 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answer and new patch below.
On 30.03.2021 02:06, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 3701741ef..078de0e62 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -604,6 +604,15 @@ mem_set_pointer(struct Mem *mem, void *ptr)
>> mem->field_type = field_type_MAX;
>> }
>>
>> +void
>> +mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
>> +{
>> + mem_clear(mem);
>> + mem->flags = MEM_Frame;
>> + mem->u.pFrame = frame;
>> + mem->field_type = field_type_MAX;
>
> The type is already installed by mem_clear().
Fixed.
New patch:
commit e88babfbe7dc13059b3bfe4d2d721f5a37f8fb7c
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 14:40:37 2021 +0300
sql: introduce mem_set_frame()
This patch introduces mem_set_frame() function. This function clears the
MEM and sets a frame to MEM. Frames used for internal VDBE operations.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 4091161ce..b2598816d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -604,6 +604,14 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Frame;
+ mem->u.pFrame = frame;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index ae5d78ecb..ffab0d616 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -360,6 +360,10 @@ mem_set_invalid(struct Mem *mem);
void
mem_set_ptr(struct Mem *mem, void *ptr);
+/** Clear MEM and set frame to be its value. */
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index c21a6576f..7f1e0bcbe 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4163,8 +4163,7 @@ case OP_Program: { /* jump */
goto no_mem;
}
mem_destroy(pRt);
- pRt->flags = MEM_Frame;
- pRt->u.pFrame = pFrame;
+ mem_set_frame(pRt, pFrame);
pFrame->v = p;
pFrame->nChildMem = nMem;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame()
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:19 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:37 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Good job on the patch!
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index c21a6576f..7f1e0bcbe 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -4163,8 +4163,7 @@ case OP_Program: { /* jump */
> goto no_mem;
> }
> mem_destroy(pRt);
You can remove this destroy call.
> - pRt->flags = MEM_Frame;
> - pRt->u.pFrame = pFrame;
> + mem_set_frame(pRt, pFrame);
>
> pFrame->v = p;
> pFrame->nChildMem = nMem;
>
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame()
2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:19 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:19 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Tue, Apr 13, 2021 at 01:37:08AM +0200, Vladislav Shpilevoy wrote:
> Good job on the patch!
>
> > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > index c21a6576f..7f1e0bcbe 100644
> > --- a/src/box/sql/vdbe.c
> > +++ b/src/box/sql/vdbe.c
> > @@ -4163,8 +4163,7 @@ case OP_Program: { /* jump */
> > goto no_mem;
> > }
> > mem_destroy(pRt);
>
> You can remove this destroy call.
>
Fixed.
> > - pRt->flags = MEM_Frame;
> > - pRt->u.pFrame = pFrame;
> > + mem_set_frame(pRt, pFrame);
> >
> > pFrame->v = p;
> > pFrame->nChildMem = nMem;
> >
Diff:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7a3170614..d5b8033a4 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4206,7 +4206,6 @@ case OP_Program: { /* jump */
if (!pFrame) {
goto no_mem;
}
- mem_destroy(pRt);
mem_set_frame(pRt, pFrame);
pFrame->v = p;
New patch:
commit 24a0ee0f12e15b8504c04ea34d973bcfc6a922a1
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 14:40:37 2021 +0300
sql: introduce mem_set_frame()
This patch introduces mem_set_frame() function. This function clears the
MEM and sets a frame to MEM. Frames used for internal VDBE operations.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 6c5c4f391..8e13131f1 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -461,6 +461,14 @@ mem_set_ptr(struct Mem *mem, void *ptr)
mem->u.p = ptr;
}
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Frame;
+ mem->u.pFrame = frame;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f4cdbafd2..36b7d3eca 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -542,6 +542,10 @@ mem_set_invalid(struct Mem *mem);
void
mem_set_ptr(struct Mem *mem, void *ptr);
+/** Clear MEM and set frame to be its value. */
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e3d7456f3..d5b8033a4 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4206,9 +4206,7 @@ case OP_Program: { /* jump */
if (!pFrame) {
goto no_mem;
}
- mem_destroy(pRt);
- pRt->flags = MEM_Frame;
- pRt->u.pFrame = pFrame;
+ mem_set_frame(pRt, pFrame);
pFrame->v = p;
pFrame->nChildMem = nMem;
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (34 preceding siblings ...)
2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:25 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear() Mergen Imeev via Tarantool-patches
` (5 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:25 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:06, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 078de0e62..0211069c6 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -613,6 +613,29 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
>> mem->field_type = field_type_MAX;
>> }
>>
>> +int
>> +mem_prepare_aggregate(struct Mem *mem, struct func *func, int size)
>
> 1. Why is this called 'prepare' when the other setters are called 'set'?
> Maybe use 'set' here as well?
>
Fixed.
>> +{
>> + if (size <= 0) {
>> + mem_clear(mem);
>> + return 0;
>> + }
>> + if (sqlVdbeMemGrow(mem, size, 0) != 0)
>> + return -1;
>> + memset(mem->z, 0, size);
>> + mem->n = size;
>> + mem->flags = MEM_Agg;
>> + mem->u.func = func;
>> + mem->field_type = field_type_MAX;
>> + return 0;
>> +}
>> +
>> +void *
>> +mem_get_aggregate(struct Mem *mem)
>> +{
>> + return (void *)mem->z;
>
> 2. Void cast should work implicitly here I think. But what is
> more interesting is why do you even need the getter? The other
> mem types don't have getters for the union fields, and it is
> fine I suppose. Would be too much. This one does not even check
> if the type is an aggregate.
I moved this function from here to patch "sql: introduce mem_get_agg()". I check
MEM type there, but almost nothing else changed. This is mostly so because I
actually have no idea how to deal with all aggregate functions used standard
rules for MEM. Maybe it is worth to think about moving aggregate functions,
MEM_Ptr, MEM_Frame, etc from MEM to another internal structure? This way we will
be able to use MEMs only for data.
New patch:
commit 200616f9ee5708f14c565046e2e01eacbef2be62
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 15:02:08 2021 +0300
sql: introduce mem_set_agg()
This patch introduces mem_set_agg() function. This functions stores
given functions to MEM and allocates enough memory to hold accumulation
structure for aggregate function.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index b2598816d..af11ae1d5 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -612,6 +612,23 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
mem->u.pFrame = frame;
}
+int
+mem_set_agg(struct Mem *mem, struct func *func, int size)
+{
+ if (size <= 0) {
+ mem_clear(mem);
+ return 0;
+ }
+ if (sqlVdbeMemGrow(mem, size, 0) != 0)
+ return -1;
+ memset(mem->z, 0, size);
+ mem->n = size;
+ mem->flags = MEM_Agg;
+ mem->u.func = func;
+ mem->field_type = field_type_MAX;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index ffab0d616..cf0db62f9 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -364,6 +364,13 @@ mem_set_ptr(struct Mem *mem, void *ptr);
void
mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
+/**
+ * Clear the MEM, set the function as its value, and allocate enough memory to
+ * hold the accumulation structure for the aggregate function.
+ */
+int
+mem_set_agg(struct Mem *mem, struct func *func, int size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 6d9103ff2..c2d4b8b8a 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -403,29 +403,6 @@ sqlStmtCurrentTime(sql_context * p)
return *piTime;
}
-/*
- * Create a new aggregate context for p and return a pointer to
- * its pMem->z element.
- */
-static SQL_NOINLINE void *
-createAggContext(sql_context * p, int nByte)
-{
- Mem *pMem = p->pMem;
- assert(!mem_is_agg(pMem));
- if (nByte <= 0) {
- mem_set_null(pMem);
- pMem->z = 0;
- } else {
- sqlVdbeMemClearAndResize(pMem, nByte);
- pMem->flags = MEM_Agg;
- pMem->u.func = p->func;
- if (pMem->z) {
- memset(pMem->z, 0, nByte);
- }
- }
- return (void *)pMem->z;
-}
-
/*
* Allocate or return the aggregate context for a user function. A new
* context is allocated on the first call. Subsequent calls return the
@@ -437,12 +414,9 @@ sql_aggregate_context(sql_context * p, int nByte)
assert(p != NULL && p->func != NULL);
assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
- testcase(nByte < 0);
- if (!mem_is_agg(p->pMem)) {
- return createAggContext(p, nByte);
- } else {
- return (void *)p->pMem->z;
- }
+ if (!mem_is_agg(p->pMem) && mem_set_agg(p->pMem, p->func, nByte) != 0)
+ return NULL;
+ return p->pMem->z;
}
/*
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg()
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:46 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:37 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Thanks for the fixes!
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index b2598816d..af11ae1d5 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -612,6 +612,23 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
> mem->u.pFrame = frame;
> }
>
> +int
> +mem_set_agg(struct Mem *mem, struct func *func, int size)
> +{
> + if (size <= 0) {
> + mem_clear(mem);
> + return 0;
> + }
> + if (sqlVdbeMemGrow(mem, size, 0) != 0)
> + return -1;
> + memset(mem->z, 0, size);
> + mem->n = size;
> + mem->flags = MEM_Agg;
What if it already was MEM_Agg before? sqlVdbeMemGrow()
didn't clear the old value. It clears only MEM_Dyn.
> + mem->u.func = func;
> + mem->field_type = field_type_MAX;
> + return 0;
> +}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg()
2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:46 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:46 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Tue, Apr 13, 2021 at 01:37:49AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
>
> > diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> > index b2598816d..af11ae1d5 100644
> > --- a/src/box/sql/mem.c
> > +++ b/src/box/sql/mem.c
> > @@ -612,6 +612,23 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
> > mem->u.pFrame = frame;
> > }
> >
> > +int
> > +mem_set_agg(struct Mem *mem, struct func *func, int size)
> > +{
> > + if (size <= 0) {
> > + mem_clear(mem);
> > + return 0;
> > + }
> > + if (sqlVdbeMemGrow(mem, size, 0) != 0)
> > + return -1;
> > + memset(mem->z, 0, size);
> > + mem->n = size;
> > + mem->flags = MEM_Agg;
>
> What if it already was MEM_Agg before? sqlVdbeMemGrow()
> didn't clear the old value. It clears only MEM_Dyn.
>
You are right. Added mem_clear().
> > + mem->u.func = func;
> > + mem->field_type = field_type_MAX;
> > + return 0;
> > +}
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index fdbd15b46..172883a44 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -472,10 +472,9 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
int
mem_set_agg(struct Mem *mem, struct func *func, int size)
{
- if (size <= 0) {
- mem_clear(mem);
+ mem_clear(mem);
+ if (size <= 0)
return 0;
- }
if (sqlVdbeMemGrow(mem, size, 0) != 0)
return -1;
memset(mem->z, 0, size);
New patch:
commit f8490d6176d02e144e1e3241b04c2c09f8390e78
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 15:02:08 2021 +0300
sql: introduce mem_set_agg()
This patch introduces mem_set_agg() function. This function stores
aggregation function to MEM and allocates enough memory to hold
accumulation structure for aggregate function.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 8e13131f1..172883a44 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -469,6 +469,22 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
mem->u.pFrame = frame;
}
+int
+mem_set_agg(struct Mem *mem, struct func *func, int size)
+{
+ mem_clear(mem);
+ if (size <= 0)
+ return 0;
+ if (sqlVdbeMemGrow(mem, size, 0) != 0)
+ return -1;
+ memset(mem->z, 0, size);
+ mem->n = size;
+ mem->flags = MEM_Agg;
+ mem->u.func = func;
+ mem->field_type = field_type_MAX;
+ return 0;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 36b7d3eca..6dee83dc5 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -546,6 +546,13 @@ mem_set_ptr(struct Mem *mem, void *ptr);
void
mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
+/**
+ * Clear the MEM, set the function as its value, and allocate enough memory to
+ * hold the accumulation structure for the aggregate function.
+ */
+int
+mem_set_agg(struct Mem *mem, struct func *func, int size);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 65927910a..af1174d0a 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -383,29 +383,6 @@ sqlStmtCurrentTime(sql_context * p)
return *piTime;
}
-/*
- * Create a new aggregate context for p and return a pointer to
- * its pMem->z element.
- */
-static SQL_NOINLINE void *
-createAggContext(sql_context * p, int nByte)
-{
- Mem *pMem = p->pMem;
- assert(!mem_is_agg(pMem));
- if (nByte <= 0) {
- mem_set_null(pMem);
- pMem->z = 0;
- } else {
- sqlVdbeMemClearAndResize(pMem, nByte);
- pMem->flags = MEM_Agg;
- pMem->u.func = p->func;
- if (pMem->z) {
- memset(pMem->z, 0, nByte);
- }
- }
- return (void *)pMem->z;
-}
-
/*
* Allocate or return the aggregate context for a user function. A new
* context is allocated on the first call. Subsequent calls return the
@@ -417,12 +394,9 @@ sql_aggregate_context(sql_context * p, int nByte)
assert(p != NULL && p->func != NULL);
assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
- testcase(nByte < 0);
- if (!mem_is_agg(p->pMem)) {
- return createAggContext(p, nByte);
- } else {
- return (void *)p->pMem->z;
- }
+ if (!mem_is_agg(p->pMem) && mem_set_agg(p->pMem, p->func, nByte) != 0)
+ return NULL;
+ return p->pMem->z;
}
/*
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (35 preceding siblings ...)
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:25 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:38 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
` (4 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:25 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:07, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> I propose to rename that to mem_set_null_clear(). Because
> it is a NULL, but with special semantics.
Done.
New patch:
commit 05413fe121d62c5163593436df9fc8f1a7d2c295
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 15:14:32 2021 +0300
sql: introduce mem_set_null_clear()
This patch introduces mem_set_null_clear() function. This function sets
"cleared" NULL to MEM. This NULL is used only in internal VDBE
operations.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index af11ae1d5..2e147291f 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -629,6 +629,13 @@ mem_set_agg(struct Mem *mem, struct func *func, int size)
return 0;
}
+void
+mem_set_null_clear(struct Mem *mem)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Null | MEM_Cleared;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index cf0db62f9..f17c4bb78 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -371,6 +371,10 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
int
mem_set_agg(struct Mem *mem, struct func *func, int size);
+/** Clear MEM and set it to special, "cleared", NULL. */
+void
+mem_set_null_clear(struct Mem *mem);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 7f1e0bcbe..4566606d7 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -855,7 +855,7 @@ case OP_String: { /* out2 */
* is less than P2 (typically P3 is zero) then only register P2 is
* set to NULL.
*
- * If the P1 value is non-zero, then also set the MEM_Cleared flag so that
+ * If the P1 value is non-zero, then also set the Cleared flag so that
* NULL values will not compare equal even if SQL_NULLEQ is set on
* OP_Ne or OP_Eq.
*/
@@ -865,13 +865,14 @@ case OP_Null: { /* out2 */
cnt = pOp->p3-pOp->p2;
assert(pOp->p3<=(p->nMem+1 - p->nCursor));
if (pOp->p1 != 0)
- pOut->flags = MEM_Null | MEM_Cleared;
+ mem_set_null_clear(pOut);
while( cnt>0) {
pOut++;
memAboutToChange(p, pOut);
- mem_set_null(pOut);
if (pOp->p1 != 0)
- pOut->flags = MEM_Null | MEM_Cleared;
+ mem_set_null_clear(pOut);
+ else
+ mem_set_null(pOut);
cnt--;
}
break;
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear()
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear() Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:38 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:50 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:38 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Nice fixes!
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 7f1e0bcbe..4566606d7 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -855,7 +855,7 @@ case OP_String: { /* out2 */
> * is less than P2 (typically P3 is zero) then only register P2 is
> * set to NULL.
> *
> - * If the P1 value is non-zero, then also set the MEM_Cleared flag so that
> + * If the P1 value is non-zero, then also set the Cleared flag so that
The flag is still here. If you didn't remove it, I would propose to
keep its mentions in the comments.
> * NULL values will not compare equal even if SQL_NULLEQ is set on
> * OP_Ne or OP_Eq.
> */
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear()
2021-04-12 23:38 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:50 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:50 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answer, diff and new patch below.
On Tue, Apr 13, 2021 at 01:38:18AM +0200, Vladislav Shpilevoy wrote:
> Nice fixes!
>
> > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > index 7f1e0bcbe..4566606d7 100644
> > --- a/src/box/sql/vdbe.c
> > +++ b/src/box/sql/vdbe.c
> > @@ -855,7 +855,7 @@ case OP_String: { /* out2 */
> > * is less than P2 (typically P3 is zero) then only register P2 is
> > * set to NULL.
> > *
> > - * If the P1 value is non-zero, then also set the MEM_Cleared flag so that
> > + * If the P1 value is non-zero, then also set the Cleared flag so that
>
> The flag is still here. If you didn't remove it, I would propose to
> keep its mentions in the comments.
>
Fixed, dropped this change.
> > * NULL values will not compare equal even if SQL_NULLEQ is set on
> > * OP_Ne or OP_Eq.
> > */
Diff:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 116194fae..e07ff1be9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -855,7 +855,7 @@ case OP_String: { /* out2 */
* is less than P2 (typically P3 is zero) then only register P2 is
* set to NULL.
*
- * If the P1 value is non-zero, then also set the Cleared flag so that
+ * If the P1 value is non-zero, then also set the MEM_Cleared flag so that
* NULL values will not compare equal even if SQL_NULLEQ is set on
* OP_Ne or OP_Eq.
*/
New patch:
commit 56565414c4f332d5711d458b3896016d972c1f9c
Author: Mergen Imeev <imeevma@gmail.com>
Date: Tue Mar 16 15:14:32 2021 +0300
sql: introduce mem_set_null_clear()
This patch introduces mem_set_null_clear() function. This function sets
"cleared" NULL to MEM. This NULL is used only in internal VDBE
operations.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 172883a44..3930067e1 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -485,6 +485,13 @@ mem_set_agg(struct Mem *mem, struct func *func, int size)
return 0;
}
+void
+mem_set_null_clear(struct Mem *mem)
+{
+ mem_clear(mem);
+ mem->flags = MEM_Null | MEM_Cleared;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 6dee83dc5..9890762e0 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -553,6 +553,10 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
int
mem_set_agg(struct Mem *mem, struct func *func, int size);
+/** Clear MEM and set it to special, "cleared", NULL. */
+void
+mem_set_null_clear(struct Mem *mem);
+
/**
* Copy content of MEM from one MEM to another. In case source MEM contains
* string or binary and allocation type is not STATIC, this value is copied to
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index d5b8033a4..e07ff1be9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -865,13 +865,14 @@ case OP_Null: { /* out2 */
cnt = pOp->p3-pOp->p2;
assert(pOp->p3<=(p->nMem+1 - p->nCursor));
if (pOp->p1 != 0)
- pOut->flags = MEM_Null | MEM_Cleared;
+ mem_set_null_clear(pOut);
while( cnt>0) {
pOut++;
memAboutToChange(p, pOut);
- mem_set_null(pOut);
if (pOp->p1 != 0)
- pOut->flags = MEM_Null | MEM_Cleared;
+ mem_set_null_clear(pOut);
+ else
+ mem_set_null(pOut);
cnt--;
}
break;
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (36 preceding siblings ...)
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear() Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:25 ` Mergen Imeev via Tarantool-patches
2021-04-13 20:42 ` Mergen Imeev via Tarantool-patches
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions Mergen Imeev via Tarantool-patches
` (3 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:25 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:07, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
>> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
>> index 70224b55a..8b6f6749d 100644
>> --- a/src/box/sql/mem.h
>> +++ b/src/box/sql/mem.h
>> @@ -563,13 +511,6 @@ columnNullValue(void);
>>
>> int sqlVdbeMemTooBig(Mem *);
>>
>> -/* Return TRUE if Mem X contains dynamically allocated content - anything
>> - * that needs to be deallocated to avoid a leak.
>> - */
>> -#define VdbeMemDynamic(X) \
>> - (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
>
> Why did you remove that? And why don't you have MEM_Agg|MEM_Frame in
> mem_is_dynamic()?
Function mem_is_dynamic() shows allocation type of MEM contains STRING or
VARBINARY. Macro VdbeMemDynamic() shows that MEM should be cleared before
changing. This macro is not needed now outside of mem.c since all settings there
should be done using mem_set_*() functions, which clear MEM before changing.
In mem.c this macro may be added in case we set MEM directly, however we inline
mem_set_*() only in cases when we know that there is no clearing needed. Not
sure that this macro is needed after all.
There is one place where MEM is set without mem_set_*() outside of mem.c - in
function allocateCursor(). However, the way MEM is used here is so different
from normal way of using MEM, that I am not sure that VdbeMemDynamic() is
actually proper there.
New patch:
commit 670c9835e020f49a67ee8ade81fbf59dd0062beb
Author: Mergen Imeev <imeevma@gmail.com>
Date: Wed Mar 17 10:19:57 2021 +0300
sql: move MEM flags to mem.c
This patch moves MEM flags to mem.c. This allow us to have more control
over MEM state.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 2e147291f..52b1891aa 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -52,6 +52,57 @@
static int
sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
+/* One or more of the following flags are set to indicate the validOK
+ * representations of the value stored in the Mem struct.
+ *
+ * If the MEM_Null flag is set, then the value is an SQL NULL value.
+ * No other flags may be set in this case.
+ *
+ * If the MEM_Str flag is set then Mem.z points at a string representation.
+ * Usually this is encoded in the same unicode encoding as the main
+ * database (see below for exceptions). If the MEM_Term flag is also
+ * set, then the string is nul terminated. The MEM_Int and MEM_Real
+ * flags may coexist with the MEM_Str flag.
+ */
+#define MEM_Null 0x0001 /* Value is NULL */
+#define MEM_Str 0x0002 /* Value is a string */
+#define MEM_Int 0x0004 /* Value is an integer */
+#define MEM_Real 0x0008 /* Value is a real number */
+#define MEM_Blob 0x0010 /* Value is a BLOB */
+#define MEM_Bool 0x0020 /* Value is a bool */
+#define MEM_UInt 0x0040 /* Value is an unsigned integer */
+#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
+#define MEM_Undefined 0x0100 /* Value is undefined */
+#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
+#define MEM_TypeMask 0x83ff /* Mask of type bits */
+
+/* Whenever Mem contains a valid string or blob representation, one of
+ * the following flags must be set to determine the memory management
+ * policy for Mem.z. The MEM_Term flag tells us whether or not the
+ * string is \000 or \u0000 terminated
+ */
+#define MEM_Term 0x0400 /* String rep is nul terminated */
+#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
+#define MEM_Static 0x1000 /* Mem.z points to a static string */
+#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
+#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
+#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
+#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
+#define MEM_Ptr 0x20000 /* Value is a generic pointer */
+
+/**
+ * In contrast to Mem_TypeMask, this one allows to get
+ * pure type of memory cell, i.e. without _Dyn/_Zero and other
+ * auxiliary flags.
+ */
+enum {
+ MEM_PURE_TYPE_MASK = 0x7f
+};
+
+static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
+ MEM_Blob | MEM_Bool | MEM_UInt),
+ "value of type mask must consist of corresponding to memory "\
+ "type bits");
bool
mem_is_null(const struct Mem *mem)
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f17c4bb78..ce5076361 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -473,58 +473,6 @@ int
mem_compare(const struct Mem *left, const struct Mem *right, int *result,
enum field_type type, struct coll *coll);
-/* One or more of the following flags are set to indicate the validOK
- * representations of the value stored in the Mem struct.
- *
- * If the MEM_Null flag is set, then the value is an SQL NULL value.
- * No other flags may be set in this case.
- *
- * If the MEM_Str flag is set then Mem.z points at a string representation.
- * Usually this is encoded in the same unicode encoding as the main
- * database (see below for exceptions). If the MEM_Term flag is also
- * set, then the string is nul terminated. The MEM_Int and MEM_Real
- * flags may coexist with the MEM_Str flag.
- */
-#define MEM_Null 0x0001 /* Value is NULL */
-#define MEM_Str 0x0002 /* Value is a string */
-#define MEM_Int 0x0004 /* Value is an integer */
-#define MEM_Real 0x0008 /* Value is a real number */
-#define MEM_Blob 0x0010 /* Value is a BLOB */
-#define MEM_Bool 0x0020 /* Value is a bool */
-#define MEM_UInt 0x0040 /* Value is an unsigned integer */
-#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
-#define MEM_Undefined 0x0100 /* Value is undefined */
-#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
-#define MEM_TypeMask 0x83ff /* Mask of type bits */
-
-/* Whenever Mem contains a valid string or blob representation, one of
- * the following flags must be set to determine the memory management
- * policy for Mem.z. The MEM_Term flag tells us whether or not the
- * string is \000 or \u0000 terminated
- */
-#define MEM_Term 0x0400 /* String rep is nul terminated */
-#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
-#define MEM_Static 0x1000 /* Mem.z points to a static string */
-#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
-#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
-#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
-#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
-#define MEM_Ptr 0x20000 /* Value is a generic pointer */
-
-/**
- * In contrast to Mem_TypeMask, this one allows to get
- * pure type of memory cell, i.e. without _Dyn/_Zero and other
- * auxiliary flags.
- */
-enum {
- MEM_PURE_TYPE_MASK = 0x7f
-};
-
-static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
- MEM_Blob | MEM_Bool | MEM_UInt),
- "value of type mask must consist of corresponding to memory "\
- "type bits");
-
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -555,7 +503,7 @@ registerTrace(int iReg, Mem *p);
* Return true if a memory cell is not marked as invalid. This macro
* is for use inside assert() statements only.
*/
-#define memIsValid(M) ((M)->flags & MEM_Undefined)==0
+#define memIsValid(M) !mem_is_invalid(M)
#endif
/**
@@ -589,7 +537,7 @@ int mem_apply_integer_type(struct Mem *);
int sqlVdbeMemStringify(struct Mem *);
int sqlVdbeMemNulTerminate(struct Mem *);
int sqlVdbeMemExpandBlob(struct Mem *);
-#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
+#define ExpandBlob(P) (mem_is_zerobin(P)? sqlVdbeMemExpandBlob(P) : 0)
void sql_value_apply_type(struct Mem *val, enum field_type type);
@@ -700,13 +648,6 @@ columnNullValue(void);
int sqlVdbeMemTooBig(Mem *);
-/* Return TRUE if Mem X contains dynamically allocated content - anything
- * that needs to be deallocated to avoid a leak.
- */
-#define VdbeMemDynamic(X) \
- (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
-
-
int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
/**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4566606d7..71a827034 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -607,7 +607,6 @@ case OP_SetDiag: { /* jump */
case OP_Gosub: { /* jump */
assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
pIn1 = &aMem[pOp->p1];
- assert(VdbeMemDynamic(pIn1)==0);
memAboutToChange(p, pIn1);
mem_set_uint(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
@@ -649,7 +648,6 @@ case OP_InitCoroutine: { /* jump */
assert(pOp->p2>=0 && pOp->p2<p->nOp);
assert(pOp->p3>0 && pOp->p3<p->nOp);
pOut = &aMem[pOp->p1];
- assert(!VdbeMemDynamic(pOut));
mem_set_uint(pOut, pOp->p3 - 1);
if (pOp->p2) goto jump_to_p2;
break;
@@ -691,7 +689,6 @@ case OP_EndCoroutine: { /* in1 */
*/
case OP_Yield: { /* in1, jump */
pIn1 = &aMem[pOp->p1];
- assert(VdbeMemDynamic(pIn1)==0);
int pcDest = (int)pIn1->u.u;
mem_set_uint(pIn1, pOp - aOp);
REGISTER_TRACE(p, pOp->p1, pIn1);
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
@ 2021-04-13 20:42 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 20:42 UTC (permalink / raw)
To: v.shpilevoy, tsafin, tarantool-patches
On new branch I dropped this commit. Please see my answer to
"sql: introduce mem_is_*() functions()".
On Fri, Apr 09, 2021 at 11:25:55PM +0300, Mergen Imeev via Tarantool-patches wrote:
> Thank you for the review! My answers and new patch below.
>
>
> On 30.03.2021 02:07, Vladislav Shpilevoy wrote:
> > Thanks for the patch!
> >
> >> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> >> index 70224b55a..8b6f6749d 100644
> >> --- a/src/box/sql/mem.h
> >> +++ b/src/box/sql/mem.h
> >> @@ -563,13 +511,6 @@ columnNullValue(void);
> >>
> >> int sqlVdbeMemTooBig(Mem *);
> >>
> >> -/* Return TRUE if Mem X contains dynamically allocated content - anything
> >> - * that needs to be deallocated to avoid a leak.
> >> - */
> >> -#define VdbeMemDynamic(X) \
> >> - (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
> >
> > Why did you remove that? And why don't you have MEM_Agg|MEM_Frame in
> > mem_is_dynamic()?
> Function mem_is_dynamic() shows allocation type of MEM contains STRING or
> VARBINARY. Macro VdbeMemDynamic() shows that MEM should be cleared before
> changing. This macro is not needed now outside of mem.c since all settings there
> should be done using mem_set_*() functions, which clear MEM before changing.
> In mem.c this macro may be added in case we set MEM directly, however we inline
> mem_set_*() only in cases when we know that there is no clearing needed. Not
> sure that this macro is needed after all.
>
> There is one place where MEM is set without mem_set_*() outside of mem.c - in
> function allocateCursor(). However, the way MEM is used here is so different
> from normal way of using MEM, that I am not sure that VdbeMemDynamic() is
> actually proper there.
>
>
> New patch:
>
> commit 670c9835e020f49a67ee8ade81fbf59dd0062beb
> Author: Mergen Imeev <imeevma@gmail.com>
> Date: Wed Mar 17 10:19:57 2021 +0300
>
> sql: move MEM flags to mem.c
>
> This patch moves MEM flags to mem.c. This allow us to have more control
> over MEM state.
>
> Part of #5818
>
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 2e147291f..52b1891aa 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -52,6 +52,57 @@
> static int
> sqlVdbeMemGrow(struct Mem *pMem, int n, int preserve);
>
> +/* One or more of the following flags are set to indicate the validOK
> + * representations of the value stored in the Mem struct.
> + *
> + * If the MEM_Null flag is set, then the value is an SQL NULL value.
> + * No other flags may be set in this case.
> + *
> + * If the MEM_Str flag is set then Mem.z points at a string representation.
> + * Usually this is encoded in the same unicode encoding as the main
> + * database (see below for exceptions). If the MEM_Term flag is also
> + * set, then the string is nul terminated. The MEM_Int and MEM_Real
> + * flags may coexist with the MEM_Str flag.
> + */
> +#define MEM_Null 0x0001 /* Value is NULL */
> +#define MEM_Str 0x0002 /* Value is a string */
> +#define MEM_Int 0x0004 /* Value is an integer */
> +#define MEM_Real 0x0008 /* Value is a real number */
> +#define MEM_Blob 0x0010 /* Value is a BLOB */
> +#define MEM_Bool 0x0020 /* Value is a bool */
> +#define MEM_UInt 0x0040 /* Value is an unsigned integer */
> +#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
> +#define MEM_Undefined 0x0100 /* Value is undefined */
> +#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
> +#define MEM_TypeMask 0x83ff /* Mask of type bits */
> +
> +/* Whenever Mem contains a valid string or blob representation, one of
> + * the following flags must be set to determine the memory management
> + * policy for Mem.z. The MEM_Term flag tells us whether or not the
> + * string is \000 or \u0000 terminated
> + */
> +#define MEM_Term 0x0400 /* String rep is nul terminated */
> +#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
> +#define MEM_Static 0x1000 /* Mem.z points to a static string */
> +#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
> +#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
> +#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
> +#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
> +#define MEM_Ptr 0x20000 /* Value is a generic pointer */
> +
> +/**
> + * In contrast to Mem_TypeMask, this one allows to get
> + * pure type of memory cell, i.e. without _Dyn/_Zero and other
> + * auxiliary flags.
> + */
> +enum {
> + MEM_PURE_TYPE_MASK = 0x7f
> +};
> +
> +static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
> + MEM_Blob | MEM_Bool | MEM_UInt),
> + "value of type mask must consist of corresponding to memory "\
> + "type bits");
>
> bool
> mem_is_null(const struct Mem *mem)
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index f17c4bb78..ce5076361 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -473,58 +473,6 @@ int
> mem_compare(const struct Mem *left, const struct Mem *right, int *result,
> enum field_type type, struct coll *coll);
>
> -/* One or more of the following flags are set to indicate the validOK
> - * representations of the value stored in the Mem struct.
> - *
> - * If the MEM_Null flag is set, then the value is an SQL NULL value.
> - * No other flags may be set in this case.
> - *
> - * If the MEM_Str flag is set then Mem.z points at a string representation.
> - * Usually this is encoded in the same unicode encoding as the main
> - * database (see below for exceptions). If the MEM_Term flag is also
> - * set, then the string is nul terminated. The MEM_Int and MEM_Real
> - * flags may coexist with the MEM_Str flag.
> - */
> -#define MEM_Null 0x0001 /* Value is NULL */
> -#define MEM_Str 0x0002 /* Value is a string */
> -#define MEM_Int 0x0004 /* Value is an integer */
> -#define MEM_Real 0x0008 /* Value is a real number */
> -#define MEM_Blob 0x0010 /* Value is a BLOB */
> -#define MEM_Bool 0x0020 /* Value is a bool */
> -#define MEM_UInt 0x0040 /* Value is an unsigned integer */
> -#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
> -#define MEM_Undefined 0x0100 /* Value is undefined */
> -#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
> -#define MEM_TypeMask 0x83ff /* Mask of type bits */
> -
> -/* Whenever Mem contains a valid string or blob representation, one of
> - * the following flags must be set to determine the memory management
> - * policy for Mem.z. The MEM_Term flag tells us whether or not the
> - * string is \000 or \u0000 terminated
> - */
> -#define MEM_Term 0x0400 /* String rep is nul terminated */
> -#define MEM_Dyn 0x0800 /* Need to call Mem.xDel() on Mem.z */
> -#define MEM_Static 0x1000 /* Mem.z points to a static string */
> -#define MEM_Ephem 0x2000 /* Mem.z points to an ephemeral string */
> -#define MEM_Agg 0x4000 /* Mem.z points to an agg function context */
> -#define MEM_Zero 0x8000 /* Mem.i contains count of 0s appended to blob */
> -#define MEM_Subtype 0x10000 /* Mem.eSubtype is valid */
> -#define MEM_Ptr 0x20000 /* Value is a generic pointer */
> -
> -/**
> - * In contrast to Mem_TypeMask, this one allows to get
> - * pure type of memory cell, i.e. without _Dyn/_Zero and other
> - * auxiliary flags.
> - */
> -enum {
> - MEM_PURE_TYPE_MASK = 0x7f
> -};
> -
> -static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
> - MEM_Blob | MEM_Bool | MEM_UInt),
> - "value of type mask must consist of corresponding to memory "\
> - "type bits");
> -
> /**
> * Simple type to str convertor. It is used to simplify
> * error reporting.
> @@ -555,7 +503,7 @@ registerTrace(int iReg, Mem *p);
> * Return true if a memory cell is not marked as invalid. This macro
> * is for use inside assert() statements only.
> */
> -#define memIsValid(M) ((M)->flags & MEM_Undefined)==0
> +#define memIsValid(M) !mem_is_invalid(M)
> #endif
>
> /**
> @@ -589,7 +537,7 @@ int mem_apply_integer_type(struct Mem *);
> int sqlVdbeMemStringify(struct Mem *);
> int sqlVdbeMemNulTerminate(struct Mem *);
> int sqlVdbeMemExpandBlob(struct Mem *);
> -#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
> +#define ExpandBlob(P) (mem_is_zerobin(P)? sqlVdbeMemExpandBlob(P) : 0)
> void sql_value_apply_type(struct Mem *val, enum field_type type);
>
>
> @@ -700,13 +648,6 @@ columnNullValue(void);
>
> int sqlVdbeMemTooBig(Mem *);
>
> -/* Return TRUE if Mem X contains dynamically allocated content - anything
> - * that needs to be deallocated to avoid a leak.
> - */
> -#define VdbeMemDynamic(X) \
> - (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
> -
> -
> int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
>
> /**
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 4566606d7..71a827034 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -607,7 +607,6 @@ case OP_SetDiag: { /* jump */
> case OP_Gosub: { /* jump */
> assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
> pIn1 = &aMem[pOp->p1];
> - assert(VdbeMemDynamic(pIn1)==0);
> memAboutToChange(p, pIn1);
> mem_set_uint(pIn1, pOp - aOp);
> REGISTER_TRACE(p, pOp->p1, pIn1);
> @@ -649,7 +648,6 @@ case OP_InitCoroutine: { /* jump */
> assert(pOp->p2>=0 && pOp->p2<p->nOp);
> assert(pOp->p3>0 && pOp->p3<p->nOp);
> pOut = &aMem[pOp->p1];
> - assert(!VdbeMemDynamic(pOut));
> mem_set_uint(pOut, pOp->p3 - 1);
> if (pOp->p2) goto jump_to_p2;
> break;
> @@ -691,7 +689,6 @@ case OP_EndCoroutine: { /* in1 */
> */
> case OP_Yield: { /* in1, jump */
> pIn1 = &aMem[pOp->p1];
> - assert(VdbeMemDynamic(pIn1)==0);
> int pcDest = (int)pIn1->u.u;
> mem_set_uint(pIn1, pOp - aOp);
> REGISTER_TRACE(p, pOp->p1, pIn1);
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (37 preceding siblings ...)
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:25 ` Mergen Imeev via Tarantool-patches
2021-04-12 23:39 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 20:26 ` [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double() Mergen Imeev via Tarantool-patches
` (2 subsequent siblings)
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:25 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
Thank you for the review! My answers and new patch below.
On 30.03.2021 02:07, Vladislav Shpilevoy wrote:
> Thanks for the patch!
>
> See 2 comments below.
>
> On 23.03.2021 10:36, imeevma@tarantool.org wrote:
>> This patch introduces mem_convert_to_integer() which is used to convert
>> a MEM to a MEM that contains integer value.
>>
>> Part of #5818
>> ---
>> src/box/sql/mem.c | 114 +++++++++++++++++++++++++++++----------------
>> src/box/sql/mem.h | 7 ++-
>> src/box/sql/vdbe.c | 19 ++++----
>> 3 files changed, 88 insertions(+), 52 deletions(-)
>>
>> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
>> index 1209df1ce..b9bcd3d3a 100644
>> --- a/src/box/sql/mem.c
>> +++ b/src/box/sql/mem.c
>> @@ -695,6 +695,80 @@ mem_set_cleared(struct Mem *mem)
>
> <...>
>
>> +
>> +static inline int
>> +mem_convert_double_to_integer_lossless(struct Mem *mem)
>
> 1. Lets use 'precise'. Here and in other places. And shorten
> the names. So it would be mem_convert_bool_to_int(),
> mem_convert_to_int(), etc.
>
Thanks. Fixed. I shortened names of the functions to mem_to_int(),
mem_to_double(), etc. Also, I shortened names of static functions to
double_to_int(), int_to_string(), etc. Replaces "lossless" by "precise".
>> +{
>> + double d = mem->u.r;
>> + if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
>> + mem_set_integer(mem, (int64_t)d, true);
>> + return 0;
>> + }
>> + if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
>> + mem_set_integer(mem, (int64_t)(uint64_t)d, false);
>
> 2. Isn't mem_set_unsigned() faster? Or we can't set FIELD_TYPE_UNSIGNED?
Inlined setting of MEM here and in some other places.
New patch:
commit 8b70c648c1b907b04859cfc8bcef94870057bf9f
Author: Mergen Imeev <imeevma@gmail.com>
Date: Wed Mar 17 10:46:29 2021 +0300
sql: introduce mem_to_int*() functions
This patch introduces mem_to_int() and mem_to_int_precise() functions.
These functions are used to convert a MEM to a MEM that contains
integer value. These functions defines the rules that are used during
convertion from values of all other types to INTEGER.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 52b1891aa..d3a3215bc 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -687,6 +687,90 @@ mem_set_null_clear(struct Mem *mem)
mem->flags = MEM_Null | MEM_Cleared;
}
+static inline int
+bytes_to_int(struct Mem *mem)
+{
+ bool is_neg;
+ int64_t i;
+ if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
+ return -1;
+ mem_set_int(mem, i, is_neg);
+ return 0;
+}
+
+static inline int
+double_to_int(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+double_to_int_precise(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+bool_to_int(struct Mem *mem)
+{
+ mem->u.u = (uint64_t)mem->u.b;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+}
+
+int
+mem_to_int(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int(mem);
+ if ((mem->flags & MEM_Bool) != 0)
+ return bool_to_int(mem);
+ return -1;
+}
+
+int
+mem_to_int_precise(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & MEM_Str) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int_precise(mem);
+ return -1;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1924,26 +2008,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
}
}
-/*
- * The MEM structure is already a MEM_Real. Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
- int rc;
- i64 ix;
- assert(pMem->flags & MEM_Real);
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
- pMem->u.i = ix;
- pMem->flags = pMem->u.r <= -1 ? MEM_Int : MEM_UInt;
- pMem->field_type = FIELD_TYPE_INTEGER;
- }
- return rc;
-}
-
/*
* Add MEM_Str to the set of representations for the given Mem. Numbers
* are converted using sql_snprintf(). Converting a BLOB to a string
@@ -2215,34 +2279,6 @@ mem_convert_to_unsigned(struct Mem *mem)
return 0;
}
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
- if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
- return 0;
- if ((mem->flags & MEM_Real) == 0)
- return -1;
- double d = mem->u.r;
- if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
- return -1;
- if (d < 0.) {
- mem->u.i = (int64_t)d;
- mem->flags = MEM_Int;
- mem->field_type = FIELD_TYPE_INTEGER;
- } else {
- mem->u.u = (uint64_t)d;
- mem->flags = MEM_UInt;
- mem->field_type = FIELD_TYPE_UNSIGNED;
- }
- return 0;
-}
-
int
mem_convert_to_numeric(struct Mem *mem, enum field_type type)
{
@@ -2253,7 +2289,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
if (type == FIELD_TYPE_UNSIGNED)
return mem_convert_to_unsigned(mem);
assert(type == FIELD_TYPE_INTEGER);
- return mem_convert_to_integer(mem);
+ return mem_to_int(mem);
}
static int
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index ce5076361..d3eb04c44 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -473,6 +473,24 @@ int
mem_compare(const struct Mem *left, const struct Mem *right, int *result,
enum field_type type, struct coll *coll);
+/**
+ * Convert the given MEM to INTEGER. This function and the function below define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer may result in loss of
+ * precision.
+ */
+int
+mem_to_int(struct Mem *mem);
+
+/**
+ * Convert the given MEM to INTEGER. This function and the function above define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer is only possible if it
+ * is lossless.
+ */
+int
+mem_to_int_precise(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -533,7 +551,6 @@ int
vdbe_mem_numerify(struct Mem *mem);
int sqlVdbeMemCast(struct Mem *, enum field_type type);
-int mem_apply_integer_type(struct Mem *);
int sqlVdbeMemStringify(struct Mem *);
int sqlVdbeMemNulTerminate(struct Mem *);
int sqlVdbeMemExpandBlob(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 71a827034..e61ad4251 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1432,17 +1432,14 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if (!mem_is_int(pIn1)) {
- mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if (!mem_is_int(pIn1)) {
- if (pOp->p2==0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "integer");
- goto abort_due_to_error;
- } else {
- goto jump_to_p2;
- }
- }
+ if (mem_is_int(pIn1))
+ break;
+ if (mem_to_int_precise(pIn1) != 0) {
+ if (pOp->p2 !=0 )
+ goto jump_to_p2;
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+ mem_str(pIn1), "integer");
+ goto abort_due_to_error;
}
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-12 23:39 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-13 22:58 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-12 23:39 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
Good job on the fixes!
This is the last email for today, I will continue the review of
the patchset tomorrow.
See 2 comments below.
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 71a827034..e61ad4251 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1432,17 +1432,14 @@ case OP_AddImm: { /* in1 */
> */
> case OP_MustBeInt: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - if (!mem_is_int(pIn1)) {
> - mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
> - if (!mem_is_int(pIn1)) {
> - if (pOp->p2==0) {
> - diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> - mem_str(pIn1), "integer");
> - goto abort_due_to_error;
> - } else {
> - goto jump_to_p2;
> - }
> - }
> + if (mem_is_int(pIn1))
1. The same check is done inside mem_to_int_precise(). You
don't need to do it manually.
> + break;
> + if (mem_to_int_precise(pIn1) != 0) {
> + if (pOp->p2 !=0 )
2. Whitespace should be before '0', not after.
> + goto jump_to_p2;
> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> + mem_str(pIn1), "integer");
> + goto abort_due_to_error;
> }
> break;
> }
>
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions
2021-04-12 23:39 ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-13 22:58 ` Mergen Imeev via Tarantool-patches
2021-04-13 23:10 ` Mergen Imeev via Tarantool-patches
0 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 22:58 UTC (permalink / raw)
To: Vladislav Shpilevoy; +Cc: tarantool-patches
Thank you for the review! My answers, diff and new patch below.
On Tue, Apr 13, 2021 at 01:39:01AM +0200, Vladislav Shpilevoy wrote:
> Good job on the fixes!
>
> This is the last email for today, I will continue the review of
> the patchset tomorrow.
>
> See 2 comments below.
>
> > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > index 71a827034..e61ad4251 100644
> > --- a/src/box/sql/vdbe.c
> > +++ b/src/box/sql/vdbe.c
> > @@ -1432,17 +1432,14 @@ case OP_AddImm: { /* in1 */
> > */
> > case OP_MustBeInt: { /* jump, in1 */
> > pIn1 = &aMem[pOp->p1];
> > - if (!mem_is_int(pIn1)) {
> > - mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
> > - if (!mem_is_int(pIn1)) {
> > - if (pOp->p2==0) {
> > - diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> > - mem_str(pIn1), "integer");
> > - goto abort_due_to_error;
> > - } else {
> > - goto jump_to_p2;
> > - }
> > - }
> > + if (mem_is_int(pIn1))
>
> 1. The same check is done inside mem_to_int_precise(). You
> don't need to do it manually.
>
Thanks, fixed.
> > + break;
> > + if (mem_to_int_precise(pIn1) != 0) {
> > + if (pOp->p2 !=0 )
>
> 2. Whitespace should be before '0', not after.
>
Fixed.
> > + goto jump_to_p2;
> > + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> > + mem_str(pIn1), "integer");
> > + goto abort_due_to_error;
> > }
> > break;
> > }
> >
Diff:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 061f77952..c908f13c0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1435,10 +1435,8 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if (mem_is_int(pIn1))
- break;
if (mem_to_int_precise(pIn1) != 0) {
- if (pOp->p2 !=0 )
+ if (pOp->p2 != 0)
goto jump_to_p2;
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
mem_str(pIn1), "integer");
New patch:
commit ecf63aa8a6887b3b1ac7a67a6b60fe3a62426ed9
Author: Mergen Imeev <imeevma@gmail.com>
Date: Wed Mar 17 10:46:29 2021 +0300
sql: introduce mem_to_int*() functions
This patch introduces mem_to_int() and mem_to_int_precise() functions.
These functions are used to convert a MEM to a MEM that contains
integer value. These functions defines the rules that are used during
convertion from values of all other types to INTEGER.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 3930067e1..fd68acd6e 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -492,6 +492,90 @@ mem_set_null_clear(struct Mem *mem)
mem->flags = MEM_Null | MEM_Cleared;
}
+static inline int
+bytes_to_int(struct Mem *mem)
+{
+ bool is_neg;
+ int64_t i;
+ if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
+ return -1;
+ mem_set_int(mem, i, is_neg);
+ return 0;
+}
+
+static inline int
+double_to_int(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+double_to_int_precise(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+bool_to_int(struct Mem *mem)
+{
+ mem->u.u = (uint64_t)mem->u.b;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+}
+
+int
+mem_to_int(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int(mem);
+ if ((mem->flags & MEM_Bool) != 0)
+ return bool_to_int(mem);
+ return -1;
+}
+
+int
+mem_to_int_precise(struct Mem *mem)
+{
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & MEM_Str) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int_precise(mem);
+ return -1;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1694,26 +1778,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
}
}
-/*
- * The MEM structure is already a MEM_Real. Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
- int rc;
- i64 ix;
- assert(pMem->flags & MEM_Real);
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
- pMem->u.i = ix;
- pMem->flags = pMem->u.r <= -1 ? MEM_Int : MEM_UInt;
- pMem->field_type = FIELD_TYPE_INTEGER;
- }
- return rc;
-}
-
/*
* Add MEM_Str to the set of representations for the given Mem. Numbers
* are converted using sql_snprintf(). Converting a BLOB to a string
@@ -1985,34 +2049,6 @@ mem_convert_to_unsigned(struct Mem *mem)
return 0;
}
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
- if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
- return 0;
- if ((mem->flags & MEM_Real) == 0)
- return -1;
- double d = mem->u.r;
- if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
- return -1;
- if (d < 0.) {
- mem->u.i = (int64_t)d;
- mem->flags = MEM_Int;
- mem->field_type = FIELD_TYPE_INTEGER;
- } else {
- mem->u.u = (uint64_t)d;
- mem->flags = MEM_UInt;
- mem->field_type = FIELD_TYPE_UNSIGNED;
- }
- return 0;
-}
-
int
mem_convert_to_numeric(struct Mem *mem, enum field_type type)
{
@@ -2023,7 +2059,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
if (type == FIELD_TYPE_UNSIGNED)
return mem_convert_to_unsigned(mem);
assert(type == FIELD_TYPE_INTEGER);
- return mem_convert_to_integer(mem);
+ return mem_to_int(mem);
}
static int
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9890762e0..5aabcc1fd 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -683,6 +683,24 @@ mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
int
mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
+/**
+ * Convert the given MEM to INTEGER. This function and the function below define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer may result in loss of
+ * precision.
+ */
+int
+mem_to_int(struct Mem *mem);
+
+/**
+ * Convert the given MEM to INTEGER. This function and the function above define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer is only possible if it
+ * is lossless.
+ */
+int
+mem_to_int_precise(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -743,7 +761,6 @@ int
vdbe_mem_numerify(struct Mem *mem);
int sqlVdbeMemCast(struct Mem *, enum field_type type);
-int mem_apply_integer_type(struct Mem *);
int sqlVdbeMemStringify(struct Mem *);
int sqlVdbeMemNulTerminate(struct Mem *);
int sqlVdbeMemExpandBlob(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e07ff1be9..c908f13c0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1435,17 +1435,12 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if (!mem_is_int(pIn1)) {
- mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if (!mem_is_int(pIn1)) {
- if (pOp->p2==0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "integer");
- goto abort_due_to_error;
- } else {
- goto jump_to_p2;
- }
- }
+ if (mem_to_int_precise(pIn1) != 0) {
+ if (pOp->p2 != 0)
+ goto jump_to_p2;
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+ mem_str(pIn1), "integer");
+ goto abort_due_to_error;
}
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions
2021-04-13 22:58 ` Mergen Imeev via Tarantool-patches
@ 2021-04-13 23:10 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 23:10 UTC (permalink / raw)
To: Vladislav Shpilevoy, tarantool-patches
This is addition from review of patch "sql: introduce mem_to_str() and
mem_to_str0()". I added assert to check that only simple types may be in
mem_to_*() functions. Diff and new patch below.
On Wed, Apr 14, 2021 at 01:58:33AM +0300, Mergen Imeev via Tarantool-patches wrote:
> Thank you for the review! My answers, diff and new patch below.
>
>
> On Tue, Apr 13, 2021 at 01:39:01AM +0200, Vladislav Shpilevoy wrote:
> > Good job on the fixes!
> >
> > This is the last email for today, I will continue the review of
> > the patchset tomorrow.
> >
> > See 2 comments below.
> >
> > > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> > > index 71a827034..e61ad4251 100644
> > > --- a/src/box/sql/vdbe.c
> > > +++ b/src/box/sql/vdbe.c
> > > @@ -1432,17 +1432,14 @@ case OP_AddImm: { /* in1 */
> > > */
> > > case OP_MustBeInt: { /* jump, in1 */
> > > pIn1 = &aMem[pOp->p1];
> > > - if (!mem_is_int(pIn1)) {
> > > - mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
> > > - if (!mem_is_int(pIn1)) {
> > > - if (pOp->p2==0) {
> > > - diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> > > - mem_str(pIn1), "integer");
> > > - goto abort_due_to_error;
> > > - } else {
> > > - goto jump_to_p2;
> > > - }
> > > - }
> > > + if (mem_is_int(pIn1))
> >
> > 1. The same check is done inside mem_to_int_precise(). You
> > don't need to do it manually.
> >
> Thanks, fixed.
>
> > > + break;
> > > + if (mem_to_int_precise(pIn1) != 0) {
> > > + if (pOp->p2 !=0 )
> >
> > 2. Whitespace should be before '0', not after.
> >
> Fixed.
>
> > > + goto jump_to_p2;
> > > + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> > > + mem_str(pIn1), "integer");
> > > + goto abort_due_to_error;
> > > }
> > > break;
> > > }
> > >
>
>
> Diff:
>
>
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 061f77952..c908f13c0 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1435,10 +1435,8 @@ case OP_AddImm: { /* in1 */
> */
> case OP_MustBeInt: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - if (mem_is_int(pIn1))
> - break;
> if (mem_to_int_precise(pIn1) != 0) {
> - if (pOp->p2 !=0 )
> + if (pOp->p2 != 0)
> goto jump_to_p2;
> diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> mem_str(pIn1), "integer");
>
>
> New patch:
>
>
> commit ecf63aa8a6887b3b1ac7a67a6b60fe3a62426ed9
> Author: Mergen Imeev <imeevma@gmail.com>
> Date: Wed Mar 17 10:46:29 2021 +0300
>
> sql: introduce mem_to_int*() functions
>
> This patch introduces mem_to_int() and mem_to_int_precise() functions.
> These functions are used to convert a MEM to a MEM that contains
> integer value. These functions defines the rules that are used during
> convertion from values of all other types to INTEGER.
>
> Part of #5818
>
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 3930067e1..fd68acd6e 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -492,6 +492,90 @@ mem_set_null_clear(struct Mem *mem)
> mem->flags = MEM_Null | MEM_Cleared;
> }
>
> +static inline int
> +bytes_to_int(struct Mem *mem)
> +{
> + bool is_neg;
> + int64_t i;
> + if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
> + return -1;
> + mem_set_int(mem, i, is_neg);
> + return 0;
> +}
> +
> +static inline int
> +double_to_int(struct Mem *mem)
> +{
> + double d = mem->u.r;
> + if (d < 0 && d >= (double)INT64_MIN) {
> + mem->u.i = (int64_t)d;
> + mem->flags = MEM_Int;
> + mem->field_type = FIELD_TYPE_INTEGER;
> + return 0;
> + }
> + if (d >= 0 && d < (double)UINT64_MAX) {
> + mem->u.u = (uint64_t)d;
> + mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_UNSIGNED;
> + return 0;
> + }
> + return -1;
> +}
> +
> +static inline int
> +double_to_int_precise(struct Mem *mem)
> +{
> + double d = mem->u.r;
> + if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
> + mem->u.i = (int64_t)d;
> + mem->flags = MEM_Int;
> + mem->field_type = FIELD_TYPE_INTEGER;
> + return 0;
> + }
> + if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
> + mem->u.u = (uint64_t)d;
> + mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_UNSIGNED;
> + return 0;
> + }
> + return -1;
> +}
> +
> +static inline int
> +bool_to_int(struct Mem *mem)
> +{
> + mem->u.u = (uint64_t)mem->u.b;
> + mem->flags = MEM_UInt;
> + mem->field_type = FIELD_TYPE_UNSIGNED;
> + return 0;
> +}
> +
> +int
> +mem_to_int(struct Mem *mem)
> +{
> + if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> + return 0;
> + if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
> + return bytes_to_int(mem);
> + if ((mem->flags & MEM_Real) != 0)
> + return double_to_int(mem);
> + if ((mem->flags & MEM_Bool) != 0)
> + return bool_to_int(mem);
> + return -1;
> +}
> +
> +int
> +mem_to_int_precise(struct Mem *mem)
> +{
> + if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> + return 0;
> + if ((mem->flags & MEM_Str) != 0)
> + return bytes_to_int(mem);
> + if ((mem->flags & MEM_Real) != 0)
> + return double_to_int_precise(mem);
> + return -1;
> +}
> +
> int
> mem_copy(struct Mem *to, const struct Mem *from)
> {
> @@ -1694,26 +1778,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
> }
> }
>
> -/*
> - * The MEM structure is already a MEM_Real. Try to also make it a
> - * MEM_Int if we can.
> - */
> -int
> -mem_apply_integer_type(Mem *pMem)
> -{
> - int rc;
> - i64 ix;
> - assert(pMem->flags & MEM_Real);
> - assert(EIGHT_BYTE_ALIGNMENT(pMem));
> -
> - if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
> - pMem->u.i = ix;
> - pMem->flags = pMem->u.r <= -1 ? MEM_Int : MEM_UInt;
> - pMem->field_type = FIELD_TYPE_INTEGER;
> - }
> - return rc;
> -}
> -
> /*
> * Add MEM_Str to the set of representations for the given Mem. Numbers
> * are converted using sql_snprintf(). Converting a BLOB to a string
> @@ -1985,34 +2049,6 @@ mem_convert_to_unsigned(struct Mem *mem)
> return 0;
> }
>
> -/**
> - * Convert the numeric value contained in MEM to integer.
> - *
> - * @param mem The MEM that contains the numeric value.
> - * @retval 0 if the conversion was successful, -1 otherwise.
> - */
> -static int
> -mem_convert_to_integer(struct Mem *mem)
> -{
> - if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
> - return 0;
> - if ((mem->flags & MEM_Real) == 0)
> - return -1;
> - double d = mem->u.r;
> - if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
> - return -1;
> - if (d < 0.) {
> - mem->u.i = (int64_t)d;
> - mem->flags = MEM_Int;
> - mem->field_type = FIELD_TYPE_INTEGER;
> - } else {
> - mem->u.u = (uint64_t)d;
> - mem->flags = MEM_UInt;
> - mem->field_type = FIELD_TYPE_UNSIGNED;
> - }
> - return 0;
> -}
> -
> int
> mem_convert_to_numeric(struct Mem *mem, enum field_type type)
> {
> @@ -2023,7 +2059,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
> if (type == FIELD_TYPE_UNSIGNED)
> return mem_convert_to_unsigned(mem);
> assert(type == FIELD_TYPE_INTEGER);
> - return mem_convert_to_integer(mem);
> + return mem_to_int(mem);
> }
>
> static int
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index 9890762e0..5aabcc1fd 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -683,6 +683,24 @@ mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
> int
> mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
>
> +/**
> + * Convert the given MEM to INTEGER. This function and the function below define
> + * the rules that are used to convert values of all other types to INTEGER. In
> + * this function, the conversion from double to integer may result in loss of
> + * precision.
> + */
> +int
> +mem_to_int(struct Mem *mem);
> +
> +/**
> + * Convert the given MEM to INTEGER. This function and the function above define
> + * the rules that are used to convert values of all other types to INTEGER. In
> + * this function, the conversion from double to integer is only possible if it
> + * is lossless.
> + */
> +int
> +mem_to_int_precise(struct Mem *mem);
> +
> /**
> * Simple type to str convertor. It is used to simplify
> * error reporting.
> @@ -743,7 +761,6 @@ int
> vdbe_mem_numerify(struct Mem *mem);
>
> int sqlVdbeMemCast(struct Mem *, enum field_type type);
> -int mem_apply_integer_type(struct Mem *);
> int sqlVdbeMemStringify(struct Mem *);
> int sqlVdbeMemNulTerminate(struct Mem *);
> int sqlVdbeMemExpandBlob(struct Mem *);
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index e07ff1be9..c908f13c0 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1435,17 +1435,12 @@ case OP_AddImm: { /* in1 */
> */
> case OP_MustBeInt: { /* jump, in1 */
> pIn1 = &aMem[pOp->p1];
> - if (!mem_is_int(pIn1)) {
> - mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
> - if (!mem_is_int(pIn1)) {
> - if (pOp->p2==0) {
> - diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> - mem_str(pIn1), "integer");
> - goto abort_due_to_error;
> - } else {
> - goto jump_to_p2;
> - }
> - }
> + if (mem_to_int_precise(pIn1) != 0) {
> + if (pOp->p2 != 0)
> + goto jump_to_p2;
> + diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> + mem_str(pIn1), "integer");
> + goto abort_due_to_error;
> }
> break;
> }
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index fd68acd6e..b22b4ba21 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -553,6 +553,7 @@ bool_to_int(struct Mem *mem)
int
mem_to_int(struct Mem *mem)
{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
return 0;
if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
@@ -567,6 +568,7 @@ mem_to_int(struct Mem *mem)
int
mem_to_int_precise(struct Mem *mem)
{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
return 0;
if ((mem->flags & MEM_Str) != 0)
New patch:
commit 5723ddfbed6c9fdf7e01bbbc51a47db496e605e2
Author: Mergen Imeev <imeevma@gmail.com>
Date: Wed Mar 17 10:46:29 2021 +0300
sql: introduce mem_to_int*() functions
This patch introduces mem_to_int() and mem_to_int_precise() functions.
These functions are used to convert a MEM to a MEM that contains
integer value. These functions defines the rules that are used during
convertion from values of all other types to INTEGER.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 3930067e1..b22b4ba21 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -492,6 +492,92 @@ mem_set_null_clear(struct Mem *mem)
mem->flags = MEM_Null | MEM_Cleared;
}
+static inline int
+bytes_to_int(struct Mem *mem)
+{
+ bool is_neg;
+ int64_t i;
+ if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
+ return -1;
+ mem_set_int(mem, i, is_neg);
+ return 0;
+}
+
+static inline int
+double_to_int(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+double_to_int_precise(struct Mem *mem)
+{
+ double d = mem->u.r;
+ if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
+ mem->u.i = (int64_t)d;
+ mem->flags = MEM_Int;
+ mem->field_type = FIELD_TYPE_INTEGER;
+ return 0;
+ }
+ if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
+ mem->u.u = (uint64_t)d;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+ }
+ return -1;
+}
+
+static inline int
+bool_to_int(struct Mem *mem)
+{
+ mem->u.u = (uint64_t)mem->u.b;
+ mem->flags = MEM_UInt;
+ mem->field_type = FIELD_TYPE_UNSIGNED;
+ return 0;
+}
+
+int
+mem_to_int(struct Mem *mem)
+{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int(mem);
+ if ((mem->flags & MEM_Bool) != 0)
+ return bool_to_int(mem);
+ return -1;
+}
+
+int
+mem_to_int_precise(struct Mem *mem)
+{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return 0;
+ if ((mem->flags & MEM_Str) != 0)
+ return bytes_to_int(mem);
+ if ((mem->flags & MEM_Real) != 0)
+ return double_to_int_precise(mem);
+ return -1;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1694,26 +1780,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
}
}
-/*
- * The MEM structure is already a MEM_Real. Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
- int rc;
- i64 ix;
- assert(pMem->flags & MEM_Real);
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
- if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0) {
- pMem->u.i = ix;
- pMem->flags = pMem->u.r <= -1 ? MEM_Int : MEM_UInt;
- pMem->field_type = FIELD_TYPE_INTEGER;
- }
- return rc;
-}
-
/*
* Add MEM_Str to the set of representations for the given Mem. Numbers
* are converted using sql_snprintf(). Converting a BLOB to a string
@@ -1985,34 +2051,6 @@ mem_convert_to_unsigned(struct Mem *mem)
return 0;
}
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
- if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
- return 0;
- if ((mem->flags & MEM_Real) == 0)
- return -1;
- double d = mem->u.r;
- if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
- return -1;
- if (d < 0.) {
- mem->u.i = (int64_t)d;
- mem->flags = MEM_Int;
- mem->field_type = FIELD_TYPE_INTEGER;
- } else {
- mem->u.u = (uint64_t)d;
- mem->flags = MEM_UInt;
- mem->field_type = FIELD_TYPE_UNSIGNED;
- }
- return 0;
-}
-
int
mem_convert_to_numeric(struct Mem *mem, enum field_type type)
{
@@ -2023,7 +2061,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
if (type == FIELD_TYPE_UNSIGNED)
return mem_convert_to_unsigned(mem);
assert(type == FIELD_TYPE_INTEGER);
- return mem_convert_to_integer(mem);
+ return mem_to_int(mem);
}
static int
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9890762e0..5aabcc1fd 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -683,6 +683,24 @@ mem_cmp_str(const struct Mem *left, const struct Mem *right, int *result,
int
mem_cmp_num(const struct Mem *a, const struct Mem *b, int *result);
+/**
+ * Convert the given MEM to INTEGER. This function and the function below define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer may result in loss of
+ * precision.
+ */
+int
+mem_to_int(struct Mem *mem);
+
+/**
+ * Convert the given MEM to INTEGER. This function and the function above define
+ * the rules that are used to convert values of all other types to INTEGER. In
+ * this function, the conversion from double to integer is only possible if it
+ * is lossless.
+ */
+int
+mem_to_int_precise(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -743,7 +761,6 @@ int
vdbe_mem_numerify(struct Mem *mem);
int sqlVdbeMemCast(struct Mem *, enum field_type type);
-int mem_apply_integer_type(struct Mem *);
int sqlVdbeMemStringify(struct Mem *);
int sqlVdbeMemNulTerminate(struct Mem *);
int sqlVdbeMemExpandBlob(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e07ff1be9..c908f13c0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1435,17 +1435,12 @@ case OP_AddImm: { /* in1 */
*/
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
- if (!mem_is_int(pIn1)) {
- mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
- if (!mem_is_int(pIn1)) {
- if (pOp->p2==0) {
- diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
- mem_str(pIn1), "integer");
- goto abort_due_to_error;
- } else {
- goto jump_to_p2;
- }
- }
+ if (mem_to_int_precise(pIn1) != 0) {
+ if (pOp->p2 != 0)
+ goto jump_to_p2;
+ diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+ mem_str(pIn1), "integer");
+ goto abort_due_to_error;
}
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double()
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (38 preceding siblings ...)
2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions Mergen Imeev via Tarantool-patches
@ 2021-04-09 20:26 ` Mergen Imeev via Tarantool-patches
2021-04-13 23:21 ` Mergen Imeev via Tarantool-patches
2021-04-15 0:39 ` [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Vladislav Shpilevoy via Tarantool-patches
2021-04-15 6:49 ` Kirill Yukhin via Tarantool-patches
41 siblings, 1 reply; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 20:26 UTC (permalink / raw)
To: v.shpilevoy, tsafin; +Cc: tarantool-patches
This patch intruduces mem_to_double(). This function is used to convert
a MEM to a MEM that contains double value. This function defines the
rules that are used during conversion from values of all other types to
double.
Part of #5818
---
src/box/sql/mem.c | 81 +++++++++++++++++++++++-----------------------
src/box/sql/mem.h | 8 ++++-
src/box/sql/vdbe.c | 2 +-
3 files changed, 48 insertions(+), 43 deletions(-)
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index d3a3215bc..75d4c4d18 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -771,6 +771,42 @@ mem_to_int_precise(struct Mem *mem)
return -1;
}
+static inline int
+int_to_double(struct Mem *mem)
+{
+ double d;
+ if ((mem->flags & MEM_UInt) != 0)
+ d = (double)mem->u.u;
+ else
+ d = (double)mem->u.i;
+ mem->u.r = d;
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ return 0;
+}
+
+static inline int
+bytes_to_double(struct Mem *mem)
+{
+ double d;
+ if (sqlAtoF(mem->z, &d, mem->n) == 0)
+ return -1;
+ mem_set_double(mem, d);
+ return 0;
+}
+
+int
+mem_to_double(struct Mem *mem)
+{
+ if ((mem->flags & MEM_Real) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return int_to_double(mem);
+ if ((mem->flags & MEM_Str) != 0)
+ return bytes_to_double(mem);
+ return -1;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1861,21 +1897,6 @@ mem_apply_numeric_type(struct Mem *record)
return 0;
}
-/*
- * Convert pMem so that it is of type MEM_Real.
- * Invalidate any prior representations.
- */
-int
-sqlVdbeMemRealify(Mem * pMem)
-{
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
- double v;
- if (sqlVdbeRealValue(pMem, &v))
- return -1;
- mem_set_double(pMem, v);
- return 0;
-}
-
int
vdbe_mem_numerify(struct Mem *mem)
{
@@ -1979,7 +2000,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return -1;
return 0;
case FIELD_TYPE_DOUBLE:
- return sqlVdbeMemRealify(pMem);
+ return mem_to_double(pMem);
case FIELD_TYPE_NUMBER:
return vdbe_mem_numerify(pMem);
case FIELD_TYPE_VARBINARY:
@@ -2185,11 +2206,11 @@ mem_apply_type(struct Mem *record, enum field_type type)
case FIELD_TYPE_NUMBER:
if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
return 0;
- return sqlVdbeMemRealify(record);
+ return mem_to_double(record);
case FIELD_TYPE_DOUBLE:
if ((record->flags & MEM_Real) != 0)
return 0;
- return sqlVdbeMemRealify(record);
+ return mem_to_double(record);
case FIELD_TYPE_STRING:
/*
* Only attempt the conversion to TEXT if there is
@@ -2233,28 +2254,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
}
}
-/**
- * Convert the numeric value contained in MEM to double.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_double(struct Mem *mem)
-{
- if ((mem->flags & MEM_Real) != 0)
- return 0;
- if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
- return -1;
- double d;
- if ((mem->flags & MEM_Int) != 0)
- d = (double)mem->u.i;
- else
- d = (double)mem->u.u;
- mem_set_double(mem, d);
- return 0;
-}
-
/**
* Convert the numeric value contained in MEM to unsigned.
*
@@ -2285,7 +2284,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
assert(mem_is_num(mem) && sql_type_is_numeric(type));
assert(type != FIELD_TYPE_NUMBER);
if (type == FIELD_TYPE_DOUBLE)
- return mem_convert_to_double(mem);
+ return mem_to_double(mem);
if (type == FIELD_TYPE_UNSIGNED)
return mem_convert_to_unsigned(mem);
assert(type == FIELD_TYPE_INTEGER);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d3eb04c44..bf8c0f3b5 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -491,6 +491,13 @@ mem_to_int(struct Mem *mem);
int
mem_to_int_precise(struct Mem *mem);
+/**
+ * Convert the given MEM to DOUBLE. This function defines the rules that are
+ * used to convert values of all other types to DOUBLE.
+ */
+int
+mem_to_double(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -537,7 +544,6 @@ registerTrace(int iReg, Mem *p);
*/
int
mem_apply_numeric_type(struct Mem *record);
-int sqlVdbeMemRealify(struct Mem *);
/**
* Convert @a mem to NUMBER type, so that after conversion it has
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e61ad4251..90a901555 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1456,7 +1456,7 @@ case OP_MustBeInt: { /* jump, in1 */
case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
if (mem_is_int(pIn1)) {
- sqlVdbeMemRealify(pIn1);
+ mem_to_double(pIn1);
}
break;
}
--
2.25.1
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double()
2021-04-09 20:26 ` [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double() Mergen Imeev via Tarantool-patches
@ 2021-04-13 23:21 ` Mergen Imeev via Tarantool-patches
0 siblings, 0 replies; 107+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-13 23:21 UTC (permalink / raw)
To: v.shpilevoy, tsafin, tarantool-patches
Grouped <type>_to_<type>() functions and added assert to make sure that
mem_to_double() accepts only values of simple types. Diff and new patch below.
On Fri, Apr 09, 2021 at 11:26:00PM +0300, Mergen Imeev via Tarantool-patches wrote:
> This patch intruduces mem_to_double(). This function is used to convert
> a MEM to a MEM that contains double value. This function defines the
> rules that are used during conversion from values of all other types to
> double.
>
> Part of #5818
> ---
> src/box/sql/mem.c | 81 +++++++++++++++++++++++-----------------------
> src/box/sql/mem.h | 8 ++++-
> src/box/sql/vdbe.c | 2 +-
> 3 files changed, 48 insertions(+), 43 deletions(-)
>
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index d3a3215bc..75d4c4d18 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -771,6 +771,42 @@ mem_to_int_precise(struct Mem *mem)
> return -1;
> }
>
> +static inline int
> +int_to_double(struct Mem *mem)
> +{
> + double d;
> + if ((mem->flags & MEM_UInt) != 0)
> + d = (double)mem->u.u;
> + else
> + d = (double)mem->u.i;
> + mem->u.r = d;
> + mem->flags = MEM_Real;
> + mem->field_type = FIELD_TYPE_DOUBLE;
> + return 0;
> +}
> +
> +static inline int
> +bytes_to_double(struct Mem *mem)
> +{
> + double d;
> + if (sqlAtoF(mem->z, &d, mem->n) == 0)
> + return -1;
> + mem_set_double(mem, d);
> + return 0;
> +}
> +
> +int
> +mem_to_double(struct Mem *mem)
> +{
> + if ((mem->flags & MEM_Real) != 0)
> + return 0;
> + if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> + return int_to_double(mem);
> + if ((mem->flags & MEM_Str) != 0)
> + return bytes_to_double(mem);
> + return -1;
> +}
> +
> int
> mem_copy(struct Mem *to, const struct Mem *from)
> {
> @@ -1861,21 +1897,6 @@ mem_apply_numeric_type(struct Mem *record)
> return 0;
> }
>
> -/*
> - * Convert pMem so that it is of type MEM_Real.
> - * Invalidate any prior representations.
> - */
> -int
> -sqlVdbeMemRealify(Mem * pMem)
> -{
> - assert(EIGHT_BYTE_ALIGNMENT(pMem));
> - double v;
> - if (sqlVdbeRealValue(pMem, &v))
> - return -1;
> - mem_set_double(pMem, v);
> - return 0;
> -}
> -
> int
> vdbe_mem_numerify(struct Mem *mem)
> {
> @@ -1979,7 +2000,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
> return -1;
> return 0;
> case FIELD_TYPE_DOUBLE:
> - return sqlVdbeMemRealify(pMem);
> + return mem_to_double(pMem);
> case FIELD_TYPE_NUMBER:
> return vdbe_mem_numerify(pMem);
> case FIELD_TYPE_VARBINARY:
> @@ -2185,11 +2206,11 @@ mem_apply_type(struct Mem *record, enum field_type type)
> case FIELD_TYPE_NUMBER:
> if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
> return 0;
> - return sqlVdbeMemRealify(record);
> + return mem_to_double(record);
> case FIELD_TYPE_DOUBLE:
> if ((record->flags & MEM_Real) != 0)
> return 0;
> - return sqlVdbeMemRealify(record);
> + return mem_to_double(record);
> case FIELD_TYPE_STRING:
> /*
> * Only attempt the conversion to TEXT if there is
> @@ -2233,28 +2254,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
> }
> }
>
> -/**
> - * Convert the numeric value contained in MEM to double.
> - *
> - * @param mem The MEM that contains the numeric value.
> - * @retval 0 if the conversion was successful, -1 otherwise.
> - */
> -static int
> -mem_convert_to_double(struct Mem *mem)
> -{
> - if ((mem->flags & MEM_Real) != 0)
> - return 0;
> - if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
> - return -1;
> - double d;
> - if ((mem->flags & MEM_Int) != 0)
> - d = (double)mem->u.i;
> - else
> - d = (double)mem->u.u;
> - mem_set_double(mem, d);
> - return 0;
> -}
> -
> /**
> * Convert the numeric value contained in MEM to unsigned.
> *
> @@ -2285,7 +2284,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
> assert(mem_is_num(mem) && sql_type_is_numeric(type));
> assert(type != FIELD_TYPE_NUMBER);
> if (type == FIELD_TYPE_DOUBLE)
> - return mem_convert_to_double(mem);
> + return mem_to_double(mem);
> if (type == FIELD_TYPE_UNSIGNED)
> return mem_convert_to_unsigned(mem);
> assert(type == FIELD_TYPE_INTEGER);
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index d3eb04c44..bf8c0f3b5 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -491,6 +491,13 @@ mem_to_int(struct Mem *mem);
> int
> mem_to_int_precise(struct Mem *mem);
>
> +/**
> + * Convert the given MEM to DOUBLE. This function defines the rules that are
> + * used to convert values of all other types to DOUBLE.
> + */
> +int
> +mem_to_double(struct Mem *mem);
> +
> /**
> * Simple type to str convertor. It is used to simplify
> * error reporting.
> @@ -537,7 +544,6 @@ registerTrace(int iReg, Mem *p);
> */
> int
> mem_apply_numeric_type(struct Mem *record);
> -int sqlVdbeMemRealify(struct Mem *);
>
> /**
> * Convert @a mem to NUMBER type, so that after conversion it has
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index e61ad4251..90a901555 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1456,7 +1456,7 @@ case OP_MustBeInt: { /* jump, in1 */
> case OP_Realify: { /* in1 */
> pIn1 = &aMem[pOp->p1];
> if (mem_is_int(pIn1)) {
> - sqlVdbeMemRealify(pIn1);
> + mem_to_double(pIn1);
> }
> break;
> }
> --
> 2.25.1
>
Diff:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 9972b1f85..3685bf6b9 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -492,6 +492,20 @@ mem_set_null_clear(struct Mem *mem)
mem->flags = MEM_Null | MEM_Cleared;
}
+static inline int
+int_to_double(struct Mem *mem)
+{
+ double d;
+ if ((mem->flags & MEM_UInt) != 0)
+ d = (double)mem->u.u;
+ else
+ d = (double)mem->u.i;
+ mem->u.r = d;
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ return 0;
+}
+
static inline int
bytes_to_int(struct Mem *mem)
{
@@ -503,6 +517,16 @@ bytes_to_int(struct Mem *mem)
return 0;
}
+static inline int
+bytes_to_double(struct Mem *mem)
+{
+ double d;
+ if (sqlAtoF(mem->z, &d, mem->n) == 0)
+ return -1;
+ mem_set_double(mem, d);
+ return 0;
+}
+
static inline int
double_to_int(struct Mem *mem)
{
@@ -578,33 +602,10 @@ mem_to_int_precise(struct Mem *mem)
return -1;
}
-static inline int
-int_to_double(struct Mem *mem)
-{
- double d;
- if ((mem->flags & MEM_UInt) != 0)
- d = (double)mem->u.u;
- else
- d = (double)mem->u.i;
- mem->u.r = d;
- mem->flags = MEM_Real;
- mem->field_type = FIELD_TYPE_DOUBLE;
- return 0;
-}
-
-static inline int
-bytes_to_double(struct Mem *mem)
-{
- double d;
- if (sqlAtoF(mem->z, &d, mem->n) == 0)
- return -1;
- mem_set_double(mem, d);
- return 0;
-}
-
int
mem_to_double(struct Mem *mem)
{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
if ((mem->flags & MEM_Real) != 0)
return 0;
if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
New patch:
commit f7e8e09f0160dd0cf933e4bfc29d108603291b79
Author: Mergen Imeev <imeevma@gmail.com>
Date: Wed Mar 17 11:03:22 2021 +0300
sql: introduce mem_to_double()
This patch intruduces mem_to_double(). This function is used to convert
a MEM to a MEM that contains double value. This function defines the
rules that are used during conversion from values of all other types to
double.
Part of #5818
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index b22b4ba21..3685bf6b9 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -492,6 +492,20 @@ mem_set_null_clear(struct Mem *mem)
mem->flags = MEM_Null | MEM_Cleared;
}
+static inline int
+int_to_double(struct Mem *mem)
+{
+ double d;
+ if ((mem->flags & MEM_UInt) != 0)
+ d = (double)mem->u.u;
+ else
+ d = (double)mem->u.i;
+ mem->u.r = d;
+ mem->flags = MEM_Real;
+ mem->field_type = FIELD_TYPE_DOUBLE;
+ return 0;
+}
+
static inline int
bytes_to_int(struct Mem *mem)
{
@@ -503,6 +517,16 @@ bytes_to_int(struct Mem *mem)
return 0;
}
+static inline int
+bytes_to_double(struct Mem *mem)
+{
+ double d;
+ if (sqlAtoF(mem->z, &d, mem->n) == 0)
+ return -1;
+ mem_set_double(mem, d);
+ return 0;
+}
+
static inline int
double_to_int(struct Mem *mem)
{
@@ -578,6 +602,19 @@ mem_to_int_precise(struct Mem *mem)
return -1;
}
+int
+mem_to_double(struct Mem *mem)
+{
+ assert((mem->flags & MEM_PURE_TYPE_MASK) != 0);
+ if ((mem->flags & MEM_Real) != 0)
+ return 0;
+ if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+ return int_to_double(mem);
+ if ((mem->flags & MEM_Str) != 0)
+ return bytes_to_double(mem);
+ return -1;
+}
+
int
mem_copy(struct Mem *to, const struct Mem *from)
{
@@ -1633,21 +1670,6 @@ mem_apply_numeric_type(struct Mem *record)
return 0;
}
-/*
- * Convert pMem so that it is of type MEM_Real.
- * Invalidate any prior representations.
- */
-int
-sqlVdbeMemRealify(Mem * pMem)
-{
- assert(EIGHT_BYTE_ALIGNMENT(pMem));
- double v;
- if (sqlVdbeRealValue(pMem, &v))
- return -1;
- mem_set_double(pMem, v);
- return 0;
-}
-
int
vdbe_mem_numerify(struct Mem *mem)
{
@@ -1751,7 +1773,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
return -1;
return 0;
case FIELD_TYPE_DOUBLE:
- return sqlVdbeMemRealify(pMem);
+ return mem_to_double(pMem);
case FIELD_TYPE_NUMBER:
return vdbe_mem_numerify(pMem);
case FIELD_TYPE_VARBINARY:
@@ -1957,11 +1979,11 @@ mem_apply_type(struct Mem *record, enum field_type type)
case FIELD_TYPE_NUMBER:
if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
return 0;
- return sqlVdbeMemRealify(record);
+ return mem_to_double(record);
case FIELD_TYPE_DOUBLE:
if ((record->flags & MEM_Real) != 0)
return 0;
- return sqlVdbeMemRealify(record);
+ return mem_to_double(record);
case FIELD_TYPE_STRING:
/*
* Only attempt the conversion to TEXT if there is
@@ -2005,28 +2027,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
}
}
-/**
- * Convert the numeric value contained in MEM to double.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_double(struct Mem *mem)
-{
- if ((mem->flags & MEM_Real) != 0)
- return 0;
- if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
- return -1;
- double d;
- if ((mem->flags & MEM_Int) != 0)
- d = (double)mem->u.i;
- else
- d = (double)mem->u.u;
- mem_set_double(mem, d);
- return 0;
-}
-
/**
* Convert the numeric value contained in MEM to unsigned.
*
@@ -2057,7 +2057,7 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type)
assert(mem_is_num(mem) && sql_type_is_numeric(type));
assert(type != FIELD_TYPE_NUMBER);
if (type == FIELD_TYPE_DOUBLE)
- return mem_convert_to_double(mem);
+ return mem_to_double(mem);
if (type == FIELD_TYPE_UNSIGNED)
return mem_convert_to_unsigned(mem);
assert(type == FIELD_TYPE_INTEGER);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 5aabcc1fd..485a6fed0 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -701,6 +701,13 @@ mem_to_int(struct Mem *mem);
int
mem_to_int_precise(struct Mem *mem);
+/**
+ * Convert the given MEM to DOUBLE. This function defines the rules that are
+ * used to convert values of all other types to DOUBLE.
+ */
+int
+mem_to_double(struct Mem *mem);
+
/**
* Simple type to str convertor. It is used to simplify
* error reporting.
@@ -747,7 +754,6 @@ registerTrace(int iReg, Mem *p);
*/
int
mem_apply_numeric_type(struct Mem *record);
-int sqlVdbeMemRealify(struct Mem *);
/**
* Convert @a mem to NUMBER type, so that after conversion it has
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index c908f13c0..008148687 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1457,7 +1457,7 @@ case OP_MustBeInt: { /* jump, in1 */
case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
if (mem_is_int(pIn1)) {
- sqlVdbeMemRealify(pIn1);
+ mem_to_double(pIn1);
}
break;
}
^ permalink raw reply [flat|nested] 107+ messages in thread
* Re: [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (39 preceding siblings ...)
2021-04-09 20:26 ` [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double() Mergen Imeev via Tarantool-patches
@ 2021-04-15 0:39 ` Vladislav Shpilevoy via Tarantool-patches
2021-04-15 6:49 ` Kirill Yukhin via Tarantool-patches
41 siblings, 0 replies; 107+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-04-15 0:39 UTC (permalink / raw)
To: imeevma, tsafin; +Cc: tarantool-patches
I need to go now. The patch is in a relatively good shape. If Mergen
manages to fix all of my comments before the release time, I think it
could be pushed. To all the branches, not only to the master branch.
Because it contains bugfixes, and because it is vital for the future
bugfixes.
^ permalink raw reply [flat|nested] 107+ messages in thread* Re: [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h
2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
` (40 preceding siblings ...)
2021-04-15 0:39 ` [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-15 6:49 ` Kirill Yukhin via Tarantool-patches
41 siblings, 0 replies; 107+ messages in thread
From: Kirill Yukhin via Tarantool-patches @ 2021-04-15 6:49 UTC (permalink / raw)
To: imeevma; +Cc: tarantool-patches, v.shpilevoy
Hello,
On 09 апр 19:51, Mergen Imeev via Tarantool-patches wrote:
> This patch-set moves all MEM-related functions to mem.c and mem.h. This is done
> to encapsulate struct MEM whicl allows us to simplify addition of new field
> types.
>
> https://github.com/tarantool/tarantool/issues/5818
> https://github.com/tarantool/tarantool/tree/imeevma/gh-5818-encapsulate-mem-type-checking-and-changing
>
> Changes in v5:
> - Review fixes.
>
> Changes in v4:
> - Patch-set was completely reworked.
>
> Changes in v3:
> - Inlined most of the introduced functions to improve performance.
> - Some other fixes in code to improve performance.
>
> Changes in v2:
> - Squashed almost all patches.
> - Review fixes.
I hope you've fixed all inputs.
I've checked your patchset into 2.6, 2.7 and master.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 107+ messages in thread