From: Mergen Imeev via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: v.shpilevoy@tarantool.org, tsafin@tarantool.org Cc: tarantool-patches@dev.tarantool.org Subject: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Date: Fri, 9 Apr 2021 19:51:22 +0300 [thread overview] Message-ID: <3fda2029b186d0cea67ba01269dc3d6e209b3c89.1617984948.git.imeevma@gmail.com> (raw) In-Reply-To: <cover.1617984948.git.imeevma@gmail.com> Hi! Thank you for the review! My answer and new patch below. I didn't include diffs in answers since due to merge conflicts they are partly useless. On 30.03.2021 01:57, Vladislav Shpilevoy wrote: > Hi! I appreciate the work you did here! > > Truly huge patch. But it does the important thing which I think > should give a huge help in the task of SQL code rework. > > On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote: >> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not >> properly initialized in case msgpack contains MP_EXT, MP_MAP, or >> MP_ARRAY fields. Also, it doesn't set field_type. > > AFAIR, field type wasn't set deliberately. Because we use it only for > strictly typed values obtained from formatted space fields. It wasn't > applied to plain non-formatted values on purpose. > > Why do you change that? I didn't know about that. I thought that all MEMs that contains data should have field_type set. However, I tried to understand where did this field come from and found that it was added for two purposes: to show difference between NUMBER and INTEGER in MEM before DOUBLE was added and to show column name instead of type determined from MP-type in typeof(). I believe that both these purposes are not needed now and that this field should be removed from struct MEM. I created an issue for this: #5957. However, I was prohibited to remove this field for now by @tsafin, who believes that this field is actually important. New patch: commit 3fda2029b186d0cea67ba01269dc3d6e209b3c89 Author: Mergen Imeev <imeevma@gmail.com> Date: Thu Mar 4 17:17:18 2021 +0300 sql: enhance vdbe_decode_msgpack_into_mem() Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not properly initialized in case msgpack contains MP_EXT, MP_MAP, or MP_ARRAY fields. This patch makes it so that after execution of this function, all created MEMs are properly initialized. Needed for #5818 diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 3b3b1f01d..9a4f38bb9 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -846,16 +846,6 @@ vdbe_field_ref_fetch_data(struct vdbe_field_ref *field_ref, uint32_t fieldno) return field_begin; } -static inline enum field_type -vdbe_field_ref_fetch_type(struct vdbe_field_ref *field_ref, uint32_t fieldno) -{ - const struct tuple_field *tf = - vdbe_field_ref_fetch_field(field_ref, fieldno); - if (tf == NULL || tf->type == FIELD_TYPE_ANY) - return field_type_MAX; - return tf->type; -} - /** * Fetch field by fieldno using vdbe_field_ref and store result * in dest_mem. @@ -879,17 +869,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno, if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) return -1; - /* - * MsgPack map, array or extension (unsupported in sql). - * Wrap it in a blob verbatim. - */ - if (dest_mem->flags == 0) { - dest_mem->z = (char *) data; - dest_mem->n = vdbe_field_ref_fetch_data(field_ref, - fieldno + 1) - data; - dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; - dest_mem->subtype = SQL_SUBTYPE_MSGPACK; - } /* * Add 0 termination (at most for strings) * Not sure why do we check MEM_Ephem @@ -909,7 +888,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno, dest_mem->flags |= MEM_Term; } UPDATE_MAX_BLOBSIZE(dest_mem); - dest_mem->field_type = vdbe_field_ref_fetch_type(field_ref, fieldno); return 0; } diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c index 91b64316e..772476377 100644 --- a/src/box/sql/vdbeaux.c +++ b/src/box/sql/vdbeaux.c @@ -2793,38 +2793,62 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len) { const char *start_buf = buf; switch (mp_typeof(*buf)) { - case MP_ARRAY: - case MP_MAP: - case MP_EXT: - default: { - mem->flags = 0; + case MP_ARRAY: { + mem->z = (char *)buf; + mp_next(&buf); + mem->n = buf - mem->z; + mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; + mem->subtype = SQL_SUBTYPE_MSGPACK; + mem->field_type = FIELD_TYPE_ARRAY; + break; + } + case MP_MAP: { + mem->z = (char *)buf; + mp_next(&buf); + mem->n = buf - mem->z; + mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; + mem->subtype = SQL_SUBTYPE_MSGPACK; + mem->field_type = FIELD_TYPE_MAP; + break; + } + case MP_EXT: { + mem->z = (char *)buf; + mp_next(&buf); + mem->n = buf - mem->z; + mem->flags = MEM_Blob | MEM_Ephem; + mem->field_type = FIELD_TYPE_VARBINARY; break; } case MP_NIL: { mp_decode_nil(&buf); mem->flags = MEM_Null; + mem->field_type = field_type_MAX; break; } case MP_BOOL: { mem->u.b = mp_decode_bool(&buf); mem->flags = MEM_Bool; + mem->field_type = FIELD_TYPE_BOOLEAN; break; } case MP_UINT: { uint64_t v = mp_decode_uint(&buf); mem->u.u = v; mem->flags = MEM_UInt; + mem->field_type = FIELD_TYPE_INTEGER; break; } case MP_INT: { mem->u.i = mp_decode_int(&buf); mem->flags = MEM_Int; + mem->field_type = FIELD_TYPE_INTEGER; break; } case MP_STR: { /* XXX u32->int */ mem->n = (int) mp_decode_strl(&buf); mem->flags = MEM_Str | MEM_Ephem; + mem->field_type = FIELD_TYPE_STRING; install_blob: mem->z = (char *)buf; buf += mem->n; @@ -2834,18 +2858,33 @@ install_blob: /* XXX u32->int */ mem->n = (int) mp_decode_binl(&buf); mem->flags = MEM_Blob | MEM_Ephem; + mem->field_type = FIELD_TYPE_VARBINARY; goto install_blob; } case MP_FLOAT: { mem->u.r = mp_decode_float(&buf); - mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real; + if (sqlIsNaN(mem->u.r)) { + mem->flags = MEM_Null; + mem->field_type = FIELD_TYPE_DOUBLE; + } else { + mem->flags = MEM_Real; + mem->field_type = FIELD_TYPE_DOUBLE; + } break; } case MP_DOUBLE: { mem->u.r = mp_decode_double(&buf); - mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real; + if (sqlIsNaN(mem->u.r)) { + mem->flags = MEM_Null; + mem->field_type = FIELD_TYPE_DOUBLE; + } else { + mem->flags = MEM_Real; + mem->field_type = FIELD_TYPE_DOUBLE; + } break; } + default: + unreachable(); } *len = (uint32_t)(buf - start_buf); return 0; @@ -2868,15 +2907,8 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec pMem->z = 0; uint32_t sz = 0; vdbe_decode_msgpack_into_mem(zParse, pMem, &sz); - if (sz == 0) { - /* MsgPack array, map or ext. Treat as blob. */ - pMem->z = (char *)zParse; - mp_next(&zParse); - pMem->n = zParse - pMem->z; - pMem->flags = MEM_Blob | MEM_Ephem; - } else { - zParse += sz; - } + assert(sz != 0); + zParse += sz; pMem++; } }
next prev parent reply other threads:[~2021-04-09 16:51 UTC|newest] Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` Mergen Imeev via Tarantool-patches [this message] 2021-04-11 17:42 ` [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches 2021-04-13 12:12 ` Mergen Imeev via Tarantool-patches 2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 23:34 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 02/52] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 03/52] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 22:37 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 05/52] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 06/52] sql: refactor port_vdbemem_*() functions Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 07/52] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 08/52] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str() Mergen Imeev via Tarantool-patches 2021-04-11 17:44 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:36 ` Mergen Imeev via Tarantool-patches 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 22:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 10/52] sql: introduce mem_create() Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches 2021-04-11 17:46 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches 2021-04-11 17:59 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:09 ` Mergen Imeev via Tarantool-patches 2021-04-14 22:48 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:07 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches 2021-04-11 18:06 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:18 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:31 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:37 ` [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move() Mergen Imeev via Tarantool-patches 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:38 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches 2021-04-14 22:58 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:14 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches 2021-04-11 18:11 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:57 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:04 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:22 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM Mergen Imeev via Tarantool-patches 2021-04-11 18:13 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 17:06 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:33 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches 2021-04-11 18:16 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 18:33 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:20 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:40 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM Mergen Imeev via Tarantool-patches 2021-04-12 23:31 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 20:49 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 22/52] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 23/52] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int() Mergen Imeev via Tarantool-patches 2021-04-12 23:32 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 20:56 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 25/52] sql: introduce mem_set_uint() Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 26/52] sql: move mem_set_bool() and mem_set_double() Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions Mergen Imeev via Tarantool-patches 2021-04-12 23:34 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 21:36 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:49 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-15 1:25 ` Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0() Mergen Imeev via Tarantool-patches 2021-04-12 23:35 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:00 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:54 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-15 0:30 ` Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 29/52] sql: introduce mem_set_bin_*() functions Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin() Mergen Imeev via Tarantool-patches 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:06 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 31/52] sql: introduce mem_set_zerobin() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array Mergen Imeev via Tarantool-patches 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:08 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 33/52] sql: introduce mem_set_invalid() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 34/52] sql: refactor mem_set_ptr() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:19 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg() Mergen Imeev via Tarantool-patches 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:46 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear() Mergen Imeev via Tarantool-patches 2021-04-12 23:38 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:50 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches 2021-04-13 20:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions Mergen Imeev via Tarantool-patches 2021-04-12 23:39 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:58 ` Mergen Imeev via Tarantool-patches 2021-04-13 23:10 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:26 ` [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double() Mergen Imeev via Tarantool-patches 2021-04-13 23:21 ` Mergen Imeev via Tarantool-patches 2021-04-15 0:39 ` [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Vladislav Shpilevoy via Tarantool-patches 2021-04-15 6:49 ` Kirill Yukhin via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=3fda2029b186d0cea67ba01269dc3d6e209b3c89.1617984948.git.imeevma@gmail.com \ --to=tarantool-patches@dev.tarantool.org \ --cc=imeevma@tarantool.org \ --cc=tsafin@tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem()' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox