From: Mergen Imeev via Tarantool-patches <tarantool-patches@dev.tarantool.org> To: v.shpilevoy@tarantool.org, tsafin@tarantool.org Cc: tarantool-patches@dev.tarantool.org Subject: [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem() Date: Fri, 9 Apr 2021 20:57:14 +0300 [thread overview] Message-ID: <8c81c3e86a628e5777145b30f69392dd5a0fd873.1617984948.git.imeevma@gmail.com> (raw) In-Reply-To: <cover.1617984948.git.imeevma@gmail.com> Thank you for the review! My answers and new patch below. On 30.03.2021 02:02, Vladislav Shpilevoy wrote: > Thanks for the patch! > > See 3 comments below. > > On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote: >> The original vdbe_decode_msgpack_into_mem() returns a MEM that contains >> string and binary values as ephemeral. This patch renames this function >> to vdbe_decode_msgpack_into_mem_ephemeral() and introduces new >> vdbe_decode_msgpack_into_mem(), which returns a MEM that contains string >> and binary values in newly allocated memory. >> >> This patch actually changes behavior in this case: > > 1. Changes how? I don't see any changes in the tests. > I make this change because it doesn't affect test. Example of changed behaviour: CREATE TABLE t1(m VARBINARY primary key); INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379'); SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; Before this patch: tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; --- - metadata: - name: COLUMN_1 type: integer - name: MX type: string rows: - [2, 'y'] - [1, 'y'] ... After this patch. tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; --- - metadata: - name: COLUMN_1 type: integer - name: MX type: string rows: - [2, 'x'] - [1, 'y'] ... A bit more is written in issue #5890. >> CREATE TABLE t1(m VARBINARY primary key); >> INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379'); >> SELECT count(*), substr(m,2,1) AS m FROM t1 GROUP BY m; >> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; >> >> But it doesn't change behaviour for this: >> >> CREATE TABLE t2(m STRING primary key); >> INSERT INTO t2 VALUES('ax'), ('bx'), ('cy'); >> SELECT count(*), substr(m,2,1) AS m FROM t2 GROUP BY m; >> SELECT count(*), substr(m,2,1) AS mx FROM t2 GROUP BY mx; >> >> Part of #5818 >> Part of #5890 >> --- >> src/box/sql/mem.c | 16 +++++++++++++++- >> src/box/sql/mem.h | 17 ++++++++++++++++- >> src/box/sql/vdbe.c | 18 ------------------ >> src/box/sql/vdbeaux.c | 2 +- >> 4 files changed, 32 insertions(+), 21 deletions(-) >> >> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c >> index 3d42ac63c..a2316cc90 100644 >> --- a/src/box/sql/mem.c >> +++ b/src/box/sql/mem.c >> @@ -2253,7 +2253,8 @@ sqlVdbeRecordCompareMsgpack(const void *key1, >> } >> >> int >> -vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len) >> +vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem, >> + uint32_t *len) > > 2. The function name is getting Java vibes. I propose to rename it to > mem_from_mp_ephemeral() and mem_from_mp() correspondingly. They also should > start taking the mem as a first argument. > Thank you! Fixed. >> { >> const char *start_buf = buf; >> switch (mp_typeof(*buf)) { >> @@ -2354,6 +2355,19 @@ install_blob: >> return 0; >> } >> >> +int >> +vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len) >> +{ >> + if (vdbe_decode_msgpack_into_ephemeral_mem(buf, mem, len) != 0) >> + return -1; >> + if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) { >> + assert((mem->flags & MEM_Ephem) != 0); >> + if (sqlVdbeMemGrow(mem, mem->n, 1) != 0) >> + return -1; > > 3. Maybe it is worth adding a function like mem_materialize() or > mem_make_writable() for that kind of work. > Not sure. For now decided to not add a new function. >> + } >> + return 0; >> +} New patch: commit 8c81c3e86a628e5777145b30f69392dd5a0fd873 Author: Mergen Imeev <imeevma@gmail.com> Date: Sat Mar 13 15:43:38 2021 +0300 sql: rework vdbe_decode_msgpack_into_mem() The original vdbe_decode_msgpack_into_mem() returns a MEM that contains string and binary values as ephemeral. This patch renames this function to mem_from_mp_ephemeral() and introduces new function mem_from_mp(), which returns a MEM that contains string and binary values in newly allocated memory. This patch changes behavior for this query: CREATE TABLE t1(m VARBINARY primary key); INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379'); SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; Before this patch: tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; --- - metadata: - name: COLUMN_1 type: integer - name: MX type: string rows: - [2, 'y'] - [1, 'y'] ... After this patch. tarantool> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx; --- - metadata: - name: COLUMN_1 type: integer - name: MX type: string rows: - [2, 'x'] - [1, 'y'] ... Part of #5818 Closes #5890 diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c index d56fe56c6..7d06e256c 100644 --- a/src/box/sql/mem.c +++ b/src/box/sql/mem.c @@ -2214,7 +2214,7 @@ sqlVdbeRecordCompareMsgpack(const void *key1, } int -vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len) +mem_from_mp_ephemeral(struct Mem *mem, const char *buf, uint32_t *len) { const char *start_buf = buf; switch (mp_typeof(*buf)) { @@ -2315,6 +2315,19 @@ install_blob: return 0; } +int +mem_from_mp(struct Mem *mem, const char *buf, uint32_t *len) +{ + if (mem_from_mp_ephemeral(mem, buf, len) != 0) + return -1; + if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) { + assert((mem->flags & MEM_Ephem) != 0); + if (sqlVdbeMemGrow(mem, mem->n, 1) != 0) + return -1; + } + return 0; +} + void mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var) { diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h index 394055db9..55f8f0c9f 100644 --- a/src/box/sql/mem.h +++ b/src/box/sql/mem.h @@ -520,16 +520,30 @@ int sqlVdbeRecordCompareMsgpack(const void *key1, struct UnpackedRecord *key2); /** - * Decode msgpack and save value into VDBE memory cell. + * Decode msgpack and save value into VDBE memory cell. String and binary string + * values set as ephemeral. * + * @param mem Memory cell to write value into. * @param buf Buffer to deserialize msgpack from. + * @param len[out] Length of decoded part. + * @retval Return code: < 0 in case of error. + * @retval 0 on success. + */ +int +mem_from_mp_ephemeral(struct Mem *mem, const char *buf, uint32_t *len); + +/** + * Decode msgpack and save value into VDBE memory cell. String and binary string + * values copied to newly allocated memory. + * * @param mem Memory cell to write value into. + * @param buf Buffer to deserialize msgpack from. * @param len[out] Length of decoded part. * @retval Return code: < 0 in case of error. * @retval 0 on success. */ int -vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len); +mem_from_mp(struct Mem *mem, const char *buf, uint32_t *len); /** * Perform encoding memory variable to stream. diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 0c19acff5..378c7a043 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -405,26 +405,8 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno, assert(sqlVdbeCheckMemInvariants(dest_mem) != 0); const char *data = vdbe_field_ref_fetch_data(field_ref, fieldno); uint32_t dummy; - if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) + if (mem_from_mp(dest_mem, data, &dummy) != 0) return -1; - - /* - * Add 0 termination (at most for strings) - * Not sure why do we check MEM_Ephem - */ - if (mem_is_str(dest_mem) && mem_is_ephemeral(dest_mem)) { - int len = dest_mem->n; - if (dest_mem->szMalloc < len + 1) { - if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0) - return -1; - } else { - dest_mem->z = - memcpy(dest_mem->zMalloc, dest_mem->z, len); - dest_mem->flags &= ~MEM_Ephem; - } - dest_mem->z[len] = 0; - dest_mem->flags |= MEM_Term; - } UPDATE_MAX_BLOBSIZE(dest_mem); return 0; } diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c index bec8a532a..dff108412 100644 --- a/src/box/sql/vdbeaux.c +++ b/src/box/sql/vdbeaux.c @@ -2358,7 +2358,7 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def, /* Information about the rec pMem->szMalloc = 0; pMem->z = 0; uint32_t sz = 0; - vdbe_decode_msgpack_into_mem(zParse, pMem, &sz); + mem_from_mp_ephemeral(pMem, zParse, &sz); assert(sz != 0); zParse += sz; pMem++; diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c index 91cba9962..ba5c08a00 100644 --- a/src/box/sql/vdbemem.c +++ b/src/box/sql/vdbemem.c @@ -563,7 +563,7 @@ sql_stat4_column(struct sql *db, const char *record, uint32_t col_num, } } uint32_t unused; - return vdbe_decode_msgpack_into_mem(a, mem, &unused); + return mem_from_mp(mem, a, &unused); } /*
next prev parent reply other threads:[~2021-04-09 17:57 UTC|newest] Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-09 16:51 [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 01/52] sql: enhance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches 2021-04-11 17:42 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:01 ` Mergen Imeev via Tarantool-patches 2021-04-13 12:12 ` Mergen Imeev via Tarantool-patches 2021-04-13 23:22 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 23:34 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 02/52] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 03/52] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 04/52] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 22:37 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:51 ` [Tarantool-patches] [PATCH v5 05/52] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 06/52] sql: refactor port_vdbemem_*() functions Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 07/52] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 08/52] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 09/52] sql: introduce mem_str() Mergen Imeev via Tarantool-patches 2021-04-11 17:44 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:36 ` Mergen Imeev via Tarantool-patches 2021-04-14 22:23 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 22:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 16:59 ` [Tarantool-patches] [PATCH v5 10/52] sql: introduce mem_create() Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 11/52] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches 2021-04-11 17:46 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 12:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 12/52] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches 2021-04-11 17:59 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:09 ` Mergen Imeev via Tarantool-patches 2021-04-14 22:48 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:07 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 13/52] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches 2021-04-11 18:06 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:18 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:36 ` [Tarantool-patches] [PATCH v5 14/52] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:31 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:37 ` [Tarantool-patches] [PATCH v5 15/52] sql: rework mem_move() Mergen Imeev via Tarantool-patches 2021-04-11 18:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:38 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` Mergen Imeev via Tarantool-patches [this message] 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 17/52] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches 2021-04-14 22:58 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:14 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 18/52] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches 2021-04-11 18:11 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 16:57 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:04 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:22 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 19/52] sql: introduce arithmetic operations for MEM Mergen Imeev via Tarantool-patches 2021-04-11 18:13 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 17:06 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:10 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:33 ` Mergen Imeev via Tarantool-patches 2021-04-09 17:57 ` [Tarantool-patches] [PATCH v5 20/52] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches 2021-04-11 18:16 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 18:33 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:20 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-14 23:40 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 21/52] sql: introduce bitwise operations for MEM Mergen Imeev via Tarantool-patches 2021-04-12 23:31 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 20:49 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 22/52] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 23/52] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 24/52] sql: introduce mem_set_int() Mergen Imeev via Tarantool-patches 2021-04-12 23:32 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 20:56 ` Mergen Imeev via Tarantool-patches 2021-04-09 18:11 ` [Tarantool-patches] [PATCH v5 25/52] sql: introduce mem_set_uint() Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 26/52] sql: move mem_set_bool() and mem_set_double() Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 27/52] sql: introduce mem_set_str_*() functions Mergen Imeev via Tarantool-patches 2021-04-12 23:34 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 21:36 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:49 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-15 1:25 ` Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 28/52] sql: introduce mem_copy_str() and mem_copy_str0() Mergen Imeev via Tarantool-patches 2021-04-12 23:35 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:00 ` Mergen Imeev via Tarantool-patches 2021-04-14 23:54 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-15 0:30 ` Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 29/52] sql: introduce mem_set_bin_*() functions Mergen Imeev via Tarantool-patches 2021-04-09 19:45 ` [Tarantool-patches] [PATCH v5 30/52] sql: introduce mem_copy_bin() Mergen Imeev via Tarantool-patches 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:06 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 31/52] sql: introduce mem_set_zerobin() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 32/52] sql: introduce mem_set_*() for map and array Mergen Imeev via Tarantool-patches 2021-04-12 23:36 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:08 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 33/52] sql: introduce mem_set_invalid() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 34/52] sql: refactor mem_set_ptr() Mergen Imeev via Tarantool-patches 2021-04-09 20:05 ` [Tarantool-patches] [PATCH v5 35/52] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:19 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 36/52] sql: introduce mem_set_agg() Mergen Imeev via Tarantool-patches 2021-04-12 23:37 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:46 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 37/52] sql: introduce mem_set_null_clear() Mergen Imeev via Tarantool-patches 2021-04-12 23:38 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:50 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 38/52] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches 2021-04-13 20:42 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:25 ` [Tarantool-patches] [PATCH v5 39/52] sql: introduce mem_to_int*() functions Mergen Imeev via Tarantool-patches 2021-04-12 23:39 ` Vladislav Shpilevoy via Tarantool-patches 2021-04-13 22:58 ` Mergen Imeev via Tarantool-patches 2021-04-13 23:10 ` Mergen Imeev via Tarantool-patches 2021-04-09 20:26 ` [Tarantool-patches] [PATCH v5 40/52] sql: introduce mem_to_double() Mergen Imeev via Tarantool-patches 2021-04-13 23:21 ` Mergen Imeev via Tarantool-patches 2021-04-15 0:39 ` [Tarantool-patches] [PATCH v5 00/52] Move mem-related functions to mem.c/mem.h Vladislav Shpilevoy via Tarantool-patches 2021-04-15 6:49 ` Kirill Yukhin via Tarantool-patches
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=8c81c3e86a628e5777145b30f69392dd5a0fd873.1617984948.git.imeevma@gmail.com \ --to=tarantool-patches@dev.tarantool.org \ --cc=imeevma@tarantool.org \ --cc=tsafin@tarantool.org \ --cc=v.shpilevoy@tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH v5 16/52] sql: rework vdbe_decode_msgpack_into_mem()' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox